Hosts Network issue with vDS that has no adapters

All,

I have a home lab that I'm trying to setup and it seems to be giving me a little issues with the vDS and the two Hosts I have in the cluster.

I have vCenter 5.0 with two ESXi hosts that has four NICs in each host.

I setup each Host with a vSS for the Management Network that has two pNIC's. 

Then I created two vDSv5.0 switches, one for my Home Network (vDS-PHRAMC) and the other for my LAB environment (vDS-PGTLAB).

dvPHRAMC:

• This has two pNICs associated from each host (vmnic1 and vmnic2).
• This is has the home PC's and Domain VM's running on this network (192.168.10.0)

dvPGTLAB:

• This has no pNICs associated, because it is a internal private network. I use pfSense to get the PGTLAB access to the internet.
• This is has the Lab Domain and VM's running on this network (10.2.0.0)

No for the issue;

     I currently have all PGTLAB VMs hosted on ESXi01. the network communications work fine. It's when I migrate a VM from ESXi01 to ESXi02 that I loose my network communication. I do have DHCP on the PGTLAB network, but I tried changing to static IP and still no luck.

     If I move the VM back to ESXi01, it works fine and communicates with the other VMs.

     I also noticed the migration of the VM is fast going from ESXi01 to ESXi02, but very slow from ESXi02 to ESXi01.

Can anyone help me in fixing this issue or help me setup this in the proper way, if this is not the best way?

-Mike