Connect ESX Host To Cisco Switch With Gigabit EtherChannel

ESX 4.1 Host

The vSwitch0 virtual switch was created during installation of ESX software on the host. Service concole port (vswif0) and the "VM Network" port group are both connected to this switch.

In vSphere Client (Configuration > Networking) default setup will be depicted like this:

The same information can be displayed from the command line on ESX host as well.

# esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         3           128               1500    vmnic0

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VM Network            0        0           vmnic0
  Service Console       0        1           vmnic0

#

1. Add a VMkernel port group and configure its IP settings (required for VMotion).

# esxcfg-vswitch -A VMKernel vSwitch0
# esxcfg-vmknic -a -i 10.10.10.20 -n 255.255.255.0 VMKernel

2. Enable VMotion through the new port group.

# esxcfg-vmknic -l |awk '/VMKernel/ {print $1}'
vmk3
# vmware-vim-cmd /hostsvc/vmotion/vnic_set vmk3

3. Link three additional physical network adapters to the same virtual switch.

# esxcfg-vswitch -L vmnic1 vSwitch0
# esxcfg-vswitch -L vmnic6 vSwitch0
# esxcfg-vswitch -L vmnic7 vSwitch0
#
# esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         7           128               1500    vmnic0,vmnic1,vmnic6,vmnic7

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VM Network            0        0           vmnic0,vmnic1,vmnic6,vmnic7
  VMKernel              0        1           vmnic0,vmnic1,vmnic6,vmnic7
  Service Console       0        1           vmnic0,vmnic1,vmnic6,vmnic7

#

Corresponding graphical presentation in vSphere Client will transform to the following:

4. Configure load balancing policy based on a hash of the source and destination IP addresses.

# vmware-vim-cmd /hostsvc/net/vswitch_setpolicy --nicteaming-policy=loadbalance_ip vSwitch0

5. Disable rolling failover for faulty EtherChannel members.

# vmware-vim-cmd /hostsvc/net/vswitch_setpolicy --nicteaming-rollingorder=true vSwitch0

In vSphere Client, on the NIC Teaming tab in vSwitch0 properties, equivalent setting is now called Failback (set it to "No").

6. Optionally, adjust number of ports supported by the virtual switch. Reboot will be required for this new setting to take effect.

# vmware-vim-cmd /hostsvc/net/vswitch_setnumports vSwitch0 256

Cisco C2960G

On a Cisco switch configure static (without PAgP or LACP) four-port 802.3ad channel group (interfaces Gi0/13 thru Gi0/16 are all configured identically).

port-channel load-balance src-dst-ip
!
interface Port-channel1
 switchport access vlan 301
 switchport mode access
 spanning-tree portfast
!
interface GigabitEthernet0/13
 switchport access vlan 301
 switchport mode access
 channel-group 1 mode on
 spanning-tree portfast
!

If one VLAN is not sufficient for all the virtual machines that will be hosted on this ESX server, trunk port group can be configured on the switch (and then additional port group(s) added to the vSwitch0 virtual switch on the ESX hosts).

port-channel load-balance src-dst-ip
!
interface Port-channel1
 switchport trunk native vlan 301
 switchport mode trunk
 spanning-tree portfast trunk
!
interface GigabitEthernet0/13
 switchport trunk native vlan 301
 switchport mode trunk
 channel-group 1 mode on
 spanning-tree portfast trunk
!

See Also

​http://www.vmware.com/resources/techresources/997
VMware Virtual Networking Concepts (VMware Technical Papers)

​http://www.cisco.com/en/US/docs/switches/lan/catalyst2960/software/release/12.2_52_se/configuration/guide/swethchl.html
Configuring EtherChannels and Link-State Tracking (Catalyst 2960 Switch Software Configuration Guide)

​http://en.wikipedia.org/wiki/Link_aggregation
Link aggregation (Wikipedia)