Advertisements

Configuring Ether-Channel for Vmware ESX 3.5 on a Cisco 4503 Catalyst

Find this post interesting? Do you like interesting things? Maybe you would like my invention, a connectible candle called a WickBrick!

Get one here http://wickbrick.etsy.com/

WickBrick

I am by no means a Cisco expert, not even close… so take this post with a huge grain of salt. It hopefully will help you at least get close to what you want, which is Nic Teaming on your Vmware server.

In our case our vmware servers have 6 Nics, that are configured like this.

esx_network_settingsSo vmnic0 (zero)  is for the service console, nic1 is a service console and for vmotion, and nic2 – 5 are teamed for the virtual machines.

So, now on to the Cisco config. So by design, ESX handles the outgoing load balancing, but to get incoming load balancing you need to configure your switch. This post talks a little bit about it

http://virtrix.blogspot.com/2006/11/vmware-switch-load-balancing.html

And I think it is a similar setup. I notice that that post uses spanning-tree portfast trunk but I’m not sure what that does 🙂

So to configure your Cisco

telnet into your switch

put in your password

type en

put in your admin password

Ok, so we have three groups we wanted to setup,  group 1 on ports 1-4 group 2 on ports 5-8 group 3 on 9-12

so type config t

then

int range G2/1 – 4     (notice there is a space between the 1 and the dash and the dash and the 4) (I believe the g is for gigabit)

channel-group 1 mode on  (we had a consultant use access and then caused a network storm…. )

then type

exit

so if you type show run

you should see blade 2 ports 1 – 4 as part of channel-group 1 and the mode should be on

now type

interface port-channel1

sw tr en dot

sw mode tr

sw non

exit

and then

wr mem (to write out your settings)

Hopefully these notes help somebody out there!

You might also want to take a look at http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/

Here is what my config looks like….

interface Port-channel1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
!
interface Port-channel2
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
!
interface Port-channel3
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
!
interface GigabitEthernet1/1
!
interface GigabitEthernet1/2
!
interface GigabitEthernet2/1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
!
interface GigabitEthernet2/2
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
!
interface GigabitEthernet2/3
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
!
interface GigabitEthernet2/4
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
!
interface GigabitEthernet2/5
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 2 mode on
!
interface GigabitEthernet2/6
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 2 mode on
!
interface GigabitEthernet2/7
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 2 mode on
!
interface GigabitEthernet2/8
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 2 mode on
!
interface GigabitEthernet2/9
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 3 mode on
!
interface GigabitEthernet2/10
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 3 mode on
!
interface GigabitEthernet2/11
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 3 mode on
!
interface GigabitEthernet2/12
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 3 mode on
!

Advertisements

4 Responses

  1. I wouldn’t configure the vSwitches and NICs like that in ESX. in this way, your VMotion has no back up if vmnic1 fails, same for the port group for VMs, if all those 4 nics fail, even if your vmnic 0 and 1 are alive and having no much traffic, they can’t help the VMs at all.

    I would prefer single vSwitch with all 6 NICs, and use VLAN trunking / Port group, for different services like SC, VMotion, other VM networks. And specify active and standby vmnics for different port groups. In this way, any of the nic can back up any failed nic. and your ESX and VM will run fine with even just 1 nic left alive, and it can be any of the 6 nics. It gives you much more redunancy.

  2. BTW an undocumented 4503 issues is that groups of ports share the same chip and are limited ot an aggregate of 1Gb. I believe it’s every 8 ports so you want to spread out your connections out if you want to get more than 1Gb to your server. G2/1, G2/9, G2/17 in one port group for one ESX server etc.

  3. You may be correct in some instances. However if you use a WS-X4306 or WS-X4506 linecard they are not over subscrived with any 4500 supervisor. This also depends on the supervisor and linecards used.

  4. […] M2 with 2 processors, 64 GB of RAM. 2 nics four ports (etherchannel bonded on incoming Cisco like this)  2 nics four ports for iscsi traffic. And a nic for the service console, and another gig nic for […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: