Skip navigation
Cisco Learning Home > CCNP R&S Study Group > Discussions
4659 Views 8 Replies Latest reply: Apr 20, 2012 12:48 AM by Tim RSS

Currently Being Moderated

Distributing Traffic in EtherChannel

Dec 17, 2011 5:42 PM

Harry 93 posts since
Nov 29, 2010

Can anyone explain how does Etherchannel hash algorithm work?

 

I undestand that hash algo is required to choose a port to forward traffic out which helps to distribute traffic on a switch , now how do we calculate 0 -7 range and use to select the port , I could not understand the computation in it.

 

I was going through the blog below which clarifies my doubts bit more.

 

"" On most of the CISCO switches platform, default ether-channel load-balance algorithm is "XOR source and destination IP " to do the load-balance. The load-balancing is purely based on hashing provided by the hardware, which is in turn based on flows. You can essentially end up with a load balance setup that never changes links depending on your configured flows. You can choose whether to hash on Source or Destination Layer2 (MAC), Layer3(IP Address), or Layer4(Ports) flow information, but hash algorithm cannot be configured or changed to load balance the traffic among the ports in an Etherchannel.

Typically, you will need to play with your Source and Destination settings to find which settings used with your traffic flows creates the best load balance.

The Cisco-proprietary hash algorithm computes a value in the 0-7 range. With this value as a basis, a particular port in the Etherchannel is chosen. The port setup includes a mask which indicates which values the port accepts for transmission. With the maximum number of eight ports in a single Etherchannel, each port accepts only one value. If the Etherchannel has four ports, each port accepts two values, and soon  ""   rest on the link below :

https://supportforums.cisco.com/community/netpro/network-infrastructure/switching/blog/2011/08/09/how-etherchannel-hash-algorithm-works-and-load-distribution-happens

 


 

 

 


  • Currently Being Moderated
    1. Dec 17, 2011 9:09 PM (in response to Harry)
    Re: Distributing Traffic in EtherChannel

    Hi Harry,

     

    The XOR operation is done on the last 3 bits of the source and destination address which would then correspond to a value of 0 - 7, which is in turn assigned to one of the ports in the channel. If you have 8 ports in the channel, then you get a 1 to 1 ratio of values assigned to ports. If you have less than 8 ports in the channel then you start to get ports with multiple values assigned.

     

    For instance, if you have 4 ports in the channel, each port would be assigned 2 values. If you have 3 ports in the channel, you would get a 3-3-2 ratio. 

     

    I can give a brief explanation of the XOR operation. With XOR or exclusive-or, you take two binary numbers, perform a XOR, and get either a 1 or 0 depending on the binary values. Basically XOR will only give a 1 when you have binary values of 1 and 0 that the operation is being performed on.

     

    So when comparing bit streams, in this case, last 3 of source and destination, when you have two 1s you get a 0, two 0s you get a 0, and 1 and 0 you get a 1.

     

    Here is an example:

     

    Source IP: 192.168.1.1

    Dest IP    : 192.168.1.2

     

    Source last 3 in binary: 001

    Dest last 3 in binary:    010

     

    The switch does a bit-by-bit comparison, which corresponds, from left to right, to: 011. For a port value of 3.

     

    Now lets say we have 2 ports in the channel, this would correspond to 4 values being assigned to each port. The first port would have values of 0,2,4,6 and the second port values of 1,3,5,7.

     

    So with our value calculated above of 3, this traffic would flow over the 2nd port in the channel.

     

     

    Hope this helps,

     

    DelVonte

  • Currently Being Moderated
    3. Dec 18, 2011 9:43 AM (in response to Harry)
    Re: Distributing Traffic in EtherChannel

    So the number of ports in the channel will determine the ratio of values assigned to each port. So with a 2 port channel, we will have 4 values assigned to each port, for a 4-4 ratio.

     

    So lets call the 2 ports A and B. The values are assigned in order from port A to B, and then to A and B again. ( I don't recall how Cisco chooses which port is A and B at the moment.) So A will be assigned 0, then B will be assigned 1, then A 2, and B 3, and so on and so forth, until no more values are remaining.

     

    With a 3 port channel, you will have a 3-3-2 ratio of values assigned. So lets say we have ports A, B, and C. A will be assigned 0, B assigned 1, and C assigned 2, then A 3, B 4, C 5, A 6, B 7.

     

    Here is a table:

     

    Etherchannel Port to Value Table.JPG

     

    I hope this clarifies it.

    DelVonte

  • Warren Sullivan - CCNP 934 posts since
    Jun 4, 2010
    Currently Being Moderated
    5. Dec 18, 2011 2:36 PM (in response to DelVonte)
    Re: Distributing Traffic in EtherChannel

    Excellent answer Del

  • Tim 2 posts since
    Jan 29, 2012
    Currently Being Moderated
    6. Apr 19, 2012 8:17 AM (in response to Warren Sullivan - CCNP)
    Re: Distributing Traffic in EtherChannel

    Guys, all your comments look logic to me... just with one concern.

     

    All documents, books, articles state that EC load balancing method uses up to three last bits of the address (MAC source/destination or combination of both, or same for IP) to calculate the outbound interface. But... let me show you this

     

    SWITCH#show etherchannel load-balance

    EtherChannel Load-Balancing Configuration:

            src-mac

     

    Next, I perform testing using various of source MAC addresses. First four look ok to me.

     

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.cccc.aaaa ffff.dddd.eeee
    Would select Gi1/0/26 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.cccc.aaab ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.cccc.aaac ffff.dddd.eeee
    Would select Gi1/0/26 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.cccc.aaad ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

     

    Assuming EC load balance uses last three bits to distribute traffic across interfaces (in my case there are 2 bundle members). How can it change an outbound interface when I change other MAC address bits (without even touching last three)? See below.

     

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.ccca.aaad ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.ccca.aaab ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.cccb.aaad ffff.dddd.eeee
    Would select Gi1/0/26 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeee.accb.aaad ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeea.accb.aaad ffff.dddd.eeee
    Would select Gi1/0/26 of Po8

    SWITCH#test etherchannel load-balance interface port-channel 8 mac eeed.accb.aaad ffff.dddd.eeee
    Would select Gi3/0/1 of Po8

     

    This is tested on live environment with Cisco 3750 on both ends.

     

    That's definitely something... I will play tomorrow in my lab using 4 ports and 3750/3560.

     

    Tim

  • Currently Being Moderated
    7. Apr 19, 2012 9:51 AM (in response to Tim)
    Re: Distributing Traffic in EtherChannel

    That's interesting. You know, now that I think about it, Cisco does not really say how their hash algorithm works, or what method they use to determine what traffic flows over each port. Which makes sense because it is proprietary. When I look at what I wrote here, I'm making assumptions on how it chooses the port value, but I'm not sure they are accurate.

     

    I think I'm over simplifying it...Now what they do say is that the value assigned to the ports in the channel is 0 - 7. They could be, and most likely are, using a method to define the value of the port that is separate from the calculation of which port to use.

     

    Below is Cisco's diagram for port values:

     

    The Cisco-proprietary hash algorithm computes a value in the range 0 to 7. With this value as a basis, a particular port in the EtherChannel is chosen. The port setup includes a mask which indicates which values the port accepts for transmission.With the maximum number of ports in a single EtherChannel, which is eight ports, each port accepts only one value. If you have four ports in the EtherChannel, each port accepts two values, and so forth. This table lists the ratios of the values that each port accepts, which depends on the number of ports in the EtherChannel:

    Number of Ports in the EtherChannel Load Balancing
    81:1:1:1:1:1:1:1
    72:1:1:1:1:1:1
    62:2:1:1:1:1
    52:2:2:1:1
    42:2:2:2
    33:3:2
    24:4

    Note: This table only lists the number of values, which the hash algorithm calculates, that a particular port accepts. You cannot control the port that a particular flow uses. You can only influence the load balance with a frame distribution method that results in the greatest variety.

    Note: The hash algorithm cannot be configured or changed to load balance the traffic among the ports in an EtherChannel.

    Note: The same Cisco-proprietary hash algorithm is also implemented in Cisco Catalyst 6500/6000 Series Switches that run Cisco IOS software.

    Hence, in essence, you can only achieve perfect load balancing, even with random addresses, if you have two, four, or eight ports in the port channel.

     

     

     

     

    The first note is probably the key, where it says the hash is used to calculate the values which a port accepts. It also states that you cannot control the port that a particular flow uses.

     

     

     

     

    That would explain your results, but it still doesn't really give us much insight into how Cisco determines which port to send traffic through. Above the table, it also states that there is mask which indicates which values each port accepts. That's really all the information they give us.

     

     

    I imagine it is hardware dependent. Which means that different devices will have different ways of determining which ports to use. It may even be channel dependent, which means it can be really unpredictable. I guess the key would be to use the appropriate load-balancing method, and not really worry about the other details.

     

     

     

    Regards,

     

    DelVonte

  • Tim 2 posts since
    Jan 29, 2012
    Currently Being Moderated
    8. Apr 20, 2012 12:48 AM (in response to DelVonte)
    Re: Distributing Traffic in EtherChannel

    DelVonte, I agree with you.

    This is something proprietary and platform dependent. Moreover, I found that "test etherchannel load-balance" command does not display a correct port every time (at least on Cisco 3750) and to be honest I won't rely on its output. I was doing a troubleshooting one day for one of the office in our company, that involved load balancing of the Etherchannel and Cisco 3750. I was completely frustrated about EC related show commands on Cisco 3750 and their behavior. For example,

     

    This is Cisco 6509

     

    C6509#sh etherchannel 71 port
                    Ports in the group:
                    -------------------
    Port: Gi7/17
    ------------

    Port state    = Up Mstr In-Bndl
    Channel group = 71          Mode = On      Gcchange = -
    Port-channel  = Po71        GC   =   -         Pseudo port-channel = Po71
    Port index    = 0           Load = 0x55        Protocol =    -

    Age of the port in the current state: 282d:00h:45m:23s

    Port: Gi8/1
    ------------

    Port state    = Up Mstr In-Bndl
    Channel group = 71          Mode = On      Gcchange = -
    Port-channel  = Po71        GC   =   -         Pseudo port-channel = Po71
    Port index    = 1           Load = 0xAA        Protocol =    -

    Age of the port in the current state: 263d:09h:16m:02s

     

    As you may see, this command display EC information in a right manner - we can see port's index in the EC and we can see HASH for load balance (0x55 and 0xAA). Now, trying the same on Cisco 3750!

     

    =====================

     

    C3750#show etherchannel 8 port
                    Ports in the group:
                    -------------------
    Port: Gi1/0/26
    ------------

    Port state    = Up Mstr In-Bndl
    Channel group = 8           Mode = On              Gcchange = -
    Port-channel  = Po8         GC   =   -             Pseudo port-channel = Po8
    Port index    = 0           Load = 0x00            Protocol =    -

    Age of the port in the current state: 7d:12h:18m:30s

    Port: Gi3/0/1
    ------------

    Port state    = Up Mstr In-Bndl
    Channel group = 8           Mode = On              Gcchange = -
    Port-channel  = Po8         GC   =   -             Pseudo port-channel = Po8
    Port index    = 0           Load = 0x00            Protocol =    -

    Age of the port in the current state: 9d:10h:28m:51s

     

    Check ports indexes and load balance hashes - they are not displayed by this command. What Cisco tech recommended to use on Cisco 3750 to check hashing is "show platform pm group-masks" (btw, this command is not available on Cisco 6500 series). This is the output

     

    C3750#show platform pm group-masks

                        Etherchannel members and group masks table
    Group #ports group frame-dist slot port mask interface index
    --------------------------------------------------------------------
    ----- cut for brevity ----

                                  1    25   5555 Gi1/0/25  0
                                  2    1    AAAA Gi2/0/1   1
    8    2      8     src-mac
                                  3    1    AAAA Gi3/0/1   0
                                  1    26   5555 Gi1/0/26  1
    9    0      9     src-mac

    ----- cut for brevity ----

     

    We see the masks - 0x5555 and 0xAAAA, for channels with four members masks will be 0x7777, 0xBBBB, 0xDDDD and 0xEEEE (seen that in a lab)... so, I assume these hashes can tell us what port will be used. Although, the way they use those hashes is propritery... I tried to dig deeper but stuck, and agree with you there's no sense to do that, we just have to accept they DO load balance with accordance to the method we select

     

    =============

     

    So, DO NOT USE "test etherchannel load-balance" command on Cisco 3750 as it won't tell you the right port. I say that because I tested it in a live environment (Cisco 3750 from both ends, two ports channel, load balance - source MAC, one end was core with SVI, other end was setup as Layer 2 switch serving that SVI's subnet), I did the following

     

    1) Cleared counters on both stacks

    2) Downloaded a large file from Internet from the computer which sits behind SVI's subnet and on a remote L2 switch

    3) Checked interfaces counters and found that OUTBOUND counters had increased for interface A (on the L3 switch)

    4) Executed "test etherchannel load-balance" for affected MAC addresess (source MAC - SVI's, destination - PC, although destination MAC is not relevant here) and this command told me it has to be interface B.

     

    That was discussed with Cisco tech again, and he told this

     

    ==========

    Going forward, In regards to “Why test ether-channel load-balance gives incorrect outputs", I did some research in our database and found the following mentioned IOS bug, which is Minor and should not impact the switch stability:

     

    CSCeg68763  Win05: the test etherchannel load-balance cli gave wrong output

     

    Description:

    The "test etherchannel load-balace " command does not give correct output for some destination mac address…

     

    To solve this issue I would suggest you to use the following mentioned command to give you the right info:

     

    show platform forward <”Fast or Gig” x/y/z> vlan # Source-MAC Destination-MAC

    ==========

    Hope that helps!

    Tim

Actions

More Like This

  • Retrieving data ...

Bookmarked By (3)