Configuring Network Adapters for Redundancy

Audience: System Administrators

Date: October 15, 2001

You can protect against network adapter failure using AIX's Etherchannel support. Etherchannel allows you to create a "logical" ethernet adapter out of 2 physical adapters. If the primary network adapter fails, AIX seemlessly moves network traffic to the surviving adapter using the same IP address.

The Etherchannel function is available in AIX 4.3.3+. Documentation is scarce, so here's a quick "how to". In this example, we assume we have two physical adapters (ent1 and ent2) that we want to group as a logical interface:

smit => devices => communications => etherchannel =>"Add an Etherchannel"
(fastpath "smit etherchannel")

                              Add An Etherchannel

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

                                         [Entry Fields]
  Etherchannel Adapters                  ent1 ent2
  Enable ALTERNATE ETHERCHANNEL address  no
  ALTERNATE ETHERCHANNEL address         []
  Mode                                   netif_backup
  Internet Address to Ping               []
  Number of Retries                      [] 
  Retry Timeout (sec)                    []  

This will create a "logical" interface using the next available adapter name ("ent3"). From here, configure an IP address on the logical interface ent3.

Comments:

ALTERNATE ETHERCHANNEL = yes allows you to specify a "MAC" address. (Not necessary)

Mode = round-robin provides higher aggregate bandwidth by distributing outgoing packets over multiple network adapters. However "round-robin" does not distribute incoming packets, nor does it provide adapter failover. Use "netif_backup" if you want failover.

Internet Address to Ping: IP address for "keep alive". Optional as most failures detected at link layer

Bruce Spencer,
baspence@us.ibm.com

EtherChannel Readme File

Etherchannel

Introduction

Etherchannel is an aggregation technology that allows you to produce a single large pipe by combining the bandwidth of multiple ethernet adapters. The aggregated pipe appears to be single ethernet interface to the upper layers (IP, etc.). Each of the ethernet adapters will share a single MAC address. There is a single ethernet interface associated with the aggregated pipe, which has a single IP address as well.

Hardware

The ethernet adapters inside which are members of the etherchannel can be any supported ethernet adapter.  The ethernet adapters must be connected to a switch which supports Etherchannel

.

Configuration

In AIX 4.3.3, a single etherchannel can combine either two or four ethernet adapters in the channel.

You can configure multiple etherchannels on a single system. Be aware that each etherchannel constitutes an additional ethernet interface. The "no" option, ifsize (default 8), needs to be increased to include not only the ethernet interfaces for each adapter, but also any etherchannels that are configured. See the "no" command documentation in regards to modifying ifsize.

The adapters in an etherchannel need to be configured to the same speed (10 or 100 Mbps) and mode (half or full duplex).
 

1. Configuring Etherchannel

 

 

Etherchannel is configured through smit. I will use the following setup to show how etherchannel is configured: there will be 2 ethernet adapters, ent0 and ent1, in the etherchannel and there is no etherchannel alternate address specifed. There are 5 ethernet adapters to choose from.

Execute:

# smitty etherchannel

which produces a screen that looks like:

Etherchannel Move cursor to desired item and press Enter.   List All Etherchannels   Add An Etherchannel   Change / Show Characteristics of an Etherchannel   Remove An Etherchannel

Then choose "Add An Etherchannel" selection:

Etherchannel Move cursor to desired item and press Enter.   List All Etherchannels   Add An Etherchannel   Change / Show Characteristics of an Etherchannel   Delete An Etherchannel    --------------------------------------------------------------------------   |                        Available Network Adapters                        |   |                                                                          |   | Move cursor to desired item and press F7.                                |   |     ONE OR MORE items can be selected.                                   |   | Press Enter AFTER making all selections.                                 |   |                                                                          |   | > ent0                                                                   |   | > ent1                                                                   |   |   ent2                                                                   |   |   ent3                                                                   |   |   ent4                                                                   |   |                                                                          |   | F1=Help                 F2=Refresh              F3=Cancel                |   | F7=Select               F8=Image                F10=Exit                 | F1| Enter=Do                /=Find                  n=Find Next              | F9 --------------------------------------------------------------------------

As instructed, select one more more of the listed adapters.

NOTE

The "Available Network Adapters" displays all of the ethernet adapters. If one selects an ethernet adapter that is already being used (i.e. has an interface defined), then one will get an error similar to:  Method error (/usr/lib/methods/cfgech):         0514-001 System error: Method error (/usr/lib/methods/chgent):         0514-062 Cannot perform the requested function because the                  specified device is busy.

Add an Etherchannel Type or select values in entry fields. Press Enter AFTER making all desired changes.                                                         [Entry Fields]   Etherchannel Adapters                               ent0 ent1               +   Enable ALTERNATE ETHERCHANNEL address               no                      +   ALTERNATE ETHERCHANNEL address                     []                       +   Mode                                                standard                +   Enable GIGABIT ETHERNET JUMBO frames                no                      +   Internet Address to Ping                           []   Number of Retries                                  []                        #   Retry Timeout (sec)                                []                        #

After hitting the enter key:

COMMAND STATUS Command: OK            stdout: yes           stderr: no Before command completion, additional instructions may appear below. ent5 Available

NOTE	Since an alternate address was not specified, the config method for the etherchannel changed the alternate address of ent1 to the hardware address of ent0. If an alternate address was specified, both ent0 and ent1 would have that alternate address.

Then add an interface, en5 for standard ethernet or et5 for IEEE 802.3, through the normal means. The easiest way is to use the "Minimum Configuration & Startup" TCP/IP smit menu.

2. Listing Etherchannels

 

 

Execute:

# smitty etherchannel

and choose the "List All Etherchannels" selection:

Etherchannel Move cursor to desired item and press Enter.   List All Etherchannels   Add An Etherchannel   Change / Show Characteristics of an Etherchannel   Remove An Etherchannel                                  COMMAND STATUS Command: OK            stdout: yes           stderr: no Before command completion, additional instructions may appear below. ent5       Available               Etherchannel

3. Changing the alternate address

 

 

Execute:

# smitty etherchannel

Then choose "Change / Show Characteristics of an Etherchannel" selection:

Etherchannel Move cursor to desired item and press Enter.   List All Etherchannels   Add An Etherchannel   Change / Show Characteristics of an Etherchannel   Remove An Etherchannel    --------------------------------------------------------------------------    |                         Available Etherchannels                          |    |                                                                          |    | Move cursor to desired item and press Enter.                             |   |                                                                          |   |   ent5                                                                   |   |                                                                          |   | F1=Help                 F2=Refresh              F3=Cancel                |   | F8=Image                F10=Exit                Enter=Do                 | F1| /=Find                  n=Find Next                                      | F9 --------------------------------------------------------------------------

To change the Alternate etherchannel address to 1234deadbeef:

Change / Show Characteristics of an Etherchannel Type or select values in entry fields. Press Enter AFTER making all desired changes.                                                         [Entry Fields]   Etherchannel                                        ent5                    +   Enable ALTERNATE ETHERCHANNEL address               yes                     +   ALTERNATE ETHERCHANNEL address                     [0x1234deadbeef]         +

After hitting the enter key:

COMMAND STATUS Command: OK            stdout: yes           stderr: no Before command completion, additional instructions may appear below. ent5 changed

NOTE	If one wants to add or delete an ethernet adapter from the etherchannel, one must use the "Remove an Etherchannel" smit menu selection, then re-add the etherchannel with the "Add An Etherchannel" smit menu selection.

Unconfiguring the Etherchannel
Execute:

# smitty etherchannel

Then choose "Remove an Etherchannel" selection:

Etherchannel Move cursor to desired item and press Enter.

Choose 'ent5', then hit the enter key:

COMMAND STATUS Command: OK            stdout: yes           stderr: no Before command completion, additional instructions may appear below. ent5  deleted ent0 changed ent1 changed

Troubleshooting
tcpdump and trace can be used to troubleshoot the etherchannel. The trace hook id for the transmit packets is 2FA and for other events is 2FB. You cannot trace receive packets on the channel as a whole, but you can trace each of the adapter's receive trace hooks.

 

 

The entstat command will give the aggregate statistics of all the adapters in an etherchannel. If entstat is run with the "-d" option, the statistics of each of the individual adapters is printed.

Restrictions
Remote dump is not supported over an etherchannel.
4. Modes

 

 

There are three modes for Etherchannel.  Each control the method the channel uses to send and receive packets.
 

Standard
In this mode the channel will hash on the destination IP address and use this value to choose which adapter it will send the packet out on.  This mode will guarantee packets are sent out over the channel in the order they were sent, but may not make full use of the bandwidth.
Round Robin
In this mode the channel will rotate through the adapters, giving each adapter one packet before repeating.  In this mode packets may be sent out in a slightly different order than they were given to the channel, but the channel will make the best use of its bandwidth.
Network Interface Backup
In this mode, the channel will only activate one adapter at a time.  The intention is that the adapters are plugged into different ethernet switches, each of which is capable of getting to any other machine on the subnet/network.    When a problem is detected, either with the direct connection, or through inability to ping a machine, the channel will deactivate the current adapter, and activate a backup adapter.
The channel will poll the adapter for Link Status in "Retry Timeout" intervals.  If the Link Status is not up (either due to a cable being unplugged, switch down, or device driver problem) , the channel will switch to another adapter.
This mode is the only one that makes use of the "Internet Address to Ping", "Number of Retries", and "Retry Timeout" fields.
5. Internet Address to Ping
The address will be ping'd if the address field has a non-zero address and the mode is set to netif_backup.  If the channel is unable to ping the address for the "Number of Retries" times in "Retry Timeout" intervals.  The channel will switch adapters.
6. Number of Retries
The number of retries is the number of ping response failures before the channel switches adapters.  The default is 3 times.
7. Retry Timeout
The retry timeout is the interval in seconds between the times when the channel will send out a ping packet and poll the adapter's Link Status.  The defauls is 1 sec intervals.

 

 

"Etherchannel" is a registered trademark of Cisco Systems, Inc.