Cisco Unified Computing System (UCS) uses its Virtual Interface Cards (VIC) to connect directly to the Fabric Interconnects. At this layer, Linux operating systems can use the bonding subsystem to logically combine multiple physical interfaces. Since the VIC card is not a switch in its own right, special restrictions apply to switch-dependent modes such as LACP or static trunking. In UCS environments, only switch-independent bonding variants are therefore recommended, where load distribution is based solely on the source MAC address. This approach prevents MAC flapping between the two Fabric Interconnects and ensures deterministic failover behavior.
The Linux bonding driver supports eight modes (0–7). For UCS Fabric Interconnects, only those modes make sense that either provide simple failover or distribute load based on the source MAC address. The relevant modes are briefly explained below:
-
Mode 0 – balance-rr (Round Robin)
Packets are sent sequentially across all slave interfaces. Since the distribution is not based on the MAC address, UCS can experience uneven load and potential flapping issues. This mode is therefore only recommended in very specific scenarios where the underlying switch topology routes packets through a single Fabric Interconnect only. -
Mode 1 – active-backup
Only one slave is active, while the others are on standby. If the active link fails, a replacement takes over immediately. This method requires no switch configuration at all and is fully compatible with the UCS environment, because traffic always flows through a single Fabric Interconnect until a failover occurs. -
Mode 2 – balance-xor
The slave is selected using an XOR hash of source and destination (MAC, IP, or port). Thexmit_hash_policyoption lets you choose the layer the hash is based on. For UCS, the policy must be set tolayer2(source MAC only), otherwise the flapping problem can occur. Withxmit_hash_policy=layer2+srcmac, you get stable load balancing that only takes the source MAC into account. -
Mode 3 – broadcast
All packets are sent on every slave. This mode generates unnecessary traffic and is rarely used in production UCS environments. -
Mode 4 – 802.3ad (LACP)
Requires a port channel on the remote side. Since the Fabric Interconnect cannot act as an LACP partner for the VIC card, this mode leads to MAC flapping and is not supported. -
Mode 5 – balance-tlb (Adaptive Transmit Load Balancing)
Outgoing traffic is distributed based on the current load of each slave, while incoming traffic is received by the current slave. Since no special switch configuration is required and the method only works on the outgoing side, it is compatible with UCS as long as incoming traffic is handled through a single Fabric Interconnect. -
Mode 6 – balance-alb (Adaptive Load Balancing)
Combines TLB with receive load balancing via ARP negotiation. Like TLB, it requires nothing on the switch side and works with UCS as long as ARP handling does not cause MAC addresses to migrate between the Fabric Interconnects. -
Mode 7 – balance-tcp (not part of the standard kernel, occasionally available as a patch)
Not part of the official UCS documentation and therefore not considered further.
To avoid known issues, Cisco recommends the kernel option fail_over_mac=1 for modes 2, 5, and 6. This setting ensures that during a failover the MAC address of the new active slave is used, which prevents stale MAC entries from being recorded on the Fabric Interconnects. Without this option, mode 2 with an incorrect xmit_hash_policy in particular can lead to persistent MAC flapping, which may cause performance degradation or even loss of network connectivity.
In practice, the following configurations have proven effective:
- Active-backup (mode 1) for maximum simplicity and the highest failover reliability.
- Balance-xor with
xmit_hash_policy=layer2+srcmac(mode 2) for active-active load balancing, as long as packets are routed through a single Fabric Interconnect only. - Balance-tlb (mode 5) or balance-alb (mode 6) when dynamic load distribution on the sending side is desired without having to configure the switch side.
None of the modes mentioned require any special switch or UCS configuration beyond the bonding setup itself. This allows Linux hosts to be connected to UCS Fabric Interconnects while retaining the benefits of redundancy and, if desired, load balancing. For environments connected through a single Fabric Interconnect only (e.g. in smaller edge deployments), these options provide a stable and easy-to-manage network connection.