Configure Link Aggregation

This chapter describes the procedures to configure Link Aggregation on Cisco NCS 4000 Series routers.

Link Aggregation Overview

Link Aggregation (LAG) is a mechanism used to aggregate physical interfaces or ports to create a logical entity called link bundle.

Traditionally LAG is a trunking technology that groups together multiple full-duplex IEEE 802.3 Ethernet interfaces to provide fault-tolerant high-speed links between switches, routers, and servers. LAG forms a single higher bandwidth routing or bridging endpoint and was designed primarily for host-to-switch connectivity. Following are the benefits:

  • Logical aggregation of bandwidth

  • Load balancing

  • Fault tolerance

In NCS 4000 Series Routers, primary application of LAG is to provide connectivity to Access devices like NCS4200 Series and on the core side provide connectivity to Multi-service Edge (NCS 6000 Series) and Core Routers (like NCS 6000 Series).

Understanding Link Bundle

A link bundle is a group of one or more ports that are aggregated or bundled together and act as a single link. This single link can be treated as a main interface or as a VLAN subinterface.

The advantages of link bundles are these:

  • Multiple links can span several line cards to form a single interface. Thus, the failure of a single link does not cause a loss of connectivity.

  • Bundled interfaces increase bandwidth availability, because traffic is forwarded over all available members of the bundle. Therefore, traffic can flow on the available links if one of the links within a bundle fails. Bandwidth can be added without interrupting packet flow.

NCS 4000 Series XR software supports following methods of forming bundles of Ethernet interfaces:

  • IEEE 802.3ad—Standard technology that employs a Link Aggregation Control Protocol (LACP) to ensure that all the member links in a bundle are compatible. Links that are incompatible or have failed are automatically removed from a bundle.

  • Static-LAG—Cisco proprietary technology that allows the user to configure links to join a bundle, but has no mechanisms to check whether the links in a bundle are compatible.

Characteristics and Limitations of Link Bundles

This list describes the properties and limitations of link bundles:

  • 10 Gigabit, 40 Gigabit, and 100 Gigabit Ethernet interfaces can be bundled, with or without the use of LACP (Link Aggregation Control Protocol).

  • Bundle membership can span across several line cards that are installed in the same chassis for NCS4000.

  • The Cisco NCS 4000 Series Router supports a maximum of 128 Ethernet link bundles, 1000 Ethernet link bundles sub-interfaces. Each link bundle can have a maximum of 16 physical links.

  • All the members in a link bundle shall be of same speed.

  • Physical layer and link layer configuration are performed on individual member at physical interface layer.

  • Configuration of network layer protocols and higher layer applications is performed on the bundle itself.

  • A bundle can be administratively enabled or disabled.

  • Each individual link within a bundle can be administratively enabled or disabled.

  • Bundle member links are not supported on OTN terminated interfaces.

  • Load balancing (the distribution of data between member links) is done with source and destination mac address.


    Note


    Load balancing is done based on each stream, so each interface in the bundle may not carry equal traffic.


  • QoS is supported and can be applied on the bundle interface and sub interfaces.

  • LAG CFM is supported and can be applied on the bundle interface and subinterfaces.

  • LAG is only supported for both L2 and L3 interfaces.

  • Link layer protocols, such as LLDP and Link OAM , work independently on each link within a bundle.

  • Upper layer protocols, such as routing updates and hellos, are sent over any member link of an interface bundle.

  • Bundled interfaces are point to point.

  • All links within a single bundle must be configured either to run 802.3ad (LACP) or Static-LAG (non-LACP). Mixed links within a single bundle are not supported.

  • Only default LACP timer (30sec) is supported.

  • To provision EVPL service with Bundle AC, user has to provision the bundle main interface along with the L2 bundle sub-interfaces. QOS or any other feature over the bundle main interface needs to be configured once the EVPL service is provisioned.

  • When link-OAM is configured on the bundle interface, its recommended to configure one of the following command options:

    • RP/0/RP0:hostname(config)# ethernet oam profile <profile name> action wiring-conflict disable
    • RP/0/RP0:hostname(config)# ethernet oam profile <profile name> action wiring-conflict efd
    • RP/0/RP0:hostname(config)# ethernet oam profile <profile name> action wiring-conflict log
  • While performing RPVM Switch Over or RP OIR or ISSU, the packet transmission stops for a duration of 3 to 20 seconds and causes CFM sessions with CCM interval 1 second and 10 seconds to flap (session goes down and recovers back).

IEEE 802.3ad Standard

The IEEE 802.3ad standard typically defines a method of forming Ethernet link bundles.

For each link configured as bundle member, this information is exchanged between the systems that host each end of the link bundle:

  • A globally unique local system identifier

  • An identifier (operational key) for the bundle of which the link is a member

  • An identifier (port ID) for the link

  • The current aggregation status of the link

This information is used to form the link aggregation group identifier (LAG ID). Links that share a common LAG ID can be aggregated. Individual links have unique LAG IDs.

The system identifier distinguishes one router from another, and its uniqueness is guaranteed through the use of a MAC address from the system. The bundle and link identifiers have significance only to the router assigning them, which must guarantee that no two links have the same identifier, and that no two bundles have the same identifier.

The information from the peer system is combined with the information from the local system to determine the compatibility of the links configured to be members of a bundle.

Bundle MAC addresses in the routers come from a set of reserved MAC addresses in the backplane. This MAC address stays with the bundle as long as the bundle interface exists. The bundle uses this MAC address until the user configures a different MAC address. The bundle MAC address is used by all member links when passing bundle traffic. Any unicast or multicast addresses set on the bundle are also set on all the member links.


Note


We recommend that you avoid modifying the MAC address, because changes in the MAC address can affect packet forwarding.

Prerequisites for Configuring LAG

Before configuring LAG, be sure that these tasks and conditions are met:

  • You must be in a user group associated with a task group that includes the proper task IDs. The command reference guides include the task IDs required for each command.

    If you suspect user group assignment is preventing you from using a command, contact your AAA administrator for assistance.

  • You know the interface IP address (Layer 3 only).

  • You know which links should be included in the bundle you are configuring.

  • If you are configuring an Ethernet link bundle, you should have NCS4K-4H-OPW-QC2 line card installed in the router.

VLAN Subinterfaces on an Ethernet Link Bundle

802.1Q VLAN subinterfaces can be configured on 802.3ad Ethernet link bundles. The maximum number of VLAN subinterfaces allowed per router is 1024 minus the number of main interface(s) configured. Example if one main bundle is configured then maximum 1023 VLAN subinterface bundles can be configured on the router.


Note


The memory requirement for bundle VLANs is slightly higher than standard physical interfaces.

To create a VLAN subinterface on a bundle, include the VLAN subinterface instance with the interface Bundle-Ether command:

interface Bundle-Ether instance.subinterface

After you create a VLAN on an Ethernet link bundle, all physical VLAN subinterface configuration is supported on that link bundle.

Link Aggregation Through LACP

Aggregating interfaces on different line cards provides redundancy, allowing traffic to be quickly redirected to other member links when an interface or modular services card failure occurs.

The optional Link Aggregation Control Protocol (LACP) is defined in the IEEE 802 standard. LACP communicates between two directly connected systems (or peers) to verify the compatibility of bundle members. For the Cisco NCS 4000 Series Routers, the peer can be either another router or a switch. LACP monitors the operational state of link bundles to ensure these:
  • All links terminate on the same two systems.

  • Both systems consider the links to be part of the same bundle.

  • All links have the appropriate settings on the peer.

LACP transmits frames containing the local port state and the local view of the partner system’s state. These frames are analyzed to ensure both systems are in agreement.

How to Configure Link Bundling

Configuring Ethernet Link Bundles

This section describes how to configure a Ethernet link bundle.


Note


MAC accounting is not supported on Ethernet link bundles.

Note


In order for an Ethernet bundle to be active, you must perform the same configuration on both connection endpoints of the bundle.

The creation of an Ethernet link bundle involves creating a bundle and adding member interfaces to that bundle, as shown in the steps that follow.

Procedure


Step 1

configure

Example:

RP/0/# configure

Enters the XR Config mode.

Step 2

interface Bundle-Ether bundle-id

Example:

RP/0/(config)# interface Bundle-Ether 3

Creates and names a new Ethernet link bundle.

Step 3

bundle minimum-active bandwidth kbps

Example:

RP/0/(config-if)# bundle minimum-active bandwidth 580000

(Optional) Sets the minimum amount of bandwidth required before a user can bring up a bundle.

Step 4

bundle minimum-active links links

Example:

RP/0/(config-if)# bundle minimum-active links 2

(Optional) Sets the number of active links required before you can bring up a specific bundle.

Step 5

bundle maximum-active links links

Example:

RP/0/(config-if)# bundle maximum-active links 1

(Optional) Designates one active link and one link in standby mode that can take over immediately for a bundle if the active link fails (1:1 protection).

The default number of active links in a single bundle is 8.

Note

 
If the bundle maximum-active command is issued, then only the highest-priority link within the bundle is active. The priority is based on the value from the bundle port-priority command, where a lower value is a higher priority. Therefore, we recommend that you configure a higher priority on the link that you want to be the active link.

Step 6

bundle maximum-active links links hot-standby

Example:

RP/0/(config-if)# bundle maximum-active links 1 hot-standby

The hot-standby keyword helps to avoid bundle flaps on a switchover or switchback event during which the bundle temporarily falls below the minimum links or bandwidth threshold.

It sets default values for the wait-while timer and suppress-flaps timer to achieve this.

Step 7

l2transport

Example:

RP/0/(config-if)# l2transport

Note

 
Bundled interfaces are supported only in L2transport mode.

Step 8

lacp system priority value

Example:

RP/0/RP0:hostname(config-if)# lacp system priority ?
<1-65535>  Bundle LACP system priority. Lower value is higher priority.
RP/0/RP0:hostname(config-if)#lacp system priority 1000
RP/0/RP0:hostname(config-if)#commit
RP/0/RP0:HEADRP/0/RP0:hostname(config-if)#end
RP/0/RP0:hostname#

Sets the system priority for Ethernet link bundle.

Step 9

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration submode for the Ethernet link bundle.

Step 10

interface { TenGigE | HundredGigE } instance

Example:

RP/0/(config)# interface TenGigE 1/0/0/0

Enters the interface configuration mode for the specified interface.

Mixed bandwidth bundle member configuration is only supported when 1:1 redundancy is configured.

Note

 
Mixed link bundle mode is supported only when active-standby operation is configured (usually with the lower speed link in standby mode).

Step 11

bundle id bundle-id [ mode { active | on | passive} ]

Example:

RP/0/(config-if)# bundle-id 3

Adds the link to the specified bundle.

To enable active or passive LACP on the bundle, include the optional mode active or mode passive keywords in the command string.

To add the link to the bundle without LACP support, include the optional mode on keywords with the command string.

Note

 
If you do not specify the mode keyword, the default mode is on (LACP is not run over the port).

Step 12

no shutdown(optional)

Example:

RP/0/(config-if)# no shutdown

If a link is in the down state, bring it up. The no shutdown command returns the link to an up or down state depending on the configuration and state of the link.

Step 13

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration submode for the Ethernet link bundle.

Step 14

Repeat Step 8 through Step 11 to add more links to the bundle you created in Step 2.

Step 15

Use the commit or end command.

commit - Saves the configuration changes and remains within the configuration session.

end - Prompts user to take one of these actions:

  • Yes - Saves configuration changes and exits the configuration session.

  • No - Exits the configuration session without committing the configuration changes.

  • Cancel - Remains in the configuration mode, without committing the configuration changes.

Step 16

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration mode.

Step 17

exit

Example:

RP/0/(config)# exit

Exits the XR Config mode.

Step 18

Perform Step 1 through Step 15 on the remote end of the connection.

Brings up the other end of the link bundle.

Step 19

show bundle Bundle-Ether bundle-id [ reasons ] (optional)

Example:

RP/0/# show bundle Bundle-Ether 3 reasons

Shows information about the specified Ethernet link bundle

Step 20

show lacp Bundle-Ether bundle-id

Example:

RP/0/ # show lacp Bundle-Ether 3

(Optional) Shows detailed information about LACP ports and their peers.


Configuring VLAN Bundles

This section describes how to configure a VLAN bundle. The creation of a VLAN bundle involves three main tasks:

  1. Create an Ethernet bundle.

  2. Create VLAN subinterfaces and assign them to the Ethernet bundle.

  3. Assign Ethernet links to the Ethernet bundle.

These tasks are describe in detail in the procedure that follows.


Note


In order for a VLAN bundle to be active, you must perform the same configuration on both ends of the bundle connection.

Note


Bundled interfaces are supported only in L2transport mode.

The creation of a VLAN link bundle is described in the steps that follow.

Procedure


Step 1

configure

Example:

RP/0/# configure

Enters XR Config mode.

Step 2

interface Bundle-Ether bundle-id

Example:

RP/0/#(config)# interface Bundle-Ether 3 

Creates and names a new Ethernet link bundle.

This interface Bundle-Ether command enters you into the interface configuration submode, where you can enter interface specific configuration commands are entered. Use the exit command to exit from the interface configuration submode back to the normal XR Config mode.

Step 3

bundle minimum-active bandwidth kbps

Example:

RP/0/(config-if) # bundle minimum-active bandwidth 580000

(Optional) Sets the minimum amount of bandwidth required before a user can bring up a bundle.

Step 4

bundle minimum-active links links

Example:

RP/0/(config-if)# bundle minimum-active links 2

(Optional) Sets the number of active links required before you can bring up a specific bundle.

Step 5

bundle maximum-active links links

Example:

RP/0/(config-if)# bundle maximum-active links 1

(Optional) Designates one active link and one link in standby mode that can take over immediately for a bundle if the active link fails (1:1 protection).

Note

 
The default number of active links allowed in a single bundle is 8.

Note

 
If the bundle maximum-active command is issued, then only the highest-priority link within the bundle is active. The priority is based on the value from the bundle port-priority command, where a lower value is a higher priority. Therefore, we recommend that you configure a higher priority on the link that you want to be the active link.

Step 6

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration submode.

Step 7

interface Bundle-Ether bundle-id.vlan-id l2transport

Example:

RP/0/#(config)#interface Bundle-Ether 3.1 l2transport

Creates a new VLAN, and assigns the VLAN to the Ethernet bundle you created in Step 2.

Replace the bundle-id argument with the bundle-id you created in Step 2.

Replace the vlan-id with a subinterface identifier.

Note

 
When you include the vlan-id argument with the interface Bundle-Ether bundle-id command, you enter subinterface configuration mode.

Step 8

encapsulation dot1q vlan-id

Example:

RP/0/#(config-subif)# encapsulation dot1q 10

Assigns a VLAN to the subinterface.

Replace the vlan-id argument with a subinterface identifier.

Step 9

no shutdown

Example:

RP/0/(config-subif) # no shutdown

(Optional) If a link is in the down state, bring it up. The no shutdown command returns the link to an up state.

Step 10

exit

Example:

RP/0/(config-subif)#exit

Exits subinterface configuration mode for the VLAN subinterface.

Step 11

Repeat Step 7 through Step 12 to add more VLANs to the bundle you created in Step 2.

(Optional) Adds more subinterfaces to the bundle.

Step 12

Use the commit or end command.

commit - Saves the configuration changes and remains within the configuration session.

end - Prompts user to take one of these actions:

  • Yes - Saves configuration changes and exits the configuration session.
  • No - Exits the configuration session without committing the configuration changes.
  • Cancel - Remains in the configuration mode, without committing the configuration changes.

Step 13

exit

Example:

RP/0/ (config-subif)# exit

Exits interface configuration mode.

Step 14

exit

Example:

RP/0/ (config)# exit

Exits XR Config mode.

Step 15

show ethernet trunk bundle-Ether instance

Example:

RP/0/router# show ethernet trunk bundle-ether 5

(Optional) Displays the interface configuration.

The Ethernet bundle instance range is from 1 through 65535.

Step 16

configure

Example:

RP/0/#  configure

Enters XR Config mode.

Step 17

interface { GigabitEthernet | HundredGigabitE } instance

Example:

RP/0/(config)# interface TenGigE 1/0/0/0

Enters the interface configuration mode for the specified interface.

Replace the instance argument with the node-id in the rack/slot/module format.

Note

 
A VLAN bundle is not active until you add an Ethernet interface on both ends of the link bundle.

Step 18

bundle id bundle-id [mode {active | on | passive}]

Example:

RP/0/(config-if)#  bundle-id 3

Adds an Ethernet interface to the bundle you configured in Step 2 through Step 13.

To enable active or passive LACP on the bundle, include the optional mode active or mode passive keywords in the command string.

To add the interface to the bundle without LACP support, include the optional mode on keywords with the command string.

Note

 
If you do not specify the mode keyword, the default mode is on (LACP is not run over the port).

Step 19

no shutdown

Example:

RP/0/(config-if)# no shutdown

(Optional) If a link is in the down state, bring it up. The no shutdown command returns the link to an up or down state depending on the configuration and state of the link.

Step 20

Repeat Step 19 through Step 21 to add more Ethernet interfaces to the bundle you created in Step 2 .

Step 21

Use the commit or end command.

commit - Saves the configuration changes and remains within the configuration session.

end - Prompts user to take one of these actions:

  • Yes - Saves configuration changes and exits the configuration session.
  • No - Exits the configuration session without committing the configuration changes.
  • Cancel - Remains in the configuration mode, without committing the configuration changes.

Step 22

Perform Step 1 through Step 23 on the remote end of the connection.

Brings up the other end of the link bundle.

Step 23

show bundle Bundle-Ether bundle-id

Example:

RP/0/#show bundle Bundle-Ether 3 

(Optional) Shows information about the specified Ethernet link bundle.

The show bundle Bundle-Ether command displays information about the specified bundle. If your bundle has been configured properly and is carrying traffic, the State field in the show bundle Bundle-Ether command output will show the number “4,” which means the specified VLAN bundle port is “distributing.”

Step 24

show ethernet trunk bundle-Ether instance

Example:

RP/0/# show ethernet trunk bundle-ether 5

(Optional) Displays the interface configuration.

The Ethernet bundle instance range is from 1 through 65535.


Configuring L3 Ethernet Link Bundles

This section describes how to configure a Layer 3 Ethernet link bundle.


Note


In order for an Ethernet bundle to be active, you must perform the same configuration on both connection endpoints of the bundle.

The creation of an Ethernet link bundle involves creating a bundle and adding member interfaces to that bundle, as shown in the steps that follow.

Procedure


Step 1

configure

Example:

RP/0/# configure

Enters the XR Config mode.

Step 2

interface Bundle-Ether bundle-id

Example:

RP/0/(config)# interface Bundle-Ether 3

Creates and names a new Ethernet link bundle.

Step 3

ipv4 address ipv4-address mask

Sets the IP address and mask.

Step 4

bundle minimum-active bandwidth kbps

Example:

RP/0/(config-if)# bundle minimum-active bandwidth 580000

(Optional) Sets the minimum amount of bandwidth required before a user can bring up a bundle.

Step 5

bundle minimum-active links links

Example:

RP/0/(config-if)# bundle minimum-active links 2

(Optional) Sets the number of active links required before you can bring up a specific bundle.

Step 6

bundle maximum-active links links

Example:

RP/0/(config-if)# bundle maximum-active links 1

(Optional) Designates one active link and one link in standby mode that can take over immediately for a bundle if the active link fails (1:1 protection).

The default number of active links in a single bundle is 8.

Note

 
If the bundle maximum-active command is issued, then only the highest-priority link within the bundle is active. The priority is based on the value from the bundle port-priority command, where a lower value is a higher priority. Therefore, we recommend that you configure a higher priority on the link that you want to be the active link.

Step 7

bundle maximum-active links links hot-standby

Example:

RP/0/(config-if)# bundle maximum-active links 1 hot-standby

The hot-standby keyword helps to avoid bundle flaps on a switchover or switchback event during which the bundle temporarily falls below the minimum links or bandwidth threshold.

It sets default values for the wait-while timer and suppress-flaps timer to achieve this.

Step 8

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration submode for the Ethernet link bundle.

Step 9

interface { TenGigE | HundredGigE | FortyGigE } instance

Example:

RP/0/(config)# interface fortyGigE 0/6/0/4

Enters the interface configuration mode for the specified interface.

Mixed bandwidth bundle member configuration is only supported when 1:1 redundancy is configured.

Note

 
Mixed link bundle mode is supported only when active-standby operation is configured (usually with the lower speed link in standby mode).

Step 10

bundle id bundle-id [ mode { active | on | passive} ]

Example:

RP/0/(config-if)# bundle-id 3

Adds the link to the specified bundle.

Note

 
If you do not specify the mode keyword, the default mode is on

Step 11

no shutdown(optional)

Example:

RP/0/(config-if)# no shutdown

If a link is in the down state, bring it up. The no shutdown command returns the link to an up or down state depending on the configuration and state of the link.

Step 12

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration submode for the Ethernet link bundle.

Step 13

Repeat Step 8 through Step 11 to add more links to the bundle you created in Step 2.

Step 14

Use the commit or end command.

commit - Saves the configuration changes and remains within the configuration session.

end - Prompts user to take one of these actions:

  • Yes - Saves configuration changes and exits the configuration session.
  • No - Exits the configuration session without committing the configuration changes.
  • Cancel - Remains in the configuration mode, without committing the configuration changes.

Step 15

exit

Example:

RP/0/(config-if)# exit

Exits interface configuration mode.

Step 16

exit

Example:

RP/0/(config)# exit

Exits the XR Config mode.

Step 17

Perform Step 1 through Step 15 on the remote end of the connection.

Brings up the other end of the link bundle.

Step 18

show bundle Bundle-Ether bundle-id [ reasons ] (optional)

Example:

RP/0/# show bundle Bundle-Ether 3 reasons

Shows information about the specified Ethernet link bundle

Step 19

show lacp Bundle-Ether bundle-id

Example:

RP/0/ # show lacp Bundle-Ether 3

(Optional) Shows detailed information about LACP ports and their peers.


Example


RP/0/# config
RP/0/(config)# interface Bundle-Ether 1
RP/0/(config-if)# ipv4 address 100.110.100.2/24
RP/0/(config-if)# bundle minimum-active bandwidth 620000
RP/0/(config-if)# bundle minimum-active links 1
RP/0/(config-if)# exit
RP/0/(config)# interface FortyGigE 0/6/0/4
RP/0/(config-if)# bundle id 3 mode active
RP/0/(config-if)# no shutdown
RP/0/(config)# exit
RP/0/(config)# interface FortyGigE 0/6/0/9
RP/0/(config-if)# bundle id 3 mode active
RP/0/(config-if)# no shutdown
RP/0/(config-if)# exit

Configuration Examples for Link Bundles

Configuring Ethernet Channel Bundle with LACP mode: Example

This example shows how to join two ports to form an Ethernet Channel bundle running LACP:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 3
RP/0/RP0:hostname(config-if)# bundle minimum-active bandwidth 620000
RP/0/RP0:hostname(config-if)# bundle minimum-active links 1
RP/0/RP0:hostname(config-if)# l2transport
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 3
RP/0/RP0:hostname(config)# interface TenGigE 0/3/0/0
RP/0/RP0:hostname(config-if)# bundle id 3 mode active
RP/0/RP0:hostname(config-if)# no shutdown
RP/0/RP0:hostname(config)# exit
RP/0/RP0:hostname(config)# interface TenGigE 0/3/0/1
RP/0/RP0:hostname(config-if)# bundle id 3 mode active
RP/0/RP0:hostname(config-if)# no shutdown
RP/0/RP0:hostname(config-if-l2)# exit

Configuring Ethernet Channel Bundle with Non LACP or Static Mode : Example

This example shows how to join two ports to form an Ethernet Channel bundle with non-LACP/static mode :


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 2002
RP/0/RP0:hostname(config-if)# bundle minimum-active links 1
RP/0/RP0:hostname(config-if)# l2transport
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/2/0/9/4
RP/0/RP0:hostname(config-if)# bundle id 2002 mode on
RP/0/RP0:hostname(config-if)# no shutdown
RP/0/RP0:hostname(config)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/3/0/9/4
RP/0/RP0:hostname(config-if)# bundle id 2002 mode on
RP/0/RP0:hostname(config-if)# no shutdown
RP/0/RP0:hostname(config)# exit

Creating VLAN Subinterface on a Ethernet Bundle: Example

This example shows how to create and bring up two VLANs on an Ethernet bundle:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 1
RP/0/RP0:hostname(config-if)# bundle minimum-active bandwidth 620000
RP/0/RP0:hostname(config-if)# bundle minimum-active links 1
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# no shutdown
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.2 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 20
RP/0/RP0:hostname(config-subif)# no shutdown
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)#interface tengige 0/1/5/7
RP/0/RP0:hostname(config-if)# bundle-id 1 mode act
RP/0/RP0:hostname(config-if)# commit
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# exit
RP/0/RP0:hostname # show ethernet trunk bundle-ether 1

Configure L2VPN with Ethernet Bundle as Attachment Circuit : Examples

Following example shows how to configure local switching with bundled interface:



RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 1
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/0/0/2
RP/0/RP0:hostname(config-if)# bundle-id 1 mode on
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 2
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 2.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/5/0/7
RP/0/RP0:hostname(config-if)# bundle-id 2 mode on
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)#controller Optics 0/0/0/2 
RP/0/RP0:hostname(config-Optics)# port-mode Ethernet framing packet rate 10GE
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)#controller Optics 0/5/0/7 
RP/0/RP0:hostname(config-Optics)# port-mode Ethernet framing packet rate 10GE
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# l2vpn
RP/0/RP0:hostname(config-l2vpn)# xconnect group XCON2
RP/0/RP0:hostname(config-l2vpn-xc)# p2p xc2  
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 1.1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 2.1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# commit

Following example shows how to configure dynamic point-to-point cross-connect with bundled interface:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 1
RP/0/RP0:hostname(config-if)# bundle minimum-active bandwidth 620000
RP/0/RP0:hostname(config-if)# bundle minimum-active links 1
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# no shutdown
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface tengige0/1/5/7
RP/0/RP0:hostname(config-if)# bundle-id 1 mode on
RP/0/RP0:hostname(config-if)# commit
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# l2vpn
RP/0/RP0:hostname(config-l2vpn)# pw-class dyn-mpls
RP/0/RP0:hostname(config-l2vpn-pwc)# encapsulation mpls 
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# protocol ldp
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# ipv4 source 106.0.0.1
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# preferred-path interface tunnel-te 1
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# exit
RP/0/RP0:hostname(config-l2vpn-pwc)# exit
RP/0/RP0:hostname(config-l2vpn)# xconnect group XCON1
RP/0/RP0:hostname(config-l2vpn-xc)# p2p xc1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 1.1
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# neighbor ipv4 107.0.0.1 pw-id 1
RP/0/RP0:hostname(config-l2vpn-xc-p2p-pw)# pw-class dyn-mpls
RP/0/RP0:hostname(config-l2vpn-xc-p2p-pw)# commit

Configure CFM with Ethernet Bundle: Examples


Note


The possible intervals for transmitting Continuity Check Messages (CCMs), that can be used with bundles are : 1s, 10s, 1m ,and 10s.

Note


For more details on CFM refer section .

Example1:



RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 1
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# ethernet cfm mep domain d1 service s1 mep-id 1
RP/0/RP0:hostname(config-if-cfm-mep)# exit
RP/0/RP0:hostname(config-if-cfm)# exit
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/0/0/2
RP/0/RP0:hostname(config-if)# bundle-id 1 mode on
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 2
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 2.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# ethernet cfm mep domain d2 service s2 mep-id 2
RP/0/RP0:hostname(config-if-cfm-mep)# exit
RP/0/RP0:hostname(config-if-cfm)# exit
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface TenGigE0/5/0/7
RP/0/RP0:hostname(config-if)# bundle-id 2 mode on
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)#controller Optics 0/0/0/2 
RP/0/RP0:hostname(config-Optics)# port-mode Ethernet framing packet rate 10GE
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)#controller Optics 0/5/0/7 
RP/0/RP0:hostname(config-Optics)# port-mode Ethernet framing packet rate 10GE
RP/0/RP0:hostname(config-if)# exit

RP/0/RP0:hostname(config)# l2vpn
RP/0/RP0:hostname(config-l2vpn)# xconnect group XCON2
RP/0/RP0:hostname(config-l2vpn-xc)# p2p xc2  
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 1.1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 2.1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# commit

Example2:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)# interface Bundle-Ether 1
RP/0/RP0:hostname(config-if)# bundle minimum-active bandwidth 620000
RP/0/RP0:hostname(config-if)# bundle minimum-active links 1
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# interface Bundle-Ether 1.1 l2transport
RP/0/RP0:hostname(config-subif)# encapsulation dot1q 10
RP/0/RP0:hostname(config-subif)# ethernet cfm mep domain d1 service s1 mep-id 1
RP/0/RP0:hostname(config-if-cfm-mep)# exit
RP/0/RP0:hostname(config-if-cfm)# exit
RP/0/RP0:hostname(config-subif)# no shutdown
RP/0/RP0:hostname(config-subif)# exit
RP/0/RP0:hostname(config)# interface tengige0/1/5/7
RP/0/RP0:hostname(config-if)# bundle-id 1 mode on
RP/0/RP0:hostname(config-if)# commit
RP/0/RP0:hostname(config-if)# exit
RP/0/RP0:hostname(config)# l2vpn
RP/0/RP0:hostname(config-l2vpn)# pw-class dyn-mpls
RP/0/RP0:hostname(config-l2vpn-pwc)# encapsulation mpls 
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# protocol ldp
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# ipv4 source 106.0.0.1
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# preferred-path interface tunnel-te 1
RP/0/RP0:hostname(config-l2vpn-pwc-mpls)# exit
RP/0/RP0:hostname(config-l2vpn-pwc)# exit
RP/0/RP0:hostname(config-l2vpn)# xconnect group XCON1
RP/0/RP0:hostname(config-l2vpn-xc)# p2p xc1 
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# interface Bundle-Ether 1.1
RP/0/RP0:hostname(config-l2vpn-xc-p2p)# neighbor ipv4 107.0.0.1 pw-id 1
RP/0/RP0:hostname(config-l2vpn-xc-p2p-pw)# pw-class dyn-mpls
RP/0/RP0:hostname(config-l2vpn-xc-p2p-pw)# commit

Configure AIS for CFM with Ethernet Bundle: Examples


Note


The possible intervals for transmitting Continuity Check Messages (CCMs), that can be used with bundles are : 1s, 10s, 1m ,and 10s.

Example1:



RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)#ethernet cfm 
RP/0/RP0:hostname(config-cfm)#domain dup3 level 3 id null
RP/0/RP0:hostname(config-cfm-dmn)#service sup3 down-meps id icc-based cisco u3
RP/0/RP0:hostname(config-cfm-dmn-svc)#continuity-check interval 1s
RP/0/RP0:hostname(config-cfm-dmn-svc)#mep crosscheck 
RP/0/RP0:hostname(config-cfm-xcheck)#mep-id 3
RP/0/RP0:hostname(config-cfm-xcheck)#exit
RP/0/RP0:hostname(config-cfm-dmn-svc)#ais transmission 
RP/0/RP0:hostname(config-cfm-dmn-svc)#exit

Example2:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)#ethernet cfm 
RP/0/RP0:hostname(config-cfm)#domain dup4 level 4 id null
RP/0/RP0:hostname(config-cfm-dmn)#service sup3 xconnect group arw-g3 p2p arw_p3$
RP/0/RP0:hostname(config-cfm-dmn-svc)#mip auto-create all
RP/0/RP0:hostname(config-cfm-dmn-svc)#continuity-check interval 1s
RP/0/RP0:hostname(config-cfm-dmn-svc)#mep crosscheck 
RP/0/RP0:hostname(config-cfm-xcheck)#mep-id 3
RP/0/RP0:hostname(config-cfm-xcheck)#exit
RP/0/RP0:hostname(config-cfm-dmn-svc)#ais transmission
RP/0/RP0:hostname(config-cfm-dmn-svc)#exit

Ethernet CFM Show Command for Ethernet Bundle: Examples

Example1:



RP/0/RP0:hostname# show ethernet cfm local meps interface bundle-Ether 1.3
Thu Sep 20 22:53:01.969 UTC
Defects (from at least one peer MEP):
A - AIS received                I - Wrong interval
R - Remote Defect received      V - Wrong level
L - Loop (our MAC received)     T - Timed out
C - Config (our ID received)    M - Missing (cross-check)
X - Cross-connect (wrong MAID)  U - Unexpected (cross-check)
P - Peer port down

Domain dup3 (level 3), Service sup3
   ID Interface (State)        Dir MEPs/Err RD Defects AIS
----- ------------------------ --- -------- -- ------- ---
3003 BE1.3 (Up)               Dn     0/0   Y  TM      L4

Example 2:


RP/0/RP0:hostname# show ethernet cfm peer meps 
Thu Sep 20 22:53:36.337 UTC
Flags:
> - Ok                          I - Wrong interval
R - Remote Defect received      V - Wrong level
L - Loop (our MAC received)     T - Timed out
C - Config (our ID received)    M - Missing (cross-check)
X - Cross-connect (wrong MAID)  U - Unexpected (cross-check)
* - Multiple errors received    S - Standby

Domain MD1 (level 1), Service down_mep_customer_20001
Down MEP on Bundle-Ether2000.1 MEP-ID 6001
================================================================================
St    ID MAC Address    Port    Up/Downtime   CcmRcvd SeqErr   RDI Error
-- ----- -------------- ------- ----------- --------- ------ ----- -----
>  7001 00af.1fd6.0021 Up      00:01:56          136     0     0     0
Domain MD1 (level 1), Service down_mep_customer_20002

Down MEP on Bundle-Ether2000.2 MEP-ID 6002
================================================================================
St    ID MAC Address    Port    Up/Downtime   CcmRcvd SeqErr   RDI Error
-- ----- -------------- ------- ----------- --------- ------ ----- -----
>  7002 00af.1fd6.0021 Up      00:01:56          139     0     0     0
Domain MD2 (level 2), Service up_mep_customer_1001
Up MEP on Bundle-Ether2000.1 MEP-ID 4001
================================================================================
St    ID MAC Address    Port    Up/Downtime   CcmRcvd SeqErr   RDI Error
-- ----- -------------- ------- ----------- --------- ------ ----- -----
>  5501 7ef2.fe69.3123 Up      00:01:01           62      0     0     0
 
Domain MD2 (level 2), Service up_mep_customer_1002
Up MEP on Bundle-Ether2000.2 MEP-ID 4002
================================================================================
St    ID MAC Address    Port    Up/Downtime   CcmRcvd SeqErr   RDI Error
-- ----- -------------- ------- ----------- --------- ------ ----- -----
>  5502 7ef2.fe69.3123 Up      00:01:01           62      0     0     0

Example 3:


RP/0/RP0:hostname# show ethernet cfm peer meps interface bundle-Ether 1.3 detail 
Thu Sep 20 22:53:52.899 UTC
Domain dup3 (level 3), Service sup3
Down MEP on Bundle-Ether1.3 MEP-ID 3003
================================================================================
Peer MEP-ID 3, MAC 92bd.4a00.0023
   CFM state: Timed out, for 00:01:50
   Port state: Up
   CCM defects detected:    T - Timed out
   CCMs received: 1673
     Out-of-sequence:             0
     Remote Defect received:      0
     Wrong level:                 0
     Cross-connect (wrong MAID):  0
     Wrong interval:              0
     Loop (our MAC received):     0
     Config (our ID received):    0
   Last CCM received 00:01:53 ago:
     Level: 3, Version: 0, Interval: 1s
     Sequence number: 0, MEP-ID: 3
     MAID: NULL, ICC-based: ciscou3
     Chassis ID: Local: ios; Management address: 'Not specified'
     Port status: Up, Interface status: Up

Peer MEP-ID 3, MAC 
   CFM state: Missing (cross-check), no CCMs received
   CCM defects detected:    M - Missing (cross-check)
   CCMs received: 0
     Out-of-sequence:             0
    Remote Defect received:      0
     Wrong level:                 0
     Cross-connect (wrong MAID):  0
     Wrong interval:              0
     Loop (our MAC received):     0
     Config (our ID received):    0

Example 4:


RP/0/RP0:hostname# show ethernet cfm local meps interface bundle-Ether 1.3 verbose 
Thu Sep 20 22:55:18.149 UTC
Domain dup3 (level 3), Service sup3
Down MEP on Bundle-Ether1.3 MEP-ID 3003
================================================================================
  Interface state: Up      MAC address: 4481.9800.0023
  Peer MEPs: 0 up, 0 with errors, 1 timed out (archived)
  Cross-check errors: 1 missing, 0 unexpected

  CCM generation enabled:  Yes, 1s (Remote Defect detected: Yes)
  CCM defects detected:    T - Timed out
                           M - Missing (cross-check)
  AIS generation enabled:  Yes (level: 4, interval: 1s)
  Sending AIS:             Yes (started 00:03:15 ago)
  Receiving AIS:           No

  Packet        Sent      Received
  ------  ----------  ---------------------------------------------------------
  CCM           6594          1673  (out of seq: 0)
  AIS            196             0

Example 5:


RP/0/RP0:hostname# show ethernet cfm interfaces ais 
Thu Sep 20 22:52:10.824 UTC
Defects (from at least one peer MEP):
A - AIS received                I - Wrong interval
R - Remote Defect received      V - Wrong level
L - Loop (our MAC received)     T - Timed out
C - Config (our ID received)    M - Missing (cross-check)
X - Cross-connect (wrong MAID)  U - Unexpected (cross-check)
P - Peer port down              D - Local port down

                               Trigger                    Transmission        
                         AIS  ---------   Via     -----------------------------
Interface (State)        Dir  L Defects  Levels   L Int   Last Started Packets
------------------------ ---  - -------  -------  - ----- ------------ --------
BE1.3 (Up)               Up   3 TM                4 1s    00:00:08 ago        9

Configuring ISIS for L3 Link Bundle : Example

This example shows how to configure ISIS for layer3 link bundles:


RP/0/RP0:hostname# config
RP/0/RP0:hostname(config)#router isis 100
RP/0/RP0:hostname(config-isis)#instance-id 789
RP/0/RP0:hostname(config-isis)#interface bundle-ether 1
RP/0/RP0:hostname(config-isis-if)#address-family ipv4 unicast 
RP/0/RP0:hostname(config-isis-if-af)# commit

Show Command for L3 Ethernet Bundle: Examples

Example1:


RP/0/RP0:hostname# show bundle bundle-ether 1
Thu Sep  6 08:31:06.471 UTC

Bundle-Ether1
  Status:                                    Up
  Local links <active/standby/configured>:   2 / 0 / 2
  Local bandwidth <effective/available>:     80000000 (80000000) kbps
  MAC address (source):                      5a79.5b00.0023 (Chassis pool)
  Inter-chassis link:                        No
  Minimum active links / bandwidth:          1 / 1 kbps
  Maximum active links:                      16
  Wait while timer:                          2000 ms
  Load balancing:                            
    Link order signaling:                    Not configured
    Hash type:                               Default
    Locality threshold:                      None
  LACP:                                      Operational
    Flap suppression timer:                  Off
    Cisco extensions:                        Disabled
    Non-revertive:                           Disabled
  mLACP:                                     Not configured
  IPv4 BFD:                                  Not configured
  IPv6 BFD:                                  Not configured

  Port                  Device           State        Port ID         B/W, kbps
  --------------------  ---------------  -----------  --------------  ----------
  Fo0/6/0/4             Local            Active       0x8000, 0x0001    40000000
      Link is Active
  Fo0/6/0/9             Local            Active       0x8000, 0x0002    40000000
      Link is Active

Example 2:


RP/0/RP0:hostname# show arp bundle-Ether 1 location 0/lc0
Thu Sep  6 08:31:32.032 UTC

Address         Age        Hardware Addr   State      Type  Interface
100.110.100.2   -          5a79.5b00.0023  Interface  ARPA  Bundle-Ether1
RP/0/RP0:KK05#show arp bundle-Ether 1 location 0/lc0
Thu Sep  6 08:31:53.523 UTC

Address         Age        Hardware Addr   State      Type  Interface
100.110.100.1   00:00:07   5a79.5d00.0023  Dynamic    ARPA  Bundle-Ether1
100.110.100.2   -          5a79.5b00.0023  Interface  ARPA  Bundle-Ether1

Example 3:


RP/0/RP0:hostname# show isis neighbors 
Thu Sep  6 08:32:37.399 UTC

IS-IS 100 neighbors:
System Id      Interface        SNPA           State Holdtime Type IETF-NSF
MM10           BE1              5a79.5d00.0023 Up    8        L1L2 Capable 
IORNMAN-BACKUP Te0/6/0/7/1      b026.803a.3011 Up    9        L1L2 Capable 

Total neighbor count: 2

IS-IS jkcore neighbors:
System Id      Interface        SNPA           State Holdtime Type IETF-NSF
0000.0000.0004 Te0/6/0/5/1      a80c.0d7b.f7aa Up    8        L2   Capable 

Total neighbor count: 1