Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Problem: VLAN goes into suspended state due to LDB allocation fail on a Nexus 7000 with connected Nexus 2000 fabric extenders.

Solution

Introduction

This document describes how to troubleshoot suspended VLANs due to Light Distribution Box (LDB) allocation fail when using fabric extender (FEX) with M1 parent cards on a Nexus 7000 series switch.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these sofware and hardware versions:

- Cisco NX-OS Version 6.2(x)

- Cisco Nexus 7000 Series switch

- Cisco Nexus 2000 Series fabric extender

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Background Information

LDB is a hardware table that stores properties for frames received by the forwarding engine, including trunk to vlan mapping for FEX portchannels. Failed LDB allocation indicates that this databse has been exhausted.

When a FEX module is connected to a N7K-M132XP-12 (non XL) the maximum number of LDB entries is approximately 228k. With the XL module, the limit is approximately 512K.

Total LDB entries are the sum of VLAN ranges defined on each FEX HIF (difference between highest and lowest VLAN defined on a HIF).

For example:

switchport trunk allowed vlan 1-4 = 4 LDB entries

switchport trunk allowed vlan 1-4, 70-80, 800 = 800 LDB entries

switchport trunk allowed vlan 200-800 = 600 LDB entries

switchport trunk allowed vlan 200, 800 = 600 LDB entries

If ports 101/1/1 to 101/1/10 are configured with switchport trunk allowed vlan 200, 800 the total number of LDB entries consumed would be 6000 (10x600).

Access ports consumes 1 LDB entry (the access VLAN defined on the HIF).

Check LDB entries for each module by using this command:

N7K-A# attach mod 1

Attaching to module 1 ...

To exit type 'exit', to abort type '$.'

module-1#

module-1# show system internal eltmc info ldb summary

LDB allocation summary:

Max dynamic ldb entries:      203776

Total number of entries:      199680

Number of free entries:       197

Number of free regions:       2

Number of allocated entries:  199483

Number of allocated regions:  280

Number of fail allocations:   21

In this example there have been 21 failed LDB allocations. Due to the hashing mechanism used to hash particular entries to particular parts of the database, it is not necessary for the total number of entries to reach the maximum dynamic LDB entries in order to see failed allocations.

This output displays the number of LDB entries (in hexadecimal) used by each port-channel or interface:

module-1# show system internal eltmc info ldb all

LDB allocation maps :

base          size        allocation

0xd400        0x1000        Shared

0xe400        0x1000        Shared

0xf400        0x1000        Shared

0x10400        0x1000        Ethernet1/4

0x11400        0x1000        Ethernet1/7

0x12400        0x1        Ethernet179/1/30

0x12401        0x1        port-channel1093

0x12402        0x1        port-channel1564

0x12403        0x1        port-channel1550

0x12404        0x1        port-channel1527

0x12405        0x1        port-channel1546

0x12426        0x1        Ethernet169/1/47

0x12427        0x1        Ethernet169/1/48

0x12428        0x1        Ethernet181/1/33

0x12429        0x1        Ethernet181/1/34

0x1242a        0x1        Ethernet163/1/4

0x1242b        0x1        Ethernet163/1/5

0x1242c        0x506        Ethernet183/1/7

0x12932        0x1        port-channel1096

0x12933        0x1        port-channel1095

0x12934        0x1        port-channel1092

0x12935        0x2c8        port-channel1084

0x12bfd        0x506        Ethernet183/1/8

0x13103        0x2c8        port-channel1086

0x133cb        0x1        port-channel1589

0x133cc        0x1        port-channel1063

0x133cd        0x1        port-channel1654

0x133ce        0x1        port-channel1652

0x133d4        0x1        port-channel1520

0x133d5        0x1        port-channel1560

0x133d6        0x1        port-channel1561

0x133d7        0x506        Ethernet167/1/4

0x138dd        0x506        Ethernet167/1/2

0x13de3        0x403        Ethernet165/1/2

0x141e6        0x403        Ethernet151/1/1

<snip>

Note: The two above commands provide incorrect LDB values for N7K-M132XP-12 (non XL) in NX-OS 6.0.3 and 5.2.4.

NX-OS 5.2.5 and 6.1 will correct this.

Problem: VLAN goes into suspended state due to LDB allocation fail on a Nexus 7000 with connected Nexus 2000 fabric extenders.

Symptoms:

1. Error message in logs indicate LDB Allocation Failed

2015 Feb  3 00:01:27.260 N7k1 %ETHPORT-5-IF_SEQ_ERROR: Error ("LDB Allocation Failed") communicating with MTS_SAP_ELTM for opcode MTS_OPC_ETHPM_PORT_LOGICAL_BRINGUP (RID_PORT: port-channel1001)

2015 Feb  3 00:01:27.261 N7k1 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 268,1261-1262,1268 on Interface port-channel1001 are being suspended. (Reason: LDB Allocation Failed)

2. Connectivity lost to multiple hosts connected to FEX

3. Output of show interface status err-vlans shows VLAN's suspended due to LDB Failed Allocation

N7kA# show interface status err-vlans

--------------------------------------------------------------------------------

Port         Name               Err-Vlans                     Status

--------------------------------------------------------------------------------

Po1001       ***dcn2pclx01a**   268,1261-1262,1268            LDB Allocation

*LOG                                             Failed

Solution

This is a hardware limitation associated with the linecard, as such this issue is not addressed by software upgrades.

The recommendation is to prune VLAN's from HIF or reduce the VLAN ranges on FEX HIF to reduce the total number of LDB entries.

Each VLAN instance on each interface will consume LDB entries (eg if portchannel 1 has 100 VLAN's defined and four physical ports in the portchannel, the total number of LDB entries consumed will be 400, 100 instances per port).