DCX-No ACK in 100 PDUs Error Message

Introduction

This document describes this error message and how to identify the root cause: "%ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet115/1/17 is down (Error disabled. Reason CX-No ACK in 100 PDUs)."

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Nexus CLI
• Fibre Channel over Ethernet (FCoE) Protocol

Components Used

The information in this document is based on all Nexus 5000 and 5500 Series switch platforms.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Problem

Data Center Bridging Capability Exchange (DCBX) Type Length Values (TLV) are packaged within a Link Layer Discovery Protocol (LLDP) frame that is exchanged between the switch and the converged network adapter (CNA). One such Control Sub-TLV is used for acknowledgement (ACK), which is sequence-based. For example, the switch sends a Control Sub-TLV with a SeqNo of 1 and an AckNo of 2. The host is supposed to inverse this, and send an LLDP frame with a Control Sub-TLV with a SeqNo of 2 and an AckNo of 1. Refer to the Packet Captures section of this article for more details.

The switch expects this exchange from the host every 30 seconds. If the switch does not see this exchange for 100 Protocol Data Units (PDUs) , which is 3000 seconds or 50 minutes, the switch disables with this error:

N5k %ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet115/1/17 is down
(Error disabled. Reason:DCX-No ACK in 100 PDUs)
N5k %ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet116/1/16 is down
(Error disabled. Reason:DCX-No ACK in 100 PDUs)

Solution

You can resolve this issue if you disable LLDP. However, if you run FCoE, LLDP is required because the virtual fiber-channel port does not come up without it. In order to disable LLDP, enter these commands:

N5k(config)# interface E1/1
N5k(config-if)# no lldp receive 
N5k(config-if)# no lldp send

Here are some commands on the switch that help to narrow down the root cause.

N5k# show lldp interface ethernet 1/22
Interface Information:
  Enable (tx/rx/dcbx): Y/Y/Y    Port Mac address: 00:05:73:ab:29:bd

Peer's LLDP TLVs:
Type Length Value
---- ------ -----
001  007    040000c9 9d2372
002  007    030000c9 9d2372
003  002    0078
006  045    456d756c 6578204f 6e65436f 6e6e6563 74203130 4762204d 756c7469
            2066756e 6374696f 6e204164 61707465 72
007  004    00800080
127  055    001b2102 020a0000 00000002 00000001 04110000 c0000001 00003232
            00000000 00000206 060000c0 00080808 0a0000c0 00890600 1b2108
000  000    

N5k# show lldp dcbx interface ethernet 1/22

Local DCBXP Control information:
Operation version: 00  Max version: 00  Seq no: 1  Ack no: 2  <<---Our sequence
# and Ack #
Type/ 
Subtype    Version    En/Will/Adv Config
003/000     000        Y/N/Y      0808
004/000     000        Y/N/Y      8906001b21 08
002/000     000        Y/N/Y      0001000032 32000000 00000002

Peer's DCBXP Control information:
Operation version: 00  Max version: 00  Seq no: 2  Ack no: 1  <<---Peer sequence #
and Ack # should be reversed.
Type/      Max/Oper
Subtype    Version    En/Will/Err Config
002/000     000/000    Y/Y/N      0001000032 32000000 00000002 
003/000     000/000    Y/Y/N      0808
004/000     000/000    Y/Y/N      8906001b21 08

The root cause for this problem in most cases is CNA/server misbehavior or an incorrect firmware/driver on the CNA. A command was introduced for the Nexus 5000 Series switch platforms in Releases 5.2(1)N1(1) and later in order to recover from this error-disabled state automatically.

N5k(config)# errdisable recovery cause dcbx-no-ack

Note: Cisco Bug ID CSCtq30118 Enh: DCX-No ACK in 100 PDUs was filed in order to enhance the capabilities in order to troubleshoot this issue. This fix also allows customers to enable recovery from this condition.

Packet Views

Inline Packet Capture of Nexus 5000 Sending LLDP Frame DCBX Control Sub-TLV of SeqNo 1 and AckNo 2

Inline Packet Capture of CNA Sending LLDP Frame DCBX Control Sub-TLV of SeqNo 2 and AckNo 1

Wireshark does not decode LLDP Sub-TLVs. They are shown as an "Unknown Subtype" in the LLDP Header. Use the sequence numbers from the commands in the previous section in order to locate them in the Wireshark trace. Here are traces from a Switched Port Analyzer (SPAN) session.

Wireshark Capture of Nexus 5000 Sending LLDP Frame DCBX Control Sub-TLV of SeqNo 1 and AckNo 2

Wireshark Capture of CNA Sending LLDP Frame DCBX Control Sub-TLV of SeqNo 2 and AckNo 1

Alternatively, use the built-in sniffer in the Nexus 5000 Series switch platform in order to see the LLDP frames as well. Use the source MAC address as a display filter.

Ethanalyzer Capture of CNA Sending LLDP Frame DCBX Control Sub-TLV of SeqNo 2 and AckNo 1.

N5k# ethanalyzer local interface inbound-hi det display-filter eth.src==
00:00:c9:9d:23:72
Capturing on eth4
Frame 1215 (152 bytes on wire, 152 bytes captured)
    Arrival Time: Aug 31, 2011 09:06:25.549049000
    [Time delta from previous captured frame: 0.021367000 seconds]
    [Time delta from previous displayed frame: 1314795985.549049000 seconds]
    [Time since reference or first frame: 1314795985.549049000 seconds]
    Frame Number: 1215
    Frame Length: 152 bytes
    Capture Length: 152 bytes
    [Frame is marked: False]
    [Protocols in frame: eth:vlan:lldp]
Ethernet II, Src: 00:00:c9:9d:23:72 (00:00:c9:9d:23:72), Dst: 01:80:c2:00:00:0e
(01:80:c2:00:00:0e)
    Destination: 01:80:c2:00:00:0e (01:80:c2:00:00:0e)
        Address: 01:80:c2:00:00:0e (01:80:c2:00:00:0e)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
    Source: 00:00:c9:9d:23:72 (00:00:c9:9d:23:72)
        Address: 00:00:c9:9d:23:72 (00:00:c9:9d:23:72)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
    Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN
    000. .... .... .... = Priority: 0
    ...0 .... .... .... = CFI: 0
    .... 0000 0001 0100 = ID: 20
    Type: 802.1 Link Layer Discovery Protocol (LLDP) (0x88cc)
Link Layer Discovery Protocol
    Chassis Subtype = MAC address
        0000 001. .... .... = TLV Type: Chassis Id (1)
        .... ...0 0000 0111 = TLV Length: 7
        Chassis Id Subtype: MAC address (4)
        Chassis Id: 00:00:c9:9d:23:72 (00:00:c9:9d:23:72)
    Port Subtype = MAC address
        0000 010. .... .... = TLV Type: Port Id (2)
        .... ...0 0000 0111 = TLV Length: 7
        Port Id Subtype: MAC address (3)
        Port Id: 00:00:c9:9d:23:72 (00:00:c9:9d:23:72)
    Time To Live = 120 sec
        0000 011. .... .... = TLV Type: Time to Live (3)
        .... ...0 0000 0010 = TLV Length: 2
        Seconds: 120
    System Description = Emulex OneConnect 10Gb Multi function Adapter
        0000 110. .... .... = TLV Type: System Description (6)
        .... ...0 0010 1101 = TLV Length: 45
        System Description = Emulex OneConnect 10Gb Multi function Adapter
    Capabilities
        0000 111. .... .... = TLV Type: System Capabilities (7)
        .... ...0 0000 0100 = TLV Length: 4
        Capabilities: 0x0080
            .... .... 1... .... = Station only
        Enabled Capabilities: 0x0080
            .... .... 1... .... = Station only
    Unknown - Unknown
        1111 111. .... .... = TLV Type: Organization Specific (127)
        .... ...0 0011 0111 = TLV Length: 55
        Organization Unique Code: Unknown (0x001b21)
        Unknown Subtype Content: 02020A0000000000020000000104110000C0000001000032... <<<<<
    End of LLDPDU
        0000 000. .... .... = TLV Type: End of LLDPDU (0)
        .... ...0 0000 0000 = TLV Length: 0

N5k# 1 packets captured