Troubleshooting VXLAN Issues
The VXLAN data path includes the following paths:
- Multicast encapsulation path—Native Layer 2 packets are encapsulated in VXLAN in the access to network (Layer 2 to Layer 3) direction
- Multicast decapsulation path—Native Layer 2 packets are decapsulated in VXLAN in the network to access (Layer 3 to Layer 2) direction
- Unicast encapsulation path—Native Layer 2 packets are encapsulated in VXLAN in the access to network (Layer 2 to Layer 3) direction
- Unicast decapsulation path—Native Layer 2 packets are decapsulated in VXLAN in the network to access (Layer 3 to Layer 2) direction
Understanding these data paths can help you troubleshoot VXLAN issues.
Caution |
To troubleshoot VXLAN issues, you need to run Broadcom shell commands. Use these Broadcom shell commands with caution and only under the direct supervision or request of Cisco Support personnel. |
Note |
The Cisco Nexus 9300 Series switches support VXLAN. The Cisco Nexus 9500 Series switches do not. |
Packets Dropped in the Multicast Encapsulation Path
Follow these steps if ARP requests or multicast packets are being dropped on the device in the access to network direction.
SUMMARY STEPS
- Access the Broadcom shell.
- Check the output of the stg show command to see if the ports are in the STP forward state for a given VLAN.
- Verify if ports are part of the VLAN.
- Check the output of the mc show command to see if the local VLAN ports and encapsulation port are part of the encapsulation flood list.
- If the output of the mc show command is incorrect, exit the Broadcom shell mode, run the following commands, and view the output: show tech-support pixm, show tech-support pixm-all, and show tech-support pixmc-all.
DETAILED STEPS
Step 1 |
Access the Broadcom shell. Example:
|
Step 2 |
Check the output of the stg show command to see if the ports are in the STP forward state for a given VLAN. Example:
In this example, VLAN 3 has eth1/24 and uplink tunnel port is eth2/2, so we would expect to see xe23 (1/24) and hg in the output. |
Step 3 |
Verify if ports are part of the VLAN. Example:
In this example, xe23 needs to be part of VLAN 3. |
Step 4 |
Check the output of the mc show command to see if the local VLAN ports and encapsulation port are part of the encapsulation flood list. |
Step 5 |
If the output of the mc show command is incorrect, exit the Broadcom shell mode, run the following commands, and view the output: show tech-support pixm, show tech-support pixm-all, and show tech-support pixmc-all. Example:
|
Packets Dropped in the Multicast Decapsulation Path
Follow these steps if ARP requests or multicast packets are being dropped on the device in the network to access direction.
SUMMARY STEPS
- Check if the packets were sent to the supervisor and if remote VXLAN tunnel endpoint (VTEP) discovery occurred.
- If the mpls_entry is present in the hardware, check the vlan_xlate table.
- If the vlan_xlate table has the correct entry for the multicast DIP, check if the VLAN flood list shows the correct members (that is, the members of the VLAN excluding the encapsulation tunnel port).
DETAILED STEPS
Step 1 |
Check if the packets were sent to the supervisor and if remote VXLAN tunnel endpoint (VTEP) discovery occurred. |
Step 2 |
If the mpls_entry is present in the hardware, check the vlan_xlate table. Example:
The vlan_xlate table should have one entry for the multicast destination IP address (DIP) of the packet. This example shows such an when multicast packets are sent to 225.0.0.3. |
Step 3 |
If the vlan_xlate table has the correct entry for the multicast DIP, check if the VLAN flood list shows the correct members (that is, the members of the VLAN excluding the encapsulation tunnel port). |
Packets Dropped in the Unicast Encapsulation Path
Unicast Packets Dropped When VTEP Is Reachable Through a Single Next Hop
Follow these steps if unicast packets are being dropped on the device in the access to network direction and VTEP is reachable through a single next hop.
SUMMARY STEPS
- Check if the remote peer is discovered in the hardware.
- Get the mapping of the source virtual port (SVP) to the next hop.
- Get the port number from the next-hop index.
- Get the mapping from the port number to the physical port on the chip.
- Get the egress port to next-hop index mapping.
- Check the tunnel parameters to make sure that the EGR IP tunnel shows the correct local VTEP IP address in the SIP field.
- Make sure that the tunnel DIP is programmed.
DETAILED STEPS
Step 1 |
Check if the remote peer is discovered in the hardware. Example:
Make sure a valid source IP address (SIP) exists. In this example, 102.102.102.102 is the remote VTEP IP address. |
Step 2 |
Get the mapping of the source virtual port (SVP) to the next hop. Example:
In this example, the next-hop index is 0x18. |
Step 3 |
Get the port number from the next-hop index. Example:
In this example, the port number is 8. |
Step 4 |
Get the mapping from the port number to the physical port on the chip. Example:
In this example, port number 8 is hg7. |
Step 5 |
Get the egress port to next-hop index mapping. Example:
In this example, next-hop index 0x18 points to hg7. |
Step 6 |
Check the tunnel parameters to make sure that the EGR IP tunnel shows the correct local VTEP IP address in the SIP field. Example:
In this example, SIP is the local VTEP IP address (101.101.101.101), L4_DEST_PORT is 0x2118 (port 8472), and DSCP_SEL = 1 means that the inner DSCP packet will be copied to the outer DSCP packet. |
Step 7 |
Make sure that the tunnel DIP is programmed. Example:
|
Unicast Packets Dropped When VTEP Is Reachable Through an ECMP Path
Follow these steps if unicast packets are being dropped on the device in the access to network direction and VTEP is reachable through an ECMP path.
SUMMARY STEPS
- Get the ECMP next hop for a given remote peer virtual port (VP).
- Convert the ECMP_PTR to decimal and add 200000 to get the port number.
- Get the list of interfaces in the ECMP next-hop set.
- Find the members of the port channel.
- Find the physical next-hop interfaces for the given next-hop index.
DETAILED STEPS
Step 1 |
Get the ECMP next hop for a given remote peer virtual port (VP). Example:
In this example, 0x1751 is the VP number for the remote peer IP address derived from using the d chg mpls_entry output.
|
||
Step 2 |
Convert the ECMP_PTR to decimal and add 200000 to get the port number. Example:
In this example, the port number is 200264. |
||
Step 3 |
Get the list of interfaces in the ECMP next-hop set. Example:
In this example, the next-hop interfaces are 1t, 2t, and 3t, which are port channels. |
||
Step 4 |
Find the members of the port channel. Example:
|
||
Step 5 |
Find the physical next-hop interfaces for the given next-hop index. Example:
In this example, next-hop index 0x5f7 points to hg4, 0x9b3 points to hg6, and 0x5f8 points hg7. |
Packets Dropped in the Unicast Decapsulation Path
Follow these steps if unicast packets are being dropped on the device in the network to access direction.
SUMMARY STEPS
- Check if the packets were sent to the supervisor and if remote VXLAN tunnel endpoint (VTEP) discovery occurred.
- If the mpls_entry is present in the hardware, check the vlan_xlate table.
- Check if the unicast DIP entry is present in the vlan_xlate table.
- Check if the unicast DIP entry is present in the vlan_xlate table.
- Make sure that the destination MAC address appears in the Layer 2 MAC address table.
DETAILED STEPS
Step 1 |
Check if the packets were sent to the supervisor and if remote VXLAN tunnel endpoint (VTEP) discovery occurred. |
Step 2 |
If the mpls_entry is present in the hardware, check the vlan_xlate table. Example:
The vlan_xlate table should have one entry for the multicast destination IP address (DIP) of the packet. This example shows such an when multicast packets are sent to 225.0.0.3. |
Step 3 |
Check if the unicast DIP entry is present in the vlan_xlate table. Example:
If the entry is present, decapsulation should occur. |
Step 4 |
Check if the unicast DIP entry is present in the vlan_xlate table. Example:
If the entry is present, decapsulation should occur. |
Step 5 |
Make sure that the destination MAC address appears in the Layer 2 MAC address table. Example:
If the destination MAC address is present, Layer 2 forwarding occurs. Otherwise, packets will be flooded using the decapsulation flood list. |