Q-in-VNI over VxLAN Fabric Deployment Guide

Available Languages

Download Options

  • PDF
    (1.2 MB)
    View with Adobe Reader on a variety of devices
Updated:May 2, 2023

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Available Languages

Download Options

  • PDF
    (1.2 MB)
    View with Adobe Reader on a variety of devices
Updated:May 2, 2023
 

 

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

Summary

In traditional network segmentation, the number of segments that can be created is limited to 4096 VLANs. This can be insufficient in situations, such as cloud computing environments where multiple tenants or customers may require their own virtual networks. VxLAN solves the problem of limited network segmentation and isolation in traditional VLAN-based networks.

Q-in-VNI over VxLAN addresses the requirement of limited network segmentation and isolation by stacking multiple VLANs above each other, allowing for even more virtual network segmentation and isolation. This provides a large number of virtual networks to be created, with the necessary flexibility and scalability for cloud computing environments and other situations where many virtual networks are required.

In summary, Q-in-VNI deployment using VxLAN EVPN as the transport network provides a highly efficient and scalable solution for delivering Layer 2 VPN services. It allows users to maintain their existing VLAN structure while connecting to the service provider's network and provides high availability and redundancy with VxLAN EVPN's robust control plane.

Introduction to Q-in-Q

When a service provider offers services, each user’s network segmentation is preserved using Q-in-Q, also known as "double tagging" or "stacking VLANs”. This is a technique used to extend VLANs (Virtual LANs) across a transport network. It allows multiple customer VLANs to be tagged with a provider vlan, allowing service providers to offer Layer 2 VPN services to their customers. This allows customers to maintain their existing vlan structure while connecting to the service provider's network.

In a Q-in-Q deployment, customer frames are tagged with two vlan tags: a customer-facing vlan tag and a provider-facing vlan tag. The customer-facing vlan tag is used to identify the customer's vlan, while the provider-facing vlan tag is used to identify the provider's vlan. This allows the service provider to maintain the customer's vlan structure while connecting the customer's network to the provider's network. It can also be used to provide additional security by isolating customer VLANs from each other.

DiagramDescription automatically generated

Figure 1.  Q-in-Q packet header

Introduction to VxLAN (Virtual extensible LAN)

VxLAN (Virtual Extensible LAN) is a network virtualization technology that extends Layer 2 connections across a Layer 3 network. It uses a VxLAN header, which encapsulates the original Layer-2 frames, allowing them to be transmitted over a Layer 3 network. VxLAN ID (also known as the VxLAN Network Identifier or VNI) is a 24-bit identifier used to uniquely identify a virtual network.

The VxLAN ID in the header serves as the basis for building the VxLAN segment It is used to map the virtual network to a specific VxLAN segment, and to identify the destination of the traffic in the VxLAN network. All the endpoints that are part of the same VxLAN segment have the same VxLAN ID.

VxLAN addresses scalability limitations of traditional VLANs by allowing a large number of VLANs to be created. While traditional VLANs are limited to 4094, VxLANs can support up to 16 million unique VxLAN IDs (VNIs). This allows for a large number of virtual networks to be created, making it well suited for large-scale data center deployments.

This segmentation approach enables multi-tenancy by allowing different virtual networks to be isolated from each other, even if they are using the same physical infrastructure. This can be useful in a cloud computing environment, where different tenants may require their own virtual networks. Additionally, it can be useful where the network needs to be extended across multiple geographic locations, such as in disaster recovery scenarios.

Graphical user interface, applicationDescription automatically generated

Figure 2.  VxLAN Packet Header

Introduction to EVPN (Ethernet Virtual Private Network)

EVPN (Ethernet Virtual Private Network) is a control-plane technology that allows for the automatic discovery and configuration of virtual private networks (VPNs) using Multi-Protocol BGP (MP-BGP) routing protocol. It provides a way to exchange Layer 2 and Layer 3 information. The transport can be either VxLAN or MPLS. In this case study, VxLAN is used as a transport mechanism for the fabric infrastructure.

EVPN provides several benefits over traditional Layer 2 VPN technologies, including:

      Multi-homing support – EVPN allows for multiple links to be used for a single VPN, providing redundancy and high availability.

      Layer 3 and Layer 2 VPN services – EVPN supports both Layer 3 and Layer 2 VPN services, making it more versatile than traditional Layer 2 VPN technologies.

      Automatic discovery and configuration – EVPN uses a control-plane to automatically discover and configure VPNs, reducing the need for manual configuration and improving scalability.

      Interoperability – EVPN is designed to work with existing Layer 2 and Layer 3 protocols, making it easier to integrate into existing networks.

      EVPN can be used in different scenarios, such as data center interconnect, WAN connectivity, and service provider infrastructure. It can also be used in conjunction with VxLAN, which is known as VxLAN EVPN. This combination provides a way to extend Layer 2 connections across a Layer 3 network, while also providing the control-plane functionality using EVPN route types. EVPN route types are used to carry different types of information over the network. EVPN route types are carried in MP-BGP (Multiprotocol Border Gateway Protocol) updates and are used by the network's routing and switching equipment to make forwarding decisions.

TableDescription automatically generated

Figure 3.  EVPN Route Types

Q-in-VNI with EVPN and VxLAN as transport

A TELCO service provider VxLAN fabric infrastructure is CLOS-based architecture with a Leaf and Spine layer switches. All the leaf nodes will be connected to all the spine switches but the spine switches will not be connected to each other. Each leaf node or node pair will act as VxLAN Tunnel Endpoint (VTEP). The traffic entering the fabric will have a fixed latency with 3 hop to reach the end service.

DiagramDescription automatically generated

Figure 4.  VxLAN topology

In the above figure the Service provider VxLAN fabric consists of 2 Leaf and 2 Spine Switches. The leaf switches are the point of presence for the service provider offering Q-in-Q services to customer Red, Blue and Green.

In this case study,

      Leaf1 and Leaf2 loopback addresses are advertised using OSPF routing protocol. The other recommended option is to use IS-IS as the Interior gateway protocol(IGP).

      On both Leaf switches, Loopback 1 is source interface for VxLAN NVE interface.

      BGP EVPN is configured between the Leaf and Spine nodes.

      Spine nodes act as the route reflector (RR) for the EVPN sessions.

      Customers Red, Blue, and green are configured with a unique Provider-vlan mapped to a unique layer2 VNI across the VxLAN fabric.

      On the local leaf node, every customer endpoint (MAC) received on an interface configured with a Provider-vlan, is learned and advertised using BGP EVPN as type-2 route (MAC-Route).

      This type-2 route will also contain layer2 VNI and the next hop information which is the VTEP IP address of the local leaf.

VxLAN EVPN fabric configuration sample shown below is from Leaf and Spine switches. Note that EVPN configuration under BGP when not explicitly configured, the route target for every VNI, import a,nd export will be set to auto by default.

Note:      Partial configuration is shown below with few VLANs and VNIs but a few hundred VLANs and VNIs are used as part of this case study.

Table 1.        VxLAN Configuration sample

LEAF-1 Configuration

LEAF-2 Configuration

SPINE-1 Configuration

vlan Configuration

vlan 1300

  vn-segment 13000

vlan 1301

  vn-segment 13001

vlan 1302

  vn-segment 13002

BGP Configuration

router bgp 65001

  router-id 10.102.0.3

  neighbor 10.102.0.1

    remote-as 65001

    update-source loopback0

    address-family l2vpn evpn

      send-community

      send-community extended

evpn

OSPF Configuration

router ospf UNDER

  router-id 10.102.0.3

interface loopback0

  ip router ospf UNDER area 0.0.0.0

interface loopback1

  ip router ospf UNDER area 0.0.0.0

interface Ethernet1/49

  ip ospf network point-to-point

  ip router ospf UNDER area 0.0.0.0

VxLAN Configuration

interface nve1

switch(config-if)# no shutdown

  host-reachability protocol bgp

  source-interface loopback1

  member vni 13000

    mcast-group 239.1.1.1

  member vni 13001

    mcast-group 239.1.1.1

  member vni 13002

    mcast-group 239.1.1.1

vlan Configuration

vlan 1300

  vn-segment 13000

vlan 1301

  vn-segment 13001

vlan 1302

  vn-segment 13002

BGP Configuration

router bgp 65001

  router-id 10.102.0.2

  neighbor 10.102.0.1

    remote-as 65001

    update-source loopback0

    address-family l2vpn evpn

      send-community

      send-community extended

evpn

OSPF Configuration

router ospf UNDER

  router-id 10.102.0.2

interface loopback0

  ip router ospf UNDER area 0.0.0.0

interface loopback1

  ip router ospf UNDER area 0.0.0.0

interface Ethernet1/49

  ip ospf network point-to-point

  ip router ospf UNDER area 0.0.0.0

VxLAN Configuration

interface nve1

switch(config-if)# no shutdown

  host-reachability protocol bgp

  source-interface loopback1

  member vni 13000

    mcast-group 239.1.1.1

  member vni 13001

    mcast-group 239.1.1.1

  member vni 13002

    mcast-group 239.1.1.1

BGP Configuration

router bgp 65001

  router-id 10.102.0.1

  neighbor 10.102.0.2

    remote-as 65001

    update-source loopback0

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

  neighbor 10.102.0.3

    remote-as 65001

    update-source loopback0

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

OSPF Configuration

router ospf UNDER

  router-id 10.102.0.1

interface loopback0

  ip router ospf UNDER area 0.0.0.0

interface loopback254

  ip router ospf UNDER area 0.0.0.0

interface Ethernet1/5

  ip ospf network point-to-point

  ip router ospf UNDER area 0.0.0.0

interface Ethernet1/6

  ip ospf network point-to-point

  ip router ospf UNDER area 0.0.0.0

The subsequent topics will cover various techniques used by service providers using Q-in-VNI over a VxLAN fabric as transport to provide VPN services.

Q-in-VNI

Q-in-VNI allows you to categorize network traffic by assigning it to a specific port. In a multi-tenant setup, you can designate a port to a tenant/customer and transmit/receive data using the VxLAN overlay. The provider interface is configured as an access port that uses ‘switchport mode dot1q tunnel’, which receives a dot1q tagged packet from the customer switch. The VxLAN encapsulated packet holds the customer's VLAN tag in its inner frame's L2 header.

Interface configuration:

switch(config-if)# interface Ethernet1/51

switch(config-if)# description Q-in-VNI for Customer Red

switch(config-if)# switchport

switch(config-if)# switchport mode dot1q-tunnel

switch(config-if)# switchport access vlan 300

switch(config-if)# spanning-tree port type edge

switch(config-if)# spanning-tree bpdufilter enable

switch(config-if)# mtu 9216

switch(config-if)# no shutdown

 

switch(config-if)# interface Ethernet1/52

switch(config-if)# description Q-in-VNI for Customer Blue

switch(config-if)# switchport

switch(config-if)# switchport mode dot1q-tunnel

switch(config-if)# switchport access vlan 301

switch(config-if)# spanning-tree port type edge

switch(config-if)# spanning-tree bpdufilter enable

switch(config-if)# mtu 9216

switch(config-if)# no shutdown

 

switch(config)# vlan 300

switch(config-vlan)# vn-segment 33000

switch(config)# vlan 300

switch(config-vlan)# vn-segment 33001

DiagramDescription automatically generated

Figure 5.  Q-in-VNI model

In the above figure, the user Red’s traffic is forwarded over a VxLAN overlay network, which uses VxLAN headers to encapsulate the original dot1q frame on VLAN 10. This allows the traffic to be transmitted over a layer-3 network, while preserving the original layer-2 information. The VxLAN header includes information such as the Source VTEP and Destination VTEP, as well as the layer-2 VNI, which is used to forward the traffic to its destination. When the traffic reaches the LEAF-2 switch, the VxLAN header is removed (decapsulated), and the original VLAN 10 frame is sent out on the other side. The user’s original frame is preserved throughout the journey, ensuring that the traffic is delivered to its intended destination intact.

Like user Red, user Blue communicates over VLAN 10, but it is configured as a Q-in-VNI port with provider VLAN 301 and a unique layer-2 VNI of 33001. The traffic is forwarded to the correct Q-in-VNI port-based on the matching provider VLAN 301. Despite having the same customer VLAN, each customer has a unique layer-2 VNI to ensure proper traffic forwarding and thereby maintaining multi tenancy.

A packet capture in the VxLAN fabric is shown below:

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Layer-2 VNI for Provider vlan 300 is 33000

      Encapsulated packet showing customer Red original packet with VLAN ID 10

      Capture also shows inner packet’s SRC IP 10.1.1.2 and DST IP address 10.1.1.1

   

    [Protocols in frame: eth:ethertype:ip:udp:VxLAN:eth:ethertype:vlan:ethertype:ip:data]

Ethernet II, Src: a0:b4:39:0b:9e:0b, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75

    Source: a0:b4:39:0b:9e:0b

        Address: a0:b4:39:0b:9e:0b

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.2, Dst: 10.102.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Flags: 0x0000

        Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

    Source: 10.102.1.2

    Destination: 10.102.1.1

User Datagram Protocol, Src Port: 13894, Dst Port: 4789

    Source Port: 13894

    Destination Port: 4789

Virtual eXtensible Local Area Network

    Flags: 0x0800, VxLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VxLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VxLAN Network Identifier (VNI): 33000

    Reserved: 0

Ethernet II, Src: 00:45:50:01:00:01, Dst: 00:85:54:01:01:01

    Destination: 00:85:54:01:01:01

        Address: 00:85:54:01:01:01

    Source: 00:45:50:01:00:01

        Address: 00:45:50:01:00:01

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 10

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0000 0000 1010 = ID: 10

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.1.1.2, Dst: 10.1.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

   Time to live: 255

    Protocol: Unknown (253)

    Header checksum: 0xdab1 [validation disabled]

    [Header checksum status: Unverified]

    Source: 10.1.1.2

    Destination: 10.1.1.1

Selective Q-in-VNI with multiple provider VLANs

Selective Q-in-VNI is a VxLAN tunneling feature that allows a user-specific VLAN or range of user VLANs on a port to be associated with one specific provider VLAN. Packets that come in with a VLAN tag that matches any of the configured user  VLANs on the port are tunneled across the VxLAN fabric using the properties of the service provider VNI mapped to the provider VLAN in the configuration. The VxLAN encapsulated packet carries the user VLAN tag as part of the L2 header of the inner frame.

The packets with a VLAN tag that is not present in the range of the configured user VLANs on a selective Q-in-VNI configured port are dropped. This includes the packets with a VLAN tag that matches the native VLAN on the port. A range of customer VLANs can also be selectively mapped to a single provider VLAN as shown in the configuration sample below.

Note:      See limitations for native VLAN while configuring selective Q-in-VNI given in the reference section at the end of the document.

Interface configuration sample:

switch# config terminal

switch(config-if)# interface Ethernet1/52

switch(config-if)# description Selective Q-in-VNI with multiple provider Vlans

switch(config-if)# switchport

switch(config-if)# switchport mode trunk

switch(config-if)# switchport vlan mapping 10 dot1q-tunnel 100

switch(config-if)# switchport vlan mapping 11 dot1q-tunnel 101

switch(config-if)# switchport vlan mapping 12 dot1q-tunnel 102

.

..

...

switch(config-if)# switchport vlan mapping 58 dot1q-tunnel 148

switch(config-if)# switchport vlan mapping 59 dot1q-tunnel 149

switch(config-if)# switchport vlan mapping 800-820 dot1q-tunnel 150

switch(config-if)# switchport vlan mapping 821-840 dot1q-tunnel 151

switch(config-if)# switchport trunk allowed vlan 100-149,150,151

switch(config-if)# spanning-tree port type edge trunk

switch(config-if)# spanning-tree bpdufilter enable

switch(config-if)#  mtu 9216

switch(config-if)# no shutdown

 

switch# config terminal

switch(config) vlan 100

switch(config-vlan) vn-segment 30000

switch(config) vlan 101

switch(config-vlan) vn-segment 30001

switch(config) vlan 102

switch(config-vlan) vn-segment 30002

.

..

...

switch(config) vlan 120

switch(config-vlan) vn-segment 30020

switch(config) vlan 121

switch(config-vlan) vn-segment 30021

switch(config) vlan 122

switch(config-vlan) vn-segment 30022

DiagramDescription automatically generated

Figure 6.  Selective Q-in-VNI model

In the above figure, the service provider switch is set up to map user VLAN 10 to provider VLAN 100, ensuring that traffic is forwarded to the correct destination. The VxLAN header includes the layer-2 VNI, in this case is 30000, which is used to route traffic over the network with the source VTEP Leaf-1 and destination VTEP Leaf-2. The encapsulation and decapsulation process is performed at each VTEP, ensuring that the original user frame remains intact as it travels over the VxLAN fabric. At the LEAF-2 switch, the VxLAN header is removed, and the inner frame (VLAN 10) is sent out on the egress side, facing the user. This allows the user Red's original frame to be preserved and delivered without any modifications, ensuring seamless and reliable communication over the network.

Similarly, user VLAN 11 is mapped to provider VLAN 101 with Layer-2 VNI 30001 and user VLAN 20 to provider VLAN 110 with Layer-2 VNI 30010, based on the configuration of the interface. This selective mapping of user VLANs to Q-in-VNI maintains multi-tenancy over the VxLAN network. In summary, each port on the service provider's switch can accommodate multiple customer VLANs, each connected to a distinct provider VLAN with a unique layer-2 VNI. This directs traffic to the correct Q-in-VNI port, making efficient network communication.

Packet capture in the VxLAN fabric is shown below,

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Layer-2 VNI for Provider vlan 30002, which provider VLAN 102

      Encapsulated packet showing customer Red inner frame with VLAN ID 12

      Inner frames seen with SRC IP 102.1.1.2 and DST IP address 102.1.1.1

   

   [Protocols in frame: eth:ethertype:ip:udp:VxLAN:eth:ethertype:vlan:ethertype:ip:data]

Ethernet II, Src: a0:b4:39:0b:9e:0b, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75

    Source: a0:b4:39:0b:9e:0b

        Address: a0:b4:39:0b:9e:0b

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.2, Dst: 10.102.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

  Flags: 0x0000

        Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

    Source: 10.102.1.2

    Destination: 10.102.1.1

User Datagram Protocol, Src Port: 21413, Dst Port: 4789

    Source Port: 21413

    Destination Port: 4789

Virtual eXtensible Local Area Network

    Flags: 0x0800, VxLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VxLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VxLAN Network Identifier (VNI): 30002

    Reserved: 0

Ethernet II, Src: 00:51:54:02:01:01, Dst: 00:85:56:02:01:02

    Destination: 00:85:56:02:01:02

        Address: 00:85:56:02:01:02

    Source: 00:51:54:02:01:01

        Address: 00:51:54:02:01:01

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 12

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0000 0000 1100 = ID: 12

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 102.1.1.2, Dst: 102.1.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Total Length: 490

    Identification: 0x8774 (34676)

    Flags: 0x0000

        Fragment offset: 0

    Time to live: 255

     Protocol: Unknown (253)

    Source: 102.1.1.2

    Destination: 102.1.1.1

VLAN mapping

VLAN mapping is the method of connecting a user VLAN to a service provider VLAN. In certain situations, VLAN translation may be necessary, such as when multiple users connect to the same physical switch using the same VLAN encapsulation but should not present on the same layer 2 segments. In these cases, the incoming VLAN is translated to a unique VLAN in the VxLAN fabric which is mapped to a VNI to extend the segment.

In this case study, the user VLAN 1000 is mapped to the provider VLAN 1300 on the interface. The provider VLAN 1300 has a layer-2 VNI of 13000.

Interface configuration showing multiple user VLANs mapped to translated provider VLANs:

switch# config terminal

switch(config-if)# interface Ethernet1/53

switch(config-if)# description Port Vlan Mapping

switch(config-if)# switchport

switch(config-if)# switchport mode trunk

switch(config-if)# switchport vlan mapping enable

switch(config-if)# switchport vlan mapping 1000 1300

switch(config-if)# switchport vlan mapping 1001 1301

switch(config-if)# switchport vlan mapping 1002 1302

switch(config-if)# switchport vlan mapping 1003 1303

switch(config-if)# switchport vlan mapping 1004 1304

.

..

...

switch(config-if)# switchport vlan mapping 1194 1494

switch(config-if)# switchport vlan mapping 1195 1495

switch(config-if)# switchport vlan mapping 1196 1496

switch(config-if)# switchport vlan mapping 1197 1497

switch(config-if)# switchport vlan mapping 1198 1498

switch(config-if)# switchport vlan mapping 1199 1499

switch(config-if)# switchport trunk allowed vlan 1300-1499

switch(config-if)# no shutdown

 

switch# config terminal

switch(config) vlan 1300

switch(config-vlan) vn-segment 13000

switch(config) vlan 1301

switch(config-vlan) vn-segment 13001

switch(config) vlan 1302

switch(config-vlan) vn-segment 13002

switch(config) vlan 1303

switch(config-vlan) vn-segment 13003

switch(config) vlan 1310

.

..

...

switch(config-vlan) vn-segment 13010

switch(config) vlan 1311

switch(config-vlan) vn-segment 13011

switch(config) vlan 1312

switch(config-vlan) vn-segment 13012

Vlan mapping with one Customer Tag

DiagramDescription automatically generated

Figure 7.  VLAN mapping on VxLAN infrastructure

In the above figure, the user VLANs 1000, 1001, 1002, and so on are mapped to provider VLANs 1300, 1301, 1302, respectively. The user is connected to a service provider switch set up as a VLAN-mapped Q-in-VNI port. User VLAN 1000 is mapped to provider VLAN 1300 which is configured with a layer-2 VNI of 13000.

When traffic on VLAN 1000 reaches the interface, it is encapsulated with a VxLAN header that includes layer-2 VNI 13000. This is because the interface is set up to translate user VLAN 1000 to provider VLAN 1300. The VxLAN header has a source VTEP address of the LEAF-1 switch and a destination VTEP address of the LEAF-2 switch. The packet is then sent to LEAF-2, where the VxLAN header is removed and the inner frame (on VLAN 1000) is sent out on the egress side facing the user.

This process ensures that the user VLAN 1000 traffic is preserved and transmitted securely between the LEAF-1 and LEAF-2 switches through the VxLAN fabric. The VxLAN header adds an extra layer of abstraction to the original user VLAN 1000 frame, allowing it to traverse the network and reach its destination, where the header is then removed, and the original frame is delivered to the user. This VxLAN-based approach enables efficient and secure communication between user VLANs and provider VLANs while preserving the user's original VLAN information.

A packet capture in the VxLAN fabric is shown below,

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Note that vlan 1000 which is translated is not seen packet capture

      Provider VLAN 1300, which is translated VLANs Layer-2 VNI 13000 seen

      Inner frames SRC IP 10.100.1.2 and DST IP address 10.100.1.1

    [Protocols in frame: eth:ethertype:ip:udp:VxLAN:eth:ethertype:ip:data]

Ethernet II, Src: a0:b4:39:0b:9e:0b, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75

    Source: a0:b4:39:0b:9e:0b

        Address: a0:b4:39:0b:9e:0b

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.2, Dst: 10.102.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

  Flags: 0x0000

     Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

    Source: 10.102.1.2

    Destination: 10.102.1.1

User Datagram Protocol, Src Port: 42297, Dst Port: 4789

    Source Port: 42297

    Destination Port: 4789

tual eXtensible Local Area Network

    Flags: 0x0800, VxLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VxLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VxLAN Network Identifier (VNI): 13000

    Reserved: 0

Ethernet II, Src: 00:55:53:03:03:b9, Dst: 00:85:54:03:02:f1

    Destination: 00:85:54:03:02:f1

        Address: 00:85:54:03:02:f1

    Source: 00:55:53:03:03:b9

        Address: 00:55:53:03:03:b9  

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.100.1.2, Dst: 10.100.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Total Length: 490

    Identification: 0x6f4f (28495)

    Flags: 0x0000

     Fragment offset: 0

    Time to live: 255

    Protocol: Unknown (253)

    Source: 10.100.1.2

    Destination: 10.100.1.1

Q-in-VNI Vlan mapping with two Customer Tag

In the packet capture shown below, user traffic was simulated with two C-Tags (i.e. two VLAN tags). The inner tag is VLAN 600, and the outer tag is VLAN 1000. According to the configuration, outer VLAN 1000 is mapped to provider VLAN 1300.

Packet capture in the VxLAN fabric is shown below:

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Note that VLAN 1000 which is translated is not seen packet capture

      Provider VLAN 1300, which is translated VLANs Layer-2 VNI 13000 seen

      Second user VLAN 600 is preserved inside the VxLAN packet.

      Inner frames seen with SRC IP 60.1.1.2 and DST IP address 60.1.1.1

Frame 1: 554 bytes on wire (4432 bits), 554 bytes captured (4432 bits) on interface ps-inb, id 0

    [Protocols in frame: eth:ethertype:ip:udp:VxLAN:eth:ethertype:vlan:ethertype:ip:data]

Ethernet II, Src: a0:b4:39:0b:9e:0b, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75

    Source: a0:b4:39:0b:9e:0b

        Address: a0:b4:39:0b:9e:0b

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.2, Dst: 10.102.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Flags: 0x0000

    Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

    Source: 10.102.1.2

    Destination: 10.102.1.1

User Datagram Protocol, Src Port: 50138, Dst Port: 4789

    Source Port: 50138

    Destination Port: 4789

Virtual eXtensible Local Area Network

    Flags: 0x0800, VxLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VxLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VxLAN Network Identifier (VNI): 13000

    Reserved: 0

Ethernet II, Src: 00:10:aa:00:04:85, Dst: 00:10:bb:00:04:86

    Destination: 00:10:bb:00:04:86

        Address: 00:10:bb:00:04:86

    Source: 00:10:aa:00:04:85

        Address: 00:10:aa:00:04:85

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 600

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0010 0101 1000 = ID: 600

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 60.1.1.2, Dst: 60.1.1.1

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Total Length: 486

    Identification: 0x13b8 (5048)

    Flags: 0x0000

     Fragment offset: 0

    Time to live: 255

    Protocol: Unknown (253)

    Source: 60.1.1.2

    Destination: 60.1.1.1

QinQ-QinVNI Multi tag

QinQ-QinVNI can provide an even greater level of network segmentation and flexibility. For example, a single physical network could have multiple Q-in-Q VLANs running, each with its own set of virtual networks created using Q-in-VNI. This allows for a high degree of isolation between different groups of users or applications, while still allowing them to share the same physical infrastructure.

Additionally, QinQ-QinVNI can also be used to create virtual network overlays, which can be used to extend the reach of virtual networks across multiple physical locations, such as in a multi-tenant data center or cloud environment. This allows for more efficient use of resources and greater flexibility in managing and scaling virtual networks.

Overall, the combination of QinQ-QinVNI can provide a powerful toolset for network administrators to create, manage, and scaled virtual networks in a variety of different environments.

In this case study, there are two VLAN tags. The outer VLAN tag is the provider VLAN that maps to the layer-2 VNI in the  VXLAN fabric. The inner VLAN has the user VLAN which is transparent to the interface configuration and the VXLAN fabric and is kept intact. The multi-tag allows multiple provider VLANs on the same physical interface as shown below.  

QinQ-QinVNI Multi tag with one Customer Tag

Interface configuration sample:

switch# config terminal

switch(config-if)# interface Ethernet1/54

switch(config-if)# description Q-in-Q to Q-in-VNI

switch(config-if)# switchport

switch(config-if)# switch(config-if)# switchport mode trunk

switch(config-if)# switch(config-if)# switchport trunk allow-multi-tag

switch(config-if)# switchport trunk allowed vlan 50,100-149,300-301,1300-1499

switch(config-if)# spanning-tree port type edge trunk

switch(config-if)# mtu 9216

switch(config-if)# no shutdown

 

switch# config terminal

switch(config) vlan 100

switch(config-vlan) vn-segment 30000

switch(config) vlan 101

switch(config-vlan) vn-segment 30001

switch(config) vlan 102

switch(config-vlan) vn-segment 30002

switch(config) vlan 110

.

..

...

switch(config-vlan) vn-segment 30010

switch(config) vlan 111

switch(config-vlan) vn-segment 30011

switch(config) vlan 112

switch(config-vlan) vn-segment 30012

DiagramDescription automatically generated

Figure 8.  QinQ-QinVNI multi-tag with one C-tag model

Packet capture in the VxLAN fabric is shown below:

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Provider VLAN 100, Layer-2 VNI 30000 is used for forwarding this traffic

      Customer VLAN 10 is preserved inside the VxLAN packet.

      Inner frames seen with SRC IP 100.1.1.1 and DST IP address 100.1.1.2

    [Protocols in frame: eth:ethertype:ip:udp:VxLAN:eth:ethertype:vlan:ethertype:ip:data]

Ethernet II, Src: a0:b4:39:0b:a2:57, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75      

    Source: a0:b4:39:0b:a2:57

        Address: a0:b4:39:0b:a2:57

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.1, Dst: 10.102.1.2

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Total Length: 540

    Identification: 0x55aa (21930)

    Flags: 0x0000

     Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

  Source: 10.102.1.1

    Destination: 10.102.1.2

User Datagram Protocol, Src Port: 29571, Dst Port: 4789

    Source Port: 29571

    Destination Port: 4789

Virtual eXtensible Local Area Network

    Flags: 0x0800, VxLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VxLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VxLAN Network Identifier (VNI): 30000

    Reserved: 0

Ethernet II, Src: 00:85:54:02:01:02, Dst: 00:51:52:02:01:01

    Destination: 00:51:52:02:01:01

        Address: 00:51:52:02:01:01

    Source: 00:85:54:02:01:02

        Address: 00:85:54:02:01:02

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 10

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0000 0000 1010 = ID: 10

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 100.1.1.1, Dst: 100.1.1.2

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

    Total Length: 486

    Identification: 0xef59 (61273)

    Flags: 0x0000

     Fragment offset: 0

    Time to live: 255

    Protocol: Unknown (253)

    Source: 100.1.1.1

    Destination: 100.1.1.2

QinQ-QinVNI Multi tag with two Customer Tag

As discussed above, in this case, Q-in-VNI port is shown in the packet capture, sending traffic to another Q-in-VNI port with two user VLAN tags: outer VLAN 10 and inner VLAN 500. The provider VLAN for the Q-in-VNI is 100, which is mapped to layer-2 VNI 30000. The packet capture displays both the outer and inner user VLANs, as well as the layer-2 VNI 30000. Again both the user VLANs are retained inside the fabric.

Interface Configuration is the same as the previous one:

switch# config terminal

switch(config-if)# interface Ethernet1/54

switch(config-if)# description Q-in-Q to Q-in-VNI

switch(config-if)# switchport

switch(config-if)# switch(config-if)# switchport mode trunk

switch(config-if)# switch(config-if)# switchport trunk allow-multi-tag

switch(config-if)# switchport trunk allowed vlan 50,100-149,300-301,1300-1499

switch(config-if)# spanning-tree port type edge trunk

switch(config-if)# mtu 9216

switch(config-if)# no shutdown

 

switch# config terminal

switch(config) vlan 100

switch(config-vlan) vn-segment 30000

switch(config) vlan 300

switch(config-vlan) vn-segment 33000

switch(config) vlan 301

switch(config-vlan) vn-segment 33001

DiagramDescription automatically generated

Figure 9.  QinQ-QinVNI multi tag with two C-tag model

Packet capture in the VxLAN fabric is shown below,

      SRC VTEP address 10.102.1.2 is LEAF-1 lo1 address (source-interface of NVE1 interface)

      DST VTEP address 10.102.1.1 is LEAF-2 lo1 address (source-interface of NVE1 interface)

      VxLAN packet UDP header DST Port is 4789

      Provider VLAN 300, Layer-2 VNI 33000 is used for forwarding this traffic

      User VLAN 10, first C-tag is preserved inside the VxLAN packet.

      User VLAN 500, Second C-tag is also preserved inside the VxLAN packet.

      Inner frames seen with SRC IP 10.1.1.1 and DST IP address 10.1.1.2

 [Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:vlan:ethertype:vlan:ethertype:arp]

Ethernet II, Src: a0:b4:39:0b:a2:57, Dst: 00:fd:22:8d:1b:75

    Destination: 00:fd:22:8d:1b:75

        Address: 00:fd:22:8d:1b:75

    Source: a0:b4:39:0b:a2:57

        Address: a0:b4:39:0b:a2:57    

    Type: IPv4 (0x0800)

Internet Protocol Version 4, Src: 10.102.1.1, Dst: 10.102.1.2

    0100 .... = Version: 4

    .... 0101 = Header Length: 20 bytes (5)

    Differentiated Services Field: 0xe0 (DSCP: CS7, ECN: Not-ECT)

        1110 00.. = Differentiated Services Codepoint: Class Selector 7 (56)

        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)

Flags: 0x0000

        Fragment offset: 0

    Time to live: 255

    Protocol: UDP (17)

    Source: 10.102.1.1

    Destination: 10.102.1.2

User Datagram Protocol, Src Port: 53720, Dst Port: 4789

    Source Port: 53720

    Destination Port: 4789

Virtual eXtensible Local Area Network

    Flags: 0x0800, VXLAN Network ID (VNI)

        0... .... .... .... = GBP Extension: Not defined

        .... .... .0.. .... = Don't Learn: False

        .... 1... .... .... = VXLAN Network ID (VNI): True

        .... .... .... 0... = Policy Applied: False

        .000 .000 0.00 .000 = Reserved(R): 0x0000

    Group Policy ID: 0

    VXLAN Network Identifier (VNI): 33000

    Reserved: 0

Ethernet II, Src: 00:bb:94:00:04:89, Dst: 00:bb:94:00:04:88

    Destination: 00:bb:94:00:04:88

        Address: 00:bb:94:00:04:88

    Source: 00:bb:94:00:04:89

        Address: 00:bb:94:00:04:89

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 10

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0000 0000 1010 = ID: 10

    Type: 802.1Q Virtual LAN (0x8100)

802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 500

    111. .... .... .... = Priority: Network Control (7)

    ...0 .... .... .... = DEI: Ineligible

    .... 0001 1111 0100 = ID: 500

    Type: ARP (0x0806)

    Trailer: 00000000000000000000

Address Resolution Protocol (reply)

    Hardware type: Ethernet (1)

    Protocol type: IPv4 (0x0800)

    Hardware size: 6

    Protocol size: 4

    Opcode: reply (2)

    Sender MAC address: 00:bb:94:00:04:89

    Sender IP address: 10.1.1.1

    Target MAC address: 00:bb:94:00:04:88

    Target IP address: 10.1.1.2

QinVNI with Layer 2 Protocol Tunneling

Layer 2 Protocol Tunneling is the process of encapsulating a Layer 2 (L2) protocol within a different L2 protocol, so that it can be transmitted over a network that does not support the original L2 protocol. It is aimed to extend the reach of L2 protocols over networks that would otherwise not support them, by encapsulating the protocol in a compatible format.

Spanning Tree Protocol (STP), VLAN Trunking Protocol (VTP), Link Layer Discovery Protocol (LLDP), and Cisco Discovery Protocol (CDP) are all Layer 2 protocols that can be tunneled. These protocols are typically used to manage the behavior and configuration of switches in a local area network (LAN), and tunneling them can enable their use over networks that would not otherwise support them.

Tunneling these protocols can help to maintain consistency and manageability of the network, even when the network is spread across multiple physical locations or spans multiple underlying networking technologies.

DiagramDescription automatically generated

Figure 10.  Q-in-VNI with Layer 2 Tunneling Protocol at provider edge

Interface configuration sample:

switch(config-if)# interface Ethernet1/51

switch(config-if)# description Q-in-VNI for Customer Red

switch(config-if)# switchport

switch(config-if)# switchport mode dot1q-tunnel

switch(config-if)# switchport access vlan 300

switch(config-if)# spanning-tree port type edge

switch(config-if)# l2protocol tunnel cdp

switch(config-if)# l2protocol tunnel stp

switch(config-if)# l2protocol tunnel vtp

switch(config-if)# l2protocol tunnel lldp

switch(config-if)# mtu 9216

switch(config-if)# no shutdown

 

switch# config terminal

switch(config) vlan 300

switch(config-vlan) vn-segment 33000

 

switch# show l2protocol tunnel

COS for Encapsulated Packets: 5

                          Shutdown    Drop    Encapsulation Decapsulation     Drop

  Port   Protocol         Threshold Threshold    Counter       Counter       Counter

-------- --------         --------- --------- ------------- ------------- -------------

Eth1/51  cdp               ----      ----             0             0             0

         stp               ----      ----           152             0             0

         vtp               ----      ----             0             0             0

         lldp              ----      ----            11             0             0

         ----              ----      ----          ----          ----          ----

         ----------        ----      ----          ----          ----          ----

Q-in-VNI with LACP Tunneling

LACP (Link Aggregation Control Protocol) tunneling over VxLAN is a method for transmitting LACP frames over a VxLAN network. LACP is used to bundle multiple physical links together to form a single logical link, providing increased bandwidth and redundancy.

When LACP is tunneled over VxLAN, LACP frames are encapsulated within VxLAN frames and transmitted over the VxLAN network. This allows LACP to be used to manage link aggregation in networks that use VxLAN as the underlying technology.

Tunneling LACP over VxLAN can be beneficial in large, geographically dispersed networks where multiple physical locations are connected over VxLAN. By encapsulating LACP frames within VxLAN frames, the protocol can be used to manage link aggregation across the WAN, helping to ensure high availability and resiliency of the network.

DiagramDescription automatically generated

Figure 11.  Q-in-VNI with LACP Tunneling at provider edge

Interface configuration sample:

switch# config terminal

switch(config)# interface ethernet 1/51

switch(config-if)# switchport mode dot1q-tunnel

switch(config-if)# switchport access vlan 10

switch(config-if)# spanning-tree bpdufilter enable

switch(config-if)# l2protocol tunnel lacp

switch(config-if)# interface nve1

switch(config-if)# overlay-encapsulation vxlan-with-tag tunnel-control-frames

 

switch# show l2protocol tunnel

COS for Encapsulated Packets: 5

                          Shutdown    Drop    Encapsulation Decapsulation     Drop

  Port   Protocol         Threshold Threshold    Counter       Counter       Counter

-------- --------         --------- --------- ------------- ------------- -------------

Eth1/51  cdp               ----      ----             0             0             0

         stp               ----      ----             0             0             0

         vtp               ----      ----             0             0             0

         lldp              ----      ----             0             0             0

         lacp              ----      ----             0             0             0

         ----------        ----      ----          ----          ----          ----

References

Cisco Configuration Guide for Q-in-VNI on NX-OS Release 10.3(x)

https://www.cisco.com/c/en/us/td/docs/dcn/nx-os/nexus9000/103x/configuration/vxlan/cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-release-103x/m_configuring_cross_connect_93x.html#task_abn_tj5_dhb

Note:      All guidelines and limitations when using Q-in-VNI on various nexus platforms is listed in the above URL.

VXLAN Design with Cisco Nexus 9300 Platform Switches

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-732453.html

VXLAN Network with MP-BGP EVPN Control Plane Design Guide

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html

Cisco Press: Building Data Centers with VXLAN BGP EVPN: A Cisco NX-OS Perspective

https://www.ciscopress.com/store/building-data-centers-with-vxlan-bgp-evpn-a-cisco-nx-9781587144677

Configuring VXLAN BGP EVPN

https://www.cisco.com/c/en/us/td/docs/dcn/nx-os/nexus9000/103x/configuration/vxlan/cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-release-103x/m_configuring_vxlan_bgp_evpn.html

Configuring VxLAN Multisite

https://www.cisco.com/c/en/us/td/docs/dcn/nx-os/nexus9000/103x/configuration/vxlan/cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-release-103x/m_configuring_multisite_93x.html

Legal Information

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

© 2023 Cisco Systems, Inc. All rights reserved.

Learn more