Configure and Verify Maximum Transmission Unit on Nexus Platforms

Introduction

This document describes how to configure and verify Maximum Transmission Unit (MTU) on Cisco Nexus switches.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Configure

Layer 3 MTU Configurations

All Layer 3 ports, regardless of platform, are configured on a per-port basis.

Configure MTU on a Switched Virtual Interface (SVI)

Nexus(config)#interface vlan 1
Nexus(config-if)#mtu 9216

Configure MTU on a Layer 3 Port

Nexus(config)#interface ethernet 1/1
Nexus(config-if)#no switchport
Nexus(config-if)#mtu 9216

Layer 2 MTU Configurations

Layer 2 MTU is set either through a network Quality of Service (QoS) policy or through configuration of the port itself (on switches that support per-port MTU).

Only the Nexus 7000, 9000, and certain 3000 models support per-port MTU.

Network QoS MTU Configuration

Nexus 3000: Includes Nexus 3048, 3064, 3132Q, 3132Q-X, 3132Q-XL, 3172, and 3500 Series switches

Nexus 5000: All Nexus 5000 and 5500 Series switches

Nexus 6000: All Nexus 6000 Series switches

In order to configure elevated MTU on these switches, create a network-qos policy or modify a policy that already exists in order to specify the elevated MTU. This configuration applies to all ports. This includes any Cisco Fabric Extender (FEX) ports connected to the switch.

Note: Do not use a network-qos policy to configure a per-port MTU. These policies do not support per-port MTU configuration.

policy-map type network-qos jumbo
  class type network-qos class-default
          mtu 9216
system qos
  service-policy type network-qos jumbo

Per-Port MTU Configuration

Nexus 3000: Includes Nexus 3132Q-V, 3164, 31108, 31128PQ, 3200 Series, and 36180YC-R switches

Neuxs 7000: All Nexus 7000- and 7700 Series switches

Nexus 9000: All Nexus 9200 Series switches (includes 92xxx), 9300 Series switches (includes 93xxx), and 9500 Series switches

Note: Platforms that use Per-Port MTU Configuration L2 ports can only use the System Jumbo MTU configuration on the switch or 1500.

By default System Jumbo MTU is 9216. L2 ports configured with Jumbo MTU automatically updates to the new value if it is changed.

Nexus# show running-config all | i jumbomtu
system jumbomtu 9216

Nexus(config)# system jumbomtu ?
<1500-9216> Enter jumbomtu

In order to configure MTU on a per-port basis, this configuration is needed:

Nexus(config)#interface ethernet 1/1
Nexus(config-if)#mtu 9216

If an invalid value is entered, an error is returned:

Nexus(config-if)# mtu 9000
ERROR: MTU can only be default or system jumbo MTU

Nexus 2000 Configuration

Note: The Nexus 2000 MTU is set with the configuration of jumbo frames on the parent switch. For parent switches that allow jumbo on a per-port basis, configure the FEX Fabric Port-channel (FPC). If the parent switch requires a network-qos policy, then jumbo is set with the QoS policy configuration of the parent switch. These changes are automatically pushed down to the FEX in both cases.

Per-Port Fabric Port Channel (FPC) Configuration (to be configured on the Parent Nexus Switch)

interface port-channel136
  switchport mode fex-fabric
  fex associate 136
  vpc 136
  mtu 9216

Note: The Nexus 7000 does not allow you to set FEX MTU with the FPC in Version 6.2 and later. You must instead create a custom QoS policy as the next configuration shows.

Nexus 7000 / FEX Jumbo Frame Configuration (only applies to version 6.2 and later)

Note: Modify the template currently in use. In order to find the current template in use, enter the show policy-map system type network-qos command.

7K(conf)#class-map type network-qos match-any c-nq-8e-custom 
7K(config-cmap-nqos)#match cos 0-7   

7K(config)#policy-map type network-qos nq-8e-custom template 8e
7K(config-pmap-nqos)#class type network-qos c-nq-8e-custom
7K(config-pmap-nqos-c)#congestion-control tail-drop
7K(config-pmap-nqos-c)#mtu 9216

7K(config)#system qos
7K(config-sys-qos)#service-policy type network-qos nq-8e-custom

Network QoS Policy Configuration (to be configured on the Parent Nexus Switch)

policy-map type network-qos jumbo
  class type network-qos class-default
    mtu 9216
system qos
  service-policy type network-qos jumbo

Verify

Use this section in order to confirm that your configuration works properly.

The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.

Layer 3 MTU

Verify Layer 3 MTU on all Nexus platforms with the show interface eth x/y command as this example shows:

Nexus# show interface ethernet 1/19
Ethernet1/19 is up
Dedicated Interface
Hardware: 100/1000/10000 Ethernet, address: 547f.ee5d.413c (bia 547f.ee5d.40fa)
MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec

Verify SVI MTU with the show interface vlan X command as this output shows:

Nexus# show interface vlan 1
Vlan1 is down (Non-routable VDC mode), line protocol is down
  Hardware is EtherSVI, address is  547f.eed8.ec7c
  Internet Address is 192.168.10.10/24
  MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec

Layer 2 MTU

This section describes how to verify Layer 2 MTU per platform. The commands are run from the parent switch.

Verify the MTU on Switches That Use Network QoS

Nexus# show queuing interface ethernet 1/1
Ethernet1/1 queuing information:
  TX Queuing
    qos-group  sched-type  oper-bandwidth
        0       WRR            100
  RX Queuing
    qos-group 0
    q-size: 469760, HW MTU: 9216 (9216 configured)   

-- or --

Nexus# show queuing interface ethernet 1/1

slot  1
=======


HW MTU of Ethernet1/1 : 9216 bytes

Egress Queuing for Ethernet1/1 [System]

Verify the MTU on Switches That Support Per-port MTU

Nexus# show interface ethernet 1/12
Ethernet1/12 is up
admin state is up, Dedicated Interface
  Hardware: 1000/10000 Ethernet, address: 7c0e.ceca.f183 (bia 7c0e.ceca.f183)
  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec  

Note: When the Nexus 3000 is on code earlier than 7.0(3)I2(2a), check the MTU value with the show queueing interface ethernet x/x command. Nexus 3000 switches that run 7.0(3)I2(2a) and later show the MTU size on a per-port basis.

Nexus 2000

Note: When you change the FEX MTU, the FEX increases its MTU to a higher, but predetermined, amount that does not match the configured value exactly. The parent device enforces the configured MTU at the FEX Fabric Port-Channel (FPC).

For a FEX connected to Nexus 5000, 6000, and 7000:

Nexus# show queuing interface ethernet 136/1/1
if_slot 68, ifidx 0x1f870000
Ethernet136/1/1 queuing information:
  Input buffer allocation:
  Qos-group: 0
  frh: 3
  drop-type: drop
  cos: 0 1 2 3 4 5 6 7
  xon       xoff      buffer-size
  ---------+---------+-----------
  19200     78080     90880

  Queueing:
  queue   qos-group    cos                 priority     bandwidth     mtu 
--------+------------+--------------------+---------+----------------+--------
  3                   0        0 1 2 3 4 5 6      WRR       100       9280

For a FEX connected to Nexus 9000:

9K# show interface ethernet 104/1/1
Ethernet104/1/1 is up
admin state is up,
  Hardware: 100/1000 Ethernet, address: 5475.d0e0.e5c2 (bia 5475.d0e0.e5c2)
  MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec

Troubleshoot

This section provides information you can use in order to troubleshoot your configuration.

The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.

Note: Refer to Important Information on Debug Commands before you use debug commands.

Sometimes it is necessary to verify the changes in software. In order to do this, check the Ethernet port manager (ethpm) in order to verify that the changes were pushed through software on any platform:

Switch#show system internal ethpm info interface eth 6/5 | egrep -i mtu
medium(broadcast), snmp trap(on), MTU(4000)

It is also possible on M1, M2, F1, and F2 modules to verify in the hardware that the changes were pushed down:

module-1#show hardware internal mac port 20 state | i MTU
 GD: Port speed Undecided GD MTU 10240 (fixed to max), PL MTU 9238 mode 0

Or, you can look at the QoS configuration on the linecard:

module-1# show hardware internal mac port 33 qos configuration | beg mtu
  vl  hw_mtu   pm_mtu   pm_adj  qos_mtu   qos_adj   last_mtu
   0   9238     9728      22     9216       22       9216

Impact

A mismatched MTU across a link can have an impact on routed interfaces with routing adjacencies and causes a type 1 inconsistency with VPC if both sides of the VPC do not match MTU. Configure with caution.

Known Defects

Cisco bug ID CSCuf20035 - (Nexus 7000) FEX MTU changes do not take effect on FEX queues.