This document describes how to configure N_Port ID Virtualization (NPIV) or N_Port Virtualization (NPV) with Fibre Channel over Ethernet (FCoE) between the Cisco Unified Computing System (UCS) and a Nexus 5000 switch.
The information in this document is based on these software and hardware versions:
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Figure 1. Topology Used in the Configuration
There are two procedures outlined here to help you configure the UCS:
It is important to note that the UCS does not support a non-default FC-Map when in NPV mode. For more information, reference Cisco bug IDs CSCuJ62005 and CSCud78290:
Create FCoE VSANs
You need to create VSANs and FCoE VLANs in UCS in order to carry the FCoE traffic. A best practice, but not a requirement, is to align the VSAN ID with the FCoE VLAN ID in order to help track and troubleshoot.
Name | Enter a name for the VSAN |
FC Zoning | Disabled - This is only used when you implement local zoning features |
Fabric Radio Buttons | Choose Fabric A |
VSAN ID | Enter the VSAN ID for the VSAN that is created on Fabric A - This should correspond with the VSAN ID that will be created on the Nexus 5000-A |
FCoE VLAN | Enter the FCoE VLAN ID for the VLAN which maps to this VSAN - This should correspond with the VLAN ID which will be created on the Nexus 5000-A |
Name | Enter a name for the VSAN |
FC Zoning | Disabled - This is only used when you implement local zoning features |
Fabric Radio Buttons | Choose Fabric B |
VSAN ID | Enter the VSAN ID for the VSAN that is created on Fabric B - This should correspond with the VSAN ID that will be created on the Nexus 5000-B |
FCoE VLAN | Enter the FCoE VLAN ID for the VLAN which maps to this VSAN - This should correspond with the VLAN ID which will be created on the Nexus 5000-B |
Configure UCS FCoE Port Channel
There are several procedures that are required in order to configure the Nexus 5000 for FCoE and NPIV:
Enable Feature FCoE
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# feature fcoe
FC license checked out successfully
fc_plugin extracted successfully
FC plugin loaded successfully
FCoE manager enabled successfully
FC enabled on all modules successfully
Enabled FCoE QoS policies successfully
Nexus 5000-A# show feature | include fcoe
Feature Name Instance State
-------------------- -------- --------
fcoe 1 enabled
Enable NPIV on the Nexus 5000
Nexus 5000-A# configuration terminal
Nexus 5000-A(config)# feature npiv
Nexus 5000-A# show npiv status
NPIV is enabled
Enable LACP on the Nexus 5000
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# feature lacp
Nexus 5000-A#show feature | include LACP
Feature Name Instance State
-------------------- -------- --------
lacp 1 enabled
Configure Nexus 5000 FCoE QoS Parameters
The configurable Cisco NX-OS QoS features allow you to classify the network traffic, prioritize the traffic flow, and provide congestion avoidance. The default QoS configuration on the Nexus 5000 provides lossless service for FC and FCoE traffic and best-effort service for Ethernet traffic. The Nexus 5000 FCoE QoS Parameters should already be configured by default.
Nexus 5000-A# show startup-config ipqos
. .
system qos
service-policy type queuing input fcoe-default-in-policy
service-policy type queuing output fcoe-default-out-policy
service-policy type qos input fcoe-default-in-policy
service-policy type network-qos fcoe-default-nq-policy
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# system qos
Nexus 5000-A(config-sys-qos)# service-policy type qos input
fcoe-default-in-policy
Nexus 5000-A(config-sys-qos)# service-policy type queuing input
fcoe-default-in-policy
Nexus 5000-A(config-sys-qos)# service-policy type queuing output
fcoe-default-out-policy
Nexus 5000-A(config-sys-qos)# service-policy type network-qos
fcoe-default-nq-policy
Nexus 5000-A# show startup-config ipqos
..
system qos
service-policy type queuing input fcoe-default-in-policy
service-policy type queuing output fcoe-default-out-policy
service-policy type qos input fcoe-default-in-policy
service-policy type network-qos fcoe-default-nq-policy
Create Nexus 5000 VLANs
Create two VLANs for FCoE use. The even should correlate with FI-A and the odd should correlate with FI-B. Use the FCoE VLAN IDs that are specified in the Configure the UCS section of this document.
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# vlan 100
Nexus 5000-A(config-vlan)# name FCoE-VLAN100
Nexus 5000-A# show vlan brief
VLAN Name Status Ports
---- ----------------------------- --------- -------------------------------
1 default active Eth1/1, Eth1/2, Eth1/3, Eth1/4
Eth1/5, Eth1/6, Eth1/7, Eth1/8
.
100 FCoE-VLAN100 active
Nexus 5000-B# configure terminal
Nexus 5000-B(config)# vlan 101
Nexus 5000-B(config-vlan)# name FCoE-VLAN101
Nexus 5000-B# show vlan brief
VLAN Name Status Ports
---- ----------------------------- --------- -------------------------------
1 default active Eth1/1, Eth1/2, Eth1/3, Eth1/4
Eth1/5, Eth1/6, Eth1/7, Eth1/8
.
100 FCoE-VLAN101 active
Create Nexus 5000 VSANs
Create two VSANs. The even should correlate with FI-A and the odd should correlate with FI-B. Use the VSAN IDs that are specified in the Configure the UCS section of this document.
Nexus 5000-A# configure terminal
Nexus 5000-A(config)#vsan database
Nexus 5000-A(config-vsan)# vsan 100
Nexus 5000-A(config-vsan)# vsan 100 name FCoE-VSAN100
Nexus 5000-A#show vsan 100
vsan 100 information
name:FCoE-VSAN100 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
Nexus 5000-B# configure terminal
Nexus 5000-B(config)#vsan database
Nexus 5000-B(config-vsan)# vsan 101
Nexus 5000-B(config-vsan)# vsan 101 name FCoE-VSAN101
Nexus 5000-A# show vsan 101
vsan 101 information
name:FCoE-VSAN101 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
Map the Vlan to VSAN
Nexus 5000-A#configure terminal
Nexus 5000-A(config)# vlan 100
Nexus 5000-A(config-vlan)#fcoe vsan 100
Nexus 5000-A# show vlan fcoe
Original VLAN ID Translated VLAN ID Association State
---------------- ------------------ -----------------
10 100 Operational
Nexus 5000-B# configure terminal
Nexus 5000-B(config)# vlan 101
Nexus 5000-B(config-vlan)# fcoe vsan 101
Nexus 5000-B# show vlan fcoe
Original VLAN ID Translated VLAN ID Association State
---------------- ------------------ -----------------
10 101 Operational
Configure Nexus 5000 FCoE Uplink Port and Port Channel
Nexus 5000-A#configure terminal
Nexus 5000-A(config)# interface ethernet 1/5-6
Nexus 5000-A(config-if-range)# switchport mode trunk
Nexus 5000-A(config-if-range)# switchport trunk allowed vlan 100
Nexus 5000-A(config-if-range)# channel-group 2 mode active
Nexus 5000-A(config-if-range)#no shutdown
Nexus 5000-A(config-if-range)# exit
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# interface port-channel 2
Nexus 5000-A(config-if)# no shutdown
Nexus 5000-A# show port-channel database
port-channel2
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/5
Age of the port-channel is 0d:00h:09m:19s
Time since last bundle is 0d:00h:05m:27s
Last bundled member is Ethernet1/6
Time since last unbundle is 0d:00h:05m:38s
Last unbundled member is Ethernet1/6
Ports: Ethernet1/5 [active ] [up] *
Ethernet1/6 [active ] [up]
Nexus 5000-B# configure terminal
Nexus 5000-B(config)# interface ethernet 1/5-6
Nexus 5000-B(config-if-range)# switchport mode trunk
Nexus 5000-B(config-if-range)# switchport trunk allowed vlan 101
Nexus 5000-B(config-if-range)# channel-group 3 mode active
Nexus 5000-B(config-if-range)# no shutdown
Nexus 5000-B(config-if-range)# exit
Nexus 5000-B# configure terminal
Nexus 5000-B)config)# interface port-channel 3
Nexus 5000-B(config-if)# no shutdown
Nexus 5000-B# show port-channel database
port-channel3
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/5
Age of the port-channel is 0d:00h:00m:29s
Time since last bundle is 0d:00h:00m:29s
Last bundled member is Ethernet1/6
Ports: Ethernet1/5 [active ] [up] *
Ethernet1/6 [active ] [up]
Create Nexus 5000 Virtual Fibre Channel (VFC)
VFCs need to be created and bound to the Ethernet interfaces (Port Channels) that carry the FCoE traffic.
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# interface vfc 2
Nexus 5000-A(config-if)# bind interface port-channel 2
Nexus 5000-A(config-if)# switchport trunk allowed vsan 100
Nexus 5000-A(config-if)# switchport mode F
Nexus 5000-A(config-if)# no shutdown
Nexus 5000-A# show interface vfc 2
vfc2 is trunking
Bound interface is port-channel2
Hardware is Ethernet
Port WWN is 20:01:54:7f:ee:8b:a6:ff
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 1
Trunk vsans (admin allowed and active) (100)
Trunk vsans (up) (100)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1020 frames input, 102728 bytes
0 discards, 0 errors
1154 frames output, 478968 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Tue Aug 21 06:32:40 2012
Nexus 5000-B# configure terminal
Nexus 5000-B(config)# interface vfc 3
Nexus 5000-B(config-if)# bind interface port-channel 3
Nexus 5000-B(config-if)# switchport trunk allowed vsan 101
Nexus 5000-A(config-if)# switchport mode F
Nexus 5000-B(config-if)# no shutdown
Nexus 5000-B# show interface vfc 3
vfc3 is trunking
Bound interface is port-channel3
Hardware is Ethernet
Port WWN is 20:02:54:7f:ee:8c:e2:7f
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 1
Trunk vsans (admin allowed and active) (101)
Trunk vsans (up) (101)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
738 frames input, 69404 bytes
0 discards, 0 errors
763 frames output, 427488 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Tue Aug 21 06:47:03 2012
Configure Storage Ports on the Nexus 5000
Connect the storage ports from the FCoE-enabled storage system to their designated ports on the Nexus 5000s. VFCs need to be created for these storage ports and the associated interfaces need to be configured.
Nexus 5000-A# configure terminal
Nexus 5000-A(config)# interface Ethernet 1/32
Nexus 5000-A(config-if)# switchport mode trunk
Nexus 5000-A(config-if)# switchport trunk allowed vlan 100
Nexus 5000-A(config-if)# spanning-tree port type edge trunk
Warning: Edge port type (portfast) should only be enabled on ports connected to a single host.
Connecting hubs, concentrators, switches, bridges, etc... to this interface when edge port type
(portfast) is enabled, can cause temporary bridging loops.
Use with CAUTION
Nexus 5000-A(config-if)# no shut
Nexus 5000-A(config-if)# exit
Nexus 5000-A(config)# interface vfc 132
Nexus 5000-A(config-if)# switchport trunk allowed vsan 100
Nexus 5000-A(config-if)# bind interface Ethernet 1/32
Nexus 5000-A(config-if)# no shutdown
Nexus 5000-A(config-if)# exit
Nexus 5000-A(config)# vsan database
Nexus 5000-A(config-vsan-db)# vsan 100 interface vfc 132
Nexus 5000-A(config-vsan-db)# exit
Nexus 5000-A(config)# show interface vfc 132
vfc132 is trunking
Bound interface is Ethernet1/32
Hardware is Ethernet
Port WWN is 20:83:54:7f:ee:8b:a6:ff
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 100
Trunk vsans (admin allowed and active) (100)
Trunk vsans (up) (100)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
10 frames input, 1060 bytes
0 discards, 0 errors
11 frames output, 1404 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Tue Aug 21 06:37:14 2012
Nexus 5000-B# configure terminal
Nexus 5000-B(config)# interface Ethernet 1/32
Nexus 5000-B(config-if)# switchport mode trunk
Nexus 5000-B(config-if)# switchport trunk allowed vlan 101
Nexus 5000-B(config-if)# spanning-tree port type edge trunk
Warning: Edge port type (portfast) should only be enabled on ports connected
to a single host. Connecting hubs, concentrators, switches, bridges, etc...
to this interface when edge port type (portfast) is enabled, can cause temporary
bridging loops.
Use with CAUTION
Nexus 5000-B(config-if)# no shut
Nexus 5000-B(config-if)# exit
Nexus 5000-B(config)# interface vfc 132
Nexus 5000-B(config-if)# switchport trunk allowed vsan 101
Nexus 5000-B(config-if)# bind interface Ethernet 1/32
Nexus 5000-B(config-if)# no shutdown
Nexus 5000-B(config-if)# exit
Nexus 5000-B(config)# vsan database
Nexus 5000-B(config-vsan-db)# vsan 101 interface vfc 132
Nexus 5000-B(config-vsan-db)# exit
Nexus 5000-B(config)# show interface vfc 132
vfc132 is trunking
Bound interface is Ethernet1/32
Hardware is Ethernet
Port WWN is 20:83:54:7f:ee:8c:e2:7f
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 101
Trunk vsans (admin allowed and active) (101)
Trunk vsans (up) (101)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
1 minute input rate 136 bits/sec, 17 bytes/sec, 0 frames/sec
1 minute output rate 200 bits/sec, 25 bytes/sec, 0 frames/sec
10 frames input, 1060 bytes
0 discards, 0 errors
11 frames output, 1404 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
Interface last changed at Tue Aug 21 06:50:31 2012
Fabric A Configuration
zone name ESXI01_1 vsan 100
member pwwn 10:00:e4:11:5b:65:c1:11
member pwwn 50:01:43:80:11:36:37:5c
zone name ESXI02_1 vsan 100
member pwwn 10:00:e4:11:5b:65:c0:c9
member pwwn 50:01:43:80:11:36:37:5c
zone name ESXI03_1 vsan 100
member pwwn 10:00:e4:11:5b:65:c2:01
member pwwn 50:01:43:80:11:36:37:5c
zoneset name Fabric-A vsan 100
member ESXI01_1
member ESXI02_1
member ESXI03_1
zoneset activate name Fabric-A vsan 100
Fabric A Verification
Nexus 5000-A# show flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
vfc2 100 0xab0000 10:00:e4:11:5b:65:c1:11 20:00:e4:11:5b:65:c1:11
vfc2 100 0xab0040 10:00:e4:11:5b:65:c0:c9 20:00:e4:11:5b:65:c0:c9
vfc2 100 0xab0020 10:00:e4:11:5b:65:c2:01 20:00:e4:11:5b:65:c2:01
vfc132 100 0xab00c0 50:01:43:80:11:36:37:5c 50:01:43:80:11:36:37:50
This flogi table shows three hosts (pwwns starting with 10:) and one storage pwwn
(50:01:43:80:11:36:37:5c).
Fabric B Configuration
zone name EVA_1-ESXI01_2 vsan 101
member pwwn 10:00:e4:11:5b:67:c1:12
member pwwn 50:01:43:80:11:36:37:5d
zone name EVA_1-ESXI02_2 vsan 101
member pwwn 10:00:e4:11:5b:67:c0:c8
member pwwn 50:01:43:80:11:36:37:5d
zone name EVA_1-ESXI03_2 vsan 101
member pwwn 10:00:e4:11:5b:67:c2:04
member pwwn 50:01:43:80:11:36:37:5d
zoneset name Fabric-B vsan 101
member EVA_1-ESXI01_2
member EVA_1-ESXI02_2
member EVA_1-ESXI03_2
zoneset activate name Fabric-B vsan 101
Fabric B Verification
Nexus 5000-B# show flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
vfc3 101 0xab0000 10:00:e4:11:5b:67:c1:12 20:00:e4:11:5b:65:c1:12
vfc3 101 0xab0040 10:00:e4:11:5b:67:c0:c8 20:00:e4:11:5b:65:c0:c8
vfc3 101 0xab0020 10:00:e4:11:5b:67:c2:04 20:00:e4:11:5b:65:c2:04
vfc132 101 0xab00c0 50:01:43:80:11:36:37:5d 50:01:43:80:11:36:37:50
This flogi table shows three hosts (pwwns starting with 10:) and one storage pwwn
(50:01:43:80:11:36:37:5d).
There is currently no specific troubleshooting information available for this configuration.
Revision | Publish Date | Comments |
---|---|---|
1.0 |
01-May-2014 |
Initial Release |