- Feature Information
- Supported Hardware
- Upgrading FCIP and iSCSI Interfaces
- Configuring IPS Ports on Cisco MDS 9220i Switch
- Configuring 40 Gbps Speed on Cisco MDS 24/10 port SAN Extension Module
- Configuring IPStorage Interfaces for IPv4
- Fibre Channel Module with IPS Ports Core Dumps
- About VLANs for IPStorage
- Interface Subnet Requirements
- Verifying IPStorage Connectivity
- IPStorage IPv4-ACL Guidelines
- Configuring IPStorage High Availability
- Configuring CDP
- Changing Link Speed on IPStorage Interfaces
- Displaying Statistics
- Default Settings for IPStorage Services Parameters
Configuring IPStorage Services
Cisco MDS 9000 Series IPStorage (IPS) services extend the reach of Fibre Channel SANs by using open-standard, IP-based technology. The switch connects separated SAN islands using Fibre Channel over IP (FCIP) and it allows IP hosts to access Fibre Channel storage.
Note The FCIP feature is available in the modules with IPS ports and available in Cisco MDS 9250i, MDS 9220i, and Cisco MDS 9700 Director switches.
The Cisco MDS 24/10 Port SAN Extension Module for MDS 9700 series switch allows you to use Fibre Channel and FCIP features. The Cisco MDS 24/10 port SAN Extension Module can be used in any of the Cisco MDS 9700 series switches.
This chapter includes the following sections:
- Feature Information
- Supported Hardware
- Upgrading FCIP and iSCSI Interfaces
- Configuring IPS Ports on Cisco MDS 9220i Switch
- 25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1).
- Configuring IPStorage Interfaces for IPv4
- Fibre Channel Module with IPS Ports Core Dumps
- Configuring IPStorage High Availability
- Configuring CDP
- Changing Link Speed on IPStorage Interfaces
- Default Settings for IPStorage Services Parameters
Feature Information
This section briefly describes the new and updated features for releases, starting from Cisco MDS NX-OS Release 8.x.
|
|
|
---|---|---|
25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1). |
This feature allows you to configure 40-Gbps speed on the IPS ports of Cisco MDS 24/10 port SAN Extension Module. |
|
This feature enables users to change the link speed on IPStorage interfaces between 1 Gbps and 10 Gbps on the Cisco MDS 24/10 port SAN Extension Module. |
||
Changing Link Speed on Cisco MDS 9250i Multiservice Fabric Switch |
This feature enables users to change the link speed on IPStorage interfaces between 1 Gbps and 10 Gbps on the Cisco MDS 9250i Multiservice Fabric Switch. |
Supported Hardware
You can configure the FCIP and iSCSI features on the following types of hardware:
Upgrading FCIP and iSCSI Interfaces
ISSU/D is disruptive for FCIP and iSCSI interfaces along with the update of the underlying IPS and GigabitEthernet interfaces supporting these upper layer interfaces. Fibre Channel interfaces (including FC interfaces on the same module or switch as the IPS and GigabitEthernet interfaces), FCoE Ethernet interfaces, and supervisor modules are all updated nondisruptively.
Within Cisco MDS 9700 Directors, the modules with IPS ports implement a 5 minute delay before the next module with IPS ports is upgraded.
Configuring IPS Ports on Cisco MDS 9220i Switch
Starting from Cisco MDS NX-OS Release 8.5(1), the Cisco MDS 9220i Fabric Switch is supported. Cisco MDS 9220i switch supports six IPS ports. Ports 1 through 4 support 1- and 10-Gbps speed and port 6 supports 40-Gbps speed. By default, the first two ports are operating at a 1-Gbps speed mode and are licensed. To enable the other ports and different speed modes, you must obtain a license.You cannot move the default license from the first two ports to any other ports. For information on port licensing, see the Cisco MDS 9000 Series Licensing Guide, Release 8.x.
Cisco MDS 9220i switch supports four 1 or 10 Gbps IPS ports and one 40 Gbps IPS port. All these ports are handled by a single Service Engine. Table 6-2 provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 9220i switch. For configuring the link speed, see Changing Link Speed on Cisco MDS 9220i Switch.
|
|
|
|
|
|
|
---|---|---|---|---|---|---|
Note 25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1).
Configuring 40 Gbps Speed on Cisco MDS 24/10 port SAN Extension Module
From Cisco MDS NX-OS Release 8.5(1), you can configure 40 Gbps on the IPS ports 9 and 10 of Cisco MDS 24/10 port SAN Extension Module. Cisco MDS 24/10 port SAN Extension Module supports eight 1 or 10 Gbps IPS ports and two 40 Gbps IPS ports. Ports 1 through 4 and port 9 and handled by Service Engine 1 and Ports 5 through 8 and port 10 and handled by Service Engine 2. Each Service Engine has the capability of 40 Gbps. You can only configure 40 Gbps speed on ports 9 and 10. When you configure 40 Gbps speed on port 9, ports 1 through 4 are moved to out-of-service because the Service Engine capability is 40 Gbps. However, ports 5 through 8 can be configured for 1 or 10 Gbps speed provided port 10 is not configured for 40 Gbps speed. Similarly, when you configure 40 Gbps speed on port 10, ports 5 through 8 are moved to out-of-service. However, ports 1through 4 can be configured for 1 or 10 Gbps speed provided port 9 is not configured for 40 Gbps speed. Table 6-3 provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 24/10 port SAN Extension Module. For configuring the link speed, see Changing Link Speed on Cisco MDS 9220i Switch.
The following are the recommended configurations for achieving maximum throughput on the 40 Gbps IPS port:
- The maximum bandwidth of FCIP tunnel is 10 Gbps. Therefore, configure four FCIP tunnels on the 40 Gbps IPS port. To create the FCIP tunnels, create four VLAN sub interfaces on the 40 Gbps IPS port. On each VLAN create an FCIP tunnel.
- Configure the number of TCP connections to 5.
|
|
||
---|---|---|---|
|
|
|
|
Configuring IPStorage Interfaces for IPv4
Both FCIP and iSCSI rely on TCP/IP for network connectivity. On each Fibre Channel module with IPS ports, connectivity is provided in the form of IP storage ports on Cisco MDS 9250i switches and Cisco MDS 9700 series switches with 24/10 port SAN Extension modules that are appropriately configured. This section covers the steps required to configure IP for subsequent use by FCIP and iSCSI.
A new port mode, called IPS, is defined for IP storage ports on each Fibre Channel module with IPS ports. IP storage ports are implicitly set to IPS mode, so it can only be used to perform iSCSI and FCIP storage functions. IP storage ports do not bridge Ethernet frames or route other IP packets.
Each IPS port represents a single virtual Fibre Channel host in the Fibre Channel SAN. All the iSCSI hosts connected to this IPS port are merged and multiplexed through the single Fibre Channel host.
In large scale iSCSI deployments where the Fibre Channel storage subsystems require explicit LUN access control for every host device, use of proxy-initiator mode simplifies the configuration.
Note To configure IPv6 on an IPStorage interface, see the Cisco Fabric Manager Security Configuration Guide. For information about configuring FCIP, see Chapter2, “Configuring Fibre Channel over IP” For information about configuring iSCSI, see Chapter4, “Configuring Internet Small Computer Systems Interface”
Tip IPStorage ports on any Fibre Channel module with IPS ports should not be configured in the same IPS broadcast domain as the management IPS port—they should be configured in a different broadcast domain, either by using separate standalone hubs or switches or by using separate VLANs.
Basic IPStorage Configuration
Figure 6-1 shows an example of a basic IPStorage IP version 4 (IPv4) configuration.
Figure 6-1 IPStorage IPv4 Configuration Example
Note The port on the Ethernet switch to which the IPStorage interface is connected should be configured as a host port (also known as access port) instead of a switch port. Spanning tree configuration for that port (on the Ethernet switch) should disabled. This helps avoid the delay in the management port coming up due to delay from Ethernet spanning tree processing that the Ethernet switch would run if enabled. For Cisco Ethernet switches, use either the switchport host command in Cisco IOS or the set port host command in Catalyst OS.
Fibre Channel Module with IPS Ports Core Dumps
IPS core dumps are different from the system’s kernel core dumps for other modules. When the Fibre Channel module with IPS port’s operating system (OS) unexpectedly resets, it is useful to obtain a copy of the memory image (called a IPS core dump) to identify the cause of the reset. Under that condition, the Fibre Channel module with IPS ports sends the core dump to the supervisor module for storage. Cisco MDS switches have two levels of IPS core dumps:
- Partial core dumps (default)—Each partial core dump consists of four parts (four files). All four files are saved in the active supervisor module.
In Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules, each partial core dump consists of five parts (five files). All five files are saved in the active supervisor module.
Use the show cores command to list these files.
- Full core dumps—Each full core dump of Cisco MDS 9250i Switches and SSN-16 modules consists of 64 parts (64 files), and each full core dump of Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules consists of 67 parts (67 files). The IPS core dump for MSM-18/4 modules consists of 32 parts. This dump cannot be saved on the supervisor module because of its large space requirement. They are copied directly to an external TFTP server.
Use the system cores tftp: command to configure an external TFTP server to copy the IPS core dump (and other core dumps).
To configure IPS core dumps on the Fibre Channel module with IPS ports, follow these steps:
Configuring Interface Descriptions
See the Cisco Fabric Manager Interfaces Configuration Guide for details on configuring the switch port description for any interface.
Configuring Beacon Mode
See the Cisco Fabric Manager Interfaces Configuration Guide for details on configuring the beacon mode for any interface.
Configuring Autonegotiation
By default, autonegotiation is enabled on all the IPStorage interface. You can enable or disable autonegotiation for a specified IPStorage interface. When autonegotiation is enabled, the port automatically detects the speed or pause method, and duplex of incoming signals based on the link partner. You can also detect link up conditions using the autonegotiation feature.
Configuring the MTU Frame Size
You can configure the interfaces on a switch to transfer large (or jumbo) frames on a port. The default IP maximum transmission unit (MTU) frame size is 1500 bytes for all Ethernet ports. By configuring jumbo frames on a port, the MTU size can be increased up to 9000 bytes.
Note The minimum MTU size is 576 bytes.
Tip MTU changes are disruptive, all FCIP links and iSCSI sessions flap when the software detects a change in the MTU size.
Configuring Promiscuous Mode
You can enable or disable promiscuous mode on a specific IPStorage interface. By enabling the promiscuous mode, the IPStorage interface receives all the packets and the software then filters and discards the packets that are not destined for that IPStorage interface.
About VLANs for IPStorage
Virtual LANs (VLANs) create multiple virtual Layer 2 networks over a physical LAN network. VLANs provide traffic isolation, security, and broadcast control.
IPStorage ports automatically recognize Ethernet frames with IEEE 802.1Q VLAN encapsulation. If you need to have traffic from multiple VLANs terminated on one IPStorage port, configure subinterfaces—one for each VLAN.
If the Fibre Channel module with IPS ports or MPS-14/2 module is connected to a Cisco Ethernet switch, and you need to have traffic from multiple VLANs coming to one IPS port, verify the following requirements on the Ethernet switch:
Use the VLAN ID as a subscription to the IPStorage interface name to create the subinterface name: <slot-number> / <port-number>. <VLAN-ID>
.
Interface Subnet Requirements
IPStorage interfaces (major), subinterfaces (VLAN ID), and management interfaces (mgmt 0) can be configured in the same or different subnet depending on the configuration (see Table 6-4 ).
|
|
|
|
---|---|---|---|
Yes1 |
Two major interfaces can be configured in the same or different subnets. |
||
Yes 1 |
Two subinterfaces with the same VLAN ID can be configured in the same or different subnets. |
||
Two subinterfaces with different VLAN IDs cannot be configured in the same subnet. |
|||
A subinterface cannot be configured on the same subnet as the major interface. |
|||
The mgmt0 interface cannot be configured in the same subnet as the IPStorage interfaces or subinterfaces. |
|||
1.Except IPSec enabled interfaces, which should not be configured in the same IP subnet. |
Note The configuration requirements in Table 6-4 also apply to Ethernet PortChannels.
Verifying IPStorage Connectivity
Once the IPStorage interfaces are connected with valid IP addresses, verify the interface connectivity on each switch. Ping the IP host using the IP address of the host to verify that the static IP route is configured correctly.
Note If the connection fails, verify the following, and ping the IP host again:
- The IP address for the destination (IP host) is correctly configured.
- The host is active (powered on).
- The IP route is configured correctly.
- The IP host has a route to get to the IPStorage interface subnet.
- The IPStorage interface is in the up
state.
IPStorage IPv4-ACL Guidelines
Tip If IPv4-ACLs are already configured in a IPStorage interface, you cannot add this interface to an Ethernet PortChannel group.
Follow these guidelines when configuring IPv4-ACLs for IPStorage interfaces:
Note Other protocols such as User Datagram Protocol (UDP) and HTTP are not supported in IPStorage interfaces. Applying an ACL that contains rules for these protocols to a IPStorage interface is allowed but those rules have no effect.
- Apply IPv4-ACLs to the interface before you enable an interface. This ensures that the filters are in place before traffic starts flowing.
- Be aware of the following conditions:
– If you use the log-deny option, a maximum of 50 messages are logged per second.
– The established, precedence, and fragments options are ignored when you apply IPv4-ACLs (containing these options) to IPStorage interfaces.
– If an IPv4-ACL rule applies to a preexisting TCP connection, that rule is ignored. For example if there is an existing TCP connection between A and B, and an IPv4-ACL specifies dropping all packets whose source is A and destination is B is subsequently applied, it will have no effect.
Configuring IPStorage High Availability
Virtual Router Redundancy Protocol (VRRP) and Ethernet PortChannels are two IPStorage features that provide high availability for iSCSI and FCIP services.
VRRP for iSCSI and FCIP Services
VRRP provides a redundant alternate path to the IPStorage port for iSCSI and FCIP services. VRRP provides IP address failover protection to an alternate IPStorage interface so the IP address is always available (see Figure 6-2).
In Figure 6-2, all members of the VRRP group must be IPStorage ports. VRRP group members can be one or more of the following interfaces:
- One or more interfaces in the same Fibre Channel module with IPS ports or MSM-18/4 module
- Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in one switch
- Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in different switches
- IPStorage subinterfaces
- Ethernet PortChannels and PortChannel subinterfaces
Note You can configure no more than seven VRRP groups, both IPv4 and IPv6, on a IPStorage interface, including the main interface and all subinterfaces.
Configuring VRRP for IPStorage Interfaces
To configure VRRP for IPStorage interfaces using IPv4, follow these steps:
To configure VRRP for IPStorage interfaces using IPv6, follow these steps:
Note The VRRP preempt option is not supported on IPS interfaces. However, if the virtual IPv4 IP address is also the IPv4 IP address for the interface, then preemption is implicitly applied.
Configuring CDP
The Cisco Discovery Protocol (CDP) is supported on the management Ethernet interface on the supervisor module and the IPStorage interfaces on the Fibre Channel module with IPS ports or MSM-18/4 module.
For information about configuring CDP, refer to the Cisco MDS 9000 Series NX-OS Fundamentals Configuration Guide.
Changing Link Speed on IPStorage Interfaces
Changing Link Speed on Cisco MDS 9250i Multiservice Fabric Switch
The Cisco MDS 9250i Multiservice Fabric Switch has two IPStorage interfaces that support 1 Gbps and 10 Gbps link speeds. By default, IPStorage interfaces are configured at 10 Gbps link speed.
Note Switching between different link speeds is supported on Cisco 10 Gbps IPStorage platforms starting from Cisco MDS NX-OS Release 6.2(13). An ISSD to a release earlier than Cisco MDS NX-OS Release 6.2(13) when any of the IPStorage ports are configured at 1 Gbps, is disallowed. Reconfigure such ports back to the default link speed of 10 Gbps before attempting such a downgrade.
To configure 1 Gbps link speed on an IPStorage interface, follow these steps:
|
|
|
---|---|---|
|
||
Administratively disables the interface and stops traffic through the interface. |
||
switch(config-if)# switchport speed 10002 |
Sets the link speed of the interface and all subinterfaces to 1000 Mbps (1 Gbps). This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue. |
|
Exits IPStorage interface configuration mode and returns to privileged EXEC mode. |
||
To configure 10 Gbps link speed on an IPStorage interface, follow these steps:
|
|
|
---|---|---|
|
||
Administratively disables the interface and stops traffic through the interface. |
||
switch(config-if)# switchport speed 100004 |
Sets the link speed of the interface and all subinterfaces to 10000 Mbps (10 Gbps). This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue. |
|
Exits IPStorage interface configuration mode and returns to privileged EXEC mode. |
||
If there is a mismatch between the configured link speed and the small form-factor pluggable (SFP) speed capabilities, the port goes into an Error Disabled state and a corresponding syslog message is logged. In such a scenario, either the configured link speed or the SFP should be changed. If the link speed is changed, even if the port is already enabled, the shutdown and no shutdown commands must be explicitly issued for the change to be applied.
For more information about supported 1 Gbps SFPs for a Cisco MDS 9250i Multiservice Fabric Switch, see the Cisco MDS 9000 Family Pluggable Transceivers Data Sheet.
For information about configuring FCIP tunnels with IPStorage interfaces at 1 Gbps speed, see the Configuring FCIP chapter.
Changing Link Speed on Cisco MDS 9220i Switch
Note For Cisco MDS 9220i switch, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.
To configure 1 Gbps link speed on an IPStorage interface, follow these steps:
To configure 10 Gbps link speed on an IPStorage interface, follow these steps:
To configure 25 Gbps link speed on an IPStorage interface, follow these steps:
To configure 40 Gbps link speed on an IPStorage interface, follow these steps:
Changing Link Speed on Cisco MDS 24/10 port SAN Extension Module
Note For Cisco MDS 24/10 port SAN Extension Module, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.
To configure 1 Gbps link speed on an IPStorage interface, follow these steps:
To configure 10 Gbps link speed on an IPStorage interface, follow these steps:
To configure 40 Gbps link speed on an IPStorage interface, follow these steps:
Displaying Statistics
This section provides examples to verify IPStorage interfaces and TCP/IP statistics on the IPStorage ports.
Example 6-1 Displaying the IPStorage Interface
Note In Cisco MDS NX-OS Release 7.3(0)DY(1), 40GE IPStorage interfaces are not supported.
Displaying Ethernet MAC Statistics
The show ips stats mac interface ips command takes the IPStorage interface as a parameter and returns the statistics for that interface. See Example 6-2.
Note Use the physical interface, not the subinterface, to display Ethernet MAC statistics.
Example 6-2 Displaying IPStorage Interface MAC Statistics
Displaying TCP Statistics
Use the show ips stats tcp interface ips to display and verify TCP statistics. This command takes the main Ethernet interface as a parameter, and shows TCP stats along with the connection list and TCP state. The detail option shows all information maintained by the interface. See Example 6-3 and Example 6-4.
Example 6-3 Displaying TCP Statistics
Example 6-4 Displaying Detailed TCP Statistics
Use the show ips stats icmp interface ips to display and verify IP statistics. This command takes the main Ethernet interface as a parameter and returns the ICMP statistics for that interface. See Example 6-5.
Example 6-5 Displaying ICMP Statistics
Displaying IPStorage Ports Speed
Use the show ips status command to verify the programmed speed of an IPStorage port.
Example 6-6 Displays IPStorage Port Speed
Default Settings for IPStorage Services Parameters
Table 6-5 lists the default settings for IPStorage services parameters.
|
|
---|---|