Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
The Cisco UCS Fabric Interconnects provide both network connectivity and management capabilities to the Cisco UCS system.
The fabric interconnect provides Ethernet and Fibre Channel to the servers in the system. The servers connect to the fabric
interconnect, and then to the LAN or SAN.
Each fabric interconnect runs Cisco UCS Manager software to fully manage all Cisco UCS elements. High availability redundancy
can be achieved when a fabric interconnect is connected to another fabric interconnect through the L1 or L2 port on each device.
Cisco UCS 6454 Fabric Interconnect
The Cisco UCS 6454 Fabric Interconnect (FI) is a 1-RU top-of-rack switch that mounts in a standard 19-inch rack such as the Cisco R Series rack.
The Cisco UCS 6454 Fabric Interconnect has 48 10/25 Gb SFP28 ports (16 unified ports) and 6 40/100 Gb QSFP28 ports. Each 40/100 Gb port can break out into 4 x 10/25
Gb uplink ports. The sixteen unified ports support 10/25 GbE or 8/16/32G Fibre Channel speeds.
Note
The Cisco UCS 6454 Fabric Interconnect supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager 4.0(1) and 4.0(2), but with release 4.0(4) and later it supports
16 unified ports (ports 1 - 16).
The Cisco UCS 6454 Fabric Interconnect supports:
Maximum of 8 FCoE port channels
Or 4 SAN port channels
Or a maximum of 8 SAN port channels and FCoE port channels (4 each)
The Cisco UCS 6454 Fabric Interconnect also has one network management port, one console port for setting the initial configuration, and one USB port for saving
or loading configurations. The FI also includes L1/L2 ports for connecting two fabric interconnects for high availability.
The Cisco UCS 6454 Fabric Interconnect also contains a CPU board that consists of:
Ports 1-16 (Unified Ports 10/25 Gbps Ethernet or FCoE or 8/16/32 Gbps Fibre Channel)
Note
When using Cisco UCS Manager releases earlier than 4.0(4), only ports 1-8 are Unified Ports.
2
Ports 17-44 (10/25 Gbps Ethernet or FCoE)
Note
When using Cisco UCS Manager releases earlier than 4.0(4), ports 9-44 are 10/25 Gbps Ethernet or FCoE.
3
Ports 45-48 (1/10/25 Gbps Ethernet or FCoE)
4
Uplink Ports 49-54 (40/100 Gbps Ethernet or FCoE)
Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using an appropriate breakout cable.
The Cisco UCS 6454 Fabric Interconnect chassis has two power supplies and four fans. Two of the fans provide front to rear airflow.
Figure 2. Cisco UCS 6454 Fabric Interconnect Front View
1
Power supply and power cord connector
2
Fans 1 through 4, numbered left to right, when facing the front of the chassis.
3
L1 port, L2 port, RJ45, console, USB port, and LEDs
Cisco UCS 64108 Fabric Interconnect
The Cisco UCS 64108 Fabric Interconnect (FI) is a 2-RU top-of-rack switch that mounts in a standard 19-inch rack such as the
Cisco R Series rack. This high-density FI is an ideal upgrade from the high-density Cisco UCS 6296 Fabric Interconnect.
The high-density Cisco UCS 64108 Fabric Interconnect has 96 10/25 Gb SFP28 ports and 12 40/100 Gb QSFP28 ports. Each 40/100
Gb port can break out into 4 x 10/25 Gb uplink ports. Sixteen ports (1-16) are unified ports that support 10/25 GbE or 8/16/32G
Fibre Channel speeds.The Cisco UCS 64108 Fabric Interconnect supports:
Maximum of 8 FCoE port channels
Or 4 SAN Ports
Or a maximum of 8 SAN port channels and FCoE port channels (4 each)
The Cisco UCS 64108 Fabric Interconnect also has one network management port, one RS-232 serial console port for setting the
initial configuration, and one USB port for saving or loading configurations. The FI also includes L1/L2 ports for connecting
two fabric interconnects in a high-availability configuration.
The Cisco UCS 64108 Fabric Interconnect also contains a CPU board that consists of:
Uplink Ports 97-108 (40/100 Gbps Ethernet or FCoE)
Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using a breakout cable.
5
System environment (fan fault) LED
6
System status LED
7
Beacon LED
The Cisco UCS 64108 Fabric Interconnect has two power supplies (redundant as 1+1) and three fans (redundant as 2+1).
Figure 4. Cisco UCS 64108 Fabric Interconnect Front View
1
Fan modules
(3) with slots from 1 (left) to 3 (right)
2
Management Port (1 SFP optical port)
3
USB port (1)
4
Management Port (1 RJ-45 copper port)
5
Console port (1)
6
Grounding pad for two-hole grounding lug (under protective label)
7
Power supply modules (1 or 2) (AC power supplies shown) with slots numbered 1 (top) and 2 (bottom)
Ports on the Cisco UCS Fabric Interconnects
The ports on the fabric interconnects can be configured to carry either Ethernet or Fibre Channel traffic. You can configure
only ports 1-16 to carry Fibre Channel traffic. The ports cannot be used by a Cisco UCS domain until you configure them.
Note
When you configure a port on a Fabric Interconnect, the administrative state is automatically set to enabled. If the port
is connected to another device, this may cause traffic disruption. The port can be disabled and enabled after it has been
configured.
The following table summarizes the Cisco UCS Fabric Interconnects.
Cisco UCS 6454 FI
Description
54-Port Fabric Interconnect
Form factor
1-RU
Number of fixed 10 GB Interfaces
48 10/25G interfaces
Number of Unified Ports
16
This FI supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager 4.0(1) and 4.0(2), but with Release 4.0(4) and later
it supports 16 unified ports (ports 1 - 16).
Unified Port Range
Ports 1-16
Unified Port Speeds
10/25 Gbps or 8/16/32-Gbps FC
Number of 40-Gbps ports
6 40/100 Gigabit ports
Compatibility with the IOM
UCS 2204, UCS 2208, UCS 2408
Compatibility with the FEX
Cisco Nexus 2232PP
Cisco Nexus 2232TM-E
Expansion Slots
None
Fan Modules
4
Power Supplies
2 (AC/DC/HVDC available)
Port Speeds and Types
Ports on the fabric interconnects are numbered and grouped according to their function. The ports are numbered top to bottom
and left to right. The following figures show the port numbering and define port speeds and the types of ports that can be
configured. For more information on how to configure the port modes, refer to "Configuring Port Modes for a Fabric Interconnect" in the Cisco UCS Network Management Guide, Release 4.0.
Figure 5. Rear View of Cisco UCS 6454 FI, Port Numbers
1
Ports 1–16.
Unified Ports can operate as 10/25 Gbps Ethernet or FCoE; or 8/16/32 Gbps Fibre Channel.
Port type in 8G/16G/32G FC mode: FC uplink port
Port types in 10G/25G mode:
FCoE uplink port
Server port
Appliance port (the FI must be in Ethernet-End-Host mode)
Monitor port
Note
When using Cisco UCS Manager releases earlier than 4.0(4), only ports 1-8 are Unified Ports.
2
Ports 17–44.
Each port can operate as 10G/25G Ethernet.
Port types in 10G/25G mode:
FCoE uplink port
Server port
Appliance port (the FI must be in Ethernet-End-Host mode)
Monitor port
Note
When using Cisco UCS Manager releases earlier than 4.0(4), ports 9-44 are 10/25 Gbps Ethernet or FCoE. (Only ports 1-8 were
Unified Ports in earlier releases.)
3
Ports 45–48.
Each port can operate as 1G/10G/25G Ethernet or FCoE port.
4
Uplink Ports 49–54.
Each port can operate as 40G/100G Ethernet or FCoE. With a breakout cable, each of these ports can operate as 4 x 10G or 4
x 25G Ethernet or FCoE ports.
Port types:
Uplink port
FCoE uplink port
Monitor port
Power Supplies
The fabric interconnect has two power supplies that are accessible from the front of the chassis. The two power supplies support
1+1 redundancy.
Table 1. Power Supply Models
Cisco PID
Fabric Interconnect
Source
Wattage
UCS-PSU-6332-AC
(The Cisco UCS 6454 FI shares the same AC power supply and ordering PID with the Cisco UCS 6300 Series.)
Cisco UCS 6454
110 to 240 VAC
650 W AC
UCS-PSU-64108-AC
Cisco UCS 64108
90 to 264 VAC
1200 W AC
UCS-PSU-6332-DC
(The Cisco UCS 6400 Series shares the same DC power supply and ordering PID with the Cisco UCS 6300 Series.)
Cisco UCS 6454
Cisco UCS 64108
-48 VDC
930 W DC
Figure 6. Power Supply LEDs (650 W AC Shown)
Power supplies have two LEDs: one for power status and one for a failure condition.
1
Fault or Error LED
2
Power LED
LED
State
Description
Power LED
Solid green
Power supply is on and functioning properly
Blinking green
3.3 V voltage standby (VSB) is on but the power supply is not powering up the Fabric Interconnect
Off
There is no AC power to the power supply
Fault/error LED
Solid amber
Power supply failure that indicates an over voltage, over current, or over temperature
Blinking amber
AC power is present, 3.3 VSB is on, and the power supply is off
Off
Normal operation
Fan Modules
The Cisco UCS 6454 FI and the Cisco UCS 64108 FI use different fan modules.
Cisco UCS 6454 FI: four fan modules. The fans support 3+1 redundancy. All fans are hot swappable but only one fan can be removed at a time.
Cisco UCS 64108 FI: three fan modules. The fans support 2+1 redundancy.
All fans are hot swappable but only one fan can be removed at a time.
Fabric Interconnect
Fan
Cisco UCS 6454
UCS-FAN-6332
(The Cisco UCS 6332 FI and Cisco UCS 6454 FI use the same fan modules and ordering PID.)
Cisco UCS 64108
UCS-FAN-64108
Front Panel Ports and LEDs
The FIs in the Cisco UCS 6400 Series have slightly different front panel port and LED placement,
but the icons and functions are the same. See the following figures for placement on
your FI. Also see the LED state-definition table below.
Note
When connecting L1 and L2 ports between FIs, the maximum supported length of Ethernet CAT5e or CAT6 cable is 100 meters.
Figure 7. Cisco UCS 6454 FI Front Panel Ports and LEDs
1
L1 high availability port
2
L2 high availability port
3
RJ45 Network Management Port
4
RS-232 serial console port (RJ-45 connector)
5
USB port
The USB port can be used for booting or downloading scripts.
6
Beacon LED
7
System Status LED
-
-
___
Figure 8. Cisco UCS 64108 FI Front Panel Ports and LEDs
1
Beacon LED
2
System Status LED
3
L1 and L2 high availability ports
4
RS-232 serial console port (RJ-45 connector)
5
Network Management Port (RJ-45)
6
Grounding pad for two-hole grounding lug (under protective label)
7
USB port
The USB port can be used for booting or downloading scripts.
-
-
The definition of states of the beacon and system status LEDs are as follows:
LED
Location
Function
Color
State
Description
Beacon LED
Front and rear
Indicate selected chassis
Blue
Solid On
Chassis selected
Off
Chassis not selected
System Status LED
Front and rear
Indicate system power/health at bootup and run time
Green
Solid on
Normal operation
Off
System powered off
Amber
On
System fault
Red
Solid on
Power shut down by software
Blinking
Secure boot validation has failed
Network Management Port LEDs
The states of the LEDs of the management port on the front panel are listed below.
LED Position
LED State
Description
Left
Off
No link
Solid green
Physical link
Right
Off
No activity
Blinking green
Activity
L1 and L2 Port LEDs
The states of the LEDs on the L1 and L2 front-panel ports are listed below.
LED Position
LED State
Description
Left
Off
No link
Left
Solid green
Physical link
Right
Off
No activity
Right
Blinking green
Activity
Rear Panel System Environment LED
The states of the LEDs on the system environment LED on the rear panel are listed below. The system environment LED indicates
a fan fault or alarm.
LED State
Description
Solid amber
Minor fan alarm (one fan is missing or there is a failure).
Solid red
Major fan alarm (two or more fans are missing or have failed, or there is a fan direction mismatch).
Rear Panel Port LEDs
The states of the LEDs on the rear panel ports are listed below.
Feedback