Maintaining Cisco CSP 5400

This chapter contains the following sections:

Status LEDs and Buttons

This section contains information for interpreting LED states.

Front-Panel LEDs

Figure 1. Front Panel LEDs
Table 1. Front Panel LEDs, Definition of States

LED Name

States

1

SAS

SAS/SATA drive fault

  • Off—The hard drive is operating properly.

  • Amber—Drive fault detected.

  • Amber, blinking—The device is rebuilding.

  • Amber, blinking with one-second interval—Drive locate function activated in the software.

2

SAS

SAS/SATA drive activity LED

  • Off—There is no hard drive in the hard drive tray (no access, no fault).

  • Green—The hard drive is ready.

  • Green, blinking—The hard drive is reading or writing data.

3

Power button/LED

  • Off—There is no AC power to the Cisco CSP 5400.

  • Amber—The Cisco CSP 5400 is in standby power mode. Power is supplied only to the Cisco IMC and some motherboard functions.

  • Green—The Cisco CSP 5400 is in main power mode. Power is supplied to all Cisco CSP 5400 components.

4

Unit identification

  • Off—The unit identification function is not in use.

  • Blue, blinking—The unit identification function is activated.

5

System health

  • Green—The Cisco CSP 5400 is running in normal operating condition.

  • Green, blinking—The Cisco CSP 5400 is performing system initialization and memory check.

  • Amber, steady—The Cisco CSP 5400 is in a degraded operational state (minor fault). For example:

    • Power supply redundancy is lost.

    • CPUs are mismatched.

    • At least one CPU is faulty.

    • At least one DIMM is faulty.

    • At least one drive in a RAID configuration failed.

  • Amber, 2 blinks—There is a major fault with the system board.

  • Amber, 3 blinks—There is a major fault with the memory DIMMs.

  • Amber, 4 blinks—There is a major fault with the CPUs.

6

Power supply status

  • Green—All power supplies are operating normally.

  • Amber, steady—One or more power supplies are in a degraded operational state.

  • Amber, blinking—One or more power supplies are in a critical fault state.

7

Fan status

  • Green—All fan modules are operating properly.

  • Amber, blinking—One or more fan modules breached the non-recoverable threshold.

8

Network link activity

  • Off—The Ethernet LOM port link is idle.

  • Green—One or more Ethernet LOM ports are link-active, but there is no activity.

  • Green, blinking—One or more Ethernet LOM ports are link-active, with activity.

9

Temperature status

  • Green—The Cisco CSP 5400 is operating at normal temperature.

  • Amber, steady—One or more temperature sensors breached the critical threshold.

  • Amber, blinking—One or more temperature sensors breached the non-recoverable threshold.

10

DVD drive activity

  • Off—The drive is idle.

  • Green, steady—The drive is spinning up a disk.

  • Green, blinking—The drive is accessing data.

Rear-Panel LEDs

Figure 2. Rear Panel LEDs
Table 2. Rear Panel LEDs, Definition of States

LED Name

States

1

1-Gb/10-Gb Ethernet link speed (on both LAN1 and LAN2)

  • Off—Link speed is 100 Mbps.

  • Amber—Link speed is 1 Gbps.

  • Green—Link speed is 10 Gbps.

2

1-Gb/10-Gb Ethernet link status (on both LAN1 and LAN2)

  • Off—No link is present.

  • Green—Link is active.

  • Green, blinking—Traffic is present on the active link.

3

1-Gb Ethernet dedicated management link speed

  • Off—Link speed is 10 Mbps.

  • Amber—Link speed is 100 Mbps.

  • Green—Link speed is 1 Gbps.

4

1-Gb Ethernet dedicated management link status

  • Off—No link is present.

  • Green—Link is active.

  • Green, blinking—Traffic is present on the active link.

5

Rear unit identification

  • Off—The unit identification function is not in use.

  • Blue, blinking—The unit identification function is activated.

6

Power supply status (one LED each power supply unit)

AC power supplies:

  • Off—No AC input (12 V main power off, 12 V standby power off).

  • Green, blinking—12 V main power off; 12 V standby power on.

  • Green, solid—12 V main power on; 12 V standby power on.

  • Amber, blinking—Warning threshold detected but 12 V main power on.

  • Amber, solid—Critical error detected; 12 V main power off (for example, over-current, over-voltage, or over-temperature failure).

DC power supply:

  • Off—No DC input (12 V main power off, 12 V standby power off).

  • Green, blinking—12 V main power off; 12 V standby power on.

  • Green, solid—12 V main power on; 12 V standby power on.

  • Amber, blinking—Warning threshold detected but 12 V main power on.

  • Amber, solid—Critical error detected; 12 V main power off (for example, over-current, over-voltage, or over-temperature failure).

7

SAS

SAS/SATA drive fault

  • Off—The hard drive is operating properly.

  • Amber—Drive fault detected.

  • Amber, blinking—The device is rebuilding.

  • Amber, blinking with one-second interval—Drive locate function activated in the software.

8

SAS

SAS/SATA drive activity LED

  • Off—There is no hard drive in the hard drive tray (no access, no fault).

  • Green—The hard drive is ready.

  • Green, blinking—The hard drive is reading or writing data.

Internal Diagnostic LEDs

The Cisco CSP 5400 has internal fault LEDs for CPUs, DIMMs, and fan modules.

Figure 3. Internal Diagnostic LED Locations

1

Fan module fault LEDs (one on the top of each fan module)

  • Amber—Fan has a fault or is not fully seated.

  • Green—Fan is OK.

3

DIMM fault LEDs (one behind each DIMM socket on the motherboard)

These LEDs operate only when the Cisco CSP 5400 is in standby power mode.

  • Amber—DIMM has a fault.

  • Off—DIMM is OK.

2

CPU fault LEDs (one behind each CPU socket on the motherboard).

These LEDs operate only when the Cisco CSP 5400 is in standby power mode.

  • Amber—CPU has a fault.

  • Off—CPU is OK.

-

Preparing For Component Installation

This section includes information and tasks that help prepare the Cisco CSP 5400 for component installation.

Required Equipment For Service Procedures

The following tools and equipment are used to perform the procedures in this chapter:

  • T-30 Torx driver (supplied with replacement CPUs for heatsink removal)

  • #1 flat-head screwdriver (used during CPU or heatsink replacement)

  • #1 Phillips-head screwdriver (for M.2 SSD and intrusion switch replacement)

  • Electrostatic discharge (ESD) strap or other grounding equipment such as a grounded mat

Shutting Down and Removing Power From the Cisco CSP 5400

The Cisco CSP 5400 can run in either of two power modes:

  • Main power mode—Power is supplied to all Cisco CSP 5400 components and any operating system on your drives can run.

  • Standby power mode—Power is supplied only to the service processor and certain components. It is safe for the operating system and data to remove power cords from the Cisco CSP 5400 in this mode.


Caution

After a Cisco CSP 5400 is shut down to standby power, electric current is still present in the Cisco CSP 5400. To completely remove power, you must disconnect all power cords from the power supplies in the Cisco CSP 5400, as directed in the service procedures.

You can shut down the Cisco CSP 5400 by using the front-panel power button or the software management interfaces.

Shutting Down Using the Power Button

Procedure
Step 1

Check the color of the Power button/LED:

  • Amber—The Cisco CSP 5400 is already in standby mode and you can safely remove power.

  • Green—The Cisco CSP 5400 is in main power mode and must be shut down before you can safely remove power.
Step 2

Invoke either a graceful shutdown or a hard shutdown:

Caution 
To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system.

  • Graceful shutdown—Press and release the Power button. The operating system performs a graceful shutdown and the Cisco CSP 5400 goes to standby mode, which is indicated by an amber Power button/LED.

  • Emergency shutdown—Press and hold the Power button for 4 seconds to force the main power off and immediately enter standby mode.

Step 3

If a service procedure instructs you to completely remove power from the Cisco CSP 5400, disconnect all power cords from the power supplies in the Cisco CSP 5400.

Shutting Down Using The Cisco IMC GUI

You must log in with user or admin privileges to perform this task.

Procedure
Step 1

In the Navigation pane, click the Cisco CSP 5400 tab.

Step 2

On the Cisco CSP 5400 tab, click Summary.

Step 3

In the Actions area, click Power Off Cisco CSP 5400.

Step 4

Click OK.

The operating system performs a graceful shutdown and the Cisco CSP 5400 goes to standby mode, which is indicated by an amber Power button/LED.

Step 5

If a service procedure instructs you to completely remove power from the Cisco CSP 5400, disconnect all power cords from the power supplies in the Cisco CSP 5400.

Shutting Down Using The Cisco IMC CLI

You must log in with user or admin privileges to perform this task.

Procedure
Step 1

At the Cisco CSP 5400 prompt, enter:

Example:
Cisco CSP 5400# scope chassis
Step 2

At the chassis prompt, enter:

Example:
Cisco CSP 5400/chassis# power shutdown

The operating system performs a graceful shutdown and the Cisco CSP 5400 goes to standby mode, which is indicated by an amber Power button/LED.

Step 3

If a service procedure instructs you to completely remove power from the Cisco CSP 5400, disconnect all power cords from the power supplies in the Cisco CSP 5400.

Shutting Down Using The Cisco UCS Manager Equipment Tab

You must log in with user or admin privileges to perform this task.

Procedure
Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Rack Mounts > Cisco CSP 5400s.

Step 3

Choose the Cisco CSP 5400 that you want to shut down.
Step 4

In the Work pane, click the General tab.

Step 5

In the Actions area, click Shutdown Cisco CSP 5400.

Step 6

If a confirmation dialog displays, click Yes.

The operating system performs a graceful shutdown and the Cisco CSP 5400 goes to standby mode, which is indicated by an amber Power button/LED.

Step 7

If a service procedure instructs you to completely remove power from the Cisco CSP 5400, disconnect all power cords from the power supplies in the Cisco CSP 5400.

Shutting Down Using The Cisco UCS Manager Service Profile

You must log in with user or admin privileges to perform this task.

Procedure
Step 1

In the Navigation pane, click Cisco CSP 5400s.

Step 2

Expand Cisco CSP 5400s > Service Profiles.

Step 3

Expand the node for the organization that contains the service profile of the Cisco CSP 5400 that you are shutting down.
Step 4

Choose the service profile of the Cisco CSP 5400 that you are shutting down.
Step 5

In the Work pane, click the General tab.

Step 6

In the Actions area, click Shutdown Cisco CSP 5400.

Step 7

If a confirmation dialog displays, click Yes.

The operating system performs a graceful shutdown and the Cisco CSP 5400 goes to standby mode, which is indicated by an amber Power button/LED.

Step 8

If a service procedure instructs you to completely remove power from the Cisco CSP 5400, disconnect all power cords from the power supplies in the Cisco CSP 5400.

Removing the Cisco CSP 5400 Top Cover

Procedure

Step 1

Remove the top cover:

  1. If the cover latch is locked, use a screwdriver to turn the lock 90-degrees counterclockwise to unlock it.

  2. Lift on the end of the latch that has the green finger grip. The cover is pushed back to the open position as you lift the latch.

  3. Lift the top cover straight up from the Cisco CSP 5400 and set it aside.
Step 2

Replace the top cover:

  1. With the latch in the fully open position, place the cover on top of the Cisco CSP 5400 about one-half inch (1.27 cm) behind the lip of the front cover panel. The opening in the latch should fit over the peg that sticks up from the fan tray.

  2. Press the cover latch down to the closed position. The cover is pushed forward to the closed position as you push down the latch.

  3. If desired, lock the latch by using a screwdriver to turn the lock 90-degrees clockwise.

Figure 4. Removing the Top Cover

1

Cover latch

3

Serial number label location

2

Cover lock

Hot Swap

Some components can be removed and replaced without shutting down and removing power from the Cisco CSP 5400.

  • Hot-swap replacement—You do not have to shut down the component in the software or operating system. This applies to the following components:

    • SAS/SATA hard drives

    • SAS/SATA solid state drives

    • Cooling fan modules

    • Power supplies (when redundant as 1+1)

Removing and Replacing Components


Warning

Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place.

Statement 1029

Caution
When handling Cisco CSP 5400 components, handle them only by carrier edges and use an electrostatic discharge (ESD) wrist-strap or other grounding device to avoid damage.

Tip
You can press the unit identification button on the front panel or rear panel to turn on a flashing, blue unit identification LED on both the front and rear panels of the Cisco CSP 5400. This button allows you to locate the specific Cisco CSP 5400 that you are servicing when you go to the opposite side of the rack. You can also activate these LEDs remotely by using the Cisco IMC interface.

This section describes how to install and replace Cisco CSP 5400 components.

Serviceable Component Locations

This topic shows the locations of the field-replaceable components and service-related items. The view in the following figure shows the Cisco CSP 5400 with the top cover removed.

Figure 5. Cisco CSP 5400, Serviceable Component Locations

1

Front-loading drive bays.

11

Power supplies (hot-swappable when redundant as 1+1)

2

Cooling fan modules (six, hot-swappable)

12

 Not applicable

3

DIMM sockets on motherboard (up to 12 per CPU)

Not visible under air baffle in this view.

See the "DIMM Population Rules and Memory Performance Guidelines" topic for DIMM slot numbering.

13

Trusted platform module (TPM) socket on motherboard (not visible in this view)

4

CPUs and heatsinks (up to two)

Not visible under air baffle in this view.

14

PCIe riser 2 (PCIe slots 4, 5, 6), with the following option:

  • 2B—Slots 4 (x8), 5 (x16), and 6 (x8); includes cable connector for rear-loading NVMe SSDs.

5

Supercap unit (RAID backup) mounting bracket

15

Micro-SD card socket on PCIe riser 1

6

Internal, vertical USB 3.0 port on motherboard

16

PCIe riser 1 (PCIe slot 1, 2, 3), with the following option:

  • 1A—Slots 1 (x8), 2 (x16), 3 (x8); slot 2 requires CPU2.

7

Mini storage module socket

Supports an SD card carrier with two SD card slots.

17

Modular LOM (mLOM) card bay on chassis floor (x16 PCIe lane), not visible in this view

8

Chassis intrusion switch (optional)

18

Cisco modular RAID controller PCIe slot (dedicated slot)

9

 Not applicable

19

RTC battery, vertical socket

10

Rear-drive backplane assembly

20

Securing clips for GPU cards on air baffle

For all versions of Cisco CSP 5400, which include supported component part numbers, are at Cisco UCS Cisco CSP 5400s Technical Specifications Sheets (scroll down to Technical Specifications).

Replacing Front-Loading SAS/SATA Drives


Note
You do not have to shut down the Cisco CSP 5400 or drive to replace SAS/SATA hard drives or SSDs because they are hot-swappable.

To replace rear-loading SAS/SATA drives, see Replacing a Front-Loading SAS/SATA Drive.

Front-Loading SAS/SATA Drive Population Guidelines

The following version of the server is orderable:

Small form-factor (SFF) drives, with 24-drive backplane.

  • Front-loading drive bays 1—24 support 2.5-inch SAS/SATA drives.

Drive bay numbering is shown in the following figures.

Figure 6. Small Form-Factor Drive (24-Drive) Versions, Drive Bay Numbering
Figure 7. Small Form-Factor Drive (8-Drive) Version, Drive Bay Numbering

Observe these drive population guidelines for optimum performance:

  • When populating drives, add drives to the lowest-numbered bays first.


    Note
    For diagrams of which drive bays are controlled by particular controller cables on the backplane, see Storage Controller Cable Connectors and Backplanes.

  • Keep an empty drive blanking tray in any unused bays to ensure proper airflow.

Replacing a Front-Loading SAS/SATA Drive


Note
You do not have to shut down the Cisco CSP 5400 or drive to replace SAS/SATA hard drives or SSDs because they are hot-swappable.
Procedure
Step 1

Remove the drive that you are replacing or remove a blank drive tray from the bay:

  1. Press the release button on the face of the drive tray.

  2. Grasp and open the ejector lever and then pull the drive tray out of the slot.

  3. If you are replacing an existing drive, remove the four drive-tray screws that secure the drive to the tray and then lift the drive out of the tray.

Step 2

Install a new drive:

  1. Place a new drive in the empty drive tray and install the four drive-tray screws.

  2. With the ejector lever on the drive tray open, insert the drive tray into the empty drive bay.

  3. Push the tray into the slot until it touches the backplane, and then close the ejector lever to lock the drive in place.

Figure 8. Replacing a Drive in a Drive Tray

1

Ejector lever

3

Drive tray screws (two on each side)

2

Release button

4

Drive removed from drive tray

Replacing Fan Modules

The six fan modules in the Cisco CSP 5400 are numbered as shown in Serviceable Component Locations.


Tip
There is a fault LED on the top of each fan module. This LED lights green when the fan is correctly seated and is operating OK. The LED lights amber when the fan has a fault or is not correctly seated.

Caution
You do not have to shut down or remove power from the Cisco CSP 5400 to replace fan modules because they are hot- swappable. However, to maintain proper cooling, do not operate the Cisco CSP 5400 for more than one minute with any fan module removed.

Procedure

Step 1

Remove an existing fan module:

  1. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  2. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  3. Grasp and squeeze the fan module release latches on its top. Lift straight up to disengage its connector from the motherboard.
Step 2

Install a new fan module:

  1. Set the new fan module in place. The arrow printed on the top of the fan module should point toward the rear of the Cisco CSP 5400.

  2. Press down gently on the fan module to fully engage it with the connector on the motherboard.

  3. Replace the top cover to the Cisco CSP 5400.

  4. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 9. Top View of Fan Module

1

Fan module release latches

2

Fan module fault LED

Replacing Memory DIMMs


Caution

DIMMs and their sockets are fragile and must be handled with care to avoid damage during installation.

Caution

Cisco does not support third-party DIMMs. Using non-Cisco DIMMs in the Cisco CSP 5400 might result in system problems or damage to the motherboard.


Note

To ensure the best Cisco CSP 5400 performance, it is important that you are familiar with memory performance guidelines and population rules before you install or replace DIMMs.

DIMM Population Rules and Memory Performance Guidelines

DIMM Slot Numbering

The following figure shows the numbering of the DIMM slots on the motherboard.

Figure 10. DIMM Slot Numbering
DIMM Population Rules

Observe the following guidelines when installing or replacing DIMMs for maximum performance:

  • Each CPU supports six memory channels.

    • CPU 1 supports channels A, B, C, D, E, F.

    • CPU 2 supports channels G, H, J, K, L, M.

  • Each channel has two DIMM sockets (for example, channel A = slots A1, A2).

  • In a single-CPU configuration, populate the channels for CPU1 only (A, B, C, D, E, F).

  • For optimal performance, populate DIMMs in the order shown in the following table, depending on the number of CPUs and the number of DIMMs per CPU. If your server has two CPUs, balance DIMMs evenly across the two CPUs as shown in the table.


    Note

    The table below lists recommended configurations. Using 5, 7, 9, 10, or 11 DIMMs per CPU is not recommended.
    Table 3. DIMM Population Order

    Number of DIMMs per CPU (Recommended Configurations)

    Populate CPU 1 Slot

    Populate CPU2 Slots

    Blue #1 Slots

    Black #2 Slots

    Blue #1 Slots

    Black #2 Slots

    1

    (A1)

    -

    (G1)

    -

    2

    (A1, B1)

    -

    (G1, H1)

    -

    3

    (A1, B1, C1)

    -

    (G1, H1, J1)

    -

    4

    (A1, B1); (D1, E1)

    -

    (G1, H1); (K1, L1)

    -

    6

    (A1, B1); (C1, D1); (E1, F1)

    -

    (G1, H1); (J1, K1); (L1, M1)

    -

    8

    (A1, B1); (D1, E1)

    (A2, B2); (D2, E2)

    (G1, H1); (K1, L1)

    (G2, H2); (K2, L2)

    12

    (A1, B1); (C1, D1); (E1, F1)

    (A2, B2); (C2, D2); (E2, F2)

    (G1, H1); (J1, K1); (L1, M1)

    (G2, H2); (J2, K2); (L2, M2)

Replacing DIMMs

Identifying a Faulty DIMM

Each DIMM socket has a corresponding DIMM fault LED, directly in front of the DIMM socket. See Internal Diagnostic LEDs for the locations of these LEDs. When the Cisco CSP 5400 is in standby power mode, these LEDs light amber to indicate a faulty DIMM.

Procedure
Step 1

Remove an existing DIMM:

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  4. Remove the air baffle that covers the front ends of the DIMM slots to provide clearance.

  5. Locate the DIMM that you are removing, and then open the ejector levers at each end of its DIMM slot.
Step 2

Install a new DIMM:

Note 

Before installing DIMMs, see the memory population rules for this Cisco CSP 5400: See the "DIMM Population Rules and Memory Performance Guidelines" topic.

  1. Align the new DIMM with the empty slot on the motherboard. Use the alignment feature in the DIMM slot to correctly orient the DIMM.

  2. Push down evenly on the top corners of the DIMM until it is fully seated and the ejector levers on both ends lock into place.

  3. Replace the top cover to the Cisco CSP 5400.

  4. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Replacing CPUs and Heatsinks

This section contains the following topics:

CPU Configuration Rules

This Cisco CSP 5400 has two CPU sockets on the motherboard. Each CPU supports six DIMM channels (12 DIMM slots). See the "DIMM Population Rules and Memory Performance Guidelines" topic.

  • The Cisco CSP 5400 can operate with two identical CPUs installed.

  • Install CPU 1 first, and then CPU 2.

Tools Required For CPU Replacement

You need the following tools and equipment for this procedure:

  • T-30 Torx driver—Supplied with replacement CPU.

  • #1 flat-head screwdriver—Supplied with replacement CPU.

  • CPU assembly tool—Supplied with replacement CPU. Orderable separately as Cisco PID UCS-CPUAT=.

  • Heatsink cleaning kit—Supplied with replacement CPU. Orderable separately as Cisco PID UCSX-HSCK=.

  • Thermal interface material (TIM)—Syringe supplied with replacement CPU. Use only if you are reusing your existing heatsink (new heatsinks have a pre-applied pad of TIM). Orderable separately as Cisco PID UCS-CPU-TIM=.

See also Additional CPU-Related Parts to Order with RMA Replacement CPUs.

Replacing a CPU and Heatsink


Caution

CPUs and their sockets are fragile and must be handled with extreme care to avoid damaging pins. The CPUs must be installed with heatsinks and thermal interface material to ensure cooling. Failure to install a CPU correctly might result in damage to the Cisco CSP 5400.

Procedure
Step 1

Remove the existing CPU/heatsink assembly from the Cisco CSP 5400:

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  4. Use the T-30 Torx driver that is supplied with the replacement CPU to loosen the four captive nuts that secure the assembly to the motherboard standoffs.

    Note 

    Alternate loosening the heatsink nuts evenly so that the heatsink remains level as it is raised. Loosen the heatsink nuts in the order shown on the heatsink label: 4, 3, 2, 1.

  5. Lift straight up on the CPU/heatsink assembly and set it heatsink-down on an antistatic surface.

    Figure 11. Removing the CPU/Heatsink Assembly

    1

    Heatsink

    4

    CPU socket on motherboard

    2

    Heatsink captive nuts (two on each side)

    5

    T-30 Torx driver

    3

    CPU carrier (below heatsink in this view)

    -
Step 2

Separate the heatsink from the CPU assembly (the CPU assembly includes the CPU and the plastic CPU carrier):

  1. Place the heatsink with CPU assembly so that it is oriented upside-down as shown below.

    Note the thermal-interface material (TIM) breaker location. TIM BREAKER is stamped on the CPU carrier next to a small slot.

    Figure 12. Separating the CPU Assembly From the Heatsink

    1

    CPU carrier

    4

    CPU-carrier inner-latch nearest to the TIM breaker slot

    2

    CPU

    5

    #1 flat-head screwdriver inserted into TIM breaker slot

    3

    TIM BREAKER slot in CPU carrier

    -

  2. Pinch inward on the CPU-carrier inner-latch that is nearest the TIM breaker slot and then push up to disengage the clip from its slot in the heatsink corner.

  3. Insert the blade of a #1 flat-head screwdriver into the slot marked TIM BREAKER.

    Caution 

    In the following step, do not pry on the CPU surface. Use gentle rotation to lift on the plastic surface of the CPU carrier at the TIM breaker slot. Use caution to avoid damaging the heatsink surface.

  4. Gently rotate the screwdriver to lift up on the CPU until the TIM on the heatsink separates from the CPU.

    Note 

    Do not allow the screwdriver tip to touch or damage the green CPU substrate.

  5. Pinch the CPU-carrier inner-latch at the corner opposite the TIM breaker and push up to disengage the clip from its slot in the heatsink corner.

  6. On the remaining two corners of the CPU carrier, gently pry outward on the outer-latches and then lift the CPU-assembly from the heatsink.

    Note 
    Handle the CPU-assembly by the plastic carrier only. Do not touch the CPU surface. Do not separate the CPU from the carrier.
Step 3

The new CPU assembly is shipped on a CPU assembly tool. Take the new CPU assembly and CPU assembly tool out of the carton.

If the CPU assembly and CPU assembly tool become separated, note the alignment features shown below for correct orientation. The pin 1 triangle on the CPU carrier must be aligned with the angled corner on the CPU assembly tool.

Caution 

CPUs and their sockets are fragile and must be handled with extreme care to avoid damaging pins.
Figure 13. CPU Assembly Tool, CPU Assembly, and Heatsink Alignment Features

1

CPU assembly tool

4

Angled corner on heatsink (pin 1 alignment feature)

2

CPU assembly (CPU in plastic carrier)

5

Triangle cut into carrier (pin 1 alignment feature)

3

Heatsink

6

Angled corner on CPU assembly tool (pin 1 alignment feature)
Step 4

Apply new TIM to the heatsink:

Note 
The heatsink must have new TIM on the heatsink-to-CPU surface to ensure proper cooling and performance.

  • If you are installing a new heatsink, it is shipped with a pre-applied pad of TIM. Go to step 5.

  • If you are reusing a heatsink, you must remove the old TIM from the heatsink and then apply new TIM to the CPU surface from the supplied syringe. Continue with step a below.

  1. Apply the cleaning solution that is included with the heatsink cleaning kit (UCSX-HSCK=) to the old TIM on the heatsink and let it soak for a least 15 seconds.

  2. Wipe all of the TIM off the heatsink using the soft cloth that is included with the heatsink cleaning kit. Be careful to avoid scratching the heatsink surface.

  3. Using the syringe of TIM provided with the new CPU (UCS-CPU-TIM=), apply 4 cubic centimeters of thermal interface material to the top of the CPU. Use the pattern shown below to ensure even coverage.

    Figure 14. Thermal Interface Material Application Pattern
    Caution 

    Use only the correct heatsink for your CPUs to ensure proper cooling. There are two different heatsinks: UCSC-HS-C240M5 for standard-performance CPUs 150 W and less; UCSC-HS2-C240M5 for high-performance CPUs above 150 W. Note the wattage described on the heatsink label.

Step 5

With the CPU assembly on the CPU assembly tool, set the heatsink onto the CPU assembly. Note the pin 1 alignment features for correct orientation. Push down gently until you hear the corner clips of the CPU carrier click onto the heatsink corners.

Caution 

In the following step, use extreme care to avoid touching or damaging the CPU contacts or the CPU socket pins.
Step 6

Install the CPU/heatsink assembly to the Cisco CSP 5400:

  1. Lift the heatsink with attached CPU assembly from the CPU assembly tool.

  2. Align the assembly over the CPU socket on the motherboard.

    Note the alignment features. The pin 1 angled corner on the heatsink must align with the pin 1 angled corner on the CPU socket. The CPU-socket posts must align with the guide-holes in the assembly.

    Figure 15. Installing the Heatsink/CPU Assembly to the CPU Socket

    1

    Guide hole in assembly (two)

    4

    Angled corner on heatsink (pin 1 alignment feature)

    2

    CPU socket alignment post (two)

    5

    Angled corner on socket (pin 1 alignment feature)

    3

    CPU socket leaf spring

    -

  3. Set the heatsink with CPU assembly down onto the CPU socket.

  4. Use the T-30 Torx driver that is supplied with the replacement CPU to tighten the four captive nuts that secure the heatsink to the motherboard standoffs.

    Caution 
    Alternate tightening the heatsink nuts evenly so that the heatsink remains level while it is lowered. Tighten the heatsink nuts in the order shown on the heatsink label: 1, 2, 3, 4. The captive nuts must be fully tightened so that the leaf springs on the CPU socket lie flat.

  5. Replace the top cover to the Cisco CSP 5400.

  6. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Additional CPU-Related Parts to Order with RMA Replacement CPUs

When a return material authorization (RMA) of the CPU is done on a Cisco CSP 5400, additional parts might not be included with the CPU spare. The TAC engineer might need to add the additional parts to the RMA to help ensure a successful replacement.


Note

The following items apply to CPU replacement scenarios. If you are replacing a system chassis and moving existing CPUs to the new chassis, you do not have to separate the heatsink from the CPU. See Additional CPU-Related Parts to Order with RMA Replacement System Chassis.

  • Scenario 1—You are reusing the existing heatsinks:

    • Heat sink cleaning kit (UCSX-HSCK=)

    • Thermal interface material (TIM) kit for M5 Cisco CSP 5400s (UCS-CPU-TIM=)

  • Scenario 2—You are replacing the existing heatsinks:


    Caution
    Use only the correct heatsink for your CPUs to ensure proper cooling. There are two different heatsinks: UCSC-HS-C240M5= for CPUs 150 W and less; UCSC-HS2-C240M5= for CPUs above 150 W.

    • Heat sink: UCSC-HS-C240M5= for CPUs 150 W and less; UCSC-HS2-C240M5= for CPUs above 150 W

    • Heat sink cleaning kit (UCSX-HSCK=)

  • Scenario 3—You have a damaged CPU carrier (the plastic frame around the CPU):

    • CPU Carrier: UCS-M5-CPU-CAR=

    • #1 flat-head screwdriver (for separating the CPU from the heatsink)

    • Heatsink cleaning kit (UCSX-HSCK=)

    • Thermal interface material (TIM) kit for M5 Cisco CSP 5400s (UCS-CPU-TIM=)

A CPU heat sink cleaning kit is good for up to four CPU and heat sink cleanings. The cleaning kit contains two bottles of solution, one to clean the CPU and heat sink of old TIM and the other to prepare the surface of the heat sink.

New heat sink spares come with a pre-applied pad of TIM. It is important to clean any old TIM off of the CPU surface prior to installing the heat sinks. Therefore, even when you are ordering new heat sinks, you must order the heat sink cleaning kit.

Additional CPU-Related Parts to Order with RMA Replacement System Chassis

When a return material authorization (RMA) of the system chassis is done on a Cisco UCS C-Series Cisco CSP 5400, you move existing CPUs to the new chassis.


Note

Unlike previous generation CPUs, the M5 Cisco CSP 5400 CPUs do not require you to separate the heatsink from the CPU when you move the CPU-heatsink assembly. Therefore, no additional heatsink cleaning kit or thermal-interface material items are required.

  • The only tool required for moving a CPU/heatsink assembly is a T-30 Torx driver.

To move a CPU to a new chassis, use the procedure in Moving an M5 Generation CPU.

Moving an M5 Generation CPU

Tool required for this procedure: T-30 Torx driver


Caution

When you receive a replacement Cisco CSP 5400 for an RMA, it includes dust covers on all CPU sockets. These covers protect the socket pins from damage during shipping. You must transfer these covers to the system that you are returning, as described in this procedure.

Procedure
Step 1

When moving an M5 CPU to a new Cisco CSP 5400, you do not have to separate the heatsink from the CPU. Perform the following steps:

  1. Use a T-30 Torx driver to loosen the four captive nuts that secure the assembly to the board standoffs.

    Note 
    Alternate loosening the heatsink nuts evenly so that the heatsink remains level as it is raised. Loosen the heatsink nuts in the order shown on the heatsink label: 4, 3, 2, 1.

  2. Lift straight up on the CPU/heatsink assembly to remove it from the board.

  3. Set the CPUs with heatsinks aside on an anti-static surface.

    Figure 16. Removing the CPU/Heatsink Assembly

    1

    Heatsink

    4

    CPU socket on motherboard

    2

    Heatsink captive nuts (two on each side)

    5

    T-30 Torx driver

    3

    CPU carrier (below heatsink in this view)

    -
Step 2

Transfer the CPU socket covers from the new system to the system that you are returning:

  1. Remove the socket covers from the replacement system. Grasp the two recessed finger-grip areas marked "REMOVE" and lift straight up.

    Note 

    Keep a firm grasp on the finger-grip areas at both ends of the cover. Do not make contact with the CPU socket pins.
    Figure 17. Removing a CPU Socket Dust Cover

    1

    Finger-grip areas marked "REMOVE"

    -

  2. With the wording on the dust cover facing up, set it in place over the CPU socket. Make sure that all alignment posts on the socket plate align with the cutouts on the cover.

    Caution 

    In the next step, do not press down anywhere on the cover except the two points described. Pressing elsewhere might damage the socket pins.

  3. Press down on the two circular markings next to the word "INSTALL" that are closest to the two threaded posts (see the following figure). Press until you feel and hear a click.

    Note 

    You must press until you feel and hear a click to ensure that the dust covers do not come loose during shipping.
    Figure 18. Installing a CPU Socket Dust Cover

    -

    Press down on the two circular marks next to the word INSTALL.

    -
Step 3

Install the CPUs to the new system:

  1. On the new board, align the assembly over the CPU socket, as shown below.

    Note the alignment features. The pin 1 angled corner on the heatsink must align with the pin 1 angled corner on the CPU socket. The CPU-socket posts must align with the guide-holes in the assembly.

    Figure 19. Installing the Heatsink/CPU Assembly to the CPU Socket

    1

    Guide hole in assembly (two)

    4

    Angled corner on heatsink (pin 1 alignment feature)

    2

    CPU socket alignment post (two)

    5

    Angled corner on socket (pin 1 alignment feature)

    3

    CPU socket leaf spring

    -

  2. On the new board, set the heatsink with CPU assembly down onto the CPU socket.

  3. Use a T-30 Torx driver to tighten the four captive nuts that secure the heatsink to the board standoffs.

    Note 

    Alternate tightening the heatsink nuts evenly so that the heatsink remains level while it is lowered. Tighten the heatsink nuts in the order shown on the heatsink label: 1, 2, 3, 4. The captive nuts must be fully tightened so that the leaf springs on the CPU socket lie flat.

Replacing a Mini-Storage Module

The mini-storage module plugs into a motherboard socket to provide additional internal storage. The module is available in the following version:

  • SD card carrier—provides two SD card sockets.

Replacing a Mini-Storage Module Carrier

This topic describes how to remove and replace a mini-storage module carrier. The carrier has one media socket on its top and one socket on its underside. Use the following procedure for any type of mini-storage module carrier (SD card or M.2 SSD).

Procedure
Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Remove a carrier from its socket:

  1. Locate the mini-storage module carrier in its socket just in front of power supply 1.

  2. Push outward on the securing clips that holds each end of the carrier.

  3. Lift both ends of the carrier to disengage it from the socket on the motherboard.

  4. Set the carrier on an anti-static surface.
Step 5

Install a carrier to its socket:

  1. Position carrier over socket, with the carrier's connector facing down. Two alignment pegs must match with two holes on the carrier.

  2. Gently push down the socket end of the carrier so that the two pegs go through the two holes on the carrier.

  3. Push down on the carrier so that the securing clips click over it at both ends.
Step 6

Replace the top cover to the Cisco CSP 5400.
Step 7

Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 20. Mini-Storage Module Carrier Socket

1

Location of socket on motherboard

3

Securing clips

2

Alignment pegs

-

Replacing an SD Card in a Mini-Storage Carrier For SD

This topic describes how to remove and replace an SD card in a mini-storage carrier for SD (PID CSP-MSTOR-SD). The carrier has one SD card socket on its top and one socket on its underside.

Population Rules For Mini-Storage SD Cards

  • You can use one or two SD cards in the carrier.

  • Dual SD cards can be configured in a RAID 1 array through the Cisco IMC interface.

  • SD socket 1 is on the top side of the carrier; SD socket 2 is on the underside of the carrier (the same side as the carrier's motherboard connector).

Procedure
Step 1

Power off the Cisco CSP 5400 and then remove the mini-storage module carrier from the Cisco CSP 5400 as described in Replacing a Mini-Storage Module Carrier.

Step 2

Remove an SD card:

  1. Push on the top of the SD card, and then release it to allow it to spring out from the socket.

  2. Grasp and remove the SD card from the socket.
Step 3

Install a new SD card:

  1. Insert the new SD card into the socket with its label side facing up (away from the carrier).

  2. Press on the top of the SD card until it clicks in the socket and stays in place.
Step 4

Install the mini-storage module carrier back into the Cisco CSP 5400 and then power it on as described in Replacing a Mini-Storage Module Carrier.

Replacing an Internal USB Drive

This section includes procedures for installing a USB drive and for enabling or disabling the internal USB port.

Replacing a USB Drive


Caution
We do not recommend that you hot-swap the internal USB drive while the Cisco CSP 5400 is powered on because of the potential for data loss.
Procedure
Step 1

Remove an existing internal USB drive:

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  4. Locate the USB socket on the motherboard, in front of the power supplies.

  5. Grasp the USB drive and pull it vertically to free it from the socket.

Step 2

Install a new internal USB drive:

  1. Align the USB drive with the socket.

  2. Push the USB drive vertically to fully engage it with the socket.

  3. Replace the top cover to the Cisco CSP 5400.

  4. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 21. Location of Internal USB Port <ILLO PENDING>

1

Location of vertical USB socket on motherboard

-

Enabling or Disabling the Internal USB Port

The factory default is that all USB ports on the Cisco CSP 5400 are enabled. However, the internal USB port can be enabled or disabled in the Cisco CSP 5400 BIOS.

Procedure
Step 1

Enter the BIOS Setup Utility by pressing the F2 key when prompted during bootup.

Step 2

Navigate to the Advanced tab.

Step 3

On the Advanced tab, select USB Configuration.

Step 4

On the USB Configuration page, select USB Ports Configuration.

Step 5

Scroll to USB Port: Internal, press Enter, and then choose either Enabled or Disabled from the dialog box.

Step 6

Press F10 to save and exit the utility.

Replacing the RTC Battery


Warning

There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.

[Statement 1015]

The real-time clock (RTC) battery retains system settings when the Cisco CSP 5400 is disconnected from power. The battery type is CR2032. Cisco supports the industry-standard CR2032 battery, which can be ordered from Cisco (PID N20-MBLIBATT) or purchased from most electronic stores.

Procedure

Step 1

Remove the RTC battery:

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  4. Remove PCIe riser 1 from the Cisco CSP 5400 to provide clearance to the RTC battery socket that is on the motherboard. See Replacing a PCIe Riser.

  5. Locate the horizontal RTC battery socket.

  6. Remove the battery from the socket on the motherboard. Gently pry the securing clip to the side to provide clearance, then lift up on the battery.

Step 2

Install a new RTC battery:

  1. Insert the battery into its socket and press down until it clicks in place under the clip.

    Note 

    The positive side of the battery marked “3V+” should face up.

  2. Replace PCIe riser 1 to the Cisco CSP 5400. See Replacing a PCIe Riser.

  3. Replace the top cover to the Cisco CSP 5400.

  4. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 22. RTC Battery Location on Motherboard

1

RTC battery in horizontal socket on motherboard

-

Replacing Power Supplies

The Cisco CSP 5400 can have one or two power supplies. When two power supplies are installed they are redundant as 1+1.

This section includes procedures for replacing AC and DC power supply units.

Replacing AC Power Supplies


Note
If you have ordered a Cisco CSP 5400 with power supply redundancy (two power supplies), you do not have to power off the Cisco CSP 5400 to replace a power supply because they are redundant as 1+1.

Note
Do not mix power supply types or wattages in the Cisco CSP 5400. Both power supplies must be identical.
Procedure
Step 1

Remove the power supply that you are replacing or a blank panel from an empty bay:

  1. Perform one of the following actions:

  2. Remove the power cord from the power supply that you are replacing.

  3. Grasp the power supply handle while pinching the release lever toward the handle.

  4. Pull the power supply out of the bay.
Step 2

Install a new power supply:

  1. Grasp the power supply handle and insert the new power supply into the empty bay.

  2. Push the power supply into the bay until the release lever locks.

  3. Connect the power cord to the new power supply.

  4. Only if you shut down the Cisco CSP 5400, press the Power button to boot the Cisco CSP 5400 to main power mode.

Figure 23. Replacing AC Power Supplies

1

Power supply release lever

2

Power supply handle

Replacing a PCIe Riser

This Cisco CSP 5400 has two toolless PCIe risers for horizontal installation of PCIe cards. Each riser is available in multiple versions. See PCIe Slot Specifications for detailed descriptions of the slots and features in each riser version.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Remove the PCIe riser that you are replacing:

  1. Grasp the flip-up handle on the riser and the blue forward edge, and then lift up evenly to disengage its circuit board from the socket on the motherboard. Set the riser on an antistatic surface.

  2. If the riser has a card installed, remove the card from the riser. See Replacing a PCIe Card.

Step 5

Install a new PCIe riser:

Note 

The PCIe risers are not interchangeable. If you plug a PCIe riser into the wrong socket, the Cisco CSP 5400 will not boot. Riser 1 must plug into the motherboard socket labeled “RISER1.” Riser 2 must plug into the motherboard socket labeled “RISER2.”

  1. If you removed a card from the old PCIe riser, install the card to the new riser. See Replacing a PCIe Card.

  2. Position the PCIe riser over its socket on the motherboard and over its alignment slots in the chassis.

  3. Carefully push down on both ends of the PCIe riser to fully engage its circuit board connector with the socket on the motherboard.
Step 6

Replace the top cover to the Cisco CSP 5400.
Step 7

Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 24. PCIe Riser Alignment Features

1

Riser handling points (flip-up handle and blue forward edge)

3

Riser 1 alignment features in chassis

2

Riser 2 alignment features in chassis

Replacing a PCIe Card


Note
Cisco supports all PCIe cards qualified and sold by Cisco. PCIe cards not qualified or sold by Cisco are the responsibility of the customer. Although Cisco will always stand behind and support the C-Series rack-mount Cisco CSP 5400s, customers using standard, off-the-shelf, third-party cards must go to the third-party card vendor for support if any issue with that particular card occurs.

PCIe Slot Specifications

The Cisco CSP 5400 contains two toolless PCIe risers for horizontal installation of PCIe cards. Each riser is orderable in multiple versions.

  • Riser 1 contains PCIe slots 1, 2, and 3 and is available in the following option:

    • Option 1B—Slots 1 (x8), 2 (x8), and 3 (x8). All slots are controlled by CPU 1.

  • Riser 2 contains PCIe slots 4, 5 and 6 and is available in the following options:

    • Option 2A—Slots 4 (x16), 5 (x16), and 6 (x8).

    • Option 2C—With slots 4 (x8), 5 (x8), and 6 (x8); includes one PCIe cable connector for rear-loading NVMe SSDs, plus one PCIe cable connector for front-loading NVMe SSDs.

Figure 25. Rear Panel, Showing PCIe Slot Numbering

The following tables describe the specifications for the slots.

Table 4. PCIe Riser 1B (UCSC-PCI-1B-C240M5) PCIe Expansion Slots

Slot Number

Electrical Lane Width

Connector Length

Maximum Card Length

Card Height (Rear Panel Opening)

NCSI Support

Double-Wide GPU Card Support

1

Gen-3 x8

x24 connector

¾ length

Full height

Yes 1

No

2

Gen-3 x8

x24 connector

Full length

Full height

Yes

Yes

3

Gen-3 x8

x16 connector

Full length

Full hight

No

No
1 NCSI is supported in only one slot at a time. If a GPU card is present in slot 2, NCSI support automatically moves to slot 1.
Table 5. PCIe Riser 2A (UCSC-PCI-2A-C240M5) PCIe Expansion Slots

Slot Number

Electrical Lane Width

Connector Length

Maximum Card Length

Card Height (Rear Panel Opening)

NCSI Support

Double-Wide GPU Card Support

4

Gen-3 x16

x24 connector

¾ length

Full height

Yes 2

No

5

Gen-3 x16

x24 connector

Full length

Full height

Yes

Yes

6

Gen-3 x8

x16 connector

Full length

Full hight

No

No
2 NCSI is supported in only one slot at a time. If a GPU card is present in slot 5, NCSI support automatically moves to slot 4.

Replacing a PCIe Card


Note

If you are installing a Cisco UCS Virtual Interface Card, there are prerequisite considerations. See Cisco Virtual Interface Card (VIC) Considerations.


Note

RAID controller cards install into a dedicated motherboard socket. See Replacing a SAS Storage Controller Card (RAID).


Note

For instructions on installing or replacing double-wide GPU cards, see GPU Card Installation topic.
Procedure
Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Remove the PCIe card that you are replacing:

  1. Remove any cables from the ports of the PCIe card that you are replacing.

  2. Use two hands to flip up and grasp the blue riser handle and the blue fingergrip area on the front edge of the riser, and then lift straight up.

  3. On the bottom of the riser, push the release latch that holds the securing plate, and then swing the hinged securing plate open.

  4. Open the hinged card-tab retainer that secures the rear-panel tab of the card.

  5. Pull evenly on both ends of the PCIe card to remove it from the socket on the PCIe riser.

    If the riser has no card, remove the blanking panel from the rear opening of the riser.
Step 5

Install a new PCIe card:

  1. With the hinged card-tab retainer open, align the new PCIe card with the empty socket on the PCIe riser.

  2. Push down evenly on both ends of the card until it is fully seated in the socket.

  3. Ensure that the card’s rear panel tab sits flat against the riser rear-panel opening and then close the hinged card-tab retainer over the card’s rear-panel tab.

  4. Swing the hinged securing plate closed on the bottom of the riser. Ensure that the clip on the plate clicks into the locked position.

  5. Position the PCIe riser over its socket on the motherboard and over the chassis alignment channels.

  6. Carefully push down on both ends of the PCIe riser to fully engage its connector with the sockets on the motherboard.
Step 6

Replace the top cover to the Cisco CSP 5400.
Step 7

Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 26. PCIe Riser Card Securing Mechanisms

1

Release latch on hinged securing plate

3

Hinged card-tab retainer

2

Hinged securing plate

-

Cisco Virtual Interface Card (VIC) Considerations

This section describes VIC card support and special considerations for this Cisco CSP 5400.


Note
If you use the Cisco Card NIC mode, you must also make a VIC Slot setting that matches where your VIC is installed. The options are Riser1, Riser2, and Flex-LOM.

If you want to use the Cisco UCS VIC card for Cisco UCS Manager integration, see also the Cisco UCS C-Series Cisco CSP 5400 Integration with Cisco UCS Manager Guides for details about supported configurations, cabling, and other requirements.

Table 6. VIC Support and Considerations in This Cisco CSP 5400

VIC

How Many Supported in Cisco CSP 5400

Slots That Support VICs

Primary Slot For Cisco UCS Manager Integration

Primary Slot For Cisco Card NIC Mode

Minimum Cisco IMC Firmware

Cisco UCS VIC 1385

UCSC-PCIE-C40Q-03

2 PCIe

PCIe 2

PCIe 5

PCIe 2

PCIe 2

3.1(1)

Cisco UCS VIC 1455

UCSC-PCIE-C25Q-04

2 PCIe

PCIe 2

PCIe 5

PCIe 2

PCIe 2

4.0(1)

Cisco UCS VIC 1387

UCSC-MLOM-C40Q-03

1 mLOM

mLOM

mLOM

mLOM

3.1(1)

Cisco UCS VIC 1457

UCSC-MLOM-C25Q-04

1 mLOM

mLOM

mLOM

mLOM

4.0(1)

  • A total of 3 VICs are supported in the Cisco CSP 5400: 2 PCIe style, and 1 mLOM style.


    Note

    Single wire management is supported on only one VIC at a time. If multiple VICs are installed on a Cisco CSP 5400, only one slot has NCSI enabled at a time. For single wire management, priority goes to the MLOM slot, then slot 2, then slot 5 for NCSI management traffic. When multiple cards are installed, connect the single-wire management cables in the priority order mentioned above.

  • The primary slot for a VIC card in PCIe riser 1 is is slot 2. The secondary slot for a VIC card in PCIe riser 1 is slot 1.


    Note

    The NCSI protocol is supported in only one slot at a time in each riser. If a GPU card is present in slot 2, NCSI automatically shifts from slot 2 to slot 1.

  • The primary slot for a VIC card in PCIe riser 2 is is slot 5. The secondary slot for a VIC card in PCIe riser 2 is slot 4.


    Note

    The NCSI protocol is supported in only one slot at a time in each riser. If a GPU card is present in slot 5, NCSI automatically shifts from slot 5 to slot 4.


    Note

    PCIe riser 2 is not available in a single-CPU system.

Replacing an mLOM Card

The Cisco CSP 5400 supports a modular LOM (mLOM) card to provide additional rear-panel connectivity. The mLOM socket is on the motherboard, under the storage controller card.

The mLOM socket provides a Gen-3 x16 PCIe lane. The socket remains powered when the Cisco CSP 5400 is in 12 V standby power mode and it supports the network communications services interface (NCSI) protocol.


Note
If your mLOM card is a Cisco UCS Virtual Interface Card (VIC), see Cisco Virtual Interface Card (VIC) Considerations for more information and support details.

Procedure

Step 1

Remove any existing mLOM card (or a blanking panel):

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

  4. Remove any storage controller (RAID or HBA card) to provide clearance to the mLOM socket on the motherboard. See Replacing a SAS Storage Controller Card (RAID).

  5. Loosen the single captive thumbscrew that secures the mLOM card to the threaded standoff on the chassis floor.

  6. Slide the mLOM card horizontally to free it from the socket, then lift it out of the Cisco CSP 5400.
Step 2

Install a new mLOM card:

  1. Set the mLOM card on the chassis floor so that its connector is aligned with the motherboard socket.

  2. Push the card horizontally to fully engage the card's edge connector with the socket.

  3. Tighten the captive thumbscrew to secure the card to the chassis floor.

  4. Return the storage controller card to the Cisco CSP 5400. See Replacing a SAS Storage Controller Card (RAID).

  5. Replace the top cover to the Cisco CSP 5400.

  6. Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 27. Location of the mLOM Card Socket Below the Storage Controller Card

1

Position of horizontal mLOM card socket

2

Position of mLOM card thumbscrew

Replacing a SAS Storage Controller Card (RAID)

For hardware-based storage control, the Cisco CSP 5400 can use a Cisco modular SAS RAID controller or SAS HBA that plugs into a dedicated, vertical socket on the motherboard.

Storage Controller Card Firmware Compatibility

Firmware on the storage controller (RAID) must be verified for compatibility with the current Cisco IMC and BIOS versions that are installed on the Cisco CSP 5400. If not compatible, upgrade or downgrade the storage controller firmware using the Host Upgrade Utility (HUU) for your firmware release to bring it to a compatible level.

See the HUU guide for your Cisco IMC release for instructions on downloading and using the utility to bring Cisco CSP 5400 components to compatible levels: HUU Guides.

Replacing the Supercap (RAID Backup)

This Cisco CSP 5400 supports installation of one supercap unit. The unit mounts to a bracket on the removable air baffle.

The supercap provides approximately three years of backup for the disk write-back cache DRAM in the case of a sudden power loss by offloading the cache to the NAND flash.


Warning

There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.

Statement 1015

Procedure

Step 1

Prepare the Cisco CSP 5400 for component installation:

  1. Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

  2. Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

    Caution 
    If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.

  3. Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 2

Remove an existing supercap:

  1. Disconnect the supercap cable from the existing supercap.

  2. Push aside the securing tab that holds the supercap to its bracket on the air baffle.

  3. Lift the supercap free of the bracket and set it aside.
Step 3

Install a new supercap:

  1. Set the new supercap into the mounting bracket.

  2. Push aside the black plastic tab on the air baffle and set the supercap into the bracket. Relax the tab so that it closes over the top edge of the supercap.

  3. Connect the supercap cable from the RAID controller card to the connector on the supercap cable.
Step 4

Replace the top cover to the Cisco CSP 5400.
Step 5

Replace the Cisco CSP 5400 in the rack, replace cables, and then fully power on the Cisco CSP 5400 by pressing the Power button.

Figure 28. Supercap Bracket on Air Baffle

1

Supercap bracket on removeable air baffle

2

Securing tab

Service Headers and Jumpers

This Cisco CSP 5400 includes two blocks of headers (J38, J39) that you can jumper for certain service and debug functions.

This section contains the following topics:

Figure 29. Location of Service Header Blocks J38 and J39

1

Location of header block J38

6

Location of header block J39

2

J38 pin 1 arrow printed on motherboard

7

J39 pin 1 arrow printed on motherboard

3

Clear CMOS: J38 pins 9 - 10

8

Boot Cisco IMC from alternate image: J39 pins 1 - 2

4

Recover BIOS: J38 pins 11 - 12

9

Reset Cisco IMC password to default: J39 pins 3 - 4

5

Clear password: J38 pins 13 - 14

10

Reset Cisco IMC to defaults: J39 pins 5 - 6

Using the Clear CMOS Header (J38, Pins 9 - 10)

You can use this switch to clear the Cisco CSP 5400’s CMOS settings in the case of a system hang. For example, if the Cisco CSP 5400 hangs because of incorrect settings and does not boot, use this jumper to invalidate the settings and reboot with defaults.


Caution
Clearing the CMOS removes any customized settings and might result in data loss. Make a note of any necessary customized settings in the BIOS before you use this clear CMOS procedure.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Install a two-pin jumper across J38 pins 9 and 10.
Step 5

Reinstall the top cover and reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode, indicated when the Power LED on the front panel is amber.

Step 6

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel. The Cisco CSP 5400 is in main power mode when the Power LED is green.

Note 
You must allow the entire Cisco CSP 5400 to reboot to main power mode to complete the reset. The state of the jumper cannot be determined without the host CPU running.
Step 7

Press the Power button to shut down the Cisco CSP 5400 to standby power mode, and then remove AC power cords from the Cisco CSP 5400 to remove all power.

Step 8

Remove the top cover from the Cisco CSP 5400.
Step 9

Remove the jumper that you installed.

Note 
If you do not remove the jumper, the CMOS settings are reset to the defaults every time you power-cycle the Cisco CSP 5400.
Step 10

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.

Using the BIOS Recovery Header (J38, Pins 11 - 12)

Depending on which stage the BIOS becomes corrupted, you might see different behavior.

  • If the BIOS BootBlock is corrupted, you might see the system get stuck on the following message:

    Initializing and configuring memory/hardware

  • If it is a non-BootBlock corruption, a message similar to the following is displayed:

    ****BIOS FLASH IMAGE CORRUPTED****
Flash a valid BIOS capsule file using Cisco IMC WebGUI or CLI interface.
IF Cisco IMC INTERFACE IS NOT AVAILABLE, FOLLOW THE STEPS MENTIONED BELOW.
1. Connect the USB stick with bios.cap file in root folder.
2. Reset the host.
IF THESE STEPS DO NOT RECOVER THE BIOS
1. Power off the system.
2. Mount recovery jumper.
3. Connect the USB stick with bios.cap file in root folder.
4. Power on the system.
Wait for a few seconds if already plugged in the USB stick.
REFER TO SYSTEM MANUAL FOR ANY ISSUES.

Note
As indicated by the message shown above, there are two procedures for recovering the BIOS. Try procedure 1 first. If that procedure does not recover the BIOS, use procedure 2.

Procedure 1: Reboot With bios.cap Recovery File

Procedure
Step 1

Download the BIOS update package and extract it to a temporary location.
Step 2

Copy the contents of the extracted recovery folder to the root directory of a USB drive. The recovery folder contains the bios.cap file that is required in this procedure.

Note 
The bios.cap file must be in the root directory of the USB drive. Do not rename this file. The USB drive must be formatted with either the FAT16 or FAT32 file system.
Step 3

Insert the USB drive into a USB port on the Cisco CSP 5400.
Step 4

Reboot the Cisco CSP 5400.
Step 5

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel.

The Cisco CSP 5400 boots with the updated BIOS boot block. When the BIOS detects a valid bios.cap file on the USB drive, it displays this message: 

Found a valid recovery file...Transferring to Cisco IMC
System would flash the BIOS image now...
System would restart with recovered image after a few seconds...
Step 6

Wait for Cisco CSP 5400 to complete the BIOS update, and then remove the USB drive from the Cisco CSP 5400.

Note 
During the BIOS update, Cisco IMC shuts down the Cisco CSP 5400 and the screen goes blank for about 10 minutes. Do not unplug the power cords during this update. Cisco IMC powers on the Cisco CSP 5400 after the update is complete.

Procedure 2: Use BIOS Recovery Header and bios.cap File

Procedure
Step 1

Download the BIOS update package and extract it to a temporary location.
Step 2

Copy the contents of the extracted recovery folder to the root directory of a USB drive. The recovery folder contains the bios.cap file that is required in this procedure.

Note 
The bios.cap file must be in the root directory of the USB drive. Do not rename this file. The USB drive must be formatted with either the FAT16 or FAT32 file system.
Step 3

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400. Disconnect power dords from all power supplies.

Step 4

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 5

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 6

Install a two-pin jumper across J38 pins 11 and 12.
Step 7

Reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode.
Step 8

Insert the USB thumb drive that you prepared in Step 2 into a USB port on the Cisco CSP 5400.
Step 9

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel.

The Cisco CSP 5400 boots with the updated BIOS boot block. When the BIOS detects a valid bios.cap file on the USB drive, it displays this message: 

Found a valid recovery file...Transferring to Cisco IMC
System would flash the BIOS image now...
System would restart with recovered image after a few seconds...
Step 10

Wait for Cisco CSP 5400 to complete the BIOS update, and then remove the USB drive from the Cisco CSP 5400.

Note 
During the BIOS update, Cisco IMC shuts down the Cisco CSP 5400 and the screen goes blank for about 10 minutes. Do not unplug the power cords during this update. Cisco IMC powers on the Cisco CSP 5400 after the update is complete.
Step 11

After the Cisco CSP 5400 has fully booted, power off the Cisco CSP 5400 again and disconnect all power cords.
Step 12

Remove the jumper that you installed.

Note 
If you do not remove the jumper, after recovery completion you see the prompt, “Please remove the recovery jumper.”
Step 13

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.

Using the Clear Password Header (J38, Pins 13 - 14)

You can use this switch to clear the administrator password.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400. Disconnect power dords from all power supplies.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Install a two-pin jumper across J38 pins 13 and 14.
Step 5

Reinstall the top cover and reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode, indicated when the Power LED on the front panel is amber.

Step 6

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel. The Cisco CSP 5400 is in main power mode when the Power LED is green.

Note 
You must allow the entire Cisco CSP 5400 to reboot to main power mode to complete the reset. The state of the jumper cannot be determined without the host CPU running.
Step 7

Press the Power button to shut down the Cisco CSP 5400 to standby power mode, and then remove AC power cords from the Cisco CSP 5400 to remove all power.

Step 8

Remove the top cover from the Cisco CSP 5400.
Step 9

Remove the jumper that you installed.

Note 
If you do not remove the jumper, the password is cleared every time you power-cycle the Cisco CSP 5400.
Step 10

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.

Using the Boot Alternate Cisco IMC Image Header (J39, Pins 1 - 2)

You can use this Cisco IMC debug header to force the system to boot from an alternate Cisco IMC image.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400. Disconnect power cords from all power supplies.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Install a two-pin jumper across J39 pins 1 and 2.
Step 5

Reinstall the top cover and reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode, indicated when the Power LED on the front panel is amber.

Step 6

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel. The Cisco CSP 5400 is in main power mode when the Power LED is green.

Note 

When you next log in to Cisco IMC, you see a message similar to the following:

'Boot from alternate image' debug functionality is enabled.  
CIMC will boot from alternate image on next reboot or input power cycle.
Note 
If you do not remove the jumper, the Cisco CSP 5400 will boot from an alternate Cisco IMC image every time that you power cycle the Cisco CSP 5400 or reboot Cisco IMC.
Step 7

To remove the jumper, press the Power button to shut down the Cisco CSP 5400 to standby power mode, and then remove AC power cords from the Cisco CSP 5400 to remove all power.

Step 8

Remove the top cover from the Cisco CSP 5400.
Step 9

Remove the jumper that you installed.
Step 10

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.

Using the Reset Cisco IMC Password to Default Header (J39, Pins 3 - 4)

You can use this Cisco IMC debug header to force the Cisco IMC password back to the default.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400. Disconnect power cords from all power supplies.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Install a two-pin jumper across J39 pins 3 and 4.
Step 5

Reinstall the top cover and reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode, indicated when the Power LED on the front panel is amber.

Step 6

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel. The Cisco CSP 5400 is in main power mode when the Power LED is green.

Note 

When you next log in to Cisco IMC, you see a message similar to the following:

'Reset to default CIMC password' debug functionality is enabled.  
On input power cycle, CIMC password will be reset to defaults.
Note 
If you do not remove the jumper, the Cisco CSP 5400 will reset the Cisco IMC password to the default every time that you power cycle the Cisco CSP 5400. The jumper has no effect if you reboot Cisco IMC.
Step 7

To remove the jumper, press the Power button to shut down the Cisco CSP 5400 to standby power mode, and then remove AC power cords from the Cisco CSP 5400 to remove all power.

Step 8

Remove the top cover from the Cisco CSP 5400.
Step 9

Remove the jumper that you installed.
Step 10

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.

Using the Reset Cisco IMC to Defaults Header (J39, Pins 5 - 6)

You can use this Cisco IMC debug header to force the Cisco IMC settings back to the defaults.

Procedure

Step 1

Shut down and remove power from the Cisco CSP 5400 as described in Shutting Down and Removing Power From the Cisco CSP 5400. Disconnect power cords from all power supplies.

Step 2

Slide the Cisco CSP 5400 out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.

Caution 
If you cannot safely view and access the component, remove the Cisco CSP 5400 from the rack.
Step 3

Remove the top cover from the Cisco CSP 5400 as described in Removing the Cisco CSP 5400 Top Cover.

Step 4

Install a two-pin jumper across J39 pins 5 and 6.
Step 5

Reinstall the top cover and reconnect AC power cords to the Cisco CSP 5400. The Cisco CSP 5400 powers up to standby power mode, indicated when the Power LED on the front panel is amber.

Step 6

Return the Cisco CSP 5400 to main power mode by pressing the Power button on the front panel. The Cisco CSP 5400 is in main power mode when the Power LED is green.

Note 

When you next log in to Cisco IMC, you see a message similar to the following:

'CIMC reset to factory defaults' debug functionality is enabled.  
On input power cycle, CIMC will be reset to factory defaults.
Note 
If you do not remove the jumper, the Cisco CSP 5400 will reset the Cisco IMC to the default settings every time that you power cycle the Cisco CSP 5400. The jumper has no effect if you reboot Cisco IMC.
Step 7

To remove the jumper, press the Power button to shut down the Cisco CSP 5400 to standby power mode, and then remove AC power cords from the Cisco CSP 5400 to remove all power.

Step 8

Remove the top cover from the Cisco CSP 5400.
Step 9

Remove the jumper that you installed.
Step 10

Replace the top cover, replace the Cisco CSP 5400 in the rack, replace power cords and any other cables, and then power on the Cisco CSP 5400 by pressing the Power button.