Replacing Components

Preparing a Fabric Interconnect for Removal

Removing a subordinate (standby) fabric interconnect is non-disruptive in a redundant, high-availability Cisco UCS configuration. Removing a primary (active) fabric interconnect will cause the standby fabric interconnect to become active with minimal or no disruption.

When powering down and removing clustered fabric interconnects, remove the subordinate fabric interconnect first, and then remove the primary fabric interconnect.


Caution

Be aware that if you remove both the primary and subordinate fabric interconnects, or the single fabric interconnect from a standalone system, you are shutting down the entire Cisco UCS domain.

Procedure

Step 1

Use Cisco UCS Manager to perform the following tasks:

  1. Back up your Cisco UCS Manager configuration.

  2. Shut down the OS on all servers in the Cisco UCS domain.

  3. Disable the Smart Call Home feature in the Cisco UCS domain.

  4. Decommission every attached chassis in the Cisco UCS domain.

For details, see the Configuration Guide for the version of Cisco UCS Manager that you are using. The configuration guides are available at this URL: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-installation-and-configuration-guides-list.html.

Step 2

Power down every attached chassis as described in the Cisco UCS 5108 Server Chassis Hardware Installation Guide.

Shutting Down a Fabric Interconnect

Use the following procedure when you need to replace a single fabric interconnect with the same model of fabric interconnect.

Use Cisco UCS Manager, either the GUI or the CLI, to perform the software-related tasks mentioned in the following procedure. For additional information, refer to the Cisco UCS Manager Getting Started Guide, the Cisco UCS Manager Infrastructure Management Guide, and the instructional videos available at this URL: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-installation-and-configuration-guides-list.html

As a best practice, perform a full configuration backup before replacing the fabric interconnect.


Note

Verify that all vNICs are either redundant or that the fabric failover is enabled.

Procedure

Step 1

Contact Cisco TAC or your Cisco Sales representative to have them transfer the license to the replacement fabric interconnect.

Step 2

Label the ports and the cables that you using so that you can refer to this information later. The cabling and port numbering should be same for the replacement fabric interconnect.

Step 3

Log into UCS Manager and verify the state of the A and B fabric interconnects, either UP or DOWN. From the CLI, enter the show cluster extended-state command. High availability should be running in UCS Manager, as indicated by HA is READY.

Example:

FI-A# show cluster extended-state 
Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984

A: UP, PRIMARY
B: UP, SUBORDINATE

A: memb state UP, lead state PRIMARY, mgmt services state: UP
B: memb state UP, lead state SUBORDINATE, mgmt services state: UP
   heartbeat state PRIMARY_OK 

INTERNAL NETWORK INTERFACES:
eth1, UP
eth2, UP

HA READY   <<<<<<<<<<<<<←---------------- HA is READY
Detailed state of the device selected for HA storage:
Chassis 1, serial: FOX1344G1R1, state: active
Chassis 2, serial: FOX1318GDKR, state: active
FI-A#
Step 4

Back up the software configuration.

Step 5

Use the fabric evacuation procedure on the subordinate fabric interconnect to ensure there is no data traffic impact during the hardware replacement.

Step 6

Power down the subordinate fabric interconnect by unplugging it from the power source.

Example:

FI-A# show cluster extended-state 
Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984

A: UP, PRIMARY
B: DOWN, INAPPLICABLE

A: memb state UP, lead state PRIMARY, mgmt services state: UP
B: memb state DOWN, lead state INAPPLICABLE, mgmt services state: DOWN
   heartbeat state SECONDARY_FAILED

INTERNAL NETWORK INTERFACES:
eth1, DOWN
eth2, DOWN

HA NOT READY
Peer Fabric Interconnect is down
Detailed state of the device selected for HA storage:
Chassis 1, serial: FOX1344G1R1, state: active
Chassis 2, serial: FOX1318GDKR, state: active
FI-A#
Step 7

Remove the fabric interconnect from the rack. Follow the instructions in Removing a Cisco UCS Fabric Interconnect From a Rack.

Step 8

Install the replacement fabric interconnect to the rack. Follow the instructions in Installing the Cisco UCS FI 6454 in a Cabinet or Rack.

Step 9

Connect the management and console cables to the replacement fabric interconnect.

Step 10

Connect the L1/L2 cables that were disconnected to the replacement fabric interconnect.

Step 11

Connect the data cable according to the labels that you created in Step 2.

Step 12

Connect the power cable to the fabric interconnect and it will automatically boot and run POST tests. 

         ---- Basic System Configuration Dialog ----

  This setup utility will guide you through the basic configuration of
  the system. Only minimal configuration including IP connectivity to
  the Fabric interconnect and its clustering mode is performed through these steps.

  Type Ctrl-C at any time to abort configuration and reboot system.
  To back track or make modifications to already entered values,
  complete input till end of section and answer no when prompted
  to apply configuration.

  
  Enter the configuration method. (console/gui) ? console

  Installer has detected the presence of a peer Fabric interconnect. 
  This Fabric interconnect will be added to the cluster. Continue (y/n) ? y

  Enter the admin password of the peer Fabric interconnect: 
    Connecting to peer Fabric interconnect... done
    Retrieving config from peer Fabric interconnect... done
    Peer Fabric interconnect Mgmt0 IP Address: 122.255.252.2
    Peer Fabric interconnect Mgmt0 IP Netmask: 255.255.255.0
    Cluster IP address          : 122.255.252.1

  Physical Switch Mgmt0 IPv4 address : 122.255.252.3


  Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes
  Applying configuration. Please wait.
Configuration file - Ok


Cisco UCS 6454 Fabric Interconnect

FI-B login:
Step 13

If necessary, upgrade the UCS Manager software. If the replacement fabric interconnect is not running the same firmware version as the cluster, the setup utility can upgrade the firmware.

Example:

---- Basic System Configuration Dialog ----

  This setup utility will guide you through the basic configuration of
  the system. Only minimal configuration including IP connectivity to
  the Fabric interconnect and its clustering mode is performed through 
  these steps.

  Type Ctrl-C at any time to abort configuration and reboot system.
  To back track or make modifications to already entered values,
  complete input till end of section and answer no when prompted
  to apply configuration.


  Enter the configuration method. (console/gui) ? console

  Installer has detected the presence of a peer Fabric interconnect. 
  This Fabric interconnect will be added to the cluster. Continue (y/n) ? y

  Enter the admin password of the peer Fabric interconnect:
    Connecting to peer Fabric interconnect... done
    Retrieving config from peer Fabric interconnect... done
    Installer has determined that the peer Fabric Interconnect is 
    running a different firmware version than the local Fabric. 
    Cannot join cluster.

    Local Fabric Interconnect
      UCSM version     : 3.1(2c)
      Kernel version   : 5.0(3)N2(3.12b)
      System version   : 5.0(3)N2(3.12b)
      local_model_no   : UCS-FI-6454

    Peer Fabric Interconnect
      UCSM version     : 3.1(2c)
      Kernel version   : 5.0(3)N2(3.12c)
      System version   : 5.0(3)N2(3.12c)
      peer_model_no    : UCS-FI-6454


  Do you wish to update firmware on this Fabric Interconnect to the 
  Peer's version? (y/n): y
  Updating firmware of Fabric Interconnect....... 
  [ Please don't press Ctrl+c while updating firmware ]

 Updating images
 Please wait for firmware update to complete.
 
 <output truncated>
 
 Install has been successful.
 Firmware Updation Successfully Completed. Please wait to enter the IP address
    Peer Fabric interconnect Mgmt0 IPv4 Address: xx.xx.xx.xx
    Peer Fabric interconnect Mgmt0 IPv4 Netmask: xx.xx.xx.xx
    Cluster IPv4 address          : xx.xx.xx.xx

    Peer FI is IPv4 Cluster enabled. Please Provide Local 
    Fabric Interconnect Mgmt0 IPv4 Address

  Physical Switch Mgmt0 IP address : xx.xx.xx.xx
  
  Apply and save the configuration (select 'no' if you want to re-enter)? 
  (yes/no): yes
  Applying configuration. Please wait.

  Fri Dec  9 19:57:15 UTC 2016
  Configuration file - Ok
Step 14

Enter the show cluster extended-state command to revalidate the cluster state and verify that HA is ready.

Example:

FI-A# show cluster extended-state 
Cluster Id: 0x537d0580bf9911e0-0x8955000decd07984

A: UP, PRIMARY
B: UP, SUBORDINATE

A: memb state UP, lead state PRIMARY, mgmt services state: UP
B: memb state UP, lead state SUBORDINATE, mgmt services state: UP
   heartbeat state PRIMARY_OK 

INTERNAL NETWORK INTERFACES:
eth1, UP
eth2, UP

HA READY   <<<<<<<<<<<<<←---------------- HA is READY
Detailed state of the device selected for HA storage:
Chassis 1, serial: FOX1344G1R1, state: active
Chassis 2, serial: FOX1318GDKR, state: active
FI-A#
Step 15

Disable fabric evacuation and verify that traffic is flowing in both fabric interconnects.

Step 16

Apply the port license to the new fabric interconnect.

Removing a Cisco UCS Fabric Interconnect From a Rack


Caution

Support both sides of the fabric interconnect when removing it from the rack. The slider rail and front rack-mount brackets do not have a stop mechanism. If the front of the chassis is unfastened from the rack and the chassis slides forward on the slider rails, it may slip off the end of the rails and fall out of the rack.

Procedure

Step 1

Ensure that the weight of the Cisco UCS Fabric Interconnect is fully supported and that the chassis is being held by another person.

Step 2

Remove the two screws holding the grounding cable to the chassis (if installed).

Step 3

Disconnect the power cords and any console cables.

Step 4

Disconnect all cables that are connected to SFP28 transceivers.

Step 5

Remove the screws fastening the front rack-mount brackets to the mounting rails.

Step 6

Gently slide the Cisco UCS Fabric Interconnect toward you, off of the slider rails and out of the rack.

Repacking the Cisco UCS Fabric Interconnect for Return Shipment

If you need to return the fabric interconnect, remove the fabric interconnect from the rack and repack it for shipment. If possible, use the original packing materials and container to repack the unit. Contact your Cisco customer service representative to arrange for return shipment to Cisco.