Upgrade Line Card Firmware on 12000 Series Internet Routers

Introduction

This document provides recommended upgrade procedures for the Cisco 12000 Series Internet Router that returns the router to service in the shortest timeframe.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Architecture of the Cisco 12000 Series Internet Router

• Bootup process of the Cisco 12000 Series Internet Router

  Refer to Understanding the Booting Process on the Cisco 12000 Series Internet Router for more information.

Components Used

The information in this document is based on these software and hardware versions:

• Cisco 12000 Series Internet Router

• All versions of Cisco IOS^® Software that run on this platform

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Details

Cisco IOS Software Images

Cisco IOS Software images for the 12000 routers contain not only the IOS Software, but additional images for components of the line cards and Route Processor (RP). These images are:

• The RP ROMmon code—The basic power-up, self-test, media management, and memory management control code.

• The Mbus-agent-rom code—The control code that all components within the router require to initialize and establish communication with the maintenance bus (Mbus).

• The fabric downloader code— The control code used to configure the line card to receive the full Cisco IOS Software image from the RP through the switching fabric.

An upgrade of these images speeds up the restoration of service after a router reload. The upgrade also ensures that the latest fixes are applied to the appropriate components on the line card and RP.

RP ROMmon

Cisco recommends you to upgrade the RP ROMmon to the image present within the new Cisco IOS Software that you currently run. The system does not prompt you to perform a ROMmon upgrade. Therefore, run the upgrade rom-monitor slot <x> command. If the image present on the RP is a later version than the image within the IOS Software image, the upgrade occurs. The router subsequently reloads.

Mbus Agents

The line card uses two Mbus firmware packages. The Mbus-agent held on the line card ROM is used when you power on the Mbus module. The Mbus-agent is then used to download the Mbus-agent-RAM code from the main Cisco IOS Software image on the RP. After the download, you can upgrade the ROM agent. All data transfers occur over the Mbus. Cisco recommends that you upgrade this code with the upgrade mbus-agent-rom all command. You do not need to reload the line card afterwards.

Note: If a line card has a version of Mbus that is greater than the Mbus version bundled with IOS, you do not need to downgrade the Mbus version of the line card through the force option. Issue the show bundle command to view the bundled version in IOS.

Fabric-Downloader

If the fabric-downloader code present on the line card is different from the code contained within the new Cisco IOS Software, an error message appears at the end of the show version command output. On the first reload of the new Cisco IOS Software, if a difference exists between the fabric-downloader code on the line card and the code within the Cisco IOS Software image, a new copy of the fabric-downloader is copied to line card memory and executed. The copy and execution process extends the boot-up time of the line card.

Cisco recommends that you upgrade this code with the upgrade fabric-downloader all command. The upgrade occurs over the Mbus. You do not need to reload the line card afterwards. On the next reload, the fabric-downloader is the same revision as that within the Cisco IOS Software image, and it runs directly from the line card.

Note: After the fabric upgrade, if the RP reloads (soft or warm reload), the show diag command displays Fabric Downloader version used is n/a. This is the expected behavior, and not a bug.

Line Card ROMmon Upgrade

The initialization phase of the line card uses the line card ROMmon image. Changes on the line card ROMmon code are very infrequent and do not offer many benefits to users. In case of an interruption of the line card ROMmon upgrade process, the line card can become inoperable. Therefore, Cisco recommends that you perform this upgrade only under the direct guidance of Cisco Technical Support.

Step-by-Step Upgrade Procedures

This section provides three upgrade procedures:

• Upgrade the Field Programmable Device (Shared Port Adapters)

• Single Route Processor Upgrade Process

• Dual Route Processors Upgrade Process

Upgrade the Field Programmable Device (Shared Port Adapters)

It is recommended to upgrade the Field Programmable Devices (FPDs) for the Shared Port Adapters (SPAs) when the IOS on the router is changed. There is an FPD image that corresponds to each IOS image. SPAs are supported on the Cisco 12000 series in Cisco IOS Software Releases 12.0(31)S and later.

1. Download the FPD image package for the Cisco IOS Software release that you upgrade to any Flash disk on your router.

   This is performed before you boot the new version of Cisco IOS. The FPD image package can be retrieved from the same site where you receive your Cisco IOS image. Do not change the name of the FPD image package. This allows the router to find this image package during the first IOS bootup and upgrade the FPD automatically.

2. Issue the upgrade fpd path command to direct the router to search for the FPD image package in the proper location.

   For example, if the FPD image is placed in the disk0, then the command is upgrade fpd path disk0:.

3. Boot using the new version of Cisco IOS.

   When the new Cisco IOS boots, it searches for the FPD image package in the router Flash card or disk0 by default. These images are updated automatically as part of the IOS boot process.

4. Examine the output of the show running-config command (look for the upgrade fpd auto configuration line in the output) to make sure that the FPD automatic upgrade feature is enabled.

   If there are no upgrade commands in the output, then it is due to automatic upgrades that are disabled.

5. Issue the upgrade fpd auto global configuration command to enable automatic FPD upgrades.

6. Issue the show hw-module all fpd command after the router has booted.

   This verifies that the upgrade is successful. Refer to Field-Programmable Device Upgrades for more information.

   Note: In the case of dual RP routers, also download the FPD image to secondary-disk0.

Single Route Processor Upgrade Process

Complete these steps in order to minimize the overall router outage time:

1. Make note of the slot in the chassis in which the primary RP is installed and issue the show gsr command.

   In this example, the RP is in slot 7.

   Slot 3  type  = 1 Port Packet Over SONET OC-48c/STM-16
                   state = IOS RUN   Line Card Enabled
   
   Slot 7  type  = Route Processor
                   state = ACTV RP   IOS Running  ACTIVE

2. Reload the router with the new Cisco IOS Software image.

   The Press RETURN to get started! message appears on the console.

3. Enter the enable mode and issue the upgrade rom-monitor slot <RP slot> command.

   If the routine finds that an upgrade is necessary, the new code begins to load. The line cards boot during this time, but the cards must not reach the IOS RUN state by the time the RP ROMmon upgrade is complete. The router reloads after the ROMmon upgrade is complete.

4. Wait for the router to return to full operation with Interior Gateway Protocol (IGP) and Exterior Gateway Protocol (EGP) peers established.

   This process can take a considerable amount of time. This depends on the size and complexity of the router configuration.

5. Issue the execute-on all show proc cpu | inc CPU command to check the line card CPU utilization.

   If the CPU is stable at the normal running level, proceed to the next step. Otherwise, wait five more minutes and check again.

6. Issue the upgrade mbus-agent-rom all command to upgrade the Mbus-agent-rom.

   The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 6, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   MBus agent ROM upgrade failed on slot 7 (rc=5) 
   MBus agent ROM upgrade failed on slot 8 (rc=6)

7. Issue the show version command.

   If an error message appears at the bottom of the output and indicates that a fabric-downloader code upgrade is necessary, issue the upgrade fabric-downloader all command. The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 7, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   Fabric-downloader upgrade failed on slot 7 (rc=5) 
   Fabric-downloader upgrade failed on slot 8 (rc=6)

Dual Route Processors Upgrade Process

This section provides the upgrade procedures for dual RPs.

Note: The IOS version on both RPs must be the same before you perform the ROMmon upgrade.

Prior to Cisco IOS Software Release 12.0(24)S

In a system with two RPs (RP1 and RP2), even though you can configure the secondary RP to run a new Cisco IOS Software image, you cannot upgrade the ROMmon if the RP is not active. The RPs must both fail twice before you can upgrade their ROMmon version. The upgrade occurs on the primary RP first. The secondary RP takes over control of the system when the primary RP reloads. The secondary RP is then upgraded. During the reload, control returns to the primary RP.

If you attempt an RP ROMmon upgrade on a backup RP, this message appears:

Cannot upgrade non local RP rom monitor in slot 5
When this upgrade cycle is finished, switchover to
the non-local GRP and upgrade its ROM

This is an upgrade procedure in which RP1 is initially primary while RP2 is secondary:

1. Enter the configuration to load the new Cisco IOS Software image.

2. Issue the hw-module standby reload command to reload RP2.

3. Issue the redundancy force-switchover command to switch over from RP1 to RP2.

   When RP2 becomes active, the line cards reload. The Press RETURN to get started! message appears on the console.

4. Enter the enable mode and issue the upgrade rom-monitor slot <RP slot> command.

   If the routine finds that an upgrade is necessary, the new code begins to load. The line cards boot during this time, but must not reach the IOS RUN state by the time the RP2 ROMmon upgrade is complete. The router reloads when the ROMmon upgrade is complete. RP1 now takes over. The Press RETURN to get started! message appears on the console.

5. Enter the enable mode and issue the upgrade rom-monitor slot <RP slot> command.

   If the routine finds that an upgrade is necessary, the new code begins to load. The line cards boot during this time, but must not reach the IOS RUN state by the time the RP1 ROMmon upgrade is complete. The router reloads when the ROMmon upgrade is complete. RP2 becomes primary.

6. Wait for the router to return to full operation with IGP and EGP peers established.

   This process can take a considerable amount of time. This depends on the size and complexity of the router configuration.

7. Issue the execute-on all show proc cpu | inc CPU command to check the line card CPU utilization.

   If CPU utilization is stable at the normal running level, proceed to the next step. Otherwise, wait another five minutes and check again.

8. Issue the upgrade mbus-agent-rom all command to upgrade the Mbus-agent-rom.

   The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 8, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   Bus agent ROM upgrade failed on slot 7 (rc=5) 
   MBus agent ROM upgrade failed on slot 8 (rc=6)

9. Issue the show version command.

   If there is an error message at the end of the output, which indicates that a fabric-downloader code upgrade is necessary, issue the upgrade fabric-downloader all command. The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 9, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   Fabric-downloader upgrade failed on slot 7 (rc=5) 
   Fabric-downloader upgrade failed on slot 8 (rc=6)

Cisco IOS Software Release 12.0(24)S and Later

Cisco IOS Software Release 12.0(24)S introduces new functionality that enables you to upgrade the ROMmon image of the secondary RP while in standby mode. In order for this to work, the router must already run Cisco IOS Software Release 12.0(24)S or later.

In this procedure, RP1 is initially primary while RP2 is secondary.

1. Enter the configuration to load the new Cisco IOS Software image.

2. Issue the hw-module standby reload command to reload RP2.

3. Issue the upgrade rom-monitor slot <sec-RP slot> command on RP1 when RP2 is available again.

   If the routine finds that an upgrade is necessary, the new code begins to load. When finished, issue the hw-module standby reload command to reload RP2.

4. Issue the redundancy force-switchover command to switch over from RP1 to RP2.

   When RP2 becomes active, the line cards reload.

5. Wait for the router to return to full operation with IGP and EGP peers established.

   This process can take a considerable amount of time. This depends on the size and complexity of the router configuration.

6. Issue the execute-on all show proc cpu | inc CPU command to check the line card CPU utilization.

   If CPU utilization is stable at the normal running level, proceed to the next step. Otherwise, wait another five minutes and check again.

7. Issue the upgrade mbus-agent-rom all command to upgrade the Mbus-agent-rom.

   The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 7, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   MBus agent ROM upgrade failed on slot 7 (rc=5) 
   MBus agent ROM upgrade failed on slot 8 (rc=6)

8. Issue the show version command.

   If an error message appears at the end of the output and indicates that a fabric-downloader code upgrade is necessary, issue the upgrade fabric-downloader all command. The normal forwarding operation of the line cards is not affected during the upgrade. You do not need to reload the line cards.

   Note: If any error messages appear during step 8, repeat the step before you contact Cisco Technical Support. This is an example of an error message that can appear:

   Fabric-downloader upgrade failed on slot 7 (rc=5) 
   Fabric-downloader upgrade failed on slot 8 (rc=6)

9. Issue the upgrade rom-monitor slot <sec-RP slot> command to upgrade the ROMmon image of RP1.

   If the routine finds that an upgrade is necessary, the new code begins to load.

10. When finished, issue the hw-module standby reload command to reload the standby RP.

Service Upgrade All

You can issue the service upgrade all command as part of the router configuration. When the router reloads and the command is present in the startup configuration, the line cards in the chassis have their fabric-downloader and line card ROMmon image upgraded, if necessary. The Mbus-agent-rom is not upgraded. The packet forwarding operations on the line card are delayed until the upgrade is complete.

The line card ROMmon is not a required step and can, in some circumstances, result in an inoperable line card.

Notes:

• From Cisco IOS Software Release 12.0(25)S and later, the line card ROMmon portion of the service upgrade all command is removed.

• From Cisco IOS Software Release 12.0(25)S1 and 12.0(26)S, a new service command is introduced to automatically upgrade the line card Mbus-agent-rom code. This is configured with the service upgrade mbus-agent-rom command line.

Important Note:

The service upgrade all configuration command begins to be deprecated from Cisco IOS Software Release 12.0(27)S. The command must be replaced with these commands:

• service upgrade mbus-agent-rom

• service upgrade fabric-downloader

Related Information

• Cisco 12000 Series Routers
• Cisco Line Cards
• Cisco 12000 Series Internet Routers Support Page
• Understanding the Bootup Process on the Cisco 12000 Series Internet Router
• Upgrading the FPGA Image on a Line Card
• Route Processor Redundancy Plus for the Cisco 12000 Series Internet Router
• Technical Support & Documentation - Cisco Systems