Field Notice: FN - 72016 - Replacement or Swap of Controller Card in NCS1004 System Can Cause a Reboot Loop - Workaround Provided

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	16-Dec-20	Initial Release

Products Affected

Affected Product ID	Comments
NCS1004
NCS1004=	Part Alternate

Defect Information

Defect ID	Headline
CSCvw43117	NCS1004: Disaster Recovery copied iPXE PHP script as golden ISO

Problem Description

The NCS1004 system can enter a reboot loop after the Controller card (CPU) is replaced or swapped.

Background

The SSDs on the Chassis and Controller card in the NCS1004 working system store the Golden Image of the current working software. When a Controller card is replaced or swapped, the Golden Image that is stored on the Chassis SSD will be installed on the Controller card if either of these conditions exists:

• The SSD on the new Controller card has a pre-installed software version that is different from the software version on the working Chassis SSD.
• The SSD on the new Controller card does not have any software installed.

However, a misconfiguration of the System SSD software image can cause the BIOS to be unable to find a valid image. This can cause the system to enter a reboot loop.

Problem Symptom

After the Controller card is replaced and the NCS1004 system does not boot, the console displays these error messages/logs:

<logs snip>

Version 2.19.1266. Copyright (C) 2018 American Megatrends, Inc.

BIOS Date: 11/29/2018 11:36:30 Ver: 0ACHI410

Press <DEL> or <ESC> to enter setup.

ChipDBVerified

CiscoSec: Image Signature Verified



GNU GRUB version 2.00

Press F2 to goto grub Menu..

Booting from Disaster Recovery, Chassis SSD..

Loading Kernel..

Kernel Secure Boot Validation Result: FAILED (Failure opening the file)



Press any key to reload card

</ logs snip>

Workaround/Solution

If the customer has already performed a USB or iPXE boot or software upgrade in the field, the issue is fixed and no action is needed.

If the system experiences the reboot loop issue and shows the error message that is described in the Problem Symptom section, the customer should reboot the system from USB with the desired image from this location: https://software.cisco.com/download/home/286323463/type/280805694/release

If the customer can proactively detect and fix the issue before the Controller card is replaced or swapped. Follow the instructions below in order to detect and fix the incorrect Golden Image software on the SSD:

A. Detect incorrect Golden Image software version

Follow these instructions before a Controller card is replaced or swapped:

1. Download ncs1004-mini-x.iso from https://software.cisco.com/download/home/286323463/type/280805694/release. Match the version to the current XR software version on your system.
2. Generate the md5sum value of the ncs1004-mini-x.iso that was downloaded in the previous step.
3. Use these commands in order to compare the md5sum value of the files from sysadmin#:
   • sysadmin-vm:0_RP0# run md5sum /cpu-dr-backup/cpu_image_bkp/ncs1004-mini-x.iso
   • sysadmin-vm:0_RP0# run md5sum /mb-boot/boot/install-image.iso

If the md5sum values match, replacement of the Controller card will not cause the reload loop issue.

B. Fix incorrect Golden Image software version

If the md5sum values do not match, copy the ncs1004-mini-x.iso that you downloaded in step A.1. and replace the file with the destination name as shown below:

• sysadmin-vm:0_RP0# run ls -l /mb-boot/boot/install-image.iso
• sysadmin-vm:0_RP0# run ls -l /cpu-dr-backup/cpu_image_bkp/ncs1004-mini-x.iso

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email or telephone

Receive Email Notification For New Field Notices

My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.