Resolve Incorrect Nexus 9000 SSD Partitioning

Introduction

This document describes corrective action to be taken when incorrect SSD partitioning is seen on the Nexus 9000 Series Switch running NX-OS software.

Prerequisites

Requirements

It is recommended that users are familiar with NX-OS fundamentals, and recognize the Nexus 9000 hardware version and configuration present on the NX-OS to determine the corrective action.

Components Used

The information in this document is based on the Nexus 9000 Series switches running NX-OS software release 10.5(1) and later versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Enhancements were added starting in 10.5(1) to automatically detect SSD partition size on the Nexus 9000 to match the expected configured size.  If this syslog has been seen during bootup in show logging log or show logging nvram, this indicates the NX-OS Nexus 9000 booted with an unexpected SSD partitioning size.

%PLATFORM-2-SSD_PARTITION_CHECK: Incorrect <device> partition size detected - please contact Cisco TAC for additional information

Partition issues can happen for various reasons. Smaller partitions can lead to issues where NX-OS images cannot be saved to bootflash for upgrades, show techs cannot be saved properly, loss of important log history, bootup can fail due to insufficient free space, and so on. This document focuses on recovering the SSD partition to the correct size.

Recovery Steps

Cisco recommends you use this Nexus 9000 SSD partitioning method to recover. SSD Re-Partitioning has the best chance of preserving critical files. This recovery process can only be required once, if an incorrect partition is seen.  Subsequent reloads need to preserve this configuration. If the SSD Re-partitioning method does not resolve the incorrect SSD size, contact Cisco Technical Assistance Center (TAC).

Note: This requires the Nexus 9000 to reload.

SSD Re-Partitioning

The command system flash sda resize can be used to perform a re-partition of the SSD based on the partitioning scheme specified. Standard is the default SSD configuration on Nexus 9000 Cloud Scale Switches. Extended is the default SSD configuration on Nexus 9800 Switches.

switch# system flash sda resize ?
  <CR>     
  extended  Cfg=1GB, logflash=39GB
  standard  Cfg=64MB, logflash=4|8GB

The NX-OS system makes the best effort to preserve critical files by using this SSD Re-partitioning option.  The critical files include NX-OS boot image, running-config, bootflash contents, and logflash contents. 

Caution: Cisco recommends ALWAYS backing up critical files to an external source.

Review the detailed explanation in the SSD Re-partitioning section of the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide, Release 10.4(x).

Additional Verification/Identification

This is an example of a Nexus 9000 with SSD partitioning that is incorrect.  When running the command system flash sda resize standard, you can see that the current scheme partitions do not match the target schemes for a standard partition configuration.

switch# system flash sda resize standard

!!!! WARNING !!!!

Attempts will be made to preserve drive contents during
the resize operation, but risk of data loss does exist.
Backing up of bootflash, logflash, and running configuration
is recommended prior to proceeding.

!!!! WARNING !!!!


current scheme is
sda 8:0 0 119.2G 0 disk
|-sda1 8:1 0 1.2G 0 part
|-sda2 8:2 0 9.6G 0 part /mnt/plog
|-sda3 8:3 0 1.2G 0 part /mnt/pss
|-sda4 8:4 0 11.9G 0 part /bootflash
|-sda5 8:5 0 1.2G 0 part /mnt/cfg/0
|-sda6 8:6 0 1.2G 0 part /mnt/cfg/1
|-sda7 8:7 0 39.4G 0 part /logflash
|-sda8 8:8 0 11.9G 0 part
`-sda9 8:9 0 23.9G 0 part


target scheme is
sda 8:0 0 64G|120GB|250GB 0 disk
|-sda1 8:1 0 512M 0 part
|-sda2 8:2 0 32M 0 part /mnt/plog
|-sda3 8:3 0 128M 0 part /mnt/pss
|-sda4 8:4 0 110.5G 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
|_sda7 8:7 0 8G 0 part /logflash

To verify that a Nexus 9000 SSD partitioning has been corrected, you can use the same command system flash sda resize standard to verify that the current scheme partitions match the target scheme.  Additionally, you can see a message at the bottom of the command indicating that the system is already in standard scheme. 

switch# system flash sda resize standard

!!!! WARNING !!!!

Attempts will be made to preserve drive contents during
the resize operation, but risk of data loss does exist.
Backing up of bootflash, logflash, and running configuration 
is recommended prior to proceeding.

!!!! WARNING !!!!


current scheme is 
sda 8:0 0 119.2G 0 disk 
|-sda1 8:1 0 512M 0 part 
|-sda2 8:2 0 32M 0 part /mnt/plog
|-sda3 8:3 0 128M 0 part /mnt/pss
|-sda4 8:4 0 110.5G 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
`-sda7 8:7 0 8G 0 part /logflash


target scheme is 
sda 8:0 0 64G|120GB|250GB 0 disk 
|-sda1 8:1 0 512M 0 part 
|-sda2 8:2 0 32M 0 part /mnt/plog 
|-sda3 8:3 0 128M 0 part /mnt/pss 
|-sda4 8:4 0 rem 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
|_sda7 8:7 0 8G 0 part /logflash


System is already in Standard scheme Use force option to proceed

Additionally, these commands can be used to view and verify that the Nexus 9000 SSD partitionings are configured as expected.

switch# show system internal flash | i i dev/sda
/bootflash 113795280 38647924 75147356 34 /dev/sda4
/cmn/cfg/0 58090 872 53942 2 /dev/sda5
/cmn/cfg/1 58090 872 53942 2 /dev/sda6
/cmn/pss 121299 10546 104200 10 /dev/sda3
/mnt/cfg/0 58090 872 53942 2 /dev/sda5
/mnt/cfg/1 58090 872 53942 2 /dev/sda6
/mnt/plog 27252 16025 9589 63 /dev/sda2
/mnt/pss 121299 10546 104200 10 /dev/sda3
/logflash 8107484 1501688 6186368 20 /dev/sda7

Note: feature bash-shell must be enabled globally to use bash-shell.

switch# run bash
bash-4.4$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 165.8M 0 loop /usr_ro
loop1 7:1 0 116.9M 0 loop /isan_lib_ro
loop2 7:2 0 48.2M 0 loop /isan_bin_ro
loop3 7:3 0 43.9M 0 loop /isan_bin_eth_ro
loop4 7:4 0 11.3M 0 loop /isan_lib_eth_ro
loop5 7:5 0 4.2M 0 loop /isan_lib_n9k_ro
loop6 7:6 0 4K 0 loop /isan_bin_n9k_ro
loop7 7:7 0 195.3M 0 loop /bootflash/.rpmstore/patching
loop8 7:8 0 57.6M 0 loop 
loop9 7:9 0 144.4M 0 loop 
loop10 7:10 0 221.2M 0 loop 
sda 8:0 0 119.2G 0 disk 
├─sda1 8:1 0 512M 0 part 
├─sda2 8:2 0 32M 0 part /mnt/plog
├─sda3 8:3 0 128M 0 part /mnt/pss
├─sda4 8:4 0 110.5G 0 part /bootflash
├─sda5 8:5 0 64M 0 part /mnt/cfg/0
├─sda6 8:6 0 64M 0 part /mnt/cfg/1
└─sda7 8:7 0 8G 0 part /logflash
mmcblk0 179:0 0 3.7G 0 disk 
├─mmcblk0p1 179:1 0 32M 0 part 
├─mmcblk0p2 179:2 0 32M 0 part 
├─mmcblk0p3 179:3 0 32M 0 part /mnt/pstore
└─mmcblk0p4 179:4 0 3.6G 0 part