Replacement of Nexus 9236C Spine Switch - CPAR
Available Languages
Contents
Introduction
This document describes the steps required to replace a faulty Spine switch (Nexus 9236C) in an Ultra-M setup.
This procedure applies for an Openstack environment through NEWTON version where ESC does not manage Cisco Prime Access Registrar (CPAR) and CPAR is installed directly on the VM deployed on Openstack.
Background Information
Ultra-M is a pre-packaged and validated virtualized mobile packet core solution designed to simplify the deployment of VNFs. The servers that are part of the Ultra-M setup are connected to three different types of switches :
- Catalyst Switch
- Leaf Switch
- Spine Switch
This image shows the network topology of a Ultra-M setup:
This document is intended for the Cisco personnel who are familiar with Cisco Ultra-M setup and Catalyst Switch operations.
Abbreviations
| VNF | Virtual Network Function |
| SPINE | Nexus 9236C Switch as Spine |
| MOP | Method of Procedure |
| LAN | Local Area Network |
| FTP | File Transfer Protocol |
| TFTP | Trivial File Transfer Protocol |
| CIMC | Cisco Integrated Management Controller |
Workflow of the MoP
This image shows the high level workflow of Replacement Procedure.
Spine Switch in UltraM Setup
Prerequisite
Take a backup of the configuration file from the Spine switch through FTP/TFTP before it proceedes with the switch replacement.
AL03-N9236c-POD2-spine1# copy running-config sftp:
Enter destination filename: [AL03-N9236c-POD2-spine1-running-config] backup-spine-cfg-2
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the sftp server: 10.1.1.1
Enter username: root
The authenticity of host '10.1.1.1 (10.1.1.1)' can't be established.
ECDSA key fingerprint is SHA256:dnHSXQ6fyYX5v8DU87oqVkCoA8CSPW7CpT+o2qilNI4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.1.1.1' (ECDSA) to the list of known hosts.
root@10.1.1.1's password:
Connected to 10.1.1.1.
sftp> put /var/tmp/vsh/backup-spine-cfg-2 backup-spine-cfg-2
Uploading /var/tmp/vsh/backup-spine-cfg-2 to /root/backup-spine-cfg-2
/var/tmp/vsh/backup-spine-cfg-2 0% 0 0.0KB/s --:-- ETA /var/tmp/vsh/backup-spine-cfg-2 100% 11KB 11.2KB/s 00:00
sftp> exit
Copy complete, now saving to disk (please wait)...
Copy complete.
Health Checks
Step 1. Check the current software version that runs in the switch and make a note of it.
AL03-N9236c-POD2-spine1# show ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2018, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source. This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0 or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.
Software
BIOS: version 07.59
NXOS: version 7.0(3)I7(3)
BIOS compile time: 08/26/2016
NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin
NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]
Hardware
cisco Nexus9000 C9236C chassis
Intel(R) Xeon(R) CPU @ 1.80GHz with 16400560 kB of memory.
Processor Board ID FDO211613CA
Device name: AL03-N9236c-POD2-spine1
bootflash: 53298520 kB
Kernel uptime is 134 day(s), 14 hour(s), 48 minute(s), 25 second(s)
Last reset at 705173 usecs after Fri Mar 2 17:56:31 2018
Reason: Reset due to upgrade
System version: 7.0(3)I5(2)
Service:
plugin
--More-- Core Plugin, Ethernet Plugin
Active Package(s):
Step 2. Check the current license usage.
AL03-N9236c-POD2-spine1# show license uasgesage
Feature Ins Lic Status Expiry Date Comments
Count
--------------------------------------------------------------------------------
N9K_LIC_1G No - Unused -
VPN_FABRIC No - Unused -
FCOE_NPV_PKG No - Unused -
SECURITY_PKG No 0 Unused -
N9K_UPG_EX_10G No - Unused -
TP_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_GF No - Unused -
NXOS_ADVANTAGE_M4 No - Unused -
NXOS_ADVANTAGE_XF No - Unused -
NXOS_ESSENTIALS_GF No - Unused -
NXOS_ESSENTIALS_M4 No - Unused -
NXOS_ESSENTIALS_XF No - Unused -
SAN_ENTERPRISE_PKG No - Unused -
PORT_ACTIVATION_PKG No 0 Unused -
NETWORK_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_M8-16 No - Unused -
NXOS_ESSENTIALS_M8-16 No - Unused -
FC_PORT_ACTIVATION_PKG No 0 Unused -
LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never -
--------------------------------------------------------------------------------
Step 3. Ensure that the physical cables connected to the switch port-channel, Vlan and also the port status are good as expected.
AL03-N9236c-POD2-spine1# show interface status | grep connected
mgmt0 -- connected routed full 100 --
Eth1/1 Connected to leaf1 connected trunk full 100G QSFP-100G-AOC5M
Eth1/2 Connected to leaf1 connected trunk full 100G QSFP-100G-AOC5M
Eth1/4 Connected to leaf2 connected trunk full 100G QSFP-100G-AOC5M
Eth1/5 Connected to leaf2 connected trunk full 100G QSFP-100G-AOC5M
Eth1/7 Connected to Leaf3 connected trunk full 100G QSFP-100G-AOC5M
Eth1/8 Connected to Leaf3 connected trunk full 100G QSFP-100G-AOC5M
Eth1/10 connect leaf4 connected trunk full 100G QSFP-100G-AOC5M
Eth1/11 connect leaf4 connected trunk full 100G QSFP-100G-AOC5M
Eth1/21/1 Connected to NMNET connected 101 full 10G QSFP-40G-SR4
Eth1/29 Connected to AL04- connected trunk full 100G QSFP-100G-AOC5M
Eth1/30 Connected to AL04- connected trunk full 100G QSFP-100G-AOC5M
Eth1/36/1 connected to ARG | connected routed full 10G QSFP-40G-SR4
Eth1/36/2 connected to ARG | disabled routed auto auto QSFP-40G-SR4
Eth1/36/3 connected to ARG | disabled routed auto auto QSFP-40G-SR4
Eth1/36/4 connected to ARG | disabled routed auto auto QSFP-40G-SR4
Po10 -- connected routed full 10G --
Po10.3101 -- connected routed full 10G --
Po10.3109 -- connected routed full 10G --
Po18 -- connected trunk full 100G --
Po21 -- connected trunk full 100G --
Po22 -- connected trunk full 100G --
Po25 -- connected trunk full 100G --
Po26 -- connected trunk full 100G --
Lo1 -- connected routed auto auto --
Vlan101 -- connected routed auto auto --
Vlan1025 vPCRF 1 and vPCRF connected routed auto auto --
Vlan1026 Replication for vP connected routed auto auto --
Vlan1031 External Traffic v connected routed auto auto --
Vlan1032 External External connected routed auto auto --
Vlan1033 External Traffic connected routed auto auto --
Vlan1034 External Traffic v connected routed auto auto --
Vlan1035 External Traffic v connected routed auto auto --
Vlan1036 External Traffic v connected routed auto auto --
Vlan1041 External Traffic A connected routed auto auto --
Vlan1042 External Traffic A connected routed auto auto --
Vlan1043 External Traffic A connected routed auto auto --
Vlan2601 Global Interchassi connected routed auto auto --
Vlan2602 NMNET Interchassis connected routed auto auto --
Step 4. Ensure that BGP neighbors are up.
AL03-N9236c-POD2-spine1# show ip bgp summary vrf all
BGP summary information for VRF NMNET, address family IPv4 Unicast
BGP router identifier 10.225.247.201, local AS number 65361
BGP table version is 2, IPv4 Unicast config peers 2, capable peers 1
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.0.100.6 4 65361 193834 193833 2 0 0 19w1d 0
10.250.180.185 4 65137 0 0 0 0 0 19w1d Idle
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 10.0.200.1, local AS number 65361
BGP table version is 161, IPv4 Unicast config peers 2, capable peers 2
18 network entries and 29 paths using 5700 bytes of memory
BGP attribute entries [3/480], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.0.100.2 4 65361 3868481 3868419 161 0 0 19w1d 12
10.163.140.225 4 65137 3868521 3868365 161 0 0 19w1d 5
AL03-N9236c-POD2-spine1# show ipv6 bgp summary vrf all
Step 5. Ensure the BFD neighbors state is up.
AL03-N9236c-POD2-spine1# show bfd neighbors vrf all
OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf
10.163.140.226 10.163.140.225 1090519042/1090519042 Up 107(3) Up Po10.3101 default
AL03-N9236c-POD2-spine1# show bfd ipv6 neighbor vrf all
Switch Replacement procedure
Step 1. Install the new switch in the rack and connect the cables to the switch as noted. The steps for switch installation can be found in this link: Spine Nexus 236C NX-OS mode Switch Hardware Installation Guide
Step 2. Check the software version of the Nexus Spine switch and upgrade/downgrade to the software version as per the given link: Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide, Release 7.x
Step 3. Transfer the configuration backup to the new switch and copy that configuration to the startup-config.
server-backup$ sftp admin@10.1.1.1
The authenticity of host '10.1.1.1 (10.1.1.1)' can't be established.
RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.1.1.1' (RSA) to the list of known hosts.
User Access Verification
Password:
Password:
Connected to 10.1.1.1.
sftp> ls
20170607_193430_poap_15489_1.log 20170607_193430_poap_15489_2.log 20170607_193430_poap_15489_init.log backup-spine-cfg-2 backup-spine.cfg
backup_run_02152018 bios_daemon.dbg cfg-july25th cfg-july25th-spine1 flash:
flash:cfg-aug8th-ybattina-afterNSO flash:cfg-jul28th home l3-cfg-aug10th license_FDO211406K1_16.lic
lost+found nxos.7.0.3.I4.4.bin nxos.7.0.3.I5.2.bin nxos.7.0.3.I7.3.bin platform-sdk.cmd
scripts spine-config-bkp starat virtual-instance vlan.dat 100% 33KB 21.7KB/s 00:01
sftp> put backup-spine-cfg-2
Uploading backup-spine-cfg-2 to /backup-spine-cfg-2
backup-spine-cfg-2 100% 33KB 23.5KB/s 00:01
sftp> bye
AL03-N9236c-POD2-spine1# copy bootflash:///backup-spine-cfg-2 startup-config
Copy progress 100% 33KB
Copy complete, now saving to disk (please wait)...
Copy complete.
Step 4. Reload the switch after loading the backup configuration.
AL03-N9236c-POD2-spine1#reload
Verification of Replaced Spine Switch
After the switch is available for access, perform the health checks through these commands on the spine switch to verify that replaced switch is available in expected state.
AL03-N9236c-POD2-spine1# show port-channel summary
AL03-N9236c-POD2-spine1# show ip bgp summary vrf all
AL03-N9236c-POD2-spine1# show ipv6 bgp summary vrf all
AL03-N9236c-POD2-spine1# show bfd neighbor vrf all
AL03-N9236c-POD2-spine1# show bfd ipv6 neighbor vrf all
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)