NFVIS Integration with Docker Container Lifecycle

Docker container lifecycle infrastructure is developed in NFVIS for Cisco SD-WAN Cloud OnRamp for Colocation solution. Container lifecycle APIs are developed to bring up docker containers.

Cisco Colo Manager

Cisco colo manager (CCM) is a software stack managing switches in colo. In the Cisco SD-WAN Cloud OnRamp for Colocation solution, CCM is hosted on NFVIS software in a Docker container. CCM is hosted on the CSP devices along with VNFs and there are no dedicated CSP devices for hosting CCM. CCM is used to configure and provision PNFs (switches) in this solution.

Cisco colo manager (CCM) is bundled along with the Cisco NFVIS software which is used as the base virtualization infrastructure software running on the compute platform. The NFVIS software provides programmable Rest and netconf APIs and an orchestrator can use these APIs to configure and monitor the system, instantiate virtual network functions and configure the VNF networks and service chains. As part of colo provisioning for the orchestrator, vManage selects one device in the colo and sends netconf action command to bring up the CCM container. The CCM container is connected to the colo management network. This management network is used to transfer files and images into and out of the systems. This network will not be used for the normal customer data traffic.

CCM State Transitions from the Host Side

vManage brings up CCM on one of the CSP devices in the Network Hub solution. CCM state transitions are seen on the host side, using the container life-cycle model's state operation.

The CCM state on the host side has the following states:

  • Starting : The container is booting up and starting

  • Healthy : The container health monitoring has started and container is healthy. The container operational state is set to healthy

  • Unhealthy : If the CCM does not boot properly, the CCM container is not usable and needs to be recovered. CCM in unhealthy state can be due to docker daemon not running, CCM is not configured with correct management IP address, gateway or CCM cannot respond to ping.

The starting state can only be seen when the container is brought up or re-spun. Healthy and unhealthy states can transition to each other during the lifetime of the container. A notification is also sent whenever the CCM state changes.


Note

The CCM container state is tracked through container life-cycle model as one of the containers. This is not CCM-state or CCM-status oper. The state for container named ColoMgr is used for CCM state transitions.


State

Action/config can be pushed

config status queried

oper model on host

notification for CCM state

Starting

No

No

Yes

Yes

Unhealthy

No

No

Yes

Yes

Healthy

Yes

Yes

Yes

Yes

PNF device list is sent from vManage to the NFVIS hosting CCM when CCM is in healthy state.

To verify CCM state, when Colo Manager crashes on a CSP device use support show container command:


CSP# support show container 
Possible completions:
  docker-container-ls   Lists all containers
  docker-info           Lists docker daemon info
  docker-inspect        Inspect container or volume
  docker-volume-ls      Lists all volumes
  dump                  Dumps all container related info

CCM Notifications

CCM health check sends CCM state transitions to vManage notification stream.

You can view the CCM event notifications using the show notification stream vmanageEvent command.

Event Type

Notification Trigger

Notification Output Example

ccmEvent - CCM-STATUS (init, in-progress, wait, error, ready)

notification 

 eventTime 2018-06-29T01:58:55.767142+00:00

 ccmEvent 

  severity-level minor

  host-name ccm

  user-id nso_user

  config-change false

  transaction-id 0

  status SUCCESS

  status-code 0

  status-message INIT

  details CCM status :INIT

  event-type CCM-STATUS

 !

CCM Recovery

When CCM is up, the Catalyst 9000 series switches are onboard successfully and CCM is restarted on the same or different CSP, the CCM recovery is initiated.

vManage brings down CCM and then brings it up again. vManage sends the device list with passwords for the switches along with all the service configurations. CCM then uses these configurations to sync with the device.

Recovery flag for device action list - false for day0, true for recovery (mandatory).

Static IP change for device action list - IP addresses for devices is sent all the time - day0 and recovery.

Support Commands

To verify the CCM version use support show ccm-version :


CSP# support show ccm-version
Cisco Colo Manager (CCM)
Version 0.0.1-150
Build date Tue 06 Nov 2018 09:09:28 AM UTC

To verify the firewall state use support show firewall :


CSP# support show firewall 
Possible completions:
  list-forward-ports   Lists all port forwarding rules
  state                Lists firewalld daemon status

To display information about OVS switch use support ovs vsctl show :


CSP# support ovs vsctl show 
Possible completions:
  |  <cr>
CSP2# support ovs vsctl show
107a6588-62f1-411f-b5da-fa0fd39f2500
    Bridge ovs-data-br
        Port bond-bond_data
            tag: 1
            Interface "eth2-3"
            Interface "eth2-4"
        Port ovs-data-br
            Interface ovs-data-br
                type: internal
    Bridge ovs-ha-br
        Port bond-bond_ha
            tag: 1
            Interface "eth2-2"
            Interface "eth2-1"
        Port ovs-ha-br
            Interface ovs-ha-br
                type: internal
    Bridge int-mgmt-net-br
        Port colo-mgmt
            Interface colo-mgmt
                type: internal
        Port mgmt-bond
            Interface "eth0-2"
            Interface "eth0-1"
        Port int-mgmt-net-br
            Interface int-mgmt-net-br
                type: internal
    ovs_version: "2.5.2"

To display the list of NFVIS system settings use show system:system settings-native :


system:system settings-native mgmt ip-info interface colo-mgmt
system:system settings-native mgmt ip-info ipv4_address 192.168.30.163
system:system settings-native mgmt ip-info netmask 255.255.255.0
system:system settings-native mgmt ip-info link-local ipv6 address ::
system:system settings-native mgmt ip-info link-local ipv6 prefixlen 0
system:system settings-native mgmt ip-info global ipv6 address ::
system:system settings-native mgmt ip-info global ipv6 prefixlen 0
system:system settings-native mgmt ip-info mac_address b2:5d:28:aa:f1:96
system:system settings-native mgmt ip-info mtu 1500
system:system settings-native mgmt ip-info txqueuelen 1000
system:system settings-native mgmt stats rx_packets 7140693
system:system settings-native mgmt stats rx_bytes 767558248
system:system settings-native mgmt stats rx_errors 0
system:system settings-native mgmt stats rx_dropped 2
system:system settings-native mgmt stats rx_overruns 0
system:system settings-native mgmt stats rx_frame 0
system:system settings-native mgmt stats tx_packets 5259073
system:system settings-native mgmt stats tx_bytes 1008512311
system:system settings-native mgmt stats tx_errors 0
system:system settings-native mgmt stats tx_dropped 0
system:system settings-native mgmt stats tx_overruns 0
system:system settings-native mgmt stats tx_carrier 0
system:system settings-native mgmt stats tx_collisions 0
system:system settings-native domain NA
system:system settings-native dns nameserver1 0.0.0.0
system:system settings-native dns nameserver2 0.0.0.0
system:system settings-native dns nameserver3 0.0.0.0
system:system settings-native hostname CSP2
system:system settings-native gateway ipv4_address 192.168.30.1
system:system settings-native gateway interface colo-mgmt
system:system settings-native gateway-ipv6 ipv6_address ::
system:system settings-native gateway-ipv6 interface NA
system:system settings-native trusted-source [ "not set" ]
system:system settings-native source-interface 0.0.0.0

To display information about a bond use support ovs appctl bond-show mgmt-bond


CSP2# support ovs appctl bond-show mgmt-bond
---- mgmt-bond ----
bond_mode: balance-slb
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 252 ms
lacp_status: negotiated
active slave mac: 00:fc:ba:d7:39:86(eth0-1)

slave eth0-1: enabled
        active slave
        may_enable: true
        hash 242: 8 kB load

slave eth0-2: enabled
        may_enable: true

To display the IP routing statistics use support show route :


CSP# support show route
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         gateway         0.0.0.0         UG        0 0          0 colo-mgmt
172.16.255.22   127.0.1.254     255.255.255.255 UGH       0 0          0 tun_0_0
192.168.30.0    0.0.0.0         255.255.255.0   U         0 0          0 colo-mgmt