Contents

Introduction

Background

Prerequisites

Requirements

Components Used

Troubleshooting tips

Introduction

This document helps you understand average memory utilization on Nexus 3000 device.

Contributed by Narottama Sahoo, Santhosh Shankar, Cisco TAC Engineers

Background

The memory utilization in nexus 3000 likely vary based on the version running on the device.

In newer versions due to new feature addition the memory usage will likely be high when compared to older version

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

- Nexus command line interface

- Understanding how flash and RAM memory works in computer world

Components Used

The information in this document is based on these software and hardware versions:

- Nexus 3000 series platform

- Code version 6.x and 7.x

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Troubleshooting tips

Below are some sample outputs from my lab (minimally configured N3548) demonstrates baseline memory utilization on the platform.

N3K-3548.42# sh system resources | egrep "used"

Memory usage:   4117860K total,   2250220K used,   1867640K free

N3K-3548.42# sh system internal kernel memory global | be NAME | cut -c 1-42

NAME            |TOTAL                   |

mtc_usd         | 462744  231212  223972 |

fwm             | 277132   97788   84907 |

afm             | 218180   53060   40829 |

netstack        | 380252   45592   38952 |

clis            | 219208   41924   32060 |

m2rib           | 182944   35812   31502 |

--More--

All running processes will consume some memory for text/heap/stack/data/shared-lib.   The above CLI reports proportional memory utilization per process (takes into account memory pages shared between processes).  By adding the proportional memory usage of all processes, we can get a representation of userspace processes memory usage.  On lab switch, the baseline utilization sum for all running processes was approximately 42% of overall utilization (943276 kB).

N3K-3548.42# sh system internal kernel meminfo | egrep "ShmFS|^Cached"

Cached:       1174236 kB

ShmFS:        1169840 kB

NX-OS binaries / configurations / etc are extracted from kickstart/system images during boot.  These reside in temporary file systems not backed by persistent storage and are accounted for by linux as ShmFS.

In addition to ShmFS usage, linux will temporarily cache file contents being read/referenced by processes.  The Cached counter is a sum of ShmFS and cached files.   From the above output, you can see that Cached usage accounts for approximately 52% of overall utilization (1174236 kB).

N3K-3548.42# sh system internal kernel meminfo | egrep "Slab"

Slab:           85376 kB

Finally, the kernel also uses some memory for internal data structures.  Slab accounts for kernel and kernel_module data structures.  From the above output, Slab usage accounts for approximately 4% of overall utilization (85376 kB).

There is some additional memory utilization that is not broken out by the linux kernel, but it only accounts for ~50mb.  For more information, please refer linux architecture documentation.

As a proportion of baseline utilization, temporary file systems are the largest memory consumer.  This is expected behavior and common among all NX-OS based platforms.

For NX-OS running 5.X average utlilization will be around 40%

For NX-OS running 6.X average utilization ranges between 60-70%

For NX-OS running 7.X average utlilization will bearound 80%

Therefore, the increased memory usage is likely a factor of new features in each release and a larger base footprint for the OS.

Sample output collected without any/default config on the device

Version: 6.0(2)U3(7)

switch# show system resources module all

CPU Resources:

-----------------------------------------------------------

CPU utilization: Module 5 seconds 1 minute 5 minutes

-----------------------------------------------------------

1         7        6        9

-----------------------------------------------------------

Processor memory: Module Total(KB) Free(KB) % Used

-----------------------------------------------------------

1   3785276   1450540    61             (Usage when running 6.x version)

Version:7.0.3.I2.2a

switch# show system resources module all

CPU Resources:

----------------------------------------------------------

CPU utilization: Module 5 seconds 1 minute 5 minutes

-----------------------------------------------------------

1     6        8         8

-----------------------------------------------------------

Processor memory: Module Total(KB) Free(KB) % Used

-----------------------------------------------------------

1    3903900   795600      79            (Usage when running 7.x version)

Recommendations:

Always refer Release notes to understand the system requirement for every software. The hardware and software requirement espescially the memory vary while upgrading to a newer version of code.

Useful links:

Release notes