Have you been looking for a better way to model your network infrastructure? Check out what we're doing with NetBox! Open source and widely extensible, NetBox has enabled thousands of organizations to automate their networks like never before possible.

First impressions of the Nexus 5000

By stretch | Monday, April 6, 2009 at 3:13 a.m. UTC


About a year ago, Cisco announced the Nexus 5000, a combined 10 Gbps Ethernet and Fibre Channel aggregation switch targeted for datacenter deployment. The idea is that by carrying SAN Fibre Channel traffic across a high-speed Ethernet infrastructure (using Fibre Channel over Ethernet), operational and administrative costs can be reduced.

Recently I had the opportunity to experiment with a Nexus 5020 equipped with 40 onboard 10 Gbps Ethernet interface and an expansion module sporting an additional four 10 Gbps Ethernet and four native Fibre Channel interfaces. I wanted to share my experience for those who are planning on getting more familiar with the platform in the near future.

The Nexus lines runs an entirely new operating system dubbed NX-OS, which bears zero relation to IOS, and in fact runs on a Linux kernel. As such, one can catch a few hints of its open source heritage in the initial boot sequence:

Booting kickstart image: bootflash:/n5000-uk9-kickstart.4.0.1a.N1.1.bin....
......................................................Image verification OK

Starting kernel...
Usage: init 0123456SsQqAaBbCcUu
INIT: version 2.85 booting
Starting Nexus5020 POST...
  Executing Mod 1 1 SEEPROM Test......done
  Executing Mod 1 1 GigE Port Test.......done
  Executing Mod 1 1 Inband GigE Test.....done
  Executing Mod 1 1 NVRAM Test....done
  Executing Mod 1 1 PCIE Test..............................done
  Mod 1 1 Post Completed Successfully
  Executing Mod 1 2 SEEPROM Test....done
  Mod 1 2 Post Completed Successfully
  Mod 2 Post Completed Successfully
POST is completed
Checking all filesystems..... done.


Nexus 5000 Switch
switch login: admin
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software may be covered under the GNU Public
License or the GNU Lesser General Public License. A copy of 
each such license is available at
http://www.gnu.org/licenses/gpl.html and

Although a completely different animal under the hood, NX-OS presents the user with a very IOS-like interface. If you're reasonably familiar with IOS, you're sure to find the NX-OS CLI very comfortable.

To begin with, you can view the running configuration of an NX-OS device just as with IOS. While the output has been rearranged slightly, you'll notice many lines have been ported verbatim from IOS:

switch# sh run
version 4.0(1a)N1(1)
role name access-admin
  rule 1 permit read-write  
username admin password 5 $1$nMsnY4cy$GOflFn9RQ2kZkPjTENO7T/  role network-admin
ssh key rsa 1024 force
ip host switch
snmp-server user admin network-admin auth md5 [removed] priv [removed]
snmp-server host version 2c public  udp-port 1163
vrf context management
  ip route
vlan 1
fcdomain fcid database
  vsan 1 wwn 10:00:00:00:c9:66:22:78 fcid 0x990002 dynamic

interface Ethernet1/1

interface Ethernet1/2


interface Ethernet1/39

interface Ethernet1/40

interface Ethernet2/1

interface Ethernet2/2

interface Ethernet2/3

interface Ethernet2/4

interface mgmt0
  ip address
snmp-server enable traps license
boot kickstart bootflash:/n5000-uk9-kickstart.4.0.1a.N1.1.bin
boot system bootflash:/n5000-uk9.4.0.1a.N1.1.bin 

show version provides all the information you'd expect, as well as an unnecessarily granular reset clock:

switch# sh version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at

  BIOS:      version 1.2.0
  loader:    version N/A
  kickstart: version 4.0(1a)N1(1)
  system:    version 4.0(1a)N1(1)
  BIOS compile time:       06/19/08
  kickstart image file is: bootflash:/n5000-uk9-kickstart.4.0.1a.N1.1.bin
  kickstart compile time:  12/3/2008 5:00:00 [12/03/2008 13:08:37]
  system image file is:    bootflash:/n5000-uk9.4.0.1a.N1.1.bin
  system compile time:     12/3/2008 5:00:00 [12/03/2008 13:44:18]

  cisco Nexus5020 Chassis ("40x10GE/Supervisor")
  Intel(R) Celeron(R) M CPU    with 2074308 kB of memory.
  Processor Board ID [removed]

Device name: switch
  bootflash:    1003520 kB

Kernel uptime is 0 day(s), 0 hour(s), 7 minute(s), 40 second(s)

Last reset at 516981 usecs after  Tue Mar 31 21:06:52 2009

Reason: Reset Requested by CLI command reload
  System version: 4.0(1a)N1(1)

  Core Plugin, Ethernet Plugin

Our old stomping grounds of global configuration mode are revisited as well:

switch# configure
switch(config)# hostname Nexus_5K

Notice that specifying the terminal parameter after configure is unnecessary in NX-OS, as it's default (although, thankfully, typing config t will be silently accepted just as well). There are a few other nice (albeit long overdue) improvements to the CLI over IOS, such as the ability to finally use CIDR notation (e.g. in IP address assignment and access lists:

Nexus_5K(config)# int mgmt0
Nexus_5K(config-if)# ip address

This alone seems like it could be a primary selling point for the Nexus line.

You may be wondering about the mgmt interface type. When I first saw this I thought back to CatOS' sc0 interface, a virtual layer three interface which exists solely for management purposes, and that's essentially the function of the mgmt0 interface on the Nexus 5000 (remember that the 5000 is a layer two-only device). Although none of the physical interfaces can be addressed at layer three, the 5000 does have the ability to perform per-interface and per-VLAN filtering (with access lists) up to layer four.

One of the 5000's primary features, beside raw speed, is the ability to consolidate Ethernet and Fibre Channel networks using FCoE. As mentioned above, the Nexus 5000 used in this lab has four native Fibre Channel interfaces, though an observant reader might have noted them absent from display of the running configuration.

This is because FCoE is a separately licensed, independent feature of NX-OS, and must be explicitly enabled:

Nexus_5K(config)# feature ?
  fcoe            Enable feature
  interface-vlan  Enable/Disable interface vlan
  lacp            Enable/Disable LACP
  private-vlan    Enable/Disable private-vlan
  tacacs+         Enable/Disable tacacs+
  udld            Enable/Disable UDLD

Nexus_5K(config)# feature fcoe
Nexus_5K(config)# 2009 Mar 31 21:46:26 Nexus_5K %$ VDC-1 %$ %PLATFORM-2-FC_LICENSE_DESIRED:
FCoE/FC feature will be enabled after the configuration is saved followed by a reboot

After rebooting the device (with reload), we can verify that our Fibre Channel interfaces now appear:

Nexus_5K# sh int brief

Interface  Vsan   Admin  Admin   Status          SFP    Oper  Oper   Port
              Mode   Trunk                          Mode  Speed  Channel
                     Mode                                 (Gbps)
fc2/1      1      auto   on      down             swl    --           --
fc2/2      1      auto   on      down             swl    --           --
fc2/3      1      auto   on      sfpAbsent        --     --           --
fc2/4      1      auto   on      sfpAbsent        --     --           --

Ethernet      VLAN   Type Mode   Status  Reason                   Speed     Port
Interface                                                                   Ch #
Eth1/1        1      eth  access up      none                        10G(D) --
Eth1/2        1      eth  access down    SFP not inserted            10G(D) --
Eth1/3        1      eth  access down    SFP not inserted            10G(D) --
Eth1/4        1      eth  access down    SFP not inserted            10G(D) --

Not all features are separately licensed. However, most will need to be explicitly enabled, as, unlike IOS, NX-OS is implemented in truly independent processes with their own memory space (again, it's powered by a Linux kernel). This provides for more efficient use of available memory, and provides exponentially greater protection against the emergence of rootkits targeting network infrastructure.

Obviously there's a lot more to the 5000 platform, but the goal here is to provide a measurable amount of exposure so that you have some idea what to expect if or when you encounter the platform for the first time. Check out the official Nexus 5000 configuration guide for a much more comprehensive review.

Also, if you didn't see it on Twitter, here's short video of some of the example CLI interactions above:

Posted in Hardware

Support PacketLife by buying stuff you don't need!


April 6, 2009 at 9:20 a.m. UTC

Interesting with a little introduction on NX-OS. The CIDR notation is very welcome.

April 6, 2009 at 1:09 p.m. UTC

Nice overview. From what I remember hearing a lot of the modular and linux like features were pulled in from the SAN-OS used on the fiber channel switches. And I have to agree, CIDR notation is an extremely welcome change (maybe we can look forward to that in 12.5? ;-).

April 6, 2009 at 4:05 p.m. UTC

Why not go with BSD? They could avoid all their legal troubles they've been having with their Linksys line against the Free Software Foundation.

April 6, 2009 at 4:47 p.m. UTC

For anyone interested in history, NX-OS is actually just SAN-OS renamed. SAN-OS, along with the Cisco MDS Fiberchannel switches, were developed by Nuova, one of Cisco's famous spin-off companies that it reacquired later.

April 6, 2009 at 11:17 p.m. UTC

Not that I condone this, but I wonder how long it will take someone to hack this up as its own distro. I know, it probably has some encrypted chip in it that it checks against, but it would still be interesting.

April 7, 2009 at 7:23 a.m. UTC

I noticed that IOS XR also support CIDR notation.

April 7, 2009 at 11:22 a.m. UTC

I love the look of it, also i agree it looks like it would not be hard to get used to.


love the site :)

April 8, 2009 at 1:20 p.m. UTC


Actually that's what FOSS is all about. The changes between the GPLv2 and GPLv3 (Linux still uses v2) explicitly adds that end-users must have the ability to not only change the code, but use that code to change how the device operates. I think they called it tivoization, where tivo released their code (as required by the GPL) but had hardware checks that would not allow any modified code to run. This leads back to why I'm confused why Cisco would go with Linux rather than BSD, which does not use the GPL license.

Either way nice write-up, stretch. Now that they've moved from a micro to a monolithic kernel, what kind of boot time are you seeing compared to traditional IOS-based switches?

April 8, 2009 at 2:26 p.m. UTC

you should have mentioned "show run all" which includes the defaults :)

Comments have closed for this article due to its age.