Cumulus Linux: First Impressions

Wednesday, October 1, 2014 at 2:05 a.m. UTC by stretch

Typically, when you buy a network router or switch, it comes bundled with some version of the manufacturer's operating system. Cisco routers come with IOS (or some derivative), Juniper routers come with Junos, and so on. But with the recent proliferation of merchant silicon, there seem to be fewer and fewer differences between competing devices under the hood. For instance, the Juniper QFX3500, the Cisco Nexus 3064, and the Arista 7050S are all powered by an off-the-shelf Broadcom chipset rather than custom ASICs developed in-house. Among such similar hardware platforms, the remaining differentiator is the software.

One company looking to benefit from this trend is Cumulus Networks. Cumulus does not produce or sell hardware, only a network operating system: Cumulus Linux. The Debian-based OS is built to run on whitebox hardware you can purchase from a number of partner Original Device Manufacturers (ODMs). (Their hardware compatability list includes a number of 10GE and 40GE switch models from different vendors.)

Cumulus Linux is, as the name implies, Linux. There is no "front end" CLI as on, for example, Arista platforms. Upon login you are presented with a Bash terminal and all the standard Linux utilities (plus a number of not-so-standard bits). Like any OS, Cumulus handles interactions with the underlying hardware and among processes.


Continue reading 8 comments

Preliminary Book Topics

Wednesday, August 13, 2014 at 11:46 p.m. UTC by stretch

As I announced earlier this summer, I'm working on writing a book targeted to people entering the field of computer networking. I've got a fair amount of content fleshed out already, but figured it might help to get some feedback on the tentative structure. The book is being written in a question-and-answer style, organized into chapters by subject.

Below is the preliminary table of contents. It's still very much a work in progress, but I'm curious what people think of this approach. Constructive criticism and suggestions for additional content are welcome!

Continue reading 49 comments

Replacing an MPLS WAN with an Internet VPN Overlay

Monday, July 14, 2014 at 1:03 p.m. UTC by stretch

I received an email last week from a reader seeking advice on a fairly common predicament:

Our CIO has recently told us that he wants to get rid of MPLS because it is too costly and is leaning towards big internet lines running IPSEC VPNs to connect the whole of Africa.

As you can imagine, this has caused a huge debate between the networks team and management, we run high priority services such as Lync enterprise, SAP, video conferencing etc. and networks feel we need MPLS for guaranteed quality for these services but management feels the Internet is today stable enough to run just as good as MPLS.

What is your take on the MPLS vs Internet debate from a network engineer's point of view? And more so, would running those services over Internet work?

This is something I struggled with pretty frequently in a prior job working for a managed services provider. MPLS WANs are great because they provide flexible, private connectivity with guaranteed throughput. Most MPLS providers also allow you to choose from a menu of QoS schemes and classify your traffic so that real-time voice and video services are treated higher preference during periods of congestion.

Unfortunately, MPLS WANs tend to be considerably more expensive than Internet circuits. A dedicated 3 Mbps MPLS circuit might cost three or four times as much as a 50 Mbps business class broadband Internet circuit: These numbers are hard to justify to management who may not appreciate the contexts of reliability and QoS controls. Since private connectivity can be achieved using a VPN overlay on top of plain Internet circuits, can we still justify the cost MPLS WANs? Should we?

My advice would be to stick with the MPLS WAN if you can afford it. A VPN overlaid on top of Internet circuits might work most of the time, but when it doesn't perform adequately, you'll have little immediate recourse. Should you decide on moving to a VPN overlay, do so in phases: Keep the MPLS WAN around for a few months in case the overlay strategy doesn't work out. But if you find that your Internet circuits provide sufficient throughput so that congestion of real-time services never becomes a problem, maybe that's an acceptable solution.


Beyond the Blog

Wednesday, June 25, 2014 at 2:00 a.m. UTC by stretch

I'm thinking about writing a book.

Obviously, there are a lot of networking books on the market today. Search for any mainstream certification on Amazon and you'll find titles from half a dozen publishers. The majority of these are oriented toward specific vendors (most commonly Cisco) and many parallel a given certification exam. These books are overall pretty great. Most of them.

There also exists a minority of books which cover topics outside of the vendor-driven mainstream, like Gary A. Donahue's Network Warrior published by O'Reilly, now in its second edition. I love this kind of independent title because its content isn't constrained to a particular mold. The author finds stuff he thinks is relevant and interesting, and he writes about it. This is the correct way to write a book.

But over the past few years it has become painfully evident to me that there are many areas of this field we simply don't talk about in print, at least not at the entry level where perhaps it would be most helpful. If you want a thirty-page lecture on subnetting or a terrible mnemonic for the OSI model, pick any CCNA book from the pile and you're good to go. But what if you've never set foot inside a data center and want to know what it's like? What if you're trying to decide between Cisco and Juniper for your first ever network deployment? What if you think change management means you're getting a new boss?

Continue reading 45 comments

PSA: Global IPv4 Routing Table Hits 500k Routes

Tuesday, May 6, 2014 at 10:01 p.m. UTC by stretch

Last week, the global IPv4 routing table has surpassed the 500 thousand route benchmark, according to the CIDR Report. The graph below shows its progression since the early nineties:


I last wrote about global IPv4 growth in August of 2009, when the table size was at a mere 300 thousand routes. While that benchmark was largely ceremonial, this one crosses a threshold which should may be of grave concern for many.

As has been pointed out on the NANOG mailing list, we are quickly approaching the hard forwarding plane capacity limits which exists on several very popular platforms, namely the Cisco 7600/6500 and RSP720/Sup720. The default TCAM partitioning scheme of these platforms allows for a maximum of 512 thousand IPv4 routes.

If you accept full Internet routes anywhere on your network, you'll want to verify the maximum table sizes for those platforms. On the 6500/7600 platform, the current partitioning scheme can be inspected with show mls cef maximum-routes:

Router# show mls cef maximum-routes
FIB TCAM maximum routes :
Current :
 IPv4 + MPLS         - 512k (default)
 IPv6 + IP Multicast - 256k (default)

The good news is that it's easy to repartition the default scheme (e.g. mls cef maximum-routes ip 768) to allow for more IPv4 space. Unfortunately, this requires taking the device out of production for a time to be rebooted.

Thanks to @nixgeek and the NANOG folks for inspiring this post!


Deploying Datacenter MPLS/VPN on Junos

Tuesday, April 15, 2014 at 1:17 a.m. UTC by stretch

One of my recent projects has been deploying an MPLS/VPN architecture across a pair of smallish datacenters comprised entirely of Juniper gear. While I'm no stranger to MPLS/VPN, I am still a bit green to Junos, so it was a good learning exercise. My previous articles covering MPLS/VPN on Cisco IOS have been fairly popular, so I figured it would be worthwhile to cover a similar implementation in the Juniper world.

For our datacenters, we decided to implement a simple spine and leaf topology with a pair of core routers functioning as IBGP route reflectors and a pair of layer three ToR switches in each server rack. The spine is comprised of four layer three switches which run only MPLS and OSPF; they do not participate in BGP.


This article assume some basic familiarity with MPLS/VPN, so if you're new to the game, consider reading through these previous articles for some background before continuing:

Continue reading 12 comments

The Value of a Microsecond

Wednesday, April 2, 2014 at 12:36 a.m. UTC by stretch

While perusing vendor datasheets, have you ever questioned the inclusion of seemingly insignificant latency specifications? Take a look at Arista's line-up, for instance. Their 7500 series chassis lists a port-to-port latency of up to 13 microseconds (that's thirteen thousandths of a millisecond) whereas their "ultra-low latency" 7150 series switches provide sub-microsecond latency.


But who cares? Both values can be roughly translated as "zero" for us wetware-powered humans. (For reference, 8,333 microseconds pass in the time it takes your shiny new 120 Hz HDTV to complete one screen refresh.) So, does anyone really care about such obscenely low latency?

For a certain few organizations involved in high-frequency stock trading, those shaved microseconds can add up to billions of dollars in profit. The New York Times recently published an article titled The Wolf Hunters of Wall Street by Michael Lewis, which reveals how banks have leveraged low network latency to manipulate stock prices in open markets. (Thanks to @priscillaoppy for the tip!)

The increments of time involved were absurdly small: In theory, the fastest travel time, from Katsuyama’s desk in Manhattan to the BATS exchange in Weehawken, N.J., was about two milliseconds, and the slowest, from Katsuyama’s desk to the Nasdaq exchange in Carteret, N.J., was around four milliseconds. In practice, the times could vary much more than that, depending on network traffic, static and glitches in the equipment between any two points. It takes 100 milliseconds to blink quickly — it was hard to believe that a fraction of a blink of an eye could have any real market consequences.
Continue reading 10 comments

Learn Python

Thursday, March 27, 2014 at 1:10 a.m. UTC by stretch

Around six years ago, I decided to start a website called Maybe you've heard of it. Most people turn to a purpose-built content management system like Wordpress or Drupal for such an endeavor, but I needed greater flexibility to achieve some of the projects I had in mind. This meant I needed to learn a programming language and write a good amount of the site's logic myself.

I already had some experience dabbling in PHP, but wasn't thrilled with it. I figured if I was going to learn a new language, it should be useful as a general purpose language and not just for building a web site. After a bit of research and deliberation, I chose Python (and the Django web framework).

The purpose of this post is to convince networkers with little to no experience writing code to learn Python. In the past I've encouraged fellow networkers to pick up any programming language, as it's more important to think like a programmer than it is to gain proficiency in a particular language. However, I've realized that many people get stuck on which language they want to learn, lose motivation, and end up not growing proficient in anything. So, I've started telling people to skip that first step and just learn Python.

Continue reading 14 comments

Packet Life Turns Six

Sunday, March 23, 2014 at 9:40 p.m. UTC by stretch

Today marks Packet Life's sixth birthday, and I'm celebrating by launching the new site format I talked about in January. The relaunched site is hosted on an entirely new server from Linode, which means you can (finally) access via native IPv6! The entire code base has been rewritten on Django 1.6, and should feel lighter and more responsive. The layout has been rewritten as well using the Bootstrap CSS framework.

You might have noticed that some components of the old site are now gone: The discussion forums and wiki have been axed in favor of focusing more on the site's core content. The tools armory, which was initially in jeopardy, has been maintained in response to community interest (although I do intend to spend a good amount of time cleaning it up).

There are no doubt bits of code here and there that need a tweak or three, but generally speaking the site is up and running. If you do encounter an error, rest assured that I've been alerted and should have it fixed in little time. If you feel that something is terribly amiss, give me a shout on Twitter and I'll look into it ASAP.

I want to thank everyone for their patience during the transition. I'm very excited about finally revamping the site, and paving the way for cool new things!


Selecting Shapes by Layer

Wednesday, February 26, 2014 at 12:20 a.m. UTC by stretch

Selecting shapes and connectors one-by-one in Visio can be tedious, especially when working with large or repetitive drawings. If you've been drawing for a while, you've probably gotten the hang of selecting just the right subset of shapes using the rectangular select tool, and employing the control key to add or remove any outliers as desired. This can be time-consuming though, especially when you want to pick out just a few connectors from a jumble of criss-crossing lines.

Here's a trick to try next time you find yourself excessively control-clicking: Identify each logical group of shapes or connectors that you'll likely want to tweak, and bundle them up into to their own layer. You can then use Visio's "select by layer" option to grab them all at once later. Take the drawing below, for instance.


Continue reading 6 comments

About Packet Life is the work of a network engineer named Jeremy Stretch. It began as a repository for Cisco certification study notes in 2008, but quickly grew into a popular community web site.

The site's goal is to offer free, quality technical education to networkers all over the world, regardless of skill level or background.

Blog Splotlight