In data centers, customers are interconnected with carriers and other customers using dedicated physical connections called cross-connects. These can be just about any type of medium, however most cross-connects run inside a single building are copper or fiber Ethernet. In the simplest of cases, a cross-connect can be a single CAT 5e or 6 cable running from a patch panel in one cage to a patch panel in another cage. Very little can go wrong, or so one might think.

I ordered two such Gigabit Ethernet cross-connects recently. Two lines of CAT 6 cable from a cage leased by company to a customer's cabinet down the hall. Simple. One of the cross-connects came up fine, as usual. The other did not.

This was odd, because the data center tech who installs the cross-connect is responsible for certifying its operation before making it available to the customer. I check it out with my Fluke and see that pairs 3/6 and 4/5 are crossed. No big deal, probably just needs an end re-terminated. I disconnect the patch cables from the panels at either end of the cross-connect so a tech can re-terminate it, open a trouble ticket with the data center, and go on about my day.

The next day, I get an email confirmation that ticket with the data center has been closed. Awesome. I go to plug the cables back in thinking the issue has been resolved. Same problem as before; a short on 3/6 and 4/5. Annoyed, I call up the data center help desk and ask what's up.

I've been involved with a moderate datacenter deployment of Cisco Nexus switches over the past couple months, and I have learned a good deal about the architecture along the way. Here are some of the design considerations I've encountered, and my preferred solution to each.

Drawing the Layer Three Boundary

I work for a managed services provider which offers datacenter colocation, including private line and VPN connectivity to customer sites. One of the primary requirements for network design in this field is to maintain strict customer delineation across all segments of the network. The typical go-to tools for traffic isolation are IEEE 802.1Q VLANs at layer two and MPLS/VRFs at layer three. It is preferable to extend layer three (MPLS) forwarding as close to the datacenter edge as possible, to take advantage of the flexibility and inexpensive redundancy offered by equal-cost multipath routing.

Unfortunately, the only model in the Nexus switch series which supports MPLS is the 7000 (and that wasn't until NX-OS 5.2 was released just last year). (The Nexus 5500 series offers IP-only layer three functionality with an upgraded daughter card.(This is disappointing, because it necessitates deploying expensive Nexus 7000 chassis as access switches if you need to have MPLS at the edge. Otherwise, you're stuck with an extended layer two topology by way of Nexus 5000s.

I'm sorry I haven't had much time for blogging lately. I had intended to have a new article out today but got sidetracked over the weekend. Alternatively, I thought I'd open the blog up to some discussion.

When mentoring junior engineers, I often hear something similar to "I wish someone had shown me that earlier!" This gets me thinking about how much more productive newbie networkers could be if they were shown a few tricks of the trade right out of the gate rather than learning them on the job. I'm not talking about book knowledge here but real-world practical knowledge which makes one's day go smoother.

What tip or trick do you wish someone had shown you on your first day? Nothing is too trivial or too obvious to mention. If the feedback is productive I'll create a succinct summary of the responses and publish a follow-up article at the end of the week.

Dynamic connectors in Microsoft Visio will typically have no more than one ninety-degree angle. This behavior keeps the line rendering as simple as possible. Of course, sometimes one need connectors to travel out of their way a bit across more complex drawings in order to keep things clear.

A connector can be manipulated so that it spawns additional angles by holding the shift or control key while dragging its midpoint. Holding the shift key will break out the middle of the line, creating four new right angles (the first example below). Holding the control key will create a single angle of an arbitrary degree at the midpoint (the second example below).

To create just two new right angles, you can create a breakout like the example above and then simply move one portion of the connector in line with the breakout; it should snap right in place. This will fuse the two sections of line back into one.

You can repeat this process at junctions where you want a connector to change direction to create very orderly (or in the case below, chaotic) connectors with a predictable path from end to end.

Responding to current trends in the world of IT, the International Organization for Standardization (ISO) has announced a refresh of the legacy Open Systems Interconnect (OSI) model which we've all come to know and love. The original seven-layer model is to be replaced with a simplified, sleeker six-layer model which more accurately reflects service stacks seen in today's networks.

To review, the legacy OSI model featured seven layers:

• Application
• Presentation
• Session
• Transport
• Network
• Data Link
• Physical

Obviously, this was way too complex for many people in IT to comprehend. (And really, have you ever heard a decent explanation of the presentation layer's function?) The new model, OSI 2.0, features six concise layers:

• User
• Cloud
• HTTP(S)
• Virtualization
• Fabric
• Wireless

Let's look at each layer a bit closer, starting from the bottom up.

In preparation for a major datacenter deployment, I've been re-familiarizing myself with Cisco's Nexus platform (and naturally, what I pick up on the job will make its way onto the blog). Today's lab involved attaching a Nexus 2200 fabric extender, or FEX, to a Nexus 5500.

For those unfamiliar with this setup, a Nexus 2000 is essentially a standalone line module: It requires connectivity to a Nexus 5000 switch to function. All interface configuration is performed on the Nexus 5000, where every attached Nexus 2000 is treated as an individual slot. For example, port 8 on FEX 101 is referred to as Ethernet101/1/8 on the 5000. This is very similar in concept to stacking Catalyst 2960S or 3750 switches, except that the 2000s are connected to a 5000 via 10 Gbps Ethernet rather than via a proprietary (and short) stacking cables.

The intended datacenter design is to deploy many Nexus 2000s as top-of-rack (ToR) switches and connect them back to Nexus 5000s at the end or middle of the row. This looks great on paper, however the major drawback is that Nexus 2000s don't do any local switching: A packet from port 1 on a 2000 to port 2 on the same switch must travel up to the 5000 and back. This may or may not be a problem, depending on your oversubscription rates and latency requirements.

With that said, let's look at how to connect a Nexus 2200 to a Nexus 5500.

One of my projects at work this year is bringing native IPv6 Internet transit into our datacenter. We'll be multi-homing across several Internet providers, which means we'll need to request provider-independent (PI) address space directly from ARIN, the IP registrar for North America. I haven't had any direct interaction with ARIN before, so I'm learning the process as I go, and documenting it on the blog as a reference for others.

In order to request resources or otherwise interact with ARIN, you'll need to create an organization ID for your company. You'll need to decide who your public points of contact will be and provide proof to ARIN that you are, in fact, a legitimate legal entity. The process can be a bit confusing at first, but the overall scheme looks like this:

Step 1: Create an ARIN Web Login

As with just about any other web site, your first step is to create a unique login. This is a personal account for only you to use, and doesn't bind directly to your company or appear publicly. You can register using either your work email or a personal email address.

It was four years ago today when I published my first blog article here on packetlife.net. Now, 433 posts later, I have to say it's been a remarkable experience thus far. Throughout that time, I've changed jobs twice, moved twice, and gotten married (once). When the site debuted, the global IPv4 routing table was only around 260K routes; it broke 400K earlier this year. Where has the time gone?

This blog started out as little more than a journal for my CCNP studies while I was bored out of my mind working as a DoD contractor in Iraq. Given the ample free time I had after returning stateside, the site grew into something much larger than I had originally intended. It saw the addition of a packet captures library, a discussion forum, a wiki, and of course the ever-popular community training lab, thanks to its many sponsors and donors. Today, Packet Life has over 12,000 registered members and sees around 80,000 unique visitors per month.

I've met some truly awesome people through the site, and have had people in "real life" recognize my name (or bald head) from the blog. I've even gotten to meet blogging greats like Ivan Pepelnjak and Greg Ferro in person. Most recently, I was humbled to be plugged by Jeremy Cioara in one of his CBT Nuggets videos. But the most sincere sense of accomplishment I get is from the comments and emails from readers thanking me for helping them understand some concept or shaving a few hours off their day with a particular blog article. Those have been keeping me going these last four years.

Of course, some of you will be quick to remind me that I slacked off for a bit at the end of 2011, and the frequency of posts isn't as great as it used to be. That's the price of a very engaging full-time career, I suppose. A real networking job does provide much more access to interesting topics, though; over the next few months you can expect to see a number of articles covering Cisco's Nexus switches and datacenter design practices, which correspond with some professional projects.

Where do I go from here? Several people have suggested writing a book; an interweaving of topics from the blog in a more structured form. I have always wanted to write a sort of networkers' handbook, something which encompasses all those aspects of networking newbies don't get exposure to studying for the CCNA exam. But it seems presumptuous of me to attempt a work of any real authority given my relatively limited experience at 26 years old.

I've also toyed with the notion of founding a non-profit corporation focused on aiding networkers at various stages of their careers, particularly those new to the field and in emerging economies. Of course, such an undertaking would require far more time and effort than a weekly blog posting, which I already barely manage. For the foreseeable future, I suppose I'll just keep on blogging.

Thanks for reading!

When we discuss simple network failover solutions, we typically are contrasting two fundamental options: active/active and active/standby. These describe the manner in which two redundant devices operate under normal conditions.

In active/standby implementations, only the primary device in a pair passes traffic. The standby device sits idle, ready to assume the active role should the primary device fail. The standby device may receive routing and state information from the primary device in order to facilitate stateful failover, but it doesn't actually pass traffic until the primary device fails.

In active/active implementations, both devices are online and pass traffic under normal conditions. When one device fails, all traffic flows through the remaining device.

Customers often have a difficult time coming to terms with the perceived expense of an active/standby failover solution. The notion that one is paying double for an extra device "that doesn't do anything most of the time" clashes with the nature of a responsible consumer. In an active/active implementation, the second device is at least passing traffic, which is easier to justify psychologically. However, active/active implementations present a hidden danger: artificial scalability.