In data centers, customers are interconnected with carriers and other customers using dedicated physical connections called cross-connects. These can be just about any type of medium, however most cross-connects run inside a single building are copper or fiber Ethernet. In the simplest of cases, a cross-connect can be a single CAT 5e or 6 cable running from a patch panel in one cage to a patch panel in another cage. Very little can go wrong, or so one might think.

I ordered two such Gigabit Ethernet cross-connects recently. Two lines of CAT 6 cable from a cage leased by company to a customer's cabinet down the hall. Simple. One of the cross-connects came up fine, as usual. The other did not.

This was odd, because the data center tech who installs the cross-connect is responsible for certifying its operation before making it available to the customer. I check it out with my Fluke and see that pairs 3/6 and 4/5 are crossed. No big deal, probably just needs an end re-terminated. I disconnect the patch cables from the panels at either end of the cross-connect so a tech can re-terminate it, open a trouble ticket with the data center, and go on about my day.

The next day, I get an email confirmation that ticket with the data center has been closed. Awesome. I go to plug the cables back in thinking the issue has been resolved. Same problem as before; a short on 3/6 and 4/5. Annoyed, I call up the data center help desk and ask what's up.

I've been involved with a moderate datacenter deployment of Cisco Nexus switches over the past couple months, and I have learned a good deal about the architecture along the way. Here are some of the design considerations I've encountered, and my preferred solution to each.

Drawing the Layer Three Boundary

I work for a managed services provider which offers datacenter colocation, including private line and VPN connectivity to customer sites. One of the primary requirements for network design in this field is to maintain strict customer delineation across all segments of the network. The typical go-to tools for traffic isolation are IEEE 802.1Q VLANs at layer two and MPLS/VRFs at layer three. It is preferable to extend layer three (MPLS) forwarding as close to the datacenter edge as possible, to take advantage of the flexibility and inexpensive redundancy offered by equal-cost multipath routing.

Unfortunately, the only model in the Nexus switch series which supports MPLS is the 7000 (and that wasn't until NX-OS 5.2 was released just last year). (The Nexus 5500 series offers IP-only layer three functionality with an upgraded daughter card.(This is disappointing, because it necessitates deploying expensive Nexus 7000 chassis as access switches if you need to have MPLS at the edge. Otherwise, you're stuck with an extended layer two topology by way of Nexus 5000s.

I'm sorry I haven't had much time for blogging lately. I had intended to have a new article out today but got sidetracked over the weekend. Alternatively, I thought I'd open the blog up to some discussion.

When mentoring junior engineers, I often hear something similar to "I wish someone had shown me that earlier!" This gets me thinking about how much more productive newbie networkers could be if they were shown a few tricks of the trade right out of the gate rather than learning them on the job. I'm not talking about book knowledge here but real-world practical knowledge which makes one's day go smoother.

What tip or trick do you wish someone had shown you on your first day? Nothing is too trivial or too obvious to mention. If the feedback is productive I'll create a succinct summary of the responses and publish a follow-up article at the end of the week.

Dynamic connectors in Microsoft Visio will typically have no more than one ninety-degree angle. This behavior keeps the line rendering as simple as possible. Of course, sometimes one need connectors to travel out of their way a bit across more complex drawings in order to keep things clear.

A connector can be manipulated so that it spawns additional angles by holding the shift or control key while dragging its midpoint. Holding the shift key will break out the middle of the line, creating four new right angles (the first example below). Holding the control key will create a single angle of an arbitrary degree at the midpoint (the second example below).

To create just two new right angles, you can create a breakout like the example above and then simply move one portion of the connector in line with the breakout; it should snap right in place. This will fuse the two sections of line back into one.

You can repeat this process at junctions where you want a connector to change direction to create very orderly (or in the case below, chaotic) connectors with a predictable path from end to end.

Responding to current trends in the world of IT, the International Organization for Standardization (ISO) has announced a refresh of the legacy Open Systems Interconnect (OSI) model which we've all come to know and love. The original seven-layer model is to be replaced with a simplified, sleeker six-layer model which more accurately reflects service stacks seen in today's networks.

To review, the legacy OSI model featured seven layers:

• Application
• Presentation
• Session
• Transport
• Network
• Data Link
• Physical

Obviously, this was way too complex for many people in IT to comprehend. (And really, have you ever heard a decent explanation of the presentation layer's function?) The new model, OSI 2.0, features six concise layers:

• User
• Cloud
• HTTP(S)
• Virtualization
• Fabric
• Wireless

Let's look at each layer a bit closer, starting from the bottom up.

In preparation for a major datacenter deployment, I've been re-familiarizing myself with Cisco's Nexus platform (and naturally, what I pick up on the job will make its way onto the blog). Today's lab involved attaching a Nexus 2200 fabric extender, or FEX, to a Nexus 5500.

For those unfamiliar with this setup, a Nexus 2000 is essentially a standalone line module: It requires connectivity to a Nexus 5000 switch to function. All interface configuration is performed on the Nexus 5000, where every attached Nexus 2000 is treated as an individual slot. For example, port 8 on FEX 101 is referred to as Ethernet101/1/8 on the 5000. This is very similar in concept to stacking Catalyst 2960S or 3750 switches, except that the 2000s are connected to a 5000 via 10 Gbps Ethernet rather than via a proprietary (and short) stacking cables.

The intended datacenter design is to deploy many Nexus 2000s as top-of-rack (ToR) switches and connect them back to Nexus 5000s at the end or middle of the row. This looks great on paper, however the major drawback is that Nexus 2000s don't do any local switching: A packet from port 1 on a 2000 to port 2 on the same switch must travel up to the 5000 and back. This may or may not be a problem, depending on your oversubscription rates and latency requirements.

With that said, let's look at how to connect a Nexus 2200 to a Nexus 5500.