Three Months with Google Fiber

Friday, January 6, 2017 at 2:40 p.m. UTC by stretch

I'm one of the lucky few to benefit from Google Fiber's recent expansion into new regions (before they nixed the whole thing). I've had the service fire three months now and figured I should write up my experience with it thus far.

The Installation

Google Fiber announced that it would be expanding to the Raleigh-Durham metro area, known locally as "The Triangle", in January 2015. It's been a long game of hurry-up-and-wait since then, watching crews laying fiber all over town without hearing a peep from Google regarding availability. But in the fall of 2016, people were finally able to start signing up for service. Here's how my installation went.

September 3

Google Fiber registration opens! I sign up for service and pay a paltry $10 deposit, which gets credited toward my first bill. Over the next couple weeks, various utilities swing by to mark their lines in the ground. (Here's the color code for utility markings in the US, if you're curious.)

September 24

Google's contractor arrives on site to lay fiber from the curb to my house and to many of my neighbors' houses. Surprisingly, they cut my trench by hand, possibly due to the steep incline of my side yard.


The outside installers ran a single pair of fiber to a box mounted on the side of my house, with the remainder of the installation to be completed by the indoor tech. (As with Verizon FiOS, only one strand is needed for service.)

September 27

I receive a notification to schedule my installation. Of course, I pick the first available slot: 9:40am on October 3.

Continue reading 14 comments

Legacy TLS cipher support in Firefox

Wednesday, October 12, 2016 at 5:45 p.m. UTC by stretch

After upgrading Firefox recently, I noticed that I could no longer access certain embedded devices via HTTPS. It seems that recent versions of Firefox and Chrome no longer support certain TLS ciphers due to recently discovered vulnerabilities. That's all well and good, except the error returned offers no recourse if you need to connect anyway.


Firefox returns the error SSL_ERROR_NO_CYPHER_OVERLAP with no option to temporarily allow connectivity. (Chrome reports a similar error named ERR_SSL_VERSION_OR_CIPHER_MISMATCH.) Presumably, this choice was made by the developers with the intention of forcing people to upgrade outdated devices. Unfortunately, in order to upgrade an out-of-date device, we typically must first be able to connect to it. I wasted a fair bit of time digging up a solution, so I figured I'd document the workaround here for when I inevitably run into this problem again a year from now and have forgotten what I did.

Continue reading 12 comments

The Overlay Problem: Getting In and Out

Friday, September 30, 2016 at 1:47 p.m. UTC by stretch

I've been researching overlay network strategies recently. There are plenty of competing implementations available, employing various encapsulations and control plane designs. But every design I've encountered seems ultimately hampered by the same issue: scalability at the edge.

Why Build an Overlay?

Imagine a scenario where we've got 2,000 physical servers split across 50 racks. Each server functions as a hypervisor housing on average 100 virtual machines, resulting in a total of approximately 200,000 virtual hosts (~4,000 per rack).

In an ideal world, we could allocate a /20 of IPv4 space to each rack. The top-of-rack (ToR) L3 switches in each rack would advertise this /20 northbound toward the network core, resulting in a clean, efficient routing table in the core. This is, of course, how IP was intended to function.

Unfortunately, this approach isn't usually viable in the real world because we need to preserve the ability to move a virtual machine from one hypervisor to another (often residing in a different rack) without changing its assigned IP address. Establishing the L3 boundary at the ToR switch prevents us from doing this efficiently.

Continue reading 22 comments

Taking the CCIE Lab in RTP

Friday, July 29, 2016 at 2:13 a.m. UTC by stretch

Cisco's campus in Research Triangle Park, North Carolina, is one of only two places in the United States where candidates can complete a CCIE lab exam (the other being in San Jose, California). People fly in from all over the eastern US and beyond to spend a day taking the exam. Lots of folks who've taken the exam have written up their experiences, but I haven't seen many talk at length about their time in RTP outside of Cisco's building 3.

I've lived just a few minutes away from the testing site for the past few years, and it occurred to me recently that visitors might benefit from some local knowledge.

Getting Here

Most people fly in via Raleigh-Durham International Airport (RDU). RDU is a medium-sized airport with two terminals. Most flights operate out of Terminal 2, except for Southwest Airlines, which is based in the newly-renovated Terminal 1.


As airports go, I'm a big fan of RDU. It's a very modern, clean, and well-organized facility. The interior of Terminal 2 is beautifully designed to resemble an early airplane wing and is flooded with natural light during the day. (It's also one of very few places where you can find a Five Guys that serves breakfast.) The TSA folks here tend to be very professional and polite. Wait times in security are rarely long. And this isn't just the biased opinion of a local: in 2014, RDU was ranked among the top ten airports in the US.

Continue reading 7 comments

NetBox v1.1.0 Released

Thursday, July 7, 2016 at 6:45 p.m. UTC by stretch

One year ago today, I made the first commit to a repository named "netbox" hosted internally at DigitalOcean. It was the first iteration of a tiny little app I scratched together using the Django Python framework to track IP prefix utilization. A year later, NetBox has grown into an extensive tool that we use to track IPs, racks, devices, connections, circuits, and even encrypted credentials. And I'm happy to say that it's now open source!

Continue reading 19 comments

Announcing NetBox

Wednesday, June 15, 2016 at 1:16 p.m. UTC by stretch

Update: NetBox has been released!

Several years ago, I lamented the few options available for a provider-grade IPAM solution. Specifically, I explained why building a custom application would be undesirable:

Could I create a custom IPAM solution with everything we need? Sure! The problem is that I'm a network engineer, not a programmer (a natural division of labor which, it seems, is mostly to blame for the lack of robust IPAM solutions available). Even if I had the time to undertake such a project, I have little interest in providing long-term maintenance of it.

But I suppose time makes fools of us all.

Nearly one year ago, I started developing an IPAM application as part of my day job. Leveraging my experience with the Django Python framework, I had a working proof-of-concept in just a week. Over the next several months, the project grew more mature and began to take on additional roles: data center infrastructure management, circuit tracking, and credentials storage. Today, the tool functions as our "source of truth" for many aspects of our infrastructure. We call it NetBox.

Continue reading 27 comments

Don't be Discouraged by Plagiarists

Wednesday, March 9, 2016 at 3:28 a.m. UTC by stretch

Recently, a friend pointed out that an individual had taken one of my cheat sheets, superimposed his own logo and URL on it, and published it as his own work. This is certainly not the first time I've been plagiarized, nor will it be the last, I suspect. I called out the individual on Twitter, and I'm very gratefully for the many people who helped me compel him to remove the illegitimate content. Eventually.

I wanted to write a quick post sharing my thoughts on this incident for the benefit of everyone who has expressed interest in starting their own blog or web site. I've heard plenty of people comment over the years to the effect of, "Why bother starting a blog if someone's just going to harvest the RSS feed and re-publish it on their own site to make a few bucks?" Indeed, this has always been a concern among producers of both free and paid content.

I wish I could tell you that plagiarism isn't that big a deal, or that it won't happen to you. But the truth is plagiarism is a huge problem in our industry (and across the Internet in general), and if you produce public content you will inevitably have to deal with it.

Continue reading 25 comments

Can You Keep a Secret? (Part 2)

Thursday, February 11, 2016 at 2:36 a.m. UTC by stretch

In part one, we saw how AES can be used to encrypt sensitive data so that it can be retrieved only by using an encryption key. The problem with this approach is that everyone who needs access to the data must have a copy of the key. If any one of these copies becomes compromised, the entire database must be re-encrypted using a new key, and the new key must be distributed securely to all parties involved. In this article, we'll see how symmetric encryption can be combined with asymmetric cryptography (namely RSA) to create a hybrid cryptosystem.

Let's begin by encrypting some data using AES as we did in part one. First we pad our plaintext's length to a multiple of 16 using null bytes, then generate a 256-bit encryption key and a 128-bit IV, and finally encrypt it with CFB-mode AES to generate a string of ciphertext.

>>> from Crypto.Cipher import AES
>>> import os
>>> plaintext = "Operation Neptune will launch on June 6th"
>>> plaintext += (16 - len(plaintext) % 16) * chr(0)
>>> encryption_key = os.urandom(32)
>>> iv = os.urandom(16)
>>> cipher =, AES.MODE_CFB, iv)
>>> ciphertext = cipher.encrypt(plaintext)
Continue reading 2 comments

Can You Keep a Secret?

Sunday, February 7, 2016 at 7:44 p.m. UTC by stretch

I've been developing an IPAM/DCIM tool for work over the past several months (more on that soon), and recently my focus has been on expanding it to store confidential data associated with network devices. Backup login credentials, TACACS+/RADIUS secrets, SNMP communities, and so on: Short strings that need to be stored securely.


Storing a password or other small piece of sensitive data is different from merely authenticating against it. Most password storage mechanisms never actually store a user's actual password, but rather an irreversible hash of it. (That is if you're doing it correctly, at least.)

For example, the Django Python framework (which powers by default employs salted SHA256 hashes to authenticate user passwords. When a password is saved, a random salt is generated and concatenated with the plaintext password. (A salt is used to prevent two identical passwords from producing the same hash.) The SHA256 algorithm is then run against the whole thing to produce a fixed-length hash. Here's an example in Python using Django's built-in make_password() function:

>>> from django.contrib.auth.hashers import make_password
>>> make_password("MyP@ssw0rd!")
Continue reading 4 comments

Stretch's Hierarchy of Network Needs

Monday, December 14, 2015 at 3:07 a.m. UTC by stretch

Remember Maslow's hierarchy of needs from school? The theory asserts that every human shares the same set of basic physical and psychological needs in order to be happy, with more primal needs like food and shelter taking precedence over emotional needs like love and companionship.


A while back, I was pondering what would be necessary to fully automate a network, and it occurred to me that a very similar hierarchy of needs can be laid out for a computer network to achieve its optimal state.

1. A Functioning Network

At the very bottom of the hierarchy is everything a network requires to function: Routers, switches, cabling, power, and so on, just as tier one of Maslow's hierarchy encompasses everything a human needs to stay alive. At this stage, a network can function, and can even function well, but it cannot adapt or grow.

Many small businesses operate their networks at this stage for years with no major problems. After all, when left alone, computers and networks tend to just keep chugging along. And if your entire network comprises a cable modem, a switch, and a few access points, it's entirely possible that it will run for years without needing any tweaking.

Continue reading 10 comments