Lessons Learned Writing a Custom Config Builder

A while back, I set about developing a modest configuration templating system for my employer. When I first joined the company, new network devices were being provisioned using configuration templates stored as Microsoft Word files, which, as you can imagine, was pretty painful. Each variable had to be identified and replaced by hand in a tedious and error-prone process. I wanted something better, but also cheap (or free) and simple. So I started building something.

To kick off my crazy project, I first decided to build a web application based on the Django Python framework (the same platform on which PacketLife.net runs). Django and similar frameworks handle most of the mundane tasks involved in writing a web application and allow for rapid prototyping. It also includes a built-in administration interface for creating and manipulating data independent of the front-end user interface. I spun up a modest internal VM running Ubuntu Server, installed Django, and initiated a working project in under an hour.

Once I had a development environment in place, I started to think really hard about exactly what I wanted to build. (Admittedly, this step probably should have come first.) I liked the idea of raw text-based config templates because they're nice and portable: You can copy a sample config out of a vendor's documentation or running device, tweak it, and extract the variables to produce a working template. This is a much more straightforward approach than, say, abstracting a command hierarchy into XML.

Here's a simple Cisco IOS BGP configuration snippet we'll use as an example:

router bgp {LOCAL-ASN}
 address-family ipv4
  no synchronization
  neighbor {PEER-IP} remote-as {PEER-ASN}
  neighbor {PEER-IP} activate
  neighbor {PEER-IP} soft-reconfiguration inbound
  network {LOCAL-NETWORK} mask {LOCAL-NETWORK-MASK}

The tedious part is having to replace these variables by hand when deploying a new configuration. Our primitive approach was to denote variables as capitalized text surrounded by brackets or braces, but this can be easy to miss. Even using bright yellow highlighting in a full-blown editor like Word doesn't guarantee that each variable will be caught and replaced.

The other challenge is that, even if you could ensure every variable was appropriately filled in every time, there's no contextual validation of the provided value. Someone might typo an IPv4 address as 192.0..1, and that's what gets dumped into the device when the configuration is applied. This obviously would trigger an error, which may or may not be noticed prior to deployment.

Contextual Variables

To address the first problem, detecting variables, we can use a simple regular expression. For example, the following regex could be used to match the variable naming scheme used in the sample config template above (all capital letters and hyphens surrounded by curly braces):

{([A-Z\-]+)}

We can easily write a few lines of code to parse a config template and pull out all the variables using this regex. However, it doesn't address our second problem: assigning context to each variable. For example, what if we have a variable named CUSTOMER? Does that mean we should specify a full customer name, a shorthand customer identifier, a human point of contact, or something else? This takes a bit more thought.

First we have to decide what types of contextual information we want to store about our variables. Most programming languages, for example, have different types of variables: integers, strings, lists, booleans, and so on. We need to classify variables in a similar manner: IPv4 addresses, IPv6 addresses, VLAN IDs, interface identifiers, etc. Each variable has a set of rules governing its form. For example, a VLAN ID needs to be a positive integer between 1 and 4094, whereas an IPv4 address must be expressed in dotted-decimal notation with a mask length.

I began listing the types of variables we might need, organized by the constraints needed to be placed on them:

(The table above is clearly not inclusive, and is intended only to convey the concept of classifying variables by type.)

Once I was satisfied that my list was mostly complete, I set about creating a Django model for each type of variable: NumberVar, StringVar, etc. Through the app's administration interface, I could then create arbitrary instances of each. For example, I created a NumberVar named ASN with a minimum value of 1 and a maximum value of 65535. I created a second NumberVar named PRIVATE-ASN with a minimum value of 64512 and a maximum value of 65535. I would use one or the other to represent an ASN in my templates depending on which type I wanted, providing a modest degree of context.

I began creating more and more variable types with finer and finer tuning. I could create a specific IPv4 network type with a set mask length of /30 for use on point-to-point links. I created a specific string type with a minimum length of 12 characters to represent passwords. Now I was getting somewhere.

Next, I needed to adjust my regex to accommodate the inclusion of a variable type. I decided on the form {TYPE:NAME}. Further, since this was all going to be parsed by the application anyway, I decided to use a more human-friendly form of variable naming, with mixed cases and underscores representing spaces. This would allow me to dynamically generate clean field names within the user interface (more on this in a bit). The regex grew to:

{([A-Z0-9\-]+):([A-Za-z0-9_\-]+)}

With the new scheme in place, the template snippet earlier in this article would be rewritten as:

router bgp {ASN:Local_ASN}
 address-family ipv4
  no synchronization
  neighbor {IP4-HOST:Peer_IP} remote-as {ASN:Peer_ASN}
  neighbor {IP4-HOST:Peer_IP} activate
  neighbor {IP4-HOST:Peer_IP} soft-reconfiguration inbound
  network {IP4-NETWORK:Local_Network} mask {IP4-MASK:Local_Network_Mask}

Display Filters

There's one bit that's still missing, though. Why should we have to declare the local network and its mask as two separate variables? They are, after all, two expressions of a common value. For example, for a given network -- let's say 192.168.0.0/24 -- there are several pieces of information we might want to display:

• The network ID (192.168.0.0)
• The mask length in CIDR notation (24)
• The mask length in dotted-decimal notation (255.255.255.0)
• The mask length expressed as a wildcard mask (0.0.0.255)

All of this information is already contained in the variable; we just need a way to tweak how it gets displayed in the generated template. We can add an optional third component to the variable declaration scheme, a display filter to output differing formats of a variable. We'll use a pipe character appended to the variable name to add a display filter on the local network variable:

network {CUSTOMER-IP4-NETWORK:Local_Network|ip} mask {CUSTOMER-IP4-NETWORK:Local_Network|ddmask}

Given a value of 192.168.0.0/24, the line above would generate the following:

network 192.168.0.0 mask 255.255.255.0

Our variable-matching regex is finally extended to the following to accommodate optional display filters:

{([A-Z0-9\-]+):([A-Za-z0-9_\-]+)(?:\|([a-z0-9]+))?}

At this point, we have a reliable way to assign context to named variables and display their values in arbitrary formats. The next step is dynamically generating a form to be completed by the user based on those variables.

Dynamic Forms

Django provides a pretty robust tool set for generating and processing web forms. Armed with our variable declaration regex, we can parse a configuration template and create a new form field for each variable we find and present the resulting set of fields to the user for completion. Without getting too deep into the inner-workings of Django, as I want to keep this article platform-agnostic, it's possible to dynamically generate a web form with an individual field for each variable in the template.

This bit of template config:

router bgp {ASN:Local_ASN}
 address-family ipv4
  no synchronization
  neighbor {IP4-HOST:Peer_IP} remote-as {ASN:Peer_ASN}
  neighbor {IP4-HOST:Peer_IP} activate
  neighbor {IP4-HOST:Peer_IP} soft-reconfiguration inbound
  network {IP4-NETWORK:Local_Network} mask {IP4-MASK:Local_Network_Mask}

Becomes this form:

And when the form is correctly completed, will produce output like this:

router bgp 65000
 address-family ipv4
  no synchronization
  neighbor 192.0.2.1 remote-as 65100
  neighbor 192.0.2.1 activate
  neighbor 192.0.2.1 soft-reconfiguration inbound
  network 192.168.0.0 mask 255.255.255.0

I want to stress that the definition for this form has not been defined somewhere else in the application: It has been rendered solely from the configuration template. If we add or remove a variable in the template, that change is reflected when the form is regenerated. Also note that although the peer IP variable has been declared three times in the template, we only need to specify it once. Similarly, the local network variable is prompted for only once and displayed twice using two different filters.

Reusable Configlets

What if we need to add more than one network statement to our BGP configuration? We could generate one line and then replicate it by hand, but then we're right back to where we started. A more elegant solution is to break our template into two components: The base BGP protocol configuration and the network statement. Whereas the first component is needed only once, we can replicate the second part as many times as we need. In fact, that's exactly what the green "Add another" button does in the screenshot. We can also give the user the option to include additional configlets, such as one for a BGP aggregate address.

The above form will produce the following output:

router bgp 65000
 address-family ipv4
  no synchronization
  neighbor 192.0.2.1 remote-as 65100
  neighbor 192.0.2.1 activate
  neighbor 192.0.2.1 soft-reconfiguration inbound
  network 192.168.0.0 mask 255.255.255.0
  network 192.168.1.0 mask 255.255.255.0
  network 192.168.2.0 mask 255.255.255.0
  network 192.168.3.0 mask 255.255.255.0
  aggregate-address 192.168.0.0 255.255.252.0 summary-only

As you can see, with a little thought and effort, it's possible to create a very robust and lightweight configuration templating system. Obviously, I've glossed over the mechanics behind the concepts and I am unfortunately unable to release the code as this was built on company time, but I want to encourage readers to give it a shot themselves and see what they come up with. I have no doubt that someone with stronger programming skills can come up with something much more impressive than what I did.