Writing a Custom IPAM Application
By stretch | Monday, August 10, 2015 at 1:14 a.m. UTC
Four years ago, I lamented the lackluster selection of IPAM applications available for service providers. Unfortunately, it seems not much has changed lately. I was back to exploring IPAM offerings again recently, this time with the needs of a cloud hosting provider in mind. I demoed a few tools, but none of them seemed to fit the bill (or they did, but were laughably overpriced).
So, I decided to write my own. In my rantings a few years back, I had considered this option:
Could I create a custom IPAM solution with everything we need? Sure! The problem is that I'm a network engineer, not a programmer (a natural division of labor which, it seems, is mostly to blame for the lack of robust IPAM solutions available). Even if I had the time to undertake such a project, I have little interest in providing long-term maintenance of it.
My opinion has not changed, but I've come to realize that if I want a tool that fits my requirements, I will need to build it. And after surprisingly little time, I'm happy to report that I have now have a kick-ass IPAM tool that does exactly what I want it to.
While I can't share the code, which is owned by and purpose-built for my employer, I would like to share some tips I picked up in the process with the hope of convincing others that rolling your own tools really isn't that hard. And in many cases, it may be well worth the effort.
Start with a framework
I've noticed that many people, when they set about building a tool from scratch, actually start from scratch: They write their own HTTP request handlers, their own database schema abstractions, and so on. If you're writing a tool which is to be a standalone product in its own right, this might make sense, but if all you need is a simple application for internal use, this approach will cost you hundreds of hours of time better spent on actual application logic and features.
The better option is to start with a framework like Django (Python) or Rails (Ruby). Frameworks reduce development overhead by providing the boilerplate code necessary for common functions: database ORM, authentication, templating, etc. Frameworks allow you to produce a usable bare-bones application in just a few hours.
It's no secret that I'm a big advocate of Python and Django (which powers Packetlife.net), but generally speaking, any mainstream framework should suffice for a relatively simple project like an IPAM application. Obviously, there's a learning curve associated with any platform if you're not already familiar with it, but it's a solid investment of time, especially if you might want to develop other web applications in the future.
Python's netaddr is awesome
Obviously, robust handling and representation of IP addresses and networks is crucial to the performance and integrity of an IPAM application. Fortunately, Python's awesome netaddr library comes packed with just about everything you might need. netaddr provides objects which include simple, convenient methods for manipulating IP addresses and making complex calculations.
>>> from netaddr import * >>> my_net = IPNetwork('192.0.2.0/24') >>> my_net.version 4 >>> my_net.netmask IPAddress('255.255.255.0') >>> my_net.broadcast IPAddress('192.0.2.255') >>> len(my_net) 256 >>> list(my_net[0:4]) [IPAddress('192.0.2.0'), IPAddress('192.0.2.1'), IPAddress('192.0.2.2'), IPAddress('192.0.2.3')] >>> '192.0.2.123' in my_net True >>> '188.8.131.52' in my_net False >>> list(my_net.subnet(26)) [IPNetwork('192.0.2.0/26'), IPNetwork('192.0.2.64/26'), IPNetwork('192.0.2.128/26'), IPNetwork('192.0.2.192/26')]
You can even feed netaddr a list of prefixes within an aggregate, and it will caluclate the remaining available space:
>>> aggregate = IPSet(['192.168.0.0/16']) >>> in_use = IPSet(['192.168.1.0/24', '192.168.2.0/24', '192.168.8.0/22', '192.168.192.0/18']) >>> aggregate ^ in_use IPSet(['192.168.0.0/24', '192.168.3.0/24', '192.168.4.0/22', '192.168.12.0/22', '192.168.16.0/20', '192.168.32.0/19', '192.168.64.0/18', '192.168.128.0/18'])
netaddr provides a huge chunk of the functionality any IPAM tool needs, for free. And it's even fully IPv6-compatible!
Take advantage of PostgreSQL's network address types
When I first pondered how to store IP addresses in a database, I figured it would make sense to use a 128-bit integer coupled with an integer mask length. Unfortunately, PostgreSQL, my database of choice, doesn't provide a 128-bit integer field. However, it does provide the cidr and inet address types, which were purpose-built for storing IP addresses.
Even cooler, these fields come with built-in functions and operators which allow for very flexible queries. And if you're using Django, you can leverage the django-netfields package to use these fields natively in your application.
Plan for VRF support (even if you don't need it)
Back when I was working for a managed services provider with hundreds of tenants, robust VRF support was a crucial requirement for an IPAM solution. These days, I don't have as much need for it, but based on past experience I decided to include the feature anyway. Although you might not have any immediate need to track multiple routing tables, it's trivial to add a VRF field to address objects and treat null values as belonging to the global table.
Also, be sure never to assume that an IP address will be unique in the database (even for public IP space). Never use an IP address itself to refer to an object (e.g. when forming a URL). Instead, use the primary key of its database object, which is guaranteed to be unique.
Laying the groundwork for this feature will help mitigate any issues in adding full VRF support in the future should the need arise.
Rely on native IP behavior to build hierarchies
I've seen IPAM products where each prefix is manually assigned to a parent prefix using a unique key in the database. This makes no sense. For example, if I have defined 10.10.0.0/16, I don't need to specify it as a parent network if I create 10.10.10.0/24 in the same network: This relationship is already effected by the very nature of IP (so long as both networks belong to the same routing table).
If you ever find yourself manually assigning one address object as the parent or child of another, you're probably doing something wrong.
Pay careful attention to VLAN scope
When devising a scheme to organize VLANs, remember that VLAN IDs are technically unique only within the context of a single switch. For the sake of simplicity, most people try not to reuse VLAN IDs within a given domain such as a site or building, but it happens more often than you might think. For instance, suppose you never reuse VLAN IDs, except for VLAN 999, which is used for a separate guest wireless network at each site. Well, this one exception to the rule means that you can't declare the VLAN ID field unique in the database schema. It's easy to get bit by exceptions like this, so you're best off not making any assumptions about uniqueness period.
Treat IPv4 and IPv6 equally
Any time you return a list of address objects, it should include relevant addresses belonging to both the IPv4 and IPv6 families. Use the same model to store both address types. You can always add an option to explicitly filter for one or the other, but you want to avoid having to make two separate database queries to retrieve both types.
Posted in Coding
August 10, 2015 at 11:55 a.m. UTC
If not Excel then, I like php IPAM...
August 10, 2015 at 1:20 p.m. UTC
I have been using this as a replacement for solarwinds IPAM for a few years now, I think it's excellent.
August 11, 2015 at 10:43 a.m. UTC
I have always looked for a nice IPAM tool too, and settled for phpIPAM in the end. But I'm interested in doing the same thing now, but I still green in both Python and Django.
Any chance that you can at least share screen shots of how it looks now?
August 11, 2015 at 3:04 p.m. UTC
Have you tried this http://iptrack.sourceforge.net?
It has DNS and DHCP support, it's scriptable (you can use nmap to update records), ...
August 13, 2015 at 3:11 a.m. UTC
We actually use almost this, postgres contains all data about servers and what prefixes exist in each colo and what their function is
August 17, 2015 at 9:58 a.m. UTC
Already tried Nipap ? http://spritelink.github.io/NIPAP/
August 17, 2015 at 11:12 p.m. UTC
I second that Gestio is awesome, been very reliable so far and we have it AD integrated.
August 18, 2015 at 7:22 a.m. UTC
The best IPAM I've ever used is IPAM module within “The Noc Project” (https://kb.nocproject.org/pages/viewpage.action?pageId=1507406). And it's written on Python. :)
August 31, 2015 at 9:49 a.m. UTC
NIPAP does all of that and more. If you want to improve it they're really open for community subscriptions!
September 15, 2015 at 5:29 p.m. UTC
You might ask the company you provided your IPAM to if it's okay that you offer it elsewhere so long as you remove company-specific info from it: we're all in this together, for one, and that other companies can work better / more efficiently should be good for us, as well. If 'we' need to get a job done, and I get to the destination-point (the completion of my task) first ... I just have to wait for everyone else, anyway. I understand they may say 'well, we paid him and he built it on our dime' but: imagine if we all said that? I share all my garbage with whoever wants it (of course, I don't work on top-secret stuff, or bread-and-butter-of-my-company things: I like working in more universal spheres). I have experienced exactly what you suggested on a couple of points, but not in building an IPAM but in recording information to populate a 'what's out there' database (my script is in PERL). I was just saying to a colleague - as we began to get VRF-capable information - that, while nobody's yet asked for it, we might want to start capturing VRF info. I had been storing it under a network, but after reading this article today and deciding to begin migrating to VRF-aware storage of the data today, I actually needed to record the routes under the VRFs, not VRF-assignment per route (because I'd end up clobbering the field on a route which exists under multiple VRFs with each time I encountered a different VRF's one). Took a couple of hours of re-work and debugging, and adding an additional foreach loop (to iterate through VRFs before per-VRF routes), but now it seems to work. I migrated the 'global' routing table to the 'default' VRF to be done with it even if the source device isn't VRF aware.
October 9, 2015 at 9:41 a.m. UTC
For me TeemIP works perfectly. It has IPv4 and IPv6 registration and subnet and range management, capacity planning, management of VLANs, DNS Domains, WAN Links, AS Numbers, VRFs, consistency audit to check data quality and more. It's worth giving a try if you ask me.
August 12, 2016 at 3:07 p.m. UTC
Have you heard of infoblox? DNS, DHCP, IPAM all in one. I think it is pretty great. check it out.