Multiple DMVPNs on a Single Hub

By stretch | Monday, January 9, 2012 at 1:31 a.m. UTC

I've touched on the fundamentals of DMVPN before, but today I'm going to expand upon my previous discussion and experiment with configuring multiple DMVPN clouds on a single pair of redundant headend routers. The scenario we'll use is that of a service provider offering DMVPN connectivity to two unrelated customers: Both customers need connectivity among their own sites and to the ISP, but must not be able to communicate with one another. Accordingly, each customer must use its own unique pre-shared key for authentication.

Our overall topology looks like this:

dual_DMVPN_topology.png

The 172.16.0.0/20 space represents public address space; in a real-world deployment, these would typically be effectively random public IP addresses.

The two ISP headend routers connect the four sites, two per customer, to the ISP network. Each headend router has two loopback interfaces (addressed out of 172.16.0.0/24) to which the DMVPN tunnels will be terminated. Per Cisco's DMVPN design guide, each customer will have two redundant DMVPN tunnels, one to each headend router. This adds up to a total of four DMVPN networks. Beware: this lab might get a little muddy, but I'll do my best to keep things clear.

Crypto Configuration

We'll start off by laying our crypto foundations. We'll use a single ISAKMP policy and IPsec transform-set common to all routers in the lab:

All routers

crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 2
!
crypto ipsec transform-set ESP_AES256_SHA_TRANSPORT esp-aes 256 esp-sha-hmac 
 mode transport

Next, we'll configure the necessary crypto profiles on the headend routers. Note that the keyrings and ISAKMP profiles are explicitly sourced from the loopback interfaces: This ensures that the correct pre-shared key is applied to each DMVPN spoke router based on the destination IP address of incoming encrypted packets.

Headend routers

crypto keyring CustomerA 
  local-address Loopback100
  pre-shared-key address 0.0.0.0 0.0.0.0 key MySecretKeyA
crypto keyring CustomerB 
  local-address Loopback200
  pre-shared-key address 0.0.0.0 0.0.0.0 key MySecretKeyB
!
crypto isakmp profile CustomerA
   keyring CustomerA
   match identity address 0.0.0.0 
   local-address Loopback100
crypto isakmp profile CustomerB
   keyring CustomerB
   match identity address 0.0.0.0 
   local-address Loopback200
!
crypto ipsec profile CustomerA
 set transform-set ESP_AES256_SHA_TRANSPORT 
 set isakmp-profile CustomerA
!
crypto ipsec profile CustomerB
 set transform-set ESP_AES256_SHA_TRANSPORT 
 set isakmp-profile CustomerB

The configuration of the spoke routers is very similar to that of the headend routers, but shorter since we're only including configuration applicable to the respective customers. Here we can use a single wildcard mask to match the pre-shared key.

Customer A routers

crypto keyring DMVPN 
  pre-shared-key address 0.0.0.0 0.0.0.0 key MySecretKeyA
!
crypto isakmp profile DMVPN
   keyring DMVPN
   match identity address 0.0.0.0
!
crypto ipsec profile DMVPN
 set transform-set ESP_AES256_SHA_TRANSPORT 
 set isakmp-profile DMVPN

Customer B routers

crypto keyring DMVPN 
  pre-shared-key address 0.0.0.0 0.0.0.0 key MySecretKeyB
!
crypto isakmp profile DMVPN
   keyring DMVPN
   match identity address 0.0.0.0
!
crypto ipsec profile DMVPN
 set transform-set ESP_AES256_SHA_TRANSPORT 
 set isakmp-profile DMVPN

DMVPN Configuration

With crypto out of the way, we can move on to DMVPN tunnel configuration. First we create two unique tunnels on both of the two headend routers, one in either customer VRF.

Headend1

ip vrf CustomerA
 rd 65000:100
!
ip vrf CustomerB
 rd 65000:200
!
interface Tunnel100
 ip vrf forwarding CustomerA
 ip address 192.168.1.1 255.255.255.128
 no ip redirects
 ip nhrp map multicast dynamic
 ip nhrp network-id 101
 tunnel source Loopback100
 tunnel mode gre multipoint
 tunnel key 101
 tunnel protection ipsec profile CustomerA
!
interface Tunnel200
 ip vrf forwarding CustomerB
 ip address 192.168.2.1 255.255.255.128
 no ip redirects
 ip nhrp map multicast dynamic
 ip nhrp network-id 201
 tunnel source Loopback200
 tunnel mode gre multipoint
 tunnel key 201
 tunnel protection ipsec profile CustomerB

Headend2

ip vrf CustomerA
 rd 65000:100
!
ip vrf CustomerB
 rd 65000:200
!
interface Tunnel100
 ip vrf forwarding CustomerA
 ip address 192.168.1.129 255.255.255.128
 no ip redirects
 ip nhrp map multicast dynamic
 ip nhrp network-id 102
 tunnel source Loopback100
 tunnel mode gre multipoint
 tunnel key 102
 tunnel protection ipsec profile CustomerA
!
interface Tunnel200
 ip vrf forwarding CustomerB
 ip address 192.168.2.129 255.255.255.128
 no ip redirects
 ip nhrp map multicast dynamic
 ip nhrp network-id 202
 tunnel source Loopback200
 tunnel mode gre multipoint
 tunnel key 202
 tunnel protection ipsec profile CustomerB

A few details of these tunnel configurations worth pointing out:

  • Tunnel100 is created on both headend routers for Customer A; Tunnel200 for Customer B. A separate /25 network is assigned to each of the four tunnels.
  • The tunnel interfaces are sourced from the loopback interfaces.
  • Each tunnel has been assigned a unique NHRP network ID and tunnel key. These don't necessarily need to match, but it keep things a bit tidier when they do.
  • The shared parameter has been appended to the tunnel protection ipsec profile DMVPN statement. This is necessary because both tunnels reference the same IPsec profile. Also note that the tunnels must be sourced from a physical interface identifier for this to take effect.

The spoke router tunnel interfaces are configured very similarly to the headend routers, with the addition of NHS specifications. Unlike the headend router tunnels, these are sourced from physical interfaces.

CustomerA1

interface Tunnel1
 ip address 192.168.1.2 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.1.1 172.16.0.1
 ip nhrp network-id 101
 ip nhrp nhs 192.168.1.1
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 101
 tunnel protection ipsec profile DMVPN shared
!
interface Tunnel2
 ip address 192.168.1.130 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.1.129 172.16.0.2
 ip nhrp network-id 102
 ip nhrp nhs 192.168.1.129
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 102
 tunnel protection ipsec profile DMVPN shared

CustomerA2

interface Tunnel1
 ip address 192.168.1.3 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.1.1 172.16.0.1
 ip nhrp network-id 101
 ip nhrp nhs 192.168.1.1
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 101
 tunnel protection ipsec profile DMVPN shared
!
interface Tunnel2
 ip address 192.168.1.131 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.1.129 172.16.0.2
 ip nhrp network-id 102
 ip nhrp nhs 192.168.1.129
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 102
 tunnel protection ipsec profile DMVPN shared

CustomerB1

interface Tunnel1
 ip address 192.168.2.2 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.2.1 172.16.0.3
 ip nhrp network-id 201
 ip nhrp nhs 192.168.2.1
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 201
 tunnel protection ipsec profile DMVPN shared
!
interface Tunnel2
 ip address 192.168.2.130 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.2.129 172.16.0.4
 ip nhrp network-id 202
 ip nhrp nhs 192.168.2.129
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 202
 tunnel protection ipsec profile DMVPN shared

CustomerB2

interface Tunnel1
 ip address 192.168.2.3 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.2.1 172.16.0.3
 ip nhrp network-id 201
 ip nhrp nhs 192.168.2.1
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 201
 tunnel protection ipsec profile DMVPN shared
!
interface Tunnel2
 ip address 192.168.2.131 255.255.255.128
 no ip redirects
 ip nhrp map 192.168.2.129 172.16.0.4
 ip nhrp network-id 202
 ip nhrp nhs 192.168.2.129
 tunnel source FastEthernet0/0
 tunnel mode gre multipoint
 tunnel key 202
 tunnel protection ipsec profile DMVPN shared

Whew! We're finally done with configuration. We can verify that the DMVPN spoke routers have dutifully established tunnels to both of the headend routers with the command show dmvpn. The IP address listed for each spoke router in the "Peer NBMA Addr" column is its external (public) IP address.

Headend1# show dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
    N - NATed, L - Local, X - No Socket
    # Ent --> Number of NHRP entries with same NBMA peer
    NHS Status: E --> Expecting Replies, R --> Responding
    UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel100, IPv4 NHRP Details 
Type:Hub, NHRP Peers:2,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.31.0.3     192.168.1.2    UP 00:05:50     D
     1      172.31.0.4     192.168.1.3    UP 00:05:40     D

Interface: Tunnel200, IPv4 NHRP Details 
Type:Hub, NHRP Peers:2,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.31.0.5     192.168.2.2    UP 00:05:41     D
     1      172.31.0.6     192.168.2.3    UP 00:05:45     D

We can test spoke-to-spoke connectivity by pinging from one customer router to the other. The spoke routers should be able to reach one another via either DMVPN tunnel.

CustomerA1# ping 192.168.1.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/12 ms
CustomerA1#show dmvpn      
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
    N - NATed, L - Local, X - No Socket
    # Ent --> Number of NHRP entries with same NBMA peer
    NHS Status: E --> Expecting Replies, R --> Responding
    UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel1, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:2,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.16.0.1     192.168.1.1    UP 00:06:32     S
     1      172.31.0.4     192.168.1.3    UP 00:00:04     D

Interface: Tunnel2, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:1,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.16.0.2   192.168.1.129    UP 00:06:26     S

CustomerA1# ping 192.168.1.131

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.131, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/8 ms
CustomerA1#show dmvpn        
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
    N - NATed, L - Local, X - No Socket
    # Ent --> Number of NHRP entries with same NBMA peer
    NHS Status: E --> Expecting Replies, R --> Responding
    UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel1, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:2,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.16.0.1     192.168.1.1    UP 00:06:44     S
     1      172.31.0.4     192.168.1.3    UP 00:00:16     D

Interface: Tunnel2, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:2,

# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1      172.16.0.2   192.168.1.129    UP 00:06:38     S
     1      172.31.0.4   192.168.1.131    UP 00:00:02     D

Observe how the spoke routers dynamically learn about their peers via NHRP requests sent to the NHS. Once the peer NBMA address has been learned, a new IPsec tunnel is built and the two peers communicate with one another directly.

From this point, one would move on to configuring a routing protocol to run across the DMVPN networks, but that's a topic in its own right and best left for another day.

About the Author

Jeremy Stretch is a network engineer living in the Raleigh-Durham, North Carolina area. He is known for his blog and cheat sheets here at Packet Life. You can reach him by email or follow him on Twitter.

Posted in VPN

Comments


Jay (guest)
January 9, 2012 at 1:46 a.m. UTC

Nice article Jeremy! I'm glad to see you have taken some time away from your busy schedule to post a new blog entry!

Just a heads up to users with 6500/7500s wanting to use DMVPNs - Sup720s are required to be DMVPN hub or spoke. Also, the 'tunnel key' forces the traffic to be software switched. Source: http://www.cisco.com/en/US/docs/ios/12_2t/12_2t13/feature/guide/ftgreips.html

I know it's quasi-off topic, but unfortunately I know this from experience. :)


Steve (guest)
January 9, 2012 at 2:17 a.m. UTC

Ya! Your back! I've checked in everyday, hoping.


Steve Wright (guest)
January 9, 2012 at 8:55 a.m. UTC

Does this design therefore require a specific loopback on the head end per customer? That would burn a lot of IPv4 public address space...

Would it be possible to achieve this with only one public IP?


Jamie (guest)
January 9, 2012 at 10:20 a.m. UTC

Hi Jeremy, Nice Article, Glad to see an update from you, I look forward to labbing this out, some of your previous articles (particularly on VRF and MPLS) have really helped me out, and your cheat sheets are awesome


stretch
January 9, 2012 at 1:09 p.m. UTC

Steve Wright: Great question! I've come with only two ways to configured multiple DMVPNs with unique keys on a single hub:

  • Use one loopback per DMVPN and bind a wildcard pre-shared key to that IP (as in this lab), or;
  • Explicitly define each pre-shared key per DMVPN spoke individually, which requires one to know the external IP address of every spoke ahead of time.

I'd love to hear if anyone has found a more scalable option.


daedalus01
January 9, 2012 at 3:02 p.m. UTC

I've used a third way, but it takes some backend infrastructure to power it... IPsec certificate-based authentication instead of pre-shared keys.

There's a lot more scalability and matching/filtering you can do with certificate authentication, such as matching on CN/DN, issuer name, and other items. The down side is that it's certainly more complicated than IPsec PSKs, but it gets around the "pre-shared key address 0.0.0.0 0.0.0.0" issue of matching the wrong tunnel to the wrong PSK.

For a very static DMVPN, you can define each PSK individually, but otherwise this becomes a huge pain for a very dynamic/large DMVPN.


Clorox (guest)
January 10, 2012 at 6:01 p.m. UTC

It may be off topic but one thing to note with DMVPN if using EIGRP is disabling split horizon and eigrp next-hop-self on the headend routers if they're in a hub and spoke connection. The first one for obvious reasons. The second one is to allow correct next hop resolution for spoke to spoke tunnels. Figured I'd save someone the pain of figuring it out if they look here later.


JBridgelal
April 16, 2012 at 4:38 p.m. UTC

Hey Jeremy,
Nice Work. I am trying to do something a bit similar and I am wondering if you may be able to point me in the right direction.

I have two Main Sites and about 5 or 6 branch sites.

I use a public WWAN that interconnects everyone. a DMVPN cloud is built on that WAN to secure the commenucations as it passes over the WAN.

I need to setup the second Main Site as a failover site and I also need to add Metro Ethernet to mix tp become the primary network.

Therefore, I have two main sites which are each connected to the WWAN and to the Metro E service.

I have 6 Branch Offices which are each connected to the Metro and to the WWAN.

I want to use the Metro as the Primary and the WWAN as the secondary. I am thinking to setup a separate DMVPN cloud on the Metro and connect each branch router to both clouds. Both clouds will have separate sub-nets. Each Site will also contain a Hub that is also connected to both the WWAN and the Metro Ethernet.

What is need to achieve is Redundancy. if the Metro Ethernet goes down, the branch offices should switch to the WWAN and vice versa.

Any ideas you can share?

Thanks in advance.


doncalli
July 10, 2012 at 1:26 a.m. UTC

Jeremy,

Hello, can you post the IP routing design you used for the DMVPN lab?
I have tried to duplicate your DMVPN setup but I am not getting any luck.

The question I have is what are the 172.31.0.X IPs used for? are they used for a P2P link between the customer sites.

Also are you planning on posting the same DMVPN setup but with a routing protocol setup?

Thanks!!


Rudenko Yuriy (guest)
July 19, 2013 at 6:35 a.m. UTC

Thank you. Very helpful post


Michael Whittle (guest)
March 12, 2015 at 1:15 p.m. UTC

This is a really well written article. Nice work.


Tank (guest)
March 19, 2015 at 2:09 p.m. UTC

Hi,

How would I modify this to use Dynamic DNS addresses instead of IP's?

Cheers Tank

Comments have closed for this article due to its age.