The premiere source of truth powering network automation. Open and extensible, trusted by thousands.

NetBox is now available as a managed cloud solution! Stop worrying about your tooling and get back to building networks.

Route Distinguishers and Route Targets

By stretch | Monday, June 10, 2013 at 2:27 p.m. UTC

People new to MPLS VPN are often unclear on what functions route distinguishers and route targets serve, and the difference between the two. Let's see if we can clear up some of that confusion. If you could use a refresher on VRF fundamentals, I encourage you to first check out my earlier articles on the topic, Intro to VRF Lite and Inter-VRF Routing with VRF Lite.

Route Distinguisher

As you know, VRFs allow IP address space to be reused among isolated routing domains. For example, assume you have to connect to three customer sites, all of which are using 192.168.0.0/24 as their local network. We can assign each customer its own VRF so that the overlapping networks are kept isolated from one another in their respective routing domains.

This works well, but we need a way to keep track of which 192.168.0.0/24 route belongs to which customer. This is where route distinguishers come in. As its name implies, a route distinguisher (RD) distinguishes one set of routes (one VRF) from another. It is a unique number prepended to each route within a VRF to identify it as belonging to that particular VRF or customer. An RD is carried along with a route via MP-BGP when exchanging VPN routes with other PE routers.

An RD is 64 bits in length comprising three fields: type (two bytes), administrator, and value. There are currently three defined formats which can be used by a provider:

RD_formats.png

The choice of format is largely cosmetic: It makes no difference to BGP as the RD is effectively just a flat number prepended to a route. The choice of formats exists solely to allow for flexible administration of the number space.

Here's an example IOS config showing RDs assigned to VRFs using the two-byte AS format. The type is implied by the format in which we've assigned the RD.

ip vrf Site_A
 rd 65000:10
!
ip vrf Site_B
 rd 65000:20
!
ip vrf Site_C
 rd 65000:30

When VPN routes are advertised among PE routers via MP-BGP, the RD is included as part of the route along with the IP prefix. For example, a route for 192.0.2.0/24 in VRF Site_B is effectively advertised as 65000:20:192.0.2.0/24 (ignoring some complexities concerning the actual prefix length).

Route Targets

Whereas route distinguishers are used to maintain uniqueness among identical routes in different VRFs, route targets can be used to share routes among them. We can apply route targets to a VRF to control the import and export of routes among it and other VRFs.

A route target takes the form of an extended BGP community with a structure similar to that of a route distinguisher (which is probably why the two are so easily confused). One or more route targets can be affixed to routes within a VRF using the VRF configuration command route-target export:

ip vrf Customer_A
 rd 65000:100
 route-target export 65000:100

Routes contained within this VRF will be exported with an RT of 65000:100. We can use Wireshark to examine precisely how this information is carried in an MP-BGP update:

wireshark.png

In practice, we typically want all instances of a VRF across the network to learn about all other routes within that VRF on neighboring routers. So, we add a second statement to enable importation as well:

ip vrf Customer_A
 rd 65000:100
 route-target export 65000:100
 route-target import 65000:100

(Tip: You can use the shortcut command route-target both as a macro to add both commands simultaneously. Both commands will show up separately in the running configuration.)

Now, this VRF will export its own routes with an RT of 65000:100 as well as import any routes exported from other VRFs tagged with 65000:100. It makes sense to reuse the VRF's RD as a route target here, but this isn't a hard rule. (Remember, route targets are just numeric tags: We can use whatever values we want.)

Next, let's say both customers A and B need to import routes from a third VRF named Shared. We can configure the two customer VRFs to also import routes tagged with the Shared VRF's export RT of 65000:300:

ip vrf Customer_A
 rd 65000:100
 route-target export 65000:100
 route-target import 65000:100
 route-target import 65000:300
!
ip vrf Customer_B
 rd 65000:200
 route-target export 65000:200
 route-target import 65000:200
 route-target import 65000:300
!
ip vrf Shared
 rd 65000:300
 route-target export 65000:300
 route-target import 65000:300

VRFs Customer_A and Customer_B will now import routes exported from VRF Shared in addition to their own "native" routes. To complete our VRF configuration, VRFs Customer_A and Customer_B also need to export their native routes to VRF Shared. But wait a minute. If we configure the customer VRFs to export to 65000:300, they'll end up learning each other's routes, because both are already importing 65000:300. We don't want that!

The solution here is to export using a different RT than the one used for import. You can pick whatever number you want. To illustrate this point, we'll use 65000:1234.

ip vrf Customer_A
 rd 65000:100
 route-target export 65000:100
 route-target export 65000:1234
 route-target import 65000:100
 route-target import 65000:300
!
ip vrf Customer_B
 rd 65000:200
 route-target export 65000:200
 route-target export 65000:1234
 route-target import 65000:200
 route-target import 65000:300
!
ip vrf Shared
 rd 65000:300
 route-target export 65000:300
 route-target import 65000:1234

With this approach, both customers can reach the Shared VRF but not each other, and the Shared VRF can reach both customers.

VRF_import_export.png

Alternatively, we could have configured VRF Shared to import routes with tagged with 65000:100 and 65000:200. This would have worked, however this approach doesn't scale well. Imagine if we had a hundred customer VRFs to export to: An additional community would need to be imported for each. Designating a separate RT for export to the Shared VRF allows us to scale to any number of customer VRFs using only a single pair of targets.

Posted in Routing

Comments


Stuart
June 10, 2013 at 3:24 p.m. UTC

One thing on RDs, the RD doesn't actually specify that it belongs to a VRF in itself, just that it is a unique route.

The only reason I mention is we've had some issues with a certain vendors equipment recently where route reflectors for VPNv4 routes were not installing routes correctly when the same RD is used on the two route reflectors. Using two different ones is their recommended fix (so use 65001:1 on one RR, 65001:2 on the other). Unfortunately it is a bug with their implementation of VPNv4 route reflection which isn't fixed in any later releases so we have to use this to fix the issue.

Then again we haven't seen this in practice yet so this could all go very wrong....


ddunkin
June 10, 2013 at 4:55 p.m. UTC

The shared VRF has always been a struggle for me to comprehend. If you are providing services on the shared VRF (say a public unique IPv4 subnet), and two of the VRFs you are sharing with both have a route for the same subnet, how would the router/shared service differentiate between the two without implementing NAT? I can imagine you hit a scaling issue as soon as two customers use the same subnet (all my customers have a 10.0.0.0/24 route).


stretch
June 10, 2013 at 6:48 p.m. UTC

@ddunkin: You're absolutely correct. A shared VRF can be used to allow multiple customers to access common resources (but not another), but all customers would need to be in unique address space. Or, you could NAT each customer VRF into unique space you control (likely introducing an entirely new set of problems).


killabee
June 11, 2013 at 2:07 a.m. UTC

Great article! Straight forward and to the point as always.


pro
June 11, 2013 at 11:41 a.m. UTC

"A shared VRF can be used to allow multiple customers to access common resources (but not another), but all customers would need to be in unique address space"
Could you please clarify this statement ?

Does this mean if Customer_A and Customer_B have the same ip range 10.0.0.0/24, and need to access the VRF Shared route of 192.168.1.0/24, would the routers in VRF Shared have issues routing the traffic back to the respective customers ?


stretch
June 11, 2013 at 1:14 p.m. UTC

@pro: That's correct. When BGP finds duplicate routes being imported into a VRF, it only imports the best one (per BGP's path selection process). So, VRF Shared would only have a route to customer A or customer B; not both.


Blurp
June 11, 2013 at 1:41 p.m. UTC

This may have been mentioned but we used different RDs at different sites to avoid problems with dual-homing and Route Reflectors. There's a pretty good discusion on it at IPspace called load sharing in MPLS/VPN with route reflectors. The topic is out of scope of the original article but I've picked up good info in the comments before.


ddunkin
June 12, 2013 at 8:38 p.m. UTC

@stretch Thanks for confirmation! I 'almost' saw an MPLS monitoring solution that doesn't involve NAT, or virtual machines, back to the VMs :)


andrew
June 19, 2013 at 6:15 p.m. UTC

Is there a way to specify a route-map or prefix-list with your exports?

How do you handle it if you want to only export a subset of the routes within your VRF?


neferith
June 20, 2013 at 8:17 a.m. UTC

Excellent explanation. Keep on going!


Bob
June 22, 2013 at 5:19 p.m. UTC

Hey Stretch,

I thought we were going to use IPv6 in examples nowadays ;-)

You should configure the VRFs with the multi-protocol syntax.

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_vpn_ipv4_ipv6.html


Zoltan
June 25, 2013 at 3:07 p.m. UTC

Hi Stretch,

"When BGP finds duplicate routes being imported into a VRF, it only imports the best one (per BGP's path selection process). So, VRF Shared would only have a route to customer A or customer B; not both."

Why we have duplicated routes? DR does not play here in this case? I suppose that routes come from Customer_A has different RD than routes come from Customer_B.


luismg
July 12, 2013 at 7:44 a.m. UTC

So if I need to provide a mail server or a dns server for the customers in differents vrfs this is a good trick thank you


djfader
July 17, 2013 at 10:25 p.m. UTC

For example, a route for 192.0.2.0/24 in VRF Site_B is effectively advertised as 65000:20:192.0.2.0/24 (ignoring some complexities concerning the actual prefix length).

@Stretch

What do you mean by some complexities ?


titozofito
October 10, 2013 at 8:06 p.m. UTC

Thanks stretch, you publish "good food".
Nevertheless, let me point to one serious mistake regarding RD. RD has only local significance, it is not attached to the advertised route in any way (it's the role RT plays).

RD only distinguishes routes within a box, thus providing separation between vrfs. That's all to RD.


stretch
October 10, 2013 at 9:22 p.m. UTC

@titozofito: That's incorrect. Look again at the Wireshark screenshot in the article. You can see that the RD (65000:100 in this example) is included in the BGP NLRI.


AnonGuest
November 13, 2013 at 5:58 a.m. UTC

@titozofito: stretch is right, the main reason to send the RD is to make the VRF prefixes unique when MP-BGP carries them. This is called a vpnv4 prefix and is carried by MP-BGP between the PEs.


madaha
December 28, 2013 at 10:48 a.m. UTC

I found a problem: two routers configured for ppp encapsulation type directly into the OSPF interfaces after, but I put for example: R1 interfaces configured 172.16.12.1/24, R2 interfaces configured 172.16.12.2/21, and found the two still become full adjacency, however, the book says: to form adjacencies must ensure that both interfaces belong to the same OSPF area, subnet and the subnet mask must be the same, is it not contradictory? Why?


navs
January 7, 2014 at 9:31 a.m. UTC

"Does this mean if Customer_A and Customer_B have the same ip range 10.0.0.0/24, and need to access the VRF Shared route of 192.168.1.0/24, would the routers in VRF Shared have issues routing the traffic back to the respective customers ?"

Am i correct in saying in this example the customer A and B 10.0.0.0/24 routes will be differentiated by their respective RD values ?

Half the confusion for me arises from people using the same RD and RT values, which when your learning isnt great.


Sanks
January 20, 2014 at 7:29 a.m. UTC

Thanks. your page helped me understand the use case(s) for having multiple RT exports within the same vrf, that I couldn't from elsewhere that I searched

Thank you


Surashree
April 8, 2014 at 6:57 a.m. UTC

HUGELY helpful! Thank you! :)


Deepika
May 8, 2014 at 1:20 p.m. UTC

Very interesting article!


raju
June 7, 2014 at 1:06 p.m. UTC

Can someone answer Andrew's question? I have the same question and would like to know. Is there a way to specify a route-map or prefix-list with your exports? How do you handle it if you want to only export a subset of the routes within your VRF?


Shafi
June 18, 2014 at 2:38 a.m. UTC

Great explanation.. It helps a lot.. Thanks a Ton!!


Andy
June 20, 2014 at 10:47 a.m. UTC

Good explanation - great work!


Thanzy
July 10, 2014 at 6:14 a.m. UTC

helpful thanks a lot


shankar
August 19, 2014 at 10:40 a.m. UTC

Hi, Is there any specific limit for configuring route-target. like only 64 exports can be configured like this?


ahsan828
September 11, 2014 at 5:08 a.m. UTC

Why we are importing values in RT ??? and why not exporting ?????


A guest
January 21, 2015 at 3:06 p.m. UTC

Thanks for this little guide :D really useful!!!!


Addictive
January 28, 2015 at 2:18 a.m. UTC

@Andrew and @Raju

You can leak specific routes between VRFs

This is one interesting example

https://rekrowten.wordpress.com/2012/09/24/route-leak-between-vrfs-with-import-maps-and-export-maps/


Alex G.
June 24, 2015 at 7:16 a.m. UTC

Very wonderful topic!!


Joel
July 22, 2015 at 10:19 p.m. UTC

Beautifully written explanation. Thank you!


Mehdi
July 24, 2015 at 4:56 a.m. UTC

Hi Jeremy,

Thank you so much for this outstanding explanation


Carlos
December 17, 2015 at 12:04 a.m. UTC

Hi Jeremy,

What program did you use for your VRF export/import interaction diagram?


Hashknife
February 24, 2016 at 2:27 a.m. UTC

Wow! Mr. Stretch, You've allowed an old cowboy turned network engineer to really understand the concepts of RD and RT. RD & RT are subtle but important pieces in the MPLS-VPN puzzle. Excellent Article.


Mahmood Khan
April 9, 2016 at 1:11 p.m. UTC

Thanks a lot mind blowing explanation


Nikoloz
April 25, 2016 at 7:11 p.m. UTC

Again very good article and best explanation.


Dave
May 5, 2016 at 6:25 p.m. UTC

To help understand the above example, if there were routes that originated from vrf Shared, would you need to have VRF Shared Import the 65000:300 so the VRF Shared will get routes that were pushed out from one PE to another PE? so add: route-target 65000:300 import To get routes from VRF Shared injected into VRF Shared on the other side.


Greg
May 6, 2016 at 1:57 p.m. UTC

Amazing Article ! Thanks a lot


Me
June 6, 2016 at 7:00 a.m. UTC

Guys anyone knows RT how many bits ? I tried to find the answer but still confused as in one source I read that it's the higher order 16 bits of the bgp exteneded community 64 bits , so it is 16 bits ? but in other resource I read that it the same like RD 64 bits


Girish
June 7, 2016 at 10:05 a.m. UTC

Excellent explanation. I was able to understand just by reading starting two paragraphs.


networkdj
July 23, 2016 at 6:08 p.m. UTC

to answer raju's question -

"Can someone answer Andrew's question? I have the same question and would like to know. Is there a way to specify a route-map or prefix-list with your exports? How do you handle it if you want to only export a subset of the routes within your VRF?"

You can use the command import-map and export-map and refer the command to a prefix-list to filter out what prefixes you want to import and export. You can find more information in this post here: https://supportforums.cisco.com/document/140196/mpls-vpn-importexport-maps


Nikhil
August 1, 2016 at 8:22 a.m. UTC

Hello Jeremy,

When the VRF shared is importing the routes from both the customer VRF's, it has routes to both in its routing table.

Now all these routes are again exported with 65000:300 & which are imported by both customer VRF's therefore, Wont both the customer VRF's have route for each other making VRF shared transit ?

Regards, Nikhil


abbas55
September 25, 2016 at 5:37 a.m. UTC

helpful thanks


lem
October 9, 2016 at 10:19 a.m. UTC

Hi. Just had set shared VRF with route targets import and export of customer VRFs i.e: CustA: RD 1:1, RT both 1:1 CustB: RD 1:2, RT both 1:2 Shared: RD 1:3, RT both 1:1, 1:2, 1:3 Shared VRF contains all routes from both customers but customers aren't learning each others routes. Any thoughts?


FIre
October 19, 2016 at 1:06 p.m. UTC

Is it correct that in Cisco devices I need to have at least one export route-target and an import route-target statement in a VRF for it to work well?

Comments have closed for this article due to its age.