Routed or Switched Access Layer: Why not Both?

By stretch | Friday, September 24, 2010 at 3:32 a.m. UTC

A major consideration when designing an enterprise network is whether traffic at the access layer will be switched or routed toward the distribution layer. Placing multilayer switches in the access layer yields significant advantages; chief among them is the ability to fully utilize all uplinks to the distribution layer (as loops are no longer broken by STP). However, many organizations opt to stick with simple switched (layer two) access schemes to support one or two legacy applications which require a direct layer two path to a server or management station, or to maintain a common subnet for similar devices scattered about multiple access blocks (for example, security systems or environmental monitors).

routed_vs_switched.png

In such a situation, it's quite feasible to extend both routed and switched connectivity to the access layer, provided you have multilayer access switches (such as the Catalyst 3550, 3560, or 3750). Within the IEEE 802.1Q trunks between distribution devices and access switches, one VLAN can be designated as a routed point-to-point link, and one or more additional VLANs can be added for traditional layer two connectivity.

routed_and_switched.png

In this hybrid L2/L3 access design, VLANs carrying normal routable traffic are terminated directly on the access switches as in the L3 design shown earlier. Traffic from end hosts is routed out of these access VLANs and onto one of the point-to-point VLANs shared with the distribution switches. In our example above, traffic from VLANs 100 and 101 is routed from the access switch to the distribution switch via VLAN 2 (a point-to-point /30 link). A separate point-to-point VLAN must be created for each physical link between an access switch and a distribution switch.

This design allows us to still extend some VLANs for legacy nonroutable applications up to the distribution layer. In the example above, VLAN 99 extends to the distribution layer, sharing the same 802.1Q trunks as the point-to-point VLANs.

We can see how this hybrid access design allows us to have our cake and eat it too. It's worth noting, however, that this design reinforces the need for independent layer two and layer three documentation and topology drawings.

UPDATE: I've written a follow-up article to better explain the design discussed here and address some of the comments below.

About the Author

Jeremy Stretch is a network engineer living in the Raleigh-Durham, North Carolina area. He is known for his blog and cheat sheets here at Packet Life. You can reach him by email or follow him on Twitter.

Posted in Routing, Switching

Comments


kwbales (guest)
September 24, 2010 at 3:43 a.m. UTC

This is the same way I build my networks generally. With a mixture of MST or PVST+ I have found it to be quite stable.

Its little more painful in the hosting/SP side of things if you are adding vlans constantly, but it still works (assuming your MST doesnt have a hissy-fit).

Kurt
@networkjanitor


A guest
September 24, 2010 at 8:55 a.m. UTC

Is VLAN 2 a Native VLAN?


ekenny
September 24, 2010 at 11:02 a.m. UTC

So in the hybrid version, the links from the access layer switches to the distribution are both trunks and routed links? Would you do that by defining a native vlan (in your example vlan 2) and then only allow vlans to trunk that need to extend to the access layer (in your example vlan 99)?


GT (guest)
September 24, 2010 at 12:19 p.m. UTC

Mu opinion is that this should not even be thought of when designing a enterprise network. This is really a pain to keep track of if you have like a 1000+ access switches I think even 10 with be enough if you ask me. One of the big reasons you don't do this with Cisco devices is because the pricing of switches able to do dynamic routing which perhaps would be the solution. I strongly recommend people not to do this in a enterprise environment. Instead I would say that this is possible for what ever reason a administrator would need it.. A typical scenario where this is used is to tie up a OSPF square in a datacenter solution where you might have a L2 link connecting two DCs but also using L3 on the WAN side. It's kind of hard to explain if you don't draw that exampel but use your imagination.

So come on Jeremy you can do a lot better then this!

Love your blog though, keep it up!


Santino Rizzo (guest)
September 24, 2010 at 12:32 p.m. UTC

I love this design, but for larger enterprises, I could see some concerns. Moving routing to the access layer requires switches robust enough to perform dynamic routing and maintain enterprise routing tables. Not a problem for newer equipment with the right feature set, but there's a cost and lifecycle factor to consider in larger networks.

Another concern would be complexity. In a large environment, where part your support staff may not have high level networking knowledge, the added complexity of overlapping L2/L3 topologies may become difficult for them to understand, manage, and troubleshoot effectively.

This is a great design for hierarchical campus environments though as it eliminates spanning-tree blocking from those VLANs that do not span multiple switches and potentially allows for L3 load-balancing to distribution pairs. It also allows you to move L3 security features to the access layer which can be advantageous for PCI compliance and HIPAA concerns.


Joe Astorino (guest)
September 24, 2010 at 12:41 p.m. UTC

Very nice article Jeremy! This is a great design tactic to be able to extend some "legacy" L2 VLANs around and also get the benefit of L3 convergence and load balancing as you pointed out.


Jay (guest)
September 24, 2010 at 1:18 p.m. UTC

How about access control? I believe that one could end up in a management nightmare with Access-lists or some sort of firewalling between segments. Those lists would have to end up in all sorts of places to provide necessary security :) (meaning; in the access layer, distribution layer and the core).

I think if your concern is to utilize more links one solution is to do that with multiple groups of HSRP each on different Vlan and distribute the Root Bridge priority for those Vlans across the distribution/core switches.

What do you think ?


stretch
September 24, 2010 at 1:36 p.m. UTC

Whether or not any of the VLANs you define on the trunk are native is irrelevant. For the purposes of this article, it doesn't matter whether a given VLAN is tagged or not.

@GT: Actually, routing to the access edge is the solution endorsed by Cisco, and is becoming quite common for a number of reasons. Also, I would argue that with proper documentation and change controls in place, this design actually scales very well, given that you don't extend the distribution-level L2 VLANs too far.

@Santino: One of the advantages of the hybrid L2/L3 design is that it's a cookie cutter. Admin staff only have to learn one design, which will be configured similarly across all access nodes. Further, this isn't "high-level" networking by any means; a CCNA should have no problems understanding the design.

@Jay: Access controls must exist on the distribution nodes for the switched VLANs, and can exist either on the distribution or access nodes for the VLANs carrying routed traffic. (Generally, you want to avoid performing policy enforcement on core devices.) You can still have all access lists on distribution nodes just as you would have for a L2-only access edge. Also, note that HSRP and root bridge prioritization does not achieve the real load balancing that equal cost path routing gives us.


fader (guest)
September 24, 2010 at 1:53 p.m. UTC

Hi ,all

What is really the difference between two approaches ?

Is L3 faster than L2 or what ? I don't get it

In L3 we have those point to point SVI and CEF is directing the packets to distribution layer whereas in L2 we have the same but in L2 only can you explain it to me ?


stretch
September 24, 2010 at 1:55 p.m. UTC

@fader: Routing from the access layer to the distribution layer allows us to load balance across equal cost uplinks. This is contrast to a layer two design, in which there can be no loops, meaning you can only have active uplink at a time.


whelks
September 24, 2010 at 3:16 p.m. UTC

Juniper is like this, they recommend a 2 layer architecture, the core and access, all their switch are layer3


L00pback0 (guest)
September 24, 2010 at 4:12 p.m. UTC

Stretch,

Tried to catch you on irc but you weren't there. Shouldn't there be a Layer2 connection between the two access layer switches for vlan100 to exist on both switches? How is it possible to have 10.0.100.0/24 on each access layer switch if vlan100 isn't being trunked?


welshydragon
September 24, 2010 at 4:26 p.m. UTC

The whole point about moving L3 to the access is to get rid of spanning tree and all of the trouble it can cause. If fast convergence is the business requirement then this is the way to go. I understand that BASE software images do not support dynamic routing (apart from RIP [sic.]) but most do have EIGRP stub functionality which surely gives you what you need - unless of course you are an OSPF house!!!


GLC3 (guest)
September 24, 2010 at 4:42 p.m. UTC

Cisco is moving towards layer 2 routing in their new Nexus series switches and NX-OS. Spanning tree is going away and being replaced with VPC (virtual port channels). This is really the future for enterprise wide 10G connectivity. There is a good white paper that discusses your topology here contrasted with the newer L2 Nexus model.

Link

Cheers.


Christoph (guest)
September 24, 2010 at 6:26 p.m. UTC

I think this design is not a good one and should be avoided.

First, it requires careful planning in regards of STP, VTP and VLAN pruning, much more than in the "traditional well-known" routed / switched access layer designs.

Secondly, this design might sound feasible, it has the potential to grow to a nightmare when devices are moved to another access switch and they should keep their IP addresses "because its easier and so much better" (in regards of firewall rules, application policies, ... - take a reason of your choice...). If this will be done, you're going to have one of the worst case traffic flow you can imagine, crossing the distribution layer at least twice (once at L2 + once at L3). Additionaly, troubleshooting becomes more difficult, compared to the well-known designs....

Thirdly, L3 convergence depends on STP. If STP blocks the VLAN for the L3 tranfer link between access and distribution switch for some reason, it will take quite a long time until the routing protocol will notice it.

In my opinion, it's just another kludge and kludges are bad...


stretch
September 24, 2010 at 7:32 p.m. UTC

@Christoph: The hybrid design doesn't place any more emphasis on STP planning than does a plain L2-only topology. With regard to your second point, simply which VLAN you place a device into determines whether or not it will need to change its IP when being moved to a new distribution block. Finally, yes, there will be negligible overhead from running STP on point-to-point links, but RSTP does not pose the problem you describe.

Remember, this isn't a new design; it's simply running both an L2 and L3 topology side by side. I've seen it deployed in the real world and it works quite well so long as diligence is applied when spanning L2 VLANs beyond a distribution block. It's definitely not a kludge.


James V (guest)
September 24, 2010 at 7:40 p.m. UTC

I would also avoid this type of mixed topology. Enterprises should be using cookie cutters as much as possible. The more similar your topologies, the less you need diagrams and documentation when troubleshooting, the faster you can identify and fix problems.

If you want to avoid having STP blocked ports look at a VSS, vPC, or a pure routed access model.


cosmicastaway (guest)
September 24, 2010 at 8:23 p.m. UTC

I have the same question as L00pback0: How does vlan100 exist on both access switches without a trunk between them? And would each switch need its own interface vlan100?


stretch
September 24, 2010 at 8:32 p.m. UTC

You wouldn't necessarily have the same VLAN numbers across access switches, though you could (because VLANs are locally significant). You would have separate IP subnets of course. I didn't label the other side of the block because I felt it was redundant.


alan
September 24, 2010 at 9:04 p.m. UTC

Great article. Very clever. This design also means that the access layer vlans don’t need to be on the distribution switches (or in our case the core). And anyway, if you’ve got layer 3 capable access switches – you paid for that capability, why not use it?

Hey Christoph, how does STP block a point-to-point vlan? Ou est le loop?

On the use of the pejorative term – kludge. None other than the great Radia Perlman described NAT as a kludge. Maybe it is from a Utopian technological point of view, but NAT is digging us out of a big hole at the moment. So not all kludges are bad. Just a compromise. Stretch’s design is pretty far from being a just compromise, in my opinion.


Christoph (guest)
September 25, 2010 at 12:05 a.m. UTC

@ strech

With regard to your second point, simply which VLAN you place a device into determines whether or not it will need to change its IP when being moved to a new distribution block.

Sure, but the point is that every VLAN can be made available everywhere, therefore every IP subnet. And from my expirence some people tend to do almost everything that can be done for whatever reasons.

Remember, this isn't a new design; it's simply running both an L2 and L3 topology side by side. I've seen it deployed in the real world and it works quite well so long as diligence is applied when spanning L2 VLANs beyond a distribution block. It's definitely not a kludge.

I've also seen it deployed and I agree with you that it can work quite well if you take care at some aspects (keep VLAN spreading as small as possible, use RSTP, ...) But I've also seen it failing in deployments, where no one really took care about this resulting performance issues and other strange things... Maybe "kludge" is exaggerated, compromise would be a better term

@alan

Hey Christoph, how does STP block a point-to-point vlan? Ou est le loop?

It should never do, but with STP, there should never be a loop too although loops happen. But think about a real-life network where things like accidentally mis-configuration may happen from time to time...


Dano (guest)
September 26, 2010 at 7:43 p.m. UTC

Here's a real world example of why you might want to do this:

Our network has a wifi SSID that has to be available at every site. This wifi network we have is not a Cisco wifi network so it does not use LWAPP tunnels etc. Therefore it HAS TO BE layer 2 everywhere. We would normally have a problem with this because every link back to home would have to be layer 2 just to make wifi happy. We do that with the hybrid VLAN concept with 3560's, and for our desktop hosts, we provide them a local layer 3 VLAN.

The person who made the point about the complexity of this is correct. It is kind of hard to understand looking at show runs. The way I see it is the EIGRP routing process looks in all the VLANS for interesting layer 3 stuff like those hello's and acts accordingly. All you have to do is advertise your local VLAN(s). These local VLANS do not go over the trunk, but the layer 2 VLANS do.

Dano


geert (guest)
September 27, 2010 at 1:58 p.m. UTC

If you are forced to have extended VLANs across Access Switches, of course, this is the way to go. You put your "special" devices in the special L2 VLANs and your standard PCs in a Layer3 segmented VLAN. HOWEVER, i must say this design just gives you a false sense of security. One of the main reasons to go to Layer 3, is to solve all problems with STP. By keeping one L2 Vlan on the trunks, you solve nothing. Actually, a spanning tree loop in your extended L2 VLAN will still bring down your whole network, including the L3 part. So this is a design for "lazy" people. The only good solution is: go to full L3 Access (i don't mean routed to the Access level, just VLANs/Access and a L3 connection between your core/distris), eliminate all extended L2 VLANs. If this means changing your applications, redesigning your Wifi network, so be it... In the end, you will have gained stability. You will have lost "easy ip management" and "easy moving" though :-)


dps218
September 27, 2010 at 5:26 p.m. UTC

Geert,

Come on man, we all know that Wifi is the future and is therefore more important than wired . We would sooner go back to using typewriters before re-engineering the ridiculous wifi.

I agree about the layer 2 loop possibility. Good thing we got iterations of STP protecting us. Only problem is most network people don't know how to troubleshoot STP all too much. In our network we turn off STP for some VLANS, but not on all switches where that VLAN exists. I'm not really sure how we have been able to avoid bridge loop disaster.

Dano


darenmatthews (guest)
September 28, 2010 at 7:01 p.m. UTC

I guess you could add into this VTP domains.

At Access Layer, you could use L2 switches with trunks (uplinks) to Distribution Layer where SVI's perform inter-VLAN routing. If you use two L3 switches, you can have HSRP/VRRP/GLBP for L3 SVI resilience (and pseudo or real load balancing if you alter priorities for VLANs, alternating between the two L3 Distribution switches or use GLBP).

That would be one VTP Domain (say VTP1). CIDR block would be say, 10.0.0.0 /16 further subnetted for each VLAN

Now replicate this entire model. VTP2 would be 10.1.0.0 /16

You can use the same VLAN numbers on each VTP domain, for consistency. You can add a core layer and route between subnets to reach devices on other VTP domains / subnets / vlans.

This model would be very scalable. You can also change core layer from say Frame-Relay to MPLS - in other words the design is modular. QoS trust boundaries and where you mark is easy to determine and manage, it's all good.

Normally the Access Layer connects devices to the network. The Distribution Layer performs inter-VLAN routing / packet filtering / IGP Routing

The Core Layer's job is to shift data as quickly as possible. No ACL's or other stuff, just shift data.

This is only my opinion of course.


rostov (guest)
October 1, 2010 at 2:59 p.m. UTC

Little question. In the example above, to realize a load balancing among links connecting access- distribution switch you must disable spanning tree for vlan 2 on whole network. it's right?


magik (guest)
October 1, 2010 at 7:56 p.m. UTC

I used this hybrid method at our corporate offices, for about 20 IDF closets that were Cat 4507R's. The solution was effective and allowed us to go active/active on the IDF-to-Core uplinks. However, the solution is a pain to support and is a litle more complicated to troubleshoot.

We had this in place for years with no issues, but went back to L2 when we upgraded the core to Nexus 7k's. Now we run VPC's and are back to straight L2.

If I had to do it all over again in a non-Nexus world, I'd probably have passed on the hybrid and gone straight L3.


bufo333
October 7, 2010 at 2:21 p.m. UTC

Juniper is always pushing this approach. I am not sure the slight performance boost as they put it, trumps the complexity of the design. Its not complex by itself, but when you factor in the security and business requirements, I can see this thing growing out of control if you work for a company as dynamic as mine.

If your goal is just eliminating spanning tree, and you are talking about a controlled data center environment ( not user switches) I would recommend disabling spanning tree and configuring flex links. Its a common practice in the ISP environment.

I will say that this scheme is slightly easier to manage with juniper switches, as you can manage all the switches in your company as one giant virtual switch.


GT (guest)
October 7, 2010 at 9:56 p.m. UTC

See that is wrong. Cisco does not recommend mixing L2 and L3 on enterprise environment. No matter what Cisco says I still wouldn't recommend it.


jrb (guest)
November 20, 2010 at 9:55 a.m. UTC

Ok, can I throw another scenario in for your comments? What if the distribution layer consists of a single stack of 3750s and the Access layer consists of blocks of 3560s connected redundantly to one another using the SFPs. I guess you'd need to run Spanning-Tree within the 3560 Access blocks, but I'm not sure how I would go about configuring dual uplinks to distribution layer considering there will only be the one SVI per VLAN (because the distribution layer is a single stack).

Port-channelling isn't an option here because the 3560s are not stackable and I would want the uplinks to come from different switches in the block.

Great blog btw!


iproute (guest)
February 16, 2011 at 12:11 a.m. UTC

Nice Post. Add VRF into the mix and you can have path isolation to a central firewall across L3 boundaries.

To the naysayers...study up. You will see the benefit. Do not dismiss something simply because you do not understand.

Layer 2 adjacency requirements are becoming a thing of the past, and the need to span vlans across access layers switches is going away. Keep in mind that Cisco's tried and true L2 design doesn’t have vlan's spanning across access layer switches to avoid STP convergence, solely relying on HSRP failover. With layer 3 access layer links, both links are forwarding and load balancing.


Xytek (guest)
January 29, 2012 at 5:28 a.m. UTC

OMFG! I'm glad you wrote this up ( almost two years old now), but i designed the exact same thing for a client of mine because of the constraints they have, and i was looking to see if anyone else have done this. I'm glad someone else not only have done this, but knows for a fact it works. The constraints my clients have is that they have 2960's and currently running on a totally flat network ( think 1980's - early 90's network). Now, you might think, 2960's.. layer 2 only switches.. but oh now.. the new ones as of 12.2(55)SE , with smd prefer lanbase-routing enabled turn these puppies into limited Layer3 devices! they don't do dynamic routing (and to my dismay) no routed interfaces. But once i created the routable vlan over the 802.1q trunk, all was golden. It kept me from going back to stickly layer 2 access layer, doing trunks and running all of the SVI's off the core ( which is an 8port 3560.. i did say i have constraints) :)


Kamil (guest)
October 21, 2013 at 2:38 p.m. UTC

Pseudo wires can additionally be used ro L2 connectivity over L3 network if the IOS supports it.


Mr. Mariano (guest)
September 3, 2014 at 6:03 a.m. UTC

This kind of design is very applicable for server farms where much of data are being process from time to time, campuses which taking online lessons (audio, video, and live conversation) but for some company, the old style of access-distribution-core layer design will be more consider for normal activities due to cost-budgeting.

But for companies who really rely on the internet for everything like what I've mention in the first place, it would be a good practice.

@stretch -> Thank you sir again for this additional knowledge. "Compilation of words and ideas makes a better world!"

Comments have closed for this article due to its age.