Disabling unreachables breaks PMTUD
After reading the reposting of my article on path MTU discovery in NIL's CT3 wiki, Steve Milsom wrote in to share an interesting observation. Disabling ICMP unreachable messages on a router with no ip unreachables (as is recommended by many BCP documents) has a side effect of breaking path MTU discovery. Let's take a look at what happens.
Recall that path MTU discovery is accomplished by setting the Don't Fragment (DF) bit in an outgoing IP packet, and receiving an ICMP unreachable message in the event that the packet sent exceeded the MTU at some point in the path. We can create an artificially small outgoing IP MTU on an interface with the ip mtu command in IOS to see path MTU discovery in action. We'll set the IP MTU of R2's F0/0 interface to 1200 bytes.
R2(config)# interface f0/0 R2(config-if)# ip mtu 1200
Now, by setting the DF bit in oversized ping from R1 to R3 we can see that we receive ICMP unreachable messages from R2, indicating that packet fragmentation is required to meet R2's outbound MTU.
R1# ping ip 10.0.0.6 df-bit size 1400 Type escape sequence to abort. Sending 5, 1400-byte ICMP Echos to 10.0.0.6, timeout is 2 seconds: Packet sent with the DF bit set M.M.M Success rate is 0 percent (0/5)
The letter M (for MTU, I suppose) in the output signifies the reception of an ICMP unreachable packet with code 4 (fragmentation needed). (The alternating dots indicating missed responses are due to IOS' default ICMP rate-limiting.)
Now we'll disable the transmission of ICMP unreachables out of R2's F0/1 interface with the no ip unreachables command:
R2(config)# interface f0/1 R2(config-if)# no ip unreachables
Observe what happens when we try the oversized ping from R1 again:
R1# ping ip 10.0.0.6 df-bit size 1400 Type escape sequence to abort. Sending 5, 1400-byte ICMP Echos to 10.0.0.6, timeout is 2 seconds: Packet sent with the DF bit set ..... Success rate is 0 percent (0/5)
Instead of responding with helpful ICMP messages, R2 simply drops our oversized packets. Think carefully about where you disable ICMP unreachables, as our experiment shows that path MTU discovery simply doesn't work without them.
About the Author
Jeremy Stretch is a freelance networking engineer, instructor, and the maintainer of PacketLife.net. He currently lives in Fairfax, Virginia, on the edge of the Washington, DC metro area. Although primarily an R&S guy, he likes to get into everything, and runs a free network training lab out of his basement for fun. You can contact him by email or follow him on Twitter.
Comments
Yep, firewalls can interfere with this. You need to make sure that the firewall permits ICMP unreachable (type 3, code 4).
And Cisco "autosecure" feature automatically disables "ICMP unreach" on all interfaces. ;-(
You need to be extra careful when disabling unreachables on VPN tunnel interfaces. tunnel pmtud is also broken when unreachables are disabled.
IPV6 will fix this issue....
To the commenter who blocks all ICMP traffic, I don't think this is a good practice for this very reason.
Regarding PMTUD and the no ip unreachables command, I learned this the hard way a few years ago. Cisco routers doing Lan to Lan VPN. Disabled the unreachables on all interfaces. Everything went totally to crap. Found that the inside interfaces should not have "no ip unreachables". Problem resolved.
Yeah, funny that so many people with more than a mont's experience have been bitten in the ass by this but it's still taught as default secure template stuff. The number of legacy apps no ip unreachables breaks is simply stunning, and the number of "I spent days and sleepless nights trying to figure out what had changed on our network..." stories I've read which eventually boiled down to "someone stuck this line in a config somewhere because . . . well, because." gives me heartburn - moreso because I've been there myself.
I don't see any security benefit when disabling ICMP unreachables. I am also against blindly implementing (or disabling) features because they appear in some security templates from the 90s.
Check my post about 'Why disabling ICMP unreachables is a bad thing'.
My understanding is this, and I'd welcome corrections or additions...
Disabling ICMP unreachables is useful because they are generated by the router CPU, and you are potentially exposed to a Denial of Service through CPU resource exhasustion by allowing ICMP to be generated.
In many Cisco routers the solution is simply not to punt packets which would generate ICMP messages to the CPU, and this is done using "no ip unreachables".
Unfortunately the side-effect is that Path MTU Discovery breaks, because that also uses one of the ICMP unreachable messages.
The solution is to selectively punt packets which you want ICMP messages for, but I think this feature is only available on certain 6500 models. To allow ICMP for everything but ACL-denied packets (a reasonable default) use the command:
mls rate-limit unicast ip icmp unreachable acl-drop 0
And if you search for that command on cisco.com up will come the command reference with more details.
HTH, oliver.
I also wonder how firewalls interfere with PMTUD?
We block all ICMP packets on our firewalls believing it to be security best practice. Is this incorrect?