Errdisable autorecovery

By stretch | Monday, September 14, 2009 at 7:10 a.m. UTC

There are a number of events which can disable a link on a Catalyst switch, such as the detection of a loopback, UDLD failure, or a broadcast storm. One of the most common causes of error disabling I've seen isn't technically an error, but a violation of a port security policy. Port security is a feature which allows for the restriction of incoming MAC addresses on a layer two switched interface. This is handy for mitigating the use of rogue devices customers purchase at Best Buy to help out with your network infrastructure design. In aggressive configurations, only a single MAC address (corresponding to the attached workstation) will be allowed inbound on a port; any other MAC address will trigger an error and the port will subsequently be disabled.

A default port security policy has been applied to FastEthernet0/1 in this example:

interface FastEthernet0/1
 switchport access vlan 10
 switchport mode access
 switchport port-security
 spanning-tree portfast

We can verify that the port is currently up and associated with a MAC address. Note that the violation mode is "shutdown."

Switch# show port-security interface f0/1
Port Security              : Enabled
Port Status                : Secure-up
Violation Mode             : Shutdown
Aging Time                 : 0 mins
Aging Type                 : Absolute
SecureStatic Address Aging : Disabled
Maximum MAC Addresses      : 1
Total MAC Addresses        : 1
Configured MAC Addresses   : 0
Sticky MAC Addresses       : 0
Last Source Address:Vlan   : 001d.60b3.0add:10
Security Violation Count   : 0

When a violation is detected, the switch automatically places the port in the "err-disabled" shutdown state.

%PORT_SECURITY-2-PSECURE_VIOLATION: Security violation occurred, caused by MAC address
 001d.60b3.0aff on port FastEthernet0/1.

Switch# show interface f0/1
FastEthernet0/1 is down, line protocol is down (err-disabled)
...

By default, manual intervention by an administrator is necessary to restore the interface to working order; this can be done by issuing shutdown followed by no shutdown on the interface. The idea behind requiring administrative action is so that a human engineer can intercede, assess, and (ideally) correct the issue. However, some configurations may be prone to accidental violations, and a steady recurrence of these can amount to a huge time sink for the administrative staff.

This is where autorecovery can be of great assistance. We can configure the switch to automatically re-enable any error-disabled interfaces after a specified timeout period. This gives the offending issue a chance to be cleared by the user (for example, by removing an unapproved device) without the need for administrative intervention.

Switch(config)# errdisable recovery cause psecure-violation
Switch(config)# errdisable recovery interval 300

The above configuration enables autorecovery for port security violations after five minutes. As evident in the list, autorecovery can apply to far more than just port security violations.

Switch# show errdisable recovery 
ErrDisable Reason            Timer Status
-----------------            --------------
arp-inspection               Disabled
bpduguard                    Disabled
channel-misconfig            Disabled
dhcp-rate-limit              Disabled
dtp-flap                     Disabled
gbic-invalid                 Disabled
inline-power                 Disabled
l2ptguard                    Disabled
link-flap                    Disabled
mac-limit                    Disabled
link-monitor-failure         Disabled
loopback                     Disabled
oam-remote-failure           Disabled
pagp-flap                    Disabled
port-mode-failure            Disabled
psecure-violation            Enabled
security-violation           Disabled
sfp-config-mismatch          Disabled
storm-control                Disabled
udld                         Disabled
unicast-flood                Disabled
vmps                         Disabled

Timer interval: 300 seconds

Interfaces that will be enabled at the next timeout:

Now when we force a violation on FastEthernet0/1, we see that it has been placed in the queue for automatic recovery:

Switch# show errdisable recovery
...

Interfaces that will be enabled at the next timeout:

Interface       Errdisable reason       Time left(sec)
---------       -----------------       --------------
Fa0/1          psecure-violation          237

And two hundred and thirty-seven seconds later...

%PM-4-ERR_RECOVER: Attempting to recover from psecure-violation err-disable
 state on Fa0/1

Switch# show interface f0/1
FastEthernet0/1 is up, line protocol is up (connected)
...

About the Author

Jeremy Stretch is a network engineer living in the Raleigh-Durham, North Carolina area. He is known for his blog and cheat sheets here at Packet Life. You can reach him by email or follow him on Twitter.

Posted in Switching

Comments


TacACK (guest)
September 14, 2009 at 8:28 a.m. UTC

Very useful feature, good coverage stretch! :)


Marc Poljak (guest)
September 14, 2009 at 10:40 a.m. UTC

Very helpful. Thank you!


tacack
September 21, 2009 at 3:35 a.m. UTC

First comment as a member! :)


techdocWriter
September 23, 2009 at 6:48 p.m. UTC

Said it in twitter, but for the record - great post, and nx-os works about the same so double bonus!


nick (guest)
September 24, 2009 at 2:39 a.m. UTC

I'm trying to figure out how this is a good thing. If port security is configured to shutdown the interface, why would you want it to automatically recover? Is it not effectively the same as just restricting the MAC without shutting down the interface?


Chris (guest)
September 26, 2009 at 5:19 p.m. UTC

Good question, The reason we decided to let it recover is that we do not have the staffing to log into the switch and babysit everyone that trips the port into error disable. So, what we did was have it reset itself after 10 minutes but if the person still has the violation it will go down again. So the bad device is never on the network. We will also get the alert and know that someone violated the policy and will be contacting their manager.

Self healing.


Vineet (guest)
September 30, 2009 at 7:30 p.m. UTC

I have just enabled UDLD in a setup involving an etherchannel between two 3750s. The good thing is that the ports go into errdisable mode as soon as UDLD is detected but the bad thing is that I have to manually bring the port up. I was wondering if the ports can bring themselves up as soon as UDLD gets fixed. Any say?


setttler
October 1, 2009 at 1:44 p.m. UTC

Gr8 stuff...Very helpful! THNX


Future_of_networking (guest)
October 1, 2009 at 2:47 p.m. UTC

Wow, really nice post stretch. I'm just now learning about networking, on my 2nd year and just started working on switches about 2 months ago. this really helps with the learning process since the cisco site is really lacking. Taking my ccna's in may hopefully haha. Keep blogging i'm gonna need your help xD.


Tanveer (guest)
October 13, 2011 at 7:51 a.m. UTC

Helpful and well explained..

Thanks.
Tanveer.


M.M.Baig (guest)
January 28, 2012 at 8:49 a.m. UTC

packetlife is great source to solve many problems one of them is err-disable,

great source guys

Thanks


Adam Craig (guest)
April 2, 2013 at 11:32 a.m. UTC

Great post.

Is there any way to set the "errdisable recovery" mode for bpduguard back to disabled?

I can't see a way from the configure>errdisable options.

Thanks


udit
August 9, 2013 at 12:57 p.m. UTC

Hey
so if a port has portfast enabled along with bpdugaurd . and it receives a bpdu . It moves to error disabled state . so when we do a shut no shut. does the portfast autorecover in 1) Normal STP
2) RSTP


James (guest)
February 22, 2015 at 9:27 p.m. UTC

So the question I'm trying to answer about err-disable in relation to psecure is this: when auto recovery is set, and port security is set, as in your example above, how does the switchport behave in the event of a persistent violation? Does the switch port simply keep bouncing up and down every however many seconds? Or does the errdisable recover/psecure somehow detect that the error condition is not resolved and wait to start the holddown timer until it is? If the port is shut down, I don't see how psecure could be tracking violations, unless errdisable is different that an ordinary shutdown?

Comments have closed for this article due to its age.