ERSPAN from NX-OS to IOS

By stretch | Tuesday, May 14, 2013 at 6:27 p.m. UTC

Most readers are probably familiar with the switchport analysis (SPAN) feature on Cisco's Catalyst switches. SPAN replicates all ingress and/or egress traffic from one or several interfaces to another for the purposes of packet capture or traffic monitoring. This is especially helpful when deploying a network-based IDS. Unfortunately, it's often not possible to install the IDS on the same physical switch as the ports from which you want to capture.

Remote SPAN (RSPAN) can be employed to extend a SPAN session between source and destination points on disparate switches, however it requires a layer two path end-to-end. When we need to replicate layer two traffic across a layer three network, we turn to encapsulated remote SPAN (ERSPAN). ERSPAN transports traffic inside a point-to-point GRE tunnel between arbitrary IP endpoints.

For this lab, we'll configure an ERSPAN session from an NX-OS source (a Nexus 7K) to an IOS destination (a Cisco 7600) to provide an example configuration for both platforms. MPLS transport is used between the two switches and routing of the ERSPAN tunnel will take place inside a VRF named Capture.

erspan_topology.png

NX-OS Source

First, we'll configure the Nexus switch as the ERSPAN source per the documentation. We'll use Lo1 in the Capture VRF as our ERSPAN tunnel source.

interface loopback1
  vrf member Capture
  ip address 172.16.0.2/32

The monitor session must be created of the erspan-source type. We'll assign a unique ERSPAN ID of 100 (valid values are 1 through 1023 inclusive) and assign the ERSPAN tunnel to the Capture VRF. The destination IP is set to the loopback interface of our destination switch (which must be routable within the Capture VRF) and we configure VLAN 142 as the SPAN source; this could alternatively be a physical interface or set of interfaces. Lastly, no shut enables the ERSPAN session.

monitor session 1 type erspan-source
  description ERSPAN to 7600
  erspan-id 100
  vrf Capture
  destination ip 172.16.0.1
  source vlan 142 both
  no shut

Our source configuration is almost complete, but an additional global command is necessary for ERSPAN to function. We need to designate Lo1 as the origin IP address for the GRE tunnel. The global keyword here signifies that the command applies across all Nexus virtual device contexts (VDCs).

monitor erspan origin ip-address 172.16.0.2 global

We can verify the operation of ERSPAN with the command show monitor session.

Nexus7K# show monitor session 1
   session 1
---------------
description       : ERSPAN to 7600
type              : erspan-source
state             : up
erspan-id         : 100
vrf-name          : Capture
acl-name          : acl-name not specified
ip-ttl            : 255
ip-dscp           : 0
destination-ip    : 172.16.0.1
origin-ip         : 172.16.0.2 (global)
source intf       :
    rx            :
    tx            :
    both          :
source VLANs      :
    rx            : 142
    tx            : 142
    both          : 142
filter VLANs      : filter not specified

IOS Destination

For the destination configuration, we define the physical destination interface, ERSPAN ID, transport VRF, and local IP address. Note that we are specifying the IP address used as the destination, not the origin/source IP. This tells ERSPAN what interface to listen on for incoming GRE-encapsulated traffic.

interface Loopback1
 ip vrf forwarding Capture
 ip address 172.16.0.1 255.255.255.255
!
monitor session 1 type erspan-destination
 description ERSPAN from Nexus 7K
 destination interface Gi1/48
 source
  erspan-id 100
  ip address 172.16.0.1
  vrf Capture
7600# show monitor session 1
Session 1
---------
Type                   : ERSPAN Destination Session
Status                 : Admin Enabled
Description            : ERSPAN from Nexus 7K
Destination Ports      : Gi1/48
Source IP Address      : 172.16.0.1
Source IP VRF          : Capture
Source ERSPAN ID       : 100

We can now connect our collection device to Gi1/48 and start sniffing. All traffic entering VLAN 142 on the source switch will be replicated out this interface.

About the Author

Jeremy Stretch is a network engineer living in the Raleigh-Durham, North Carolina area. He is known for his blog and cheat sheets here at Packet Life. You can reach him by email or follow him on Twitter.

Posted in Switching

Comments


Hamid (guest)
May 15, 2013 at 11:20 a.m. UTC

Thanks Jeremy.


Will (guest)
May 15, 2013 at 9:16 p.m. UTC

Thanks a lot. I'll use this as a ref if I ever need it. Few random comments below:

My biggest gripe with ERSPAN is Change Control. Most change control I see allows for ad-hoc SPAN sessions (good). Most do not allow for ad-hoc tunnel standups. That makes ERSPAN basically useless as it's so much quicker to find other means to get a SPAN.

Also when considering NX-OS related devices the interfaces are typically 10Gb or bigger. So that hinders ERSPAN even more.


Rob M (guest)
May 17, 2013 at 7:25 p.m. UTC

Thats a shitton cheaper than a bunch of APCON/GIGAMON patch chassis and a dedicated hardware sniffer.


Sujay (guest)
May 18, 2013 at 7:32 p.m. UTC

Nice info very useful for isp environment capturing. Thanks Jeremy.


Tony (guest)
May 19, 2013 at 8:08 p.m. UTC

ERSPAN is awesome. The only problem as I see it is its limited deployment. Most access switches, such as the 3560, will only support Remote-SPAN. However, the Nexus does NOT support Remote-SPAN but supports ERSPAN. I can't think of a work around...


Yasser (guest)
May 23, 2013 at 9:01 a.m. UTC

Great explanation... thanks

Regards Yasser


Nizar (guest)
May 24, 2013 at 2:13 p.m. UTC

"Unfortunately, it's often not possible to install the IDS on the same physical switch as the ports from which you want to capture."

Can you tell me which switch model that have this problem? I thought most switches support local SPAN?


Rick (guest)
May 27, 2013 at 5:19 p.m. UTC

@Nizar, lack of available ports might be the problem rather than the platform.


Barani (guest)
September 12, 2013 at 1:43 p.m. UTC

Awesome work Jeremy,

You rock.

Leave a Comment


Optional; will not be displayed publicly or given out.
No commercial links. Only personal (e.g. blog, Twitter, or LinkedIn) and/or on-topic links, please.
What protocol is typically used to send email?