Evaluating Network Gear Performance

Choosing the right equipment for your network is hard. Even ignoring the ever-growing roster of features one must account for when evaluating candidate hardware, it's important not to overlook performance limitations. Below I describe some of the most crucial characteristics to consider when doing your research.

Throughput

Throughput is the rate at which a device can convert input to output. This is different from bandwidth, which is the rate at which data travels across a medium. An Ethernet switch, for example, might have 48 ports running at an individual bandwidth of 1 Gbps each but be able to switch only a total of 12 Gbps among the ports at any given time. This is said to be the switch's maximum throughput.

Throughput is measured in two units: bits per second (bps) and packets per second (pps). Most people are most familiar with bits per second. This is the amount of data which flows through a particular point within a duration of one second, typically expressed as megabits (Mbps) or gigabits (Gbps) per second. Capitalization is important here. A lowercase 'b' indicates bits, whereas an uppercase 'B' indicates bytes. Speed is always measured in bits per second, with a lowercase 'b' (Kbps or Mbps).

Packets per second, similarly expressed most often as Kpps or Mpps, is another way of evaluating throughput. It conveys the number of packets or frames which can be processed in one second. This approach to measuring throughput is used to expose limitations of the processing power of devices, as shorter packets demand more frequent forwarding decisions. For example, a router might claim a throughput of 30 Mbps per second using full-size packets. However, it might also be limited to processing 40 Kpps. If each packet received was the minimum size of 64 bytes (512 bits), the router would be limited to just 20.48 Mbps (512 * 40,000) of throughput.

Cisco maintains often cited baseline performance measurements for its most popular routers and switches. If you work out the math, you can see that the Mbps numbers listed in the router performance document were derived using minimum-length (64 byte) packets. These numbers thus present a worst case scenario. Packets on a production network typically vary widely in size, and larger packets will yield higher bits-per-second rates.

Keep in mind that these benchmarks were taken with no features other than IP routing enabled. Adding additional features and services such as access control lists or network address translation may reduce throughput. Unfortunately, it's impractical for a vendor to list throughput rates with and without myriad features enabled, so you'll have to do some testing yourself.

Oversubscription

Ethernet switches are often built with oversubscribed backplanes. Oversubscription refers to a point of congestion within a system where the potential rate of input is greater than the potential rate of output. For example, a switch with 48 1 Gbps ports might have a backplane throughput limitation of only 16 Gbps. This means that only 16 ports can transmit at wire rate (the physical maximum throughput) at any point in time. This isn't usually a problem at the network edge, where few users or servers ever need to transmit at these speeds for a prolonged time. However, oversubscription imposes much more critical considerations in the data center or network core.

As an example, let's look at the 16-port 10 Gbps Ethernet module WS-X6816-10G-2T for the Cisco Catalyst 6500 switch. Although the module provides an aggregate of 160 Gbps of potential throughput, its connection to the chassis backplane is only 40 Gbps. The module is oversubscribed at a ratio of 4:1. This module should only be used in situations where the aggregate throughput demand from all interfaces is not expected to exceed 40 Gbps.

IP Route Capacity

The maximum number of routes a router can hold in its routing table is limited by the amount of available content-addressable memory (CAM). Although a low-end router may be able to run BGP and exchange routes with BGP peers, it likely won't have sufficient memory to accept the full IPv4 Internet routing table, which comprises over 400 thousand routes. (Of course, low-end routers should never be implemented in a position where they would need to receive the full routing table.) Virtual routing contexts, in which a router stores multiple copies of a route in separate forwarding tables, increase routing table size exponentially, further elevating the importance of properly sizing routers for the role they play.

Maximum Concurrent Sessions

Firewalls and intrusion prevention systems perform stateful inspection of traffic transiting from one trust zone to another. These devices must be able to keep up with the demand for throughput not only in terms of bits per second and packets per second but also in the number of concurrent stateful sessions. A single web request might trigger the initiation of one or two dozen TCP connections to various content servers from an internal host. The firewall or IPS must be able to track the state of and inspect potentially thousands of sessions at any point in time. If the device's maximum capacity is reached, attempts to open new sessions may be rejected until a number of current sessions are closed or expire. Such devices are likewise limited in how fast they can create new sessions.