Sometimes we have to work with very large packet captures, captures that can be several gigabytes in size. These are cumbersome (if even possible) to analyze in an application like Wireshark because the entire capture file must be loaded into running memory at once. And manual analysis typically means we're only interested in a small portion of the capture file anyway. How can we break a huge capture into smaller, more manageable chunks?
One approach is to cut the file into slices, with each slice a containing a constant number of packets or bytes or covering a given length of time. Practically speaking, this is how huge traffic captures should be performed in the first place, using a ring buffer. But we can similarly chop up large capture files after the fact using editcap (part of the Wireshark family).
Take for instance this capture file of over a quarter million packets weighing in at 256 MB:
$ capinfos lotsapackets.cap File name: lotsapackets.cap File type: Wireshark/tcpdump/... - libpcap File encapsulation: Ethernet Number of packets: 260778 File size: 267802612 bytes Data size: 263630140 bytes Capture duration: 204 seconds Start time: Mon Apr 4 22:31:52 2011 End time: Mon Apr 4 22:35:16 2011 Data byte rate: 1290166.92 bytes/sec Data bit rate: 10321335.37 bits/sec Average packet size: 1010.94 bytes Average packet rate: 1276.21 packets/sec
256 MB certainly isn't an insurmountable file size, but it's large enough that we may want something more flexible. We can split it into a few files of, say, 50,000 packets each and load each one into Wireshark individually.
$ editcap -c 50000 lotsapackets.cap fewerpackets.cap $ capinfos -c fewerpackets*.cap File name: fewerpackets_00000_20110404223152.cap Number of packets: 50000 File name: fewerpackets_00001_20110404223310.cap Number of packets: 50000 File name: fewerpackets_00002_20110404223340.cap Number of packets: 50000 File name: fewerpackets_00003_20110404223410.cap Number of packets: 50000 File name: fewerpackets_00004_20110404223440.cap Number of packets: 50000 File name: fewerpackets_00005_20110404223510.cap Number of packets: 10778
That's a pretty good solution, but as mentioned earlier, we're probably only interested in specific traffic; for example, only traffic destined for a particular host, or a particular UDP or TCP port number. Searching each of these smaller files by hand would be a tedious waste of your time.
Luckily, we can use tshark (another Wireshark tool) to extract interesting traffic from a capture file. We just need to define a display filter to match the traffic we want. For example, if we wanted to extract all DNS traffic from our large capture file, we could do this:
$ tshark -r lotsapackets.cap -R dns -w dns.cap $ capinfos dns.cap File name: dns.cap File type: Wireshark/tcpdump/... - libpcap File encapsulation: Ethernet Number of packets: 220 File size: 24664 bytes Data size: 21120 bytes Capture duration: 32 seconds Start time: Mon Apr 4 22:31:58 2011 End time: Mon Apr 4 22:32:30 2011 Data byte rate: 656.35 bytes/sec Data bit rate: 5250.76 bits/sec Average packet size: 96.00 bytes Average packet rate: 6.84 packets/sec
After just a few seconds, tshark copies every DNS packet from our original capture into the much smaller file dns.cap, which we can easily examine in Wireshark at our leisure.
Filters can be mixed and matched just like in Wireshark. The example below matches all DNS traffic and all traffic sent to or from TCP port 80.
$ tshark -r lotsapackets.cap -R "dns or tcp.port==80" -w web.cap $ capinfos web.cap File name: web.cap File type: Wireshark/tcpdump/... - libpcap File encapsulation: Ethernet Number of packets: 1559 File size: 644994 bytes Data size: 620026 bytes Capture duration: 165 seconds Start time: Mon Apr 4 22:31:55 2011 End time: Mon Apr 4 22:34:40 2011 Data byte rate: 3757.36 bytes/sec Data bit rate: 30058.86 bits/sec Average packet size: 397.71 bytes Average packet rate: 9.45 packets/sec
The Wireshark display filter cheat sheet offers an idea of the variety of filters available. With a little practice you should be able to automatically extract just about any type of traffic.