Troubleshooting Commands for IOS Devices
General Information
show version
The command show version provides basic information about a device, including hardware resources, software revision, and the configuration register. Notable portions of the command's output have been highlighted below.
Router# show version Cisco IOS Software, 3700 Software (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T5, RELEASE SOFTWARE (fc4) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2008 by Cisco Systems, Inc. Compiled Wed 30-Apr-08 18:27 by prod_rel_team ROM: System Bootstrap, Version 12.2(8r)T2, RELEASE SOFTWARE (fc1) Router uptime is 3 weeks, 2 days, 16 hours, 55 minutes System returned to ROM by power-on System image file is "slot0:c3725-adventerprisek9-mz.124-15.T5.bin" This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html If you require further assistance please contact us by sending email to export@cisco.com. Cisco 3725 (R7000) processor (revision 0.1) with 124928K/6144K bytes of memory. Processor board ID JPE0852107G R7000 CPU at 240MHz, Implementation 39, Rev 3.3, 256KB L2 Cache 2 FastEthernet interfaces 9 Serial interfaces DRAM configuration is 64 bits wide with parity disabled. 55K bytes of NVRAM. 31360K bytes of ATA System CompactFlash (Read/Write) 250880K bytes of ATA Slot0 CompactFlash (Read/Write) Configuration register is 0x2102
The highlighted portions above describe, respectively:
- Running IOS version
- Uptime
- Reason for the last boot (e.g. power-on or reload)
- Location of the running IOS image
- Available DRAM displayed as processor memory / I/O memory (add these two numbers to get the total available DRAM)
- Device serial number
- CPU description
- Installed interfaces, including modules
- Available NVRAM (default location for configuration storage)
- Available Flash
- Configuration register
show inventory
The command show inventory provides a compiled list of all hardware components present in the device and information pertinent to each, including its human-friendly name and description, part ID (PID), hardware revision (VID), and serial number (SN).
Router# show inventory NAME: "3725 chassis", DESCR: "3725 chassis" PID: , VID: 0.1, SN: JPE0852107G NAME: "WAN Interface Card - DSU/CSU T1 Fractional on Slot 0 SubSlot 0", DESCR: "WAN Interface Card - DSU/CSU T1 Fractional" PID: WIC-1DSU-T1= , VID: 1.5, SN: 31638068 NAME: "Four Port High-Speed Serial on Slot 1", DESCR: "Four Port High-Speed Serial" PID: NM-4T= , VID: 1.1, SN: 19372964 NAME: "Four Port High-Speed Serial on Slot 2", DESCR: "Four Port High-Speed Serial" PID: NM-4T= , VID: 1.1, SN: 20591382
show environment
The command show environment displays hardware diagnostic information to include power supply status, fan speed and status, and system and environmental temperatures. The output and detail provided by the command varies widely by platform.
The following example was taken from a Cisco 2811 router:
Router# show environment Main Power Supply is AC Fan 1 OK Fan 2 OK Fan 3 OK Fan Speed : Normal System Temperature: 28 Celsius (normal) Environmental information last updated 00:00:06 ago
This second example was taken from a Cisco Catalyst 3550.
Switch# show env all FAN is OK TEMPERATURE is OK POWER is OK RPS is NOT PRESENT
show tech-support
The command show tech-support (commonly shortened as show tech) is an alias for a number of other CLI commands (including many of the commands discussed on this page). It exists as a convenient method for users to pull a large amount of information about a device, typically to be submitted to Cisco TAC in the course of a support request.
As the output of this command can be quite lengthy, it is advised to perform this command via Telnet or SSH versus a physical serial console where possible. Further, it is a good idea to capture the output of this command directly to a file on disk to avoid having to go back and copy/paste the entire thing. For example, this can be accomplished by OpenSSH on Linux using the following syntax:
ssh user@IP_address show tech-support > file_name
CPU Utilization
show processes
Router# show processes CPU utilization for five seconds: 0%/0%; one minute: 0%; five minutes: 0% PID QTy PC Runtime (ms) Invoked uSecs Stacks TTY Process 1 Cwe 60027CB0 0 76 0 5412/6000 0 Chunk Manager 2 Csp 60BF4CB8 728 342627 2 2512/3000 0 Load Meter 4 Mwe 62B6F264 0 1 023360/24000 0 EDDRI_MAIN 5 Lst 60024208 1623280 203694 7969 5316/6000 0 Check heaps 6 Cwe 6002BD48 0 3 0 5436/6000 0 Pool Manager 7 Mst 6147BBD8 0 2 0 5476/6000 0 Timers 8 Mwe 6028A658 0 1 011624/12000 0 OIR Handler 9 Mwe 60550470 0 1 023556/24000 0 Crash writer 10 Msi 6069BF8C 0 57106 0 5480/6000 0 Environmental mo 11 Mwe 60DA7164 0 28553 0 5696/6000 0 IPC Dynamic Cach 12 Mwe 60D98034 0 1 0 5652/6000 0 IPC Zone Manager 13 Mwe 60D9724C 32 1713101 0 5568/6000 0 IPC Periodic Tim 14 Mwe 60D9709C 36 1713101 0 5628/6000 0 IPC Deferred Por 15 Mwe 60D97D0C 0 1 0 5524/6000 0 IPC Seat Manager 16 Mwe 60D9D620 0 1 0 5664/6000 0 IPC BackPressure 17 Mwe 60E3B560 7736 57319 134 4588/6000 0 ARP Input 18 Mwe 60E381F4 40 1786741 0 4864/6000 0 ARP Background 19 Mwe 60EFE0BC 0 2 0 5432/6000 0 ATM Idle Timer 20 Mwe 6142C12C 0 2 0 5468/6000 0 AAA high-capacit 21 Lwe 61430134 0 1 0 5760/6000 0 AAA_SERVER_DEADT 22 Mwe 614A8034 0 1 011604/12000 0 Policy Manager ...
The top line of this output shows the CPU utilization as averaged over the last five seconds, the last minute, and the last five minutes. These three separate calculations allow the administrator to determine whether heavy CPU utilization is a momentary spike or has been prolonged. The five-second CPU utilization reading is broken into two percentages: the first indicates total CPU utilization, whereas the second indicates how much CPU time was spent processing hardware interrupts (e.g. software switching).
Column Headings
- PID
- Process ID
- Q
- Queue priority (Critical, High, Medium, or Low)
- Ty
- Scheduler state
- PC (Program counter)
- Process address in memory
- Runtime
- Amount of CPU time the process has consumed, in milliseconds
- Invoked
- The number of times this process has run since boot
- uSecs
- Average time the process runs per invocation (in microseconds)
- Stacks
- The amount of stack space (executable memory) available to the process expressed as least ever available / total allocated (in bytes)
- TTY
- The terminal that controls the process (typically 0)
- Process
- Process name
More detailed information about a specific process can be retrieved by appending a PID to the command show processes:
Router# show processes 1 Process ID 1 [Chunk Manager], TTY 0 Memory usage [in bytes] Holding: 84476, Maximum: 86524, Allocated: 79456, Freed: 16016 Getbufs: 0, Retbufs: 0, Stack: 5412/6000 CPU usage PC: 60027CB0, Invoked: 76, Giveups: 0, uSec: 0 5Sec: 0.00%, 1Min: 0.00%, 5Min: 0.00%, Average: 0.00% Age: 1714492984 msec, Runtime: 0 msec State: Waiting for Event, Priority: Critical
show processes cpu
The command show processes cpu displays the 5sec/1min/5min CPU utilization averages per process, and includes much of the information displayed by show processes. This command is typically issued with the argument sorted appended, to order processes by highest CPU utilization.
Router show processes cpu sorted CPU utilization for five seconds: 1%/0%; one minute: 0%; five minutes: 0% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 5 1624768 203878 7969 0.73% 0.11% 0.07% 0 Check heaps 131 2348 3429726 0 0.08% 0.02% 0.00% 0 DHCPD Receive 81 420 53554234 0 0.08% 0.06% 0.08% 0 ACCT Periodic Pr 85 348 53554234 0 0.08% 0.05% 0.06% 0 IP ARP Retry Age 1 0 76 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 732 342945 2 0.00% 0.02% 0.00% 0 Load Meter 8 0 1 0 0.00% 0.00% 0.00% 0 OIR Handler 4 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN 6 0 3 0 0.00% 0.00% 0.00% 0 Pool Manager 7 0 2 0 0.00% 0.00% 0.00% 0 Timers 9 0 1 0 0.00% 0.00% 0.00% 0 Crash writer 13 32 1714692 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim 14 36 1714692 0 0.00% 0.00% 0.00% 0 IPC Deferred Por 15 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager 16 0 1 0 0.00% 0.00% 0.00% 0 IPC BackPressure 10 0 57159 0 0.00% 0.00% 0.00% 0 Environmental mo 11 0 28579 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach 19 0 2 0 0.00% 0.00% 0.00% 0 ATM Idle Timer 20 0 2 0 0.00% 0.00% 0.00% 0 AAA high-capacit 12 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager 17 7744 57333 135 0.00% 0.00% 0.00% 0 ARP Input ...
show processes cpu history
The command show processes cpu history provides a less detailed, longer range view of CPU utilization graphed as averages over three time periods: 60 seconds, 60 minutes, and 72 hours (3 days):
Router# show processes cpu history
Router 08:56:32 PM Wednesday Mar 20 2002 UTC
11111 1111111111
100
90
80
70
60
50
40
30
20
10
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
4
12111118111112311211111111112121111111111111111 111111111111
100
90
80
70
60
50 *
40 *
30 *
20 *
10 *
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
322323313323314323227333223133233243132222432123223323234233234233234421
696335151042143150751006562704761600717686027772983260911322873606244196
100
90
80
70 *
60 *
50 *
40 * * * * * * * * * ** * **
30 **** ** ** ** ************* ******** ******** *********** ******** ****
20 ************* **********************************************************
10 ************************************************************************
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Each column is led by a three-digit number (displayed vertically) indicating the maximum CPU utilization recorded during the time span represented by that column.
On the graph, asterisks (*) are used to indicate maximum CPU utilization, and hash marks (#) are used to indicate average CPU utilization, both rounded to the nearest ten percent. An absence of asterisks in a column indicates that maximum CPU utilization was not high enough to display (i.e., it was less than five percent). Likewise, an absence of hash marks within a column means that average CPU utilization for that period was not high enough to display.
show platform cpu packet buffered
The command show platform cpu packet buffered will display all packets being sent to the CPU. If you notice one source with a high number of repeated forwards to the CPU, it's a good place to start looking at that host.
Switch# show platform cpu packet buffered
Total Received Packets Buffered: 1024
-------------------------------------
Index 0:
296 days 1:22:45:92525 - RxVlan: 203, RxPort: Te6/2
Priority: High, Tag: Dot1Q Tag, Event: 21, Flags: 0x40, Size: 66
Eth: Src 00:24:F9:AF:10:46 Dst 01:00:5E:00:00:02 Type/Len 0x0800
Ip: ver:IpVersion4 len:20 tos:192 totLen:48 id:0 fragOffset:0 ttl:1 proto:udp
src: 172.21.0.131 dst: 224.0.0.2 firstFragment lastFragment
Remaining data:
0: 0x7 0xC1 0x7 0xC1 0x0 0x1C 0x47 0xCE 0x0 0x0
10: 0x8 0x3 0xA 0x64 0x17 0x0 0x63 0x69 0x73 0x63
20: 0x6F 0x0 0x0 0x0 0xAC 0x15 0x0 0x81 0x31 0x71
Index 1:
296 days 1:22:45:110074 - RxVlan: 1143, RxPort: Gi8/25
Priority: Normal, Tag: No Tag, Event: 20, Flags: 0x440, Size: 106
Eth: Src 00:1E:C2:14:E8:59 Dst 00:26:0B:97:B5:FF Type/Len 0x0800
Ip: ver:IpVersion4 len:20 tos:16 totLen:88 id:16393 fragOffset:0 ttl:63 proto:tcp
src: 172.21.14.141 dst: 172.21.10.25 firstFragment lastFragment
Remaining data:
0: 0xE9 0xBE 0x0 0x16 0x86 0x2D 0x4E 0xC1 0xB 0x6D
10: 0x42 0x7E 0x50 0x18 0xFF 0xFF 0xB3 0x5 0x0 0x0
20: 0x84 0x2E 0xA6 0x85 0x2D 0xBF 0xCA 0x3D 0x1A 0xA
debug platform packet all
The command debug platform packet all buffer will identify traffic that hits the CPU in a circular buffer. This is useful when a processor is being hit hard and you need to identify the cause and is used in conjunction with the show platform cpu packet buffered command.
Switch# debug platform packet all buffer platform packet debugging is on
Memory Utilization
show memory
The show memory command displays memory statistics. The topmost portion of the output shows a summary of all available memory. Statistics are then displayed individually per pool. For example, processor memory might be shown first; if you hit q, the output will jump to the next pool (typically I/O memory).
Router# show memory
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 6611C330 27130896 20030868 7100028 5737028 5471568
I/O 7A00000 6291456 4822952 1468504 1437984 1468476
Processor memory
Address Bytes Prev Next Ref PrevF NextF Alloc PC what
6611C330 0000499688 00000000 66196348 000 674C827C 657AB608 60BEE0F0 (coalesced)
66196348 0000000404 6611C330 6619650C 001 -------- -------- 600280EC *Init*
6619650C 0000020004 66196348 6619B360 001 -------- -------- 600278C4 Managed Chunk Queue Elements
6619B360 0000010004 6619650C 6619DAA4 001 -------- -------- 62AF3780 List Elements
6619DAA4 0000005004 6619B360 6619EE60 001 -------- -------- 62AF37B0 List Headers
6619EE60 0000000164 6619DAA4 6619EF34 001 -------- -------- 6362E328 *Init*
6619EF34 0000000028 6619EE60 6619EF80 001 -------- -------- 6027B140 *Init*
6619EF80 0000000028 6619EF34 6619EFCC 001 -------- -------- 6027B140 *Init*
6619EFCC 0000000164 6619EF80 6619F0A0 001 -------- -------- 6362E328 *Init*
6619F0A0 0000000240 6619EFCC 6619F1C0 001 -------- -------- 60BF6098 *Init*
6619F1C0 0000004388 6619F0A0 661A0314 001 -------- -------- 6149147C TTY data
661A0314 0000002004 6619F1C0 661A0B18 001 -------- -------- 6148C734 TTY Input Buf
661A0B18 0000001004 661A0314 661A0F34 001 -------- -------- 6148C768 TTY Output Buf
661A0F34 0000000164 661A0B18 661A1008 001 -------- -------- 6362E328 *Init*
661A1008 0000000484 661A0F34 661A121C 001 -------- -------- 62AFE568 Watched Message Queue
661A121C 0000000068 661A1008 661A1290 001 -------- -------- 636072A4 *Init*
661A1290 0000000132 661A121C 661A1344 001 -------- -------- 60288468 *Init*
661A1344 0000000164 661A1290 661A1418 001 -------- -------- 6362E328 *Init*
...
show processes memory
The command show processes memory displays memory usage per process. Like show process cpu, the argument sorted can be appended to sort processes by highest memory consumption (the Holding column).
Router# show processes memory sorted
Processor Pool Total: 27130896 Used: 20038712 Free: 7092184
I/O Pool Total: 6291456 Used: 4822944 Free: 1468512
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 38250120 14754788 20622048 0 0 *Init*
25 0 264540 0 273864 113400 0 EEM ED Syslog
195 0 247992 0 255060 0 0 QOS_MODULE_MAIN
101 0 241000 1052 242620 0 0 EAPoUDP Process
231 0 161348 10404 167804 0 0 EEM Server
131 0 104832 252 103512 0 0 DHCPD Receive
4 0 65588 0 90656 0 0 EDDRI_MAIN
1 0 79456 16016 84476 0 0 Chunk Manager
206 0 66852 0 79920 0 0 Crypto IKEv2
208 0 76288 10532 78824 0 0 IPSEC key engine
111 0 75976 2360 74100 0 0 CEF process
159 0 66544 252 73360 0 0 HTTP Process
210 0 45300 3216 67152 0 0 Crypto ACL
58 0 57904 252 64720 0 0 VNM DSPRM MAIN
193 0 940 0 62008 0 0 http client proc
81 0 49384 0 56452 0 0 ACCT Periodic Pr
85 0 49384 0 56452 0 0 IP ARP Retry Age
178 0 45988 504 54552 0 0 Crypto WUI
17 0 45980 476 52572 0 0 ARP Input
...
The topmost portion of the output displays memory utilization per pool. Total, used, and free memory is shown (in kilobytes?), such that used + free = total.
Column Headings
- PID
- Process ID
- TTY
- Terminal to which the process belongs (typically 0)
- Allocated
- Memory allocated by this process since boot (in bytes)
- Freed
- Memory freed by this process since boot (in bytes)
- Holding
- Memory currently in use by this process (in bytes)
- Getbufs
- Number of packet buffers allocated by this process since boot
- Retbufs
- Number of packet buffers returned by this process since boot
- Process
- Process name
show region
The command show region shows how physical memory has been partitioned for use by IOS.
Router# show region Region Manager: Start End Size(b) Class Media Name 0x07A00000 0x07FFFFFF 6291456 Iomem R/W iomem 0x60000000 0x679FFFFF 127926272 Local R/W main 0x60008930 0x63639A53 56824100 IText R/O main:text 0x6363BCF0 0x65778E4F 34853216 IData R/W main:data 0x650325B0 0x6512E4EF 1032000 Local R/W data:firmware 0x65778E50 0x6611C32F 10106080 IBss R/W main:bss 0x6611C330 0x679FFFFF 26098896 Local R/W main:heap 0x80000000 0x879FFFFF 127926272 Local R/W main:(main_k0) 0xA0000000 0xA79FFFFF 127926272 Local R/W main:(main_k1)
Column Headings
- Start
- Physical beginning of region
- End
- Physical end of region
- Size
- Size of the region (in bytes)
- Class
- Class of the memory, one of the following:
- Iomem - I/O memory (packet buffers)
- Local
- Itext - Image text memory
- IData - Image data memory
- IBss - Image blind source separation (BSS) memory
- Media
- Type of access (read/write or read-only)
- Name
- Name of the region
show buffers
The command show buffers displays statistics about the buffers available to hold packets in I/O memory.
Router# show buffers
Buffer elements:
1118 in free list (1119 max allowed)
1197443 hits, 0 misses, 619 created
Public buffer pools:
Small buffers, 104 bytes (total 50, permanent 50, peak 83 @ 2w6d):
45 in free list (20 min, 150 max allowed)
1213522 hits, 11 misses, 33 trims, 33 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 25, permanent 25, peak 49 @ 2w6d):
23 in free list (10 min, 150 max allowed)
51695 hits, 8 misses, 24 trims, 24 created
0 failures (0 no memory)
Big buffers, 1536 bytes (total 50, permanent 50):
50 in free list (5 min, 150 max allowed)
33995 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10):
10 in free list (0 min, 100 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Large buffers, 5024 bytes (total 0, permanent 0):
0 in free list (0 min, 10 max allowed)
...
The topmost portion of the output displays compiled statistics for all buffer types.
The following example is used for illustration to explain the output of this command:
Small buffers, 104 bytes (total 50, permanent 50, peak 83 @ 2w6d):
45 in free list (20 min, 150 max allowed)
1213522 hits, 11 misses, 33 trims, 33 created
0 failures (0 no memory)
- Each buffer in this pool is 104 bytes in length
- There are a total of 50 buffers in this pool
- Of all of the buffers currently in the pool, 50 (all) of them are permanent
-
The highest number of buffers ever in this pool since boot is 83
-
45 of the 50 buffers are currently free
- The minimum number of free buffers to be maintained in this pool is 20
-
The maximum number of free buffers to be maintained in this pool is 150
-
This pool has fulfilled 1213522 buffer requests since boot
- The number of free buffers has fallen below the minimum 11 times since boot
-
33 extra buffers were created and later trimmed from the pool (83 - 50 = 33)
-
This pool has failed to fulfill a buffer request 0 times
- 0 of these failures were due to a shortage of available memory
Assigned Categories
- None
Referenced By
- None