11 Replies Latest reply: Jul 20, 2011 8:13 AM by StuO RSS

    Total Output Drops

    StuO

      Hi All,

      Just trying to find out why I'm getting discards on a interface. Basically we have a remote office that connects to another office before connecting to the main backhaul.

       

      SWITCH A  ---->  SWITCH B ----> SWICTH C

       

      The Link between Switch A and B is 100Mbps

      The Link between Switch B and C is 1000Mbps (1GB).

       

      Looking into the history of the link between A and B we can see that majority of the time the traffic is at 60-65% but will spike upto 85% 1-2 times a week.

       

      GigabitEthernet1/0/2 is up, line protocol is up (connected)

        Hardware is Gigabit Ethernet, address is 001d.70b8.e282 (bia 001d.70b8.e282)

        Description: to SWITCH-A TO SWITCH-B

        MTU 1998 bytes, BW 100000 Kbit, DLY 100 usec,

           reliability 255/255, txload 48/255, rxload 13/255

        Encapsulation ARPA, loopback not set

        Keepalive not set

        Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP

        input flow-control is off, output flow-control is unsupported

        ARP type: ARPA, ARP Timeout 04:00:00

        Last input 00:00:00, output 00:00:00, output hang never

        Last clearing of "show interface" counters never

        Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1851383

        Queueing strategy: fifo

        Output queue: 0/40 (size/max)

        5 minute input rate 5293000 bits/sec, 2799 packets/sec

        5 minute output rate 19054000 bits/sec, 3739 packets/sec

           91760131827 packets input, 21396599373912 bytes, 0 no buffer

           Received 50352741 broadcasts (33957950 multicasts)

           36 runts, 0 giants, 0 throttles

           36 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

           0 watchdog, 33957950 multicast, 0 pause input

           0 input packets with dribble condition detected

           123201990031 packets output, 112703016614681 bytes, 0 underruns

           0 output errors, 0 collisions, 1 interface resets

           0 babbles, 0 late collision, 0 deferred

           0 lost carrier, 0 no carrier, 0 PAUSE output

           0 output buffer failures, 0 output buffers swapped out

       

      discards.JPG

       

      All switches are Cisco 3750. What would you recommend to resolve this issue as I've been advised that the QOS is fine and link will not be upgraded to a 1GB link?

       

      Now looking into this I've read that either of the following can help:

       

      - Increasing queue-limit

      - Increasing hold queue


      Cisco advising against increasing in hold queue as TCP timer can expire whilst the packet is still in the queue.


      Anyone out there that can provide further input?

        • 1. Re: Total Output Drops
          Brian

          Aloha StuO,

           

          Output drops typically indicate a congested interface.  This can occur when going from a high speed interface (such as the SW B to SW C link at 1000Mbps) to a lower speed link (SW A to SW B at 100Mbps).  You mention the traffic on the SW A to SW B link is at 60% - 65%, but what is it on the SW B to SW C link?  Also, I notice the MTU on the GE interface is at 1998 bytes, why?  What is the MTU on the FE of SW A?

           

          Output drops are not all bad, as the drops indicate the flow control mechanisms of the upper layer protocols are working to decrease the TCP window size.  Also, if the drops are due to a constant high-rate flow of traffic you cannot prevent them, but you can control them.

           

          I would like to see both the SW A interface and the SW B interface when the load is at 60% - 65% on SW A.  This I believe will give you a much better picture of what is going on.  So, outputs of the "sh run", "sh int x/x", "sh proc cpu" to name a few.

           

          HTH

           

          • 2. Re: Total Output Drops
            StuO

            I understand what you are saying as I've already read it in the Cisco documents, but I dont understand is the burst traffic is not going over 100Mbps so should easily output onto the 100Mbps link with no issues.

             

            I have another link (like above) but the avg traffic is around 25Mbps with bursts to 45% on a fixed 100Mbps link. If the data rate was bursting high then I would see it on the monitoring systems but nothing.

             

            I will try and add further outputs but need to restrict some of the output due to company security...

             

            CPU load is always around 5%

            • 3. Re: Total Output Drops
              StuO

              Hi,

              Recent they have changed the Switch A with a Juniper switch, but it is only switch B that is getting the discards. Reset the counters to see what was happening over night and captured this at 11pm:

               

              SWITCH B (Cisco)

               

              SHOW INT     

              GigabitEthernet1/0/2 is up, line protocol is up (connected)

                Hardware is Gigabit Ethernet, address is 001d.70b8.e282 (bia 001d.70b8.e282)

                MTU 1998 bytes, BW 100000 Kbit, DLY 100 usec,

                   reliability 255/255, txload 111/255, rxload 20/255

                Encapsulation ARPA, loopback not set

                Keepalive not set

                Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP

                input flow-control is off, output flow-control is unsupported

                ARP type: ARPA, ARP Timeout 04:00:00

                Last input 00:00:00, output 00:00:00, output hang never

                Last clearing of "show interface" counters 13:11:08

                Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1524

                Queueing strategy: fifo

                Output queue: 0/40 (size/max)

                5 minute input rate 7874000 bits/sec, 4027 packets/sec

                5 minute output rate 43706000 bits/sec, 5655 packets/sec

                   209007655 packets input, 44377320962 bytes, 0 no buffer

                   Received 88392 broadcasts (57397 multicasts)

                   0 runts, 0 giants, 0 throttles

                   0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

                   0 watchdog, 57397 multicast, 0 pause input

                   0 input packets with dribble condition detected

                   274804239 packets output, 249611857325 bytes, 0 underruns

                   0 output errors, 0 collisions, 0 interface resets

                   0 babbles, 0 late collision, 0 deferred

                   0 lost carrier, 0 no carrier, 0 PAUSE output

                   0 output buffer failures, 0 output buffers swapped out

               

               

              CONFIG

              !

              interface GigabitEthernet1/0/2

              switchport trunk encapsulation dot1q

              switchport trunk allowed vlan 1-3998,4000-4094

              switchport mode trunk

              speed 100

              duplex full

              srr-queue bandwidth share 30 5 15 50

              queue-set 2

              priority-queue out

              mls qos trust cos

              end

               

              show processes cpu

              CPU utilization for five seconds: 6%/0%; one minute: 6%; five minutes: 6%

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

                 1           0        14          0  0.00%  0.00%  0.00%   0 Chunk Manager  

                 2        3279   5865148          0  0.00%  0.00%  0.00%   0 Load Meter     

                 3           0         1          0  0.00%  0.00%  0.00%   0 CEF RP IPC Backg

                 4    40533766   3476856      11658  0.00%  0.14%  0.11%   0 Check heaps    

                 5        2123      1701       1248  0.00%  0.00%  0.00%   0 Pool Manager   

                 6           0         2          0  0.00%  0.00%  0.00%   0 Timers         

                 7           0         1          0  0.00%  0.00%  0.00%   0 HRPC asic-stats

                 8           0         1          0  0.00%  0.00%  0.00%   0 Crash writer   

                 9     2519160   4727647        532  0.00%  0.00%  0.00%   0 ARP Input      

                10           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API    

                11           0        13          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT

                12           0         2          0  0.00%  0.00%  0.00%   0 AAA high-capacit

                13           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager 

                14          34        20       1700  0.00%  0.00%  0.00%   0 Entity MIB API 

                15         270      4162         64  0.00%  0.00%  0.00%   0 EEM ED Syslog  

                16           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage

                17         911    489500          1  0.00%  0.00%  0.00%   0 IPC Dynamic Cach

                18           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager

                19        5512  29177823          0  0.00%  0.00%  0.00%   0 IPC Periodic Tim

                20           0         1          0  0.00%  0.00%  0.00%   0 IPC Managed Time

                21        5984  29177823          0  0.00%  0.00%  0.00%   0 IPC Deferred Por

                22        4126   1957207          2  0.00%  0.00%  0.00%   0 IPC Seat Manager

                23           0         1          0  0.00%  0.00%  0.00%   0 IPC Session Serv

                24        6129  29177823          0  0.00%  0.00%  0.00%   0 Dynamic ARP Insp

                25           0         1          0  0.00%  0.00%  0.00%   0 ARP Snoop      

                26           0         1          0  0.00%  0.00%  0.00%   0 License IPC stat

                27           0         1          0  0.00%  0.00%  0.00%   0 License IPC serv

                28        5977  29177808          0  0.00%  0.00%  0.00%   0 GraphIt        

                29           0         2          0  0.00%  0.00%  0.00%   0 XML Proxy Client

                30           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd 

                31       23833  25668537          0  0.00%  0.00%  0.00%   0 Net Background 

                32           0         1          0  0.00%  0.00%  0.00%   0 IDB Work       

                33          26      7787          3  0.00%  0.00%  0.00%   0 Logger         

                34        3069  29177804          0  0.00%  0.00%  0.00%   0 TTY Background 

                35       74355  29178441          2  0.00%  0.00%  0.00%   0 Per-Second Jobs

                36     9391719    491887      19093  0.00%  0.01%  0.00%   0 Per-minute Jobs

                37           0        17          0  0.00%  0.00%  0.00%   0 IF-MGR control p

                38           0       283          0  0.00%  0.00%  0.00%   0 IF-MGR event pro

                39           0         1          0  0.00%  0.00%  0.00%   0 AggMgr Process 

                40      103491   2347307         44  0.00%  0.00%  0.00%   0 Net Input      

                41       16981   5865446          2  0.00%  0.00%  0.00%   0 Compute load avg

                42          17     20797          0  0.00%  0.00%  0.00%   0 Transport Port A

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

                43       15275   8794523          1  0.00%  0.00%  0.00%   0 HC Counter Timer

                44       39268 583383503          0  0.00%  0.00%  0.00%   0 DownWhenLooped 

                45           0         1          0  0.00%  0.00%  0.00%   0 HRPC hdwl reques

                46           0         1          0  0.00%  0.00%  0.00%   0 HRPC lpip reques

                47           0         2          0  0.00%  0.00%  0.00%   0 HLPIP Sync Proce

                48           0         1          0  0.00%  0.00%  0.00%   0 HRPC Multi-FS Sy

                49           0        34          0  0.00%  0.00%  0.00%   0 HULC multifs pro

                50           0         1          0  0.00%  0.00%  0.00%   0 Hulc Warm Upgrad

                51           0         2          0  0.00%  0.00%  0.00%   0 MIRAGE RBCP Moni

                52           0         1          0  0.00%  0.00%  0.00%   0 Platform-IPe Pro

                53           0         1          0  0.00%  0.00%  0.00%   0 Hulc EEM Process

                54          18        19        947  0.00%  0.00%  0.00%   0 Collection proce

                55           0         1          0  0.00%  0.00%  0.00%   0 HRPC hsm request

                56           0         7          0  0.00%  0.00%  0.00%   0 Stack Mgr      

                57          76         6      12666  0.00%  0.00%  0.00%   0 Stack Mgr Notifi

                58     22856591575120702          1  0.00%  0.00%  0.00%   0 Fifo Error Detec

                59           0         3          0  0.00%  0.00%  0.00%   0 Adjust Regions 

                60        9102  29177819          0  0.00%  0.00%  0.00%   0 hrpc -> response

                61       11575   5870833          1  0.00%  0.00%  0.00%   0 hrpc -> request

                62        4894   5869726          0  0.00%  0.00%  0.00%   0 hrpc <- response

                63           0         1          0  0.00%  0.00%  0.00%   0 HRPC hcomp reque

                64           0        11          0  0.00%  0.00%  0.00%   0 HULC Device Mana

                65           0         3          0  0.00%  0.00%  0.00%   0 HRPC hdm non blo

                66           0         2          0  0.00%  0.00%  0.00%   0 HRPC hdm blockin

                67         811   5865165          0  0.00%  0.00%  0.00%   0 HIPC bkgrd proce

                68           0         1          0  0.00%  0.00%  0.00%   0 RTTYS Process  

                69          17       424         40  0.00%  0.00%  0.00%   0 Hulc Port-Securi

                70           0         1          0  0.00%  0.00%  0.00%   0 HRPC hpsecure re

                71           0         1          0  0.00%  0.00%  0.00%   0 HRPC hlfm reques

                72      433431 874735033          0  0.00%  0.00%  0.00%   0 HLFM address lea

                73       51048  29177808          1  0.00%  0.00%  0.00%   0 HLFM aging proce

                74      267131 875263684          0  0.00%  0.00%  0.00%   0 HLFM address ret

                75           0         1          0  0.00%  0.00%  0.00%   0 HRPC hrcmd reque

                76           0         1          0  0.00%  0.00%  0.00%   0 HRPC emac reques

                77        1255     11091        113  0.00%  0.00%  0.00%   0 HRPC hulc misc r

                78           0         1          0  0.00%  0.00%  0.00%   0 HRPC system mtu

                79        7996   9780495          0  0.00%  0.00%  0.00%   0 HVLAN main bkgrd

                80           0         2          0  0.00%  0.00%  0.00%   0 HVLAN Mapped Vla

                81           0         2          0  0.00%  0.00%  0.00%   0 Vlan shutdown Pr

                82           0         1          0  0.00%  0.00%  0.00%   0 HRPC vlan reques

                83           0         3          0  0.00%  0.00%  0.00%   0 HULC VLAN REF Ba

                84           0         1          0  0.00%  0.00%  0.00%   0 HRPC hfbm reques

                85           8    163180          0  0.00%  0.00%  0.00%   0 HCMP sync proces

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

                86           0         1          0  0.00%  0.00%  0.00%   0 HRPC ilp request

                87           0         1          0  0.00%  0.00%  0.00%   0 HULC PM Vector P

                88           0         1          0  0.00%  0.00%  0.00%   0 HPM Msg Retry Pr

                89          49    244755          0  0.00%  0.00%  0.00%   0 DHCPD Timer    

                90     4196320 406618340         10  0.00%  0.00%  0.00%   0 hpm main process

                91           9       798         11  0.00%  0.00%  0.00%   0 HPM Stack Sync P

                92           0         1          0  0.00%  0.00%  0.00%   0 HRPC pm request

                93           0         4          0  0.00%  0.00%  0.00%   0 HPM if_num mappi

                94      154416  29177809          5  0.00%  0.00%  0.00%   0 hpm counter proc

                95           0         1          0  0.00%  0.00%  0.00%   0 HRPC pm-counters

                96           0         1          0  0.00%  0.00%  0.00%   0 hpm vp events ca

                97           0         1          0  0.00%  0.00%  0.00%   0 HRPC hcmp reques

                98           0     13969          0  0.00%  0.00%  0.00%   0 HCEF ADJ Refresh

                99           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl3mm reque

              100           0         1          0  0.00%  0.00%  0.00%   0 hl3md_rpfq_thrl_

              101       10627  20491117          0  0.00%  0.00%  0.00%   0 hl3mm          

              102        6870  14650203          0  0.00%  0.00%  0.00%   0 hl3mm_rp       

              103           0         1          0  0.00%  0.00%  0.00%   0 HACL Queue Proce

              104           0         1          0  0.00%  0.00%  0.00%   0 HRPC acl request

              105        3204   1099667          2  0.00%  0.00%  0.00%   0 HACL Acl Manager

              106           0         1          0  0.00%  0.00%  0.00%   0 HRPC backup inte

              107      970406  14506822         66  0.00%  0.00%  0.00%   0 OSPF-10 Hello  

              108           0         1          0  0.00%  0.00%  0.00%   0 HRPC cdp request

              109           0         1          0  0.00%  0.00%  0.00%   0 HRPC lldp reques

              110           0         1          0  0.00%  0.00%  0.00%   0 HRPC dot1x reque

              111           0         3          0  0.00%  0.00%  0.00%   0 HULC DOT1X Proce

              112           0         1          0  0.00%  0.00%  0.00%   0 HRPC foxhound re

              113           0         2          0  0.00%  0.00%  0.00%   0 Hulc Foxhound Pr

              114           0         1          0  0.00%  0.00%  0.00%   0 HRPC rep request

              115           0         3          0  0.00%  0.00%  0.00%   0 REP Helper Proc

              116           0         1          0  0.00%  0.00%  0.00%   0 HRPC sdm request

              117       75870 145866167          0  0.00%  0.00%  0.00%   0 Hulc Storm Contr

              118           0         2          0  0.00%  0.00%  0.00%   0 HSTP Sync Proces

              119           0         1          0  0.00%  0.00%  0.00%   0 HRPC stp_cli req

              120           0         1          0  0.00%  0.00%  0.00%   0 HRPC stp_state_s

              121           0         2          0  0.00%  0.00%  0.00%   0 S/W Bridge Proce

              122           0         1          0  0.00%  0.00%  0.00%   0 HRPC hudld reque

              123           0         1          0  0.00%  0.00%  0.00%   0 HRPC vqpc reques

              124           0         1          0  0.00%  0.00%  0.00%   0 HRPC iec_load_ba

              125           0         1          0  0.00%  0.00%  0.00%   0 HRPC l2pt qnq rp

              126           0         1          0  0.00%  0.00%  0.00%   0 HRPC IPe request

              127           0         1          0  0.00%  0.00%  0.00%   0 HRPC hled reques

              128     3058040 699959351          4  0.00%  0.01%  0.00%   0 Hulc LED Process

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

              129      423594  21491649         19  0.00%  0.00%  0.00%   0 HL3U bkgrd proce

              130           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl3u reques

              131        4279   7073175          0  0.00%  0.00%  0.00%   0 HL3U PBR bkgrd p

              132         185    244755          0  0.00%  0.00%  0.00%   0 HL3U PBR n-h res

              133           0         1          0  0.00%  0.00%  0.00%   0 HRPC dtp request

              134           0         1          0  0.00%  0.00%  0.00%   0 HRPC show_forwar

              135           0         1          0  0.00%  0.00%  0.00%   0 HRPC snmp reques

              136       30491   5865199          5  0.00%  0.00%  0.00%   0 HQM Stack Proces

              137      689137  11730484         58  0.00%  0.00%  0.00%   0 HRPC qos request

              138           0         1          0  0.00%  0.00%  0.00%   0 HRPC span reques

              139           0         1          0  0.00%  0.00%  0.00%   0 HRPC obfl reques

              140           0         1          0  0.00%  0.00%  0.00%   0 HRPC system post

              141           0         1          0  0.00%  0.00%  0.00%   0 Hulc Reload Mana

              142           0         1          0  0.00%  0.00%  0.00%   0 HRPC hrcli-event

              143        1971       645       3055  0.00%  0.00%  0.00%   0 SpanTree Helper

              144           0         2          0  0.00%  0.00%  0.00%   0 image mgr      

              145        5145   9780476          0  0.00%  0.00%  0.00%   0 Power RPS Proces

              146          40         6       6666  0.00%  0.00%  0.00%   0 HL2MCM         

              147          43         6       7166  0.00%  0.00%  0.00%   0 HL2MCM         

              148      216717  29177807          7  0.00%  0.00%  0.00%   0 PI MATM Aging Pr

              149           0         2          0  0.00%  0.00%  0.00%   0 REP Topology cha

              150           0         2          0  0.00%  0.00%  0.00%   0 Switch Backup In

              151         748    489499          1  0.00%  0.00%  0.00%   0 MMN bkgrd proces

              152           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Mgr Proces

              153           0         2          0  0.00%  0.00%  0.00%   0 802.1x Accountin

              154           0         1          0  0.00%  0.00%  0.00%   0 MAB Framework  

              155           0        80          0  0.00%  0.00%  0.00%   0 802.1x switch  

              156        1312    489500          2  0.00%  0.00%  0.00%   0 802.1x MDA Aging

              157           0         1          0  0.00%  0.00%  0.00%   0 802.1x Webauth F

              158      454943   3890052        116  0.00%  0.00%  0.00%   0 DTP Protocol   

              159           0         1          0  0.00%  0.00%  0.00%   0 EAP Framework  

              160           0         1          0  0.00%  0.00%  0.00%   0 HRPC dai request

              161           0         1          0  0.00%  0.00%  0.00%   0 HULC DAI Process

              162           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip device t

              163           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip source g

              164           0         1          0  0.00%  0.00%  0.00%   0 HULC IP Source g

              165        4758  29372345          0  0.00%  0.00%  0.00%   0 UDLD           

              166         314    979075          0  0.00%  0.00%  0.00%   0 Port-Security  

              167          18       281         64  0.00%  0.00%  0.00%   0 Switch IP Host T

              168           0         1          0  0.00%  0.00%  0.00%   0 Link State Group

              169         575   2935678          0  0.00%  0.00%  0.00%   0 Ethchnl        

              170        8605      1178       7304  0.00%  0.00%  0.00%   0 VMATM Callback 

              171           0         1          0  0.00%  0.00%  0.00%   0 IPv6 Echo event

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

              172           0         2          0  0.00%  0.00%  0.00%   0 UNCLIP Process 

              173      495598  11784012         42  0.00%  0.00%  0.00%   0 IPe BSSM Manager

              174         115      2616         43  0.00%  0.00%  0.00%   0 AAA Server     

              175        1449     18490         78  0.00%  0.00%  0.00%   0 AAA ACCT Proc  

              176           0         1          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr

              177           0         1          0  0.00%  0.00%  0.00%   0 AAA System Acct

              178           0         1          0  0.00%  0.00%  0.00%   0 Auth-proxy AAA B

              179          85     97907          0  0.00%  0.00%  0.00%   0 IP Admin SM Proc

              180      938945   5530344        169  0.00%  0.00%  0.00%   0 CDP Protocol   

              181         857   1957205          0  0.00%  0.00%  0.00%   0 MDFS LC Download

              182           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl2mcm igmp

              183           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R

              184         313    244770          1  0.00%  0.00%  0.00%   0 DHCP Snooping  

              185           0         3          0  0.00%  0.00%  0.00%   0 CEF switching ba

              186     3545483  16071011        220  0.15%  0.02%  0.00%   0 IP Input       

              187           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl

              188       13914  56693995          0  0.00%  0.00%  0.00%   0 IP ARP Track   

              189       38798 291738085          0  0.00%  0.00%  0.00%   0 MDFS MFIB Proces

              190           8        93         86  0.00%  0.00%  0.00%   0 ADJ resolve proc

              191         572       748        764  0.00%  0.00%  0.00%   0 SpanTree Flush 

              192    84321498 937146882         89  0.63%  0.49%  0.48%   0 Spanning Tree  

              193         122    489511          0  0.00%  0.00%  0.00%   0 Spanning Tree St

              194           0         2          0  0.00%  0.00%  0.00%   0 EAPoUDP Process

              195           0         2          0  0.00%  0.00%  0.00%   0 KRB5 AAA       

              196        3102    639843          4  0.00%  0.00%  0.00%   0 CEF background p

              197           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP        

              198           0         1          0  0.00%  0.00%  0.00%   0 CEF RF HULC Conv

              199           8         3       2666  0.00%  0.00%  0.00%   0 XDR mcast      

              200           0         1          0  0.00%  0.00%  0.00%   0 IPC LC Message H

              201           0         1          0  0.00%  0.00%  0.00%   0 XDR RP Ping Back

              202          34    244756          0  0.00%  0.00%  0.00%   0 XDR RP backgroun

              203           0         1          0  0.00%  0.00%  0.00%   0 XDR RP Test Back

              204           0         3          0  0.00%  0.00%  0.00%   0 MDFS LC Process

              205        7509    624316         12  0.00%  0.00%  0.00%   0 TCP Timer      

              206         449      1279        351  0.00%  0.00%  0.00%   0 TCP Protocols  

              207       10988  29382470          0  0.00%  0.00%  0.00%   0 Socket Timers  

              208          85     97912          0  0.00%  0.00%  0.00%   0 HTTP CORE      

              209         122    367145          0  0.00%  0.00%  0.00%   0 Cluster L2     

              210         810   2935697          0  0.00%  0.00%  0.00%   0 Cluster RARP   

              211       64234   3952462         16  0.00%  0.00%  0.00%   0 Cluster Base   

              212           0         1          0  0.00%  0.00%  0.00%   0 RARP Input     

              213           0         2          0  0.00%  0.00%  0.00%   0 REP LSL Proc   

              214           0         2          0  0.00%  0.00%  0.00%   0 REP BPA/EPA Proc

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

              215           0         1          0  0.00%  0.00%  0.00%   0 802.1x Critical

              216      357868  59117821          6  0.00%  0.00%  0.00%   0 DHCPD Receive  

              217           0         2          0  0.00%  0.00%  0.00%   0 IPe BS         

              218           0         2          0  0.00%  0.00%  0.00%   0 IPe P2P        

              219       17587    489352         35  0.00%  0.00%  0.00%   0 IPe BS SAF     

              220           0         1          0  0.00%  0.00%  0.00%   0 IPe DM Main    

              221           0         1          0  0.00%  0.00%  0.00%   0 IPe DM Backgroun

              222       85710 291694905          0  0.00%  0.00%  0.00%   0 IPe Host       

              223           0         1          0  0.00%  0.00%  0.00%   0 HRPC dhcp snoopi

              224           0         4          0  0.00%  0.00%  0.00%   0 HULC DHCP Snoopi

              225        3123    367113          8  0.00%  0.00%  0.00%   0 IGMPSN L2MCM   

              226         193        60       3216  0.00%  0.00%  0.00%   0 IGMPSN MRD     

              227          59       746         79  0.00%  0.00%  0.00%   0 IGMPSN         

              228           0         1          0  0.00%  0.00%  0.00%   0 IGMPQR         

              229           0         2          0  0.00%  0.00%  0.00%   0 L2TRACE SERVER 

              230        3602    367109          9  0.00%  0.00%  0.00%   0 MLDSN L2MCM    

              231           0         1          0  0.00%  0.00%  0.00%   0 MRD            

              232           0         1          0  0.00%  0.00%  0.00%   0 MLD_SNOOP      

              233           0         1          0  0.00%  0.00%  0.00%   0 HRPC hwccp reque

              234           0         1          0  0.00%  0.00%  0.00%   0 HULC_WCCP_BACKGR

              235           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl2mcm mlds

              236        9970    489459         20  0.00%  0.00%  0.00%   0 IP RIB Update  

              237           0         1          0  0.00%  0.00%  0.00%   0 HRPC x_setup req

              238      125332 293486881          0  0.00%  0.00%  0.00%   0 MDFS RP process

              239         899    489503          1  0.00%  0.00%  0.00%   0 DHCPD Database 

              240           9       246         36  0.00%  0.00%  0.00%   0 LOCAL AAA      

              241           0         2          0  0.00%  0.00%  0.00%   0 ENABLE AAA     

              242           0         2          0  0.00%  0.00%  0.00%   0 LINE AAA       

              243       36321  59008406          0  0.00%  0.00%  0.00%   0 RAC I/F Conf.  

              244       65019    119508        544  0.00%  0.00%  0.00%   0 TPLUS          

              245           9        12        750  0.00%  0.00%  0.00%   0 crypto engine pr

              246          26         4       6500  0.00%  0.00%  0.00%   0 Crypto CA      

              247           0         1          0  0.00%  0.00%  0.00%   0 Crypto PKI-CRL 

              248           0         1          0  0.00%  0.00%  0.00%   0 Crypto SSL     

              249           0         1          0  0.00%  0.00%  0.00%   0 encrypt proc   

              250           8       122         65  0.00%  0.00%  0.00%   0 VTP Trap Process

              251           0         2          0  0.00%  0.00%  0.00%   0 VTPMIB EDIT BUFF

              252           0         2          0  0.00%  0.00%  0.00%   0 DHCP Security He

              253           0         1          0  0.00%  0.00%  0.00%   0 HCD Process    

              254           0         1          0  0.00%  0.00%  0.00%   0 HRPC cable diagn

              255           0         2          0  0.00%  0.00%  0.00%   0 DiagCard1/-1   

              256       31079  46913750          0  0.00%  0.00%  0.00%   0 PM Callback    

              257          16        35        457  0.00%  0.00%  0.00%   0 VLAN Manager   

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

              258           0         1          0  0.00%  0.00%  0.00%   0 LICENSE AGENT  

              259         531   2348518          0  0.00%  0.00%  0.00%   0 dhcp snooping sw

              260           0         1          0  0.00%  0.00%  0.00%   0 Licensing MIB pr

              261      134990  56241965          2  0.00%  0.00%  0.00%   0 OSPF-10 Router 

              262         279      1278        218  0.00%  0.00%  0.00%   0 AAA SEND STOP EV

              263       13185      3764       3502  0.00%  0.00%  0.00%   0 Syslog Traps   

              264         143      3919         36  0.00%  0.00%  0.00%   0 Syslog         

              265        3204  15663417          0  0.00%  0.00%  0.00%   0 BGP Scheduler  

              266           9        51        176  0.00%  0.00%  0.00%   0 EEM Server     

              267           0         2          0  0.00%  0.00%  0.00%   0 EEM Policy Direc

              268           0         2          0  0.00%  0.00%  0.00%   0 EEM ED CLI     

              269           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Counter 

              270           0         3          0  0.00%  0.00%  0.00%   0 EM ED GOLD     

              271           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Interface

              272           0         3          0  0.00%  0.00%  0.00%   0 EEM ED IOSWD   

              273           0         3          0  0.00%  0.00%  0.00%   0 EEM ED None    

              274           9         3       3000  0.00%  0.00%  0.00%   0 EEM ED OIR     

              276           0         3          0  0.00%  0.00%  0.00%   0 EEM ED SNMP    

              277           9         2       4500  0.00%  0.00%  0.00%   0 EEM ED Snmp_Prox

              278         573    767079          0  0.00%  0.00%  0.00%   0 EEM ED Timer   

              279           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Test    

              280           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Config  

              281           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Env     

              282           0         2          0  0.00%  0.00%  0.00%   0 STP FAST TRANSIT

              283           0         2          0  0.00%  0.00%  0.00%   0 CSRT RAPID TRANS

              284           0         3          0  0.00%  0.00%  0.00%   0 EEM ED RPC     

              285       99809  45636529          2  0.00%  0.00%  0.00%   0 CEF: IPv4 proces

              286           0         5          0  0.00%  0.00%  0.00%   0 ADJ background 

              287      162232    957751        169  0.00%  0.00%  0.00%   0 IP Background  

              288         143         5      28600  0.00%  0.00%  0.00%   0 crypto sw pk pro

              289           0        40          0  0.00%  0.00%  0.00%   0 hulc cfg mgr mas

              290        7458  29177812          0  0.00%  0.00%  0.00%   0 DHCP Client    

              291       29293  29707057          0  0.00%  0.00%  0.00%   0 NTP            

              292           0         6          0  0.00%  0.00%  0.00%   0 SNMP Timers    

              293      917280   3390363        270  0.00%  0.00%  0.00%   0 IP SNMP        

              294      394494   1693311        232  0.00%  0.01%  0.00%   0 PDU DISPATCHER 

              295     3176390   1699042       1869  0.00%  0.01%  0.00%   0 SNMP ENGINE    

              296           0         2          0  0.00%  0.00%  0.00%   0 IP SNMPV6      

              297           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro

              298        3495      1822       1918  0.00%  0.00%  0.00%   0 SNMP Traps     

              299       33850       171     197953  0.00%  0.00%  0.07%   0 hulc running con

              300       59351   1340779         44  0.00%  0.00%  0.00%   0 BGP Router     

              301       20356    533283         38  0.00%  0.00%  0.00%   0 BGP I/O        

              PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

              302      956717   1074143        890  0.00%  0.00%  0.00%   0 BGP Scanner    

              303           0         5          0  0.00%  0.00%  0.00%   0 TCP Listener   

              304         349       222       1572  0.95%  0.07%  0.01%   1 Virtual Exec  

               

               

               

               

              show int gigabitEthernet 1/0/2 counters err

               

              Port        Align-Err     FCS-Err    Xmit-Err     Rcv-Err  UnderSize  OutDiscards

              Gi1/0/2             0           0           0           0          0         1524

               

              Port      Single-Col  Multi-Col   Late-Col  Excess-Col  Carri-Sen      Runts     Giants

              Gi1/0/2            0          0          0           0          0          0          0

               

               

               

               

              show proc mem

              Processor Pool Total:   80415592 Used:   36856576 Free:   43559016

                    I/O Pool Total:   12574720 Used:    8546964 Free:    4027756

               

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

                 0   0   50804216   12539936   35382584          0          0 *Init*        

                 0   0      13728  756403484      13728          0          0 *Sched*       

                 0   0 1805425944 2202538420    3257428   12628544    1229480 *Dead*        

                 1   0      20512       9176      19804          0          0 Chunk Manager 

                 2   0        180        180       3924          0          0 Load Meter    

                 3   0        216          0       7196          0          0 CEF RP IPC Backg

                 4   0          0          0       6924          0          0 Check heaps   

                 5   0          0   18197792       6924          0   11814756 Pool Manager  

                 6   0        180        180       6924          0          0 Timers        

                 7   0          0          0       6924          0          0 HRPC asic-stats

                 8   0          0          0      24924          0          0 Crash writer  

                 9   0    5678100      86108     358748          0          0 ARP Input     

                10   0          0          0       6924          0          0 CEF MIB API   

                11   0          0       1212       6924          0          0 AAA_SERVER_DEADT

                12   0        204        204       6924          0          0 AAA high-capacit

                13   0          0          0      12924          0          0 Policy Manager

                14   0     135944      47104      95832          0          0 Entity MIB API

                15   0     230560          0     240484      99792          0 EEM ED Syslog 

                16   0          0          0       6924          0          0 IFS Agent Manage

                17   0          0          0       6924          0          0 IPC Dynamic Cach

                18   0          0          0       6924          0          0 IPC Zone Manager

                19   0          0          0       6924          0          0 IPC Periodic Tim

                20   0          0          0       6924          0          0 IPC Managed Time

                21   0          0          0       6924          0          0 IPC Deferred Por

                22   0       5232          0      12156          0          0 IPC Seat Manager

                23   0          0          0       6924          0          0 IPC Session Serv

                24   0          0          0       6924          0          0 Dynamic ARP Insp

                25   0          0          0       6924          0          0 ARP Snoop     

                26   0          0          0      12924          0          0 License IPC stat

                27   0          0          0      12924          0          0 License IPC serv

                28   0        180        180       6924          0          0 GraphIt       

                29   0        180        180      12924          0          0 XML Proxy Client

                30   0          0          0       6924          0          0 Critical Bkgnd

                31   0      21384          0      13536          0          0 Net Background

                32   0          0          0      12924          0          0 IDB Work      

                33   0        180        180      12924          0          0 Logger        

                34   0        180       1892       9924          0          0 TTY Background

                35   0  211202196  216432524      32108          0          0 Per-Second Jobs

                36   0          0        208       6924          0          0 Per-minute Jobs

                37   0     210932     146656     111244          0          0 IF-MGR control p

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

                38   0          0          0       6924          0          0 IF-MGR event pro

                39   0          0          0       6924          0          0 AggMgr Process

                40   0          0          0       6924          0          0 Net Input     

                41   0        180        180       6924          0          0 Compute load avg

                42   0          0    1987400       6924          0          0 Transport Port A

                43   0          0          0       6924          0          0 HC Counter Timer

                44   0        180        180       6924          0          0 DownWhenLooped

                45   0          0          0       6924          0          0 HRPC hdwl reques

                46   0          0          0       6924          0          0 HRPC lpip reques

                47   0        180        180       6924          0          0 HLPIP Sync Proce

                48   0          0          0       6924          0          0 HRPC Multi-FS Sy

                49   0          0          0       6924          0          0 HULC multifs pro

                50   0          0          0       6924          0          0 Hulc Warm Upgrad

                51   0        180        180       6924          0          0 MIRAGE RBCP Moni

                52   0          0          0       6924          0          0 Platform-IPe Pro

                53   0          0          0       6924          0          0 Hulc EEM Process

                54   0        396       4708      13140          0          0 Collection proce

                55   0          0          0       3924          0          0 HRPC hsm request

                56   0          0          0      12924          0          0 Stack Mgr     

                57   0     286940       5484     212456          0          0 Stack Mgr Notifi

                58   0       2496        180       9240          0          0 Fifo Error Detec

                59   0        576        576       3924          0          0 Adjust Regions

                60   0          0          0       6924          0          0 hrpc -> response

                61   0          0          0       6924          0          0 hrpc -> request

                62   0    7442876 1841080072       6924          0          0 hrpc <- response

                63   0          0          0       6924          0          0 HRPC hcomp reque

                64   0        732        180       6924          0          0 HULC Device Mana

                65   0        116        116       6924          0          0 HRPC hdm non blo

                66   0        176        176       6924          0          0 HRPC hdm blockin

                67   0          0          0      12924          0          0 HIPC bkgrd proce

                68   0          0          0       6924          0          0 RTTYS Process 

                69   0        180      45668       6924          0          0 Hulc Port-Securi

                70   0          0          0       6924          0          0 HRPC hpsecure re

                71   0          0          0       6924          0          0 HRPC hlfm reques

                72   0    2483524       3828     338920          0          0 HLFM address lea

                73   0        180        180       6924          0          0 HLFM aging proce

                74   0        180        180       6924          0          0 HLFM address ret

                75   0          0          0      12924          0          0 HRPC hrcmd reque

                76   0          0          0       6924          0          0 HRPC emac reques

                77   0    4042808    4042808       6924          0          0 HRPC hulc misc r

                78   0          0          0       3924          0          0 HRPC system mtu

                79   0          0          0      12924          0          0 HVLAN main bkgrd

                80   0        180        180       6924          0          0 HVLAN Mapped Vla

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

                81   0        180        180       6924          0          0 Vlan shutdown Pr

                82   0          0          0       6924          0          0 HRPC vlan reques

                83   0          0          0       6924          0          0 HULC VLAN REF Ba

                84   0          0          0       6924          0          0 HRPC hfbm reques

                85   0          0          0       6976          0          0 HCMP sync proces

                86   0          0          0       6924          0          0 HRPC ilp request

                87   0          0          0       6924          0          0 HULC PM Vector P

                88   0          0          0       6924          0          0 HPM Msg Retry Pr

                89   0          0          0       6948          0          0 DHCPD Timer   

                90   0   15124856   13334884     183016          0          0 hpm main process

                91   0        180        180       6924          0          0 HPM Stack Sync P

                92   0          0          0      12924          0          0 HRPC pm request

                93   0        180        180       3924          0          0 HPM if_num mappi

                94   0        180        180       6924          0          0 hpm counter proc

                95   0          0          0       6924          0          0 HRPC pm-counters

                96   0          0          0       6924          0          0 hpm vp events ca

                97   0          0          0       6924          0          0 HRPC hcmp reques

                98   0          0          0      12924          0          0 HCEF ADJ Refresh

                99   0          0          0       6924          0          0 HRPC hl3mm reque

              100   0          0          0       3924          0          0 hl3md_rpfq_thrl_

              101   0     387072     387072       6924          0          0 hl3mm         

              102   0          0          0       6924          0          0 hl3mm_rp      

              103   0          0          0       6924          0          0 HACL Queue Proce

              104   0          0          0       6924          0          0 HRPC acl request

              105   0       5224  185351584      65968          0          0 HACL Acl Manager

              106   0          0          0       6924          0          0 HRPC backup inte

              107   0      89160       7152      12788      65464          0 OSPF-10 Hello 

              108   0          0          0       6924          0          0 HRPC cdp request

              109   0          0          0       6924          0          0 HRPC lldp reques

              110   0          0          0       6924          0          0 HRPC dot1x reque

              111   0          0          0       6924          0          0 HULC DOT1X Proce

              112   0          0          0       6924          0          0 HRPC foxhound re

              113   0        472        180       7216          0          0 Hulc Foxhound Pr

              114   0          0          0       6924          0          0 HRPC rep request

              115   0        180        180       6924          0          0 REP Helper Proc

              116   0          0          0       6924          0          0 HRPC sdm request

              117   0        180        180       6924          0          0 Hulc Storm Contr

              118   0        180        180       6924          0          0 HSTP Sync Proces

              119   0          0          0       6924          0          0 HRPC stp_cli req

              120   0          0          0       6924          0          0 HRPC stp_state_s

              121   0        180        180       6924          0          0 S/W Bridge Proce

              122   0          0          0       6924          0          0 HRPC hudld reque

              123   0          0          0       6924          0          0 HRPC vqpc reques

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

              124   0          0          0       6924          0          0 HRPC iec_load_ba

              125   0          0          0       6924          0          0 HRPC l2pt qnq rp

              126   0          0          0       6924          0          0 HRPC IPe request

              127   0          0          0       6924          0          0 HRPC hled reques

              128   0        180        180       6924          0          0 Hulc LED Process

              129   0          0     590220      12924          0          0 HL3U bkgrd proce

              130   0          0          0       6924          0          0 HRPC hl3u reques

              131   0          0        544      60924          0          0 HL3U PBR bkgrd p

              132   0          0          0       6924          0          0 HL3U PBR n-h res

              133   0          0          0       6924          0          0 HRPC dtp request

              134   0          0          0       6924          0          0 HRPC show_forwar

              135   0          0          0       6924          0          0 HRPC snmp reques

              136   0 2386586760  550924320      11832          0          0 HQM Stack Proces

              137   0 3220967340 3220869984      86604      45684          0 HRPC qos request

              138   0          0          0       6924          0          0 HRPC span reques

              139   0          0          0       6924          0          0 HRPC obfl reques

              140   0          0          0       6924          0          0 HRPC system post

              141   0          0          0       6924          0          0 Hulc Reload Mana

              142   0          0          0       6924          0          0 HRPC hrcli-event

              143   0   12854368    1607476       7536          0          0 SpanTree Helper

              144   0        180        180       9924          0          0 image mgr     

              145   0        180        180       6924          0          0 Power RPS Proces

              146   0     323620      30280     294332          0          0 HL2MCM        

              147   0     322184      28292     294332          0          0 HL2MCM        

              148   0        180      23424       6924          0          0 PI MATM Aging Pr

              149   0        212        212       6924          0          0 REP Topology cha

              150   0      71308          0      78232          0          0 Switch Backup In

              151   0          0          0       6924          0          0 MMN bkgrd proces

              152   0        180        180       6924          0          0 Dot1x Mgr Proces

              153   0        180        180       9924          0          0 802.1x Accountin

              154   0          0          0       6924          0          0 MAB Framework 

              155   0          0      11948       6924          0          0 802.1x switch 

              156   0        180        180       6924          0          0 802.1x MDA Aging

              157   0          0          0       6924          0          0 802.1x Webauth F

              158   0    1984416    1528064      99236          0          0 DTP Protocol  

              159   0          0          0       6924          0          0 EAP Framework 

              160   0          0          0       6924          0          0 HRPC dai request

              161   0          0          0       3924          0          0 HULC DAI Process

              162   0          0          0       6924          0          0 HRPC ip device t

              163   0          0          0       6924          0          0 HRPC ip source g

              164   0          0          0       6924          0          0 HULC IP Source g

              165   0        472        180       7216          0          0 UDLD          

              166   0        216          0       7140          0          0 Port-Security 

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

              167   0        180        180       9924          0          0 Switch IP Host T

              168   0          0          0       6924          0          0 Link State Group

              169   0        180        180       6924          0          0 Ethchnl       

              170   0     189264    3873148     196188          0          0 VMATM Callback

              171   0          0          0       6924          0          0 IPv6 Echo event

              172   0          0          0       6924          0          0 UNCLIP Process

              173   0        216          0       7140          0          0 IPe BSSM Manager

              174   0        180        180       6924          0          0 AAA Server    

              175   0          0          0       6924          0          0 AAA ACCT Proc 

              176   0          0          0       6924          0          0 ACCT Periodic Pr

              177   0          0          0       6924          0          0 AAA System Acct

              178   0          0          0       6924          0          0 Auth-proxy AAA B

              179   0          0          0       6924          0          0 IP Admin SM Proc

              180   0  105826220   96173956      34620    8000636          0 CDP Protocol  

              181   0          0          0       6948          0          0 MDFS LC Download

              182   0          0          0       6924          0          0 HRPC hl2mcm igmp

              183   0        204        204       6924          0          0 AAA Dictionary R

              184   0          0          0       6924          0          0 DHCP Snooping 

              185   0        396        180      13140          0          0 CEF switching ba

              186   0    2724380   24060736      15972          0          0 IP Input      

              187   0          0          0       6924          0          0 ICMP event handl

              188   0        180        180       9924          0          0 IP ARP Track  

              189   0        180        180      12924          0          0 MDFS MFIB Proces

              190   0        520        180      13140          0          0 ADJ resolve proc

              191   0    1580124    1810400       6924          0          0 SpanTree Flush

              192   0    7700596     118024       7140     125736          0 Spanning Tree 

              193   0          0          0       6924          0          0 Spanning Tree St

              194   0        180        180       9924          0          0 EAPoUDP Process

              195   0        180        180      12924          0          0 KRB5 AAA      

              196   0        484        424      13140          0          0 CEF background p

              197   0          0          0       9924          0          0 IP IRDP       

              198   0          0          0       6924          0          0 CEF RF HULC Conv

              199   0          0          0       6924          0          0 XDR mcast     

              200   0          0          0       6924          0          0 IPC LC Message H

              201   0          0          0       6924          0          0 XDR RP Ping Back

              202   0          0          0       6924          0          0 XDR RP backgroun

              203   0          0          0       6924          0          0 XDR RP Test Back

              204   0       7332        180      13128          0          0 MDFS LC Process

              205   0      45364     492048      13060          0          0 TCP Timer     

              206   0   23856356          0      12924          0          0 TCP Protocols 

              207   0          0          0       6924          0          0 Socket Timers 

              208   0      16636       4784      21624          0          0 HTTP CORE     

              209   0        180        180       6924          0          0 Cluster L2    

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

              210   0          0          0       6924          0          0 Cluster RARP  

              211   0   70821760   70810052       7360       9352          0 Cluster Base  

              212   0          0          0       6924          0          0 RARP Input    

              213   0        180        180       6924          0          0 REP LSL Proc  

              214   0      10224        180      16968          0          0 REP BPA/EPA Proc

              215   0          0          0       6924          0          0 802.1x Critical

              216   0      40192        180      39032          0          0 DHCPD Receive 

              217   0        180        180       6924          0          0 IPe BS        

              218   0        180        180       6924          0          0 IPe P2P       

              219   0        180        180       3924          0          0 IPe BS SAF    

              220   0          0          0       9924          0          0 IPe DM Main   

              221   0          0          0       9924          0          0 IPe DM Backgroun

              222   0          0          0       6924          0          0 IPe Host      

              223   0          0          0       6924          0          0 HRPC dhcp snoopi

              224   0          0        384       6924          0          0 HULC DHCP Snoopi

              225   0    1361180 3060283188      90216          0          0 IGMPSN L2MCM  

              226   0     416652     418916      10488          0          0 IGMPSN MRD    

              227   0      10672       2240      13156          0          0 IGMPSN        

              228   0          0          0       6924          0          0 IGMPQR        

              229   0       2876        180       9620          0          0 L2TRACE SERVER

              230   0    1332372 3059976440     101648          0          0 MLDSN L2MCM   

              231   0          0          0       6924          0          0 MRD           

              232   0          0          0       6924          0          0 MLD_SNOOP     

              233   0          0          0       6924          0          0 HRPC hwccp reque

              234   0          0          0       6924          0          0 HULC_WCCP_BACKGR

              235   0          0          0       6924          0          0 HRPC hl2mcm mlds

              236   0     282592       1644     290872          0          0 IP RIB Update 

              237   0          0          0       6924          0          0 HRPC x_setup req

              238   0 1151275152          0       6924          0          0 MDFS RP process

              239   0        276          0       7232          0          0 DHCPD Database

              240   0      38436       6476       6924          0          0 LOCAL AAA     

              241   0        180        180       6924          0          0 ENABLE AAA    

              242   0        180        180       6924          0          0 LINE AAA      

              243   0 2224115028       4096       6924          0          0 RAC I/F Conf. 

              244   0   58457268   42934524       9596     290304          0 TPLUS         

              245   0      30948       1736      26892          0          0 crypto engine pr

              246   0     531592     525944      14572          0          0 Crypto CA     

              247   0          0          0       8924          0          0 Crypto PKI-CRL

              248   0          0          0       8924          0          0 Crypto SSL    

              249   0          0          0       6988          0          0 encrypt proc  

              250   0       4824       6112      11568          0          0 VTP Trap Process

              251   0        180        180       6924          0          0 VTPMIB EDIT BUFF

              252   0        180        180       6924          0          0 DHCP Security He

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

              253   0          0          0       6924          0          0 HCD Process   

              254   0          0          0       6924          0          0 HRPC cable diagn

              255   0        132        348       6924          0          0 DiagCard1/-1  

              256   0    1785824    2475168      49244          0          0 PM Callback   

              257   0      92048      74420      16732          0          0 VLAN Manager  

              258   0       8888          0      33836          0          0 LICENSE AGENT 

              259   0        180        180       6924          0          0 dhcp snooping sw

              260   0          0          0       9924          0          0 Licensing MIB pr

              261   0    1589208    1512408      18308      46760          0 OSPF-10 Router

              262   0     410536     958564       6924          0          0 AAA SEND STOP EV

              263   0    7780556    7780556       6924          0          0 Syslog Traps  

              264   0     938656     936136      15444          0          0 Syslog        

              265   0      14320          0       9952          0          0 BGP Scheduler 

              266   0     136152          0     143076          0          0 EEM Server    

              267   0       9676        832      15768          0          0 EEM Policy Direc

              268   0       3724          0      13648          0          0 EEM ED CLI    

              269   0       3724          0      13648          0          0 EEM ED Counter

              270   0       3724          0      13648          0          0 EM ED GOLD    

              271   0       3724          0      13648          0          0 EEM ED Interface

              272   0       3724          0      13648          0          0 EEM ED IOSWD  

              273   0       3724          0      13648          0          0 EEM ED None   

              274   0       3724          0      13648          0          0 EEM ED OIR    

              276   0       3724          0      13648          0          0 EEM ED SNMP   

              277   0       3724          0      13648          0          0 EEM ED Snmp_Prox

              278   0       3724          0      13648          0          0 EEM ED Timer  

              279   0       3724          0      13648          0          0 EEM ED Test   

              280   0       3724          0      13648          0          0 EEM ED Config 

              281   0       3724          0      13648          0          0 EEM ED Env    

              282   0        180        180       3924          0          0 STP FAST TRANSIT

              283   0        180        180       3924          0          0 CSRT RAPID TRANS

              284   0       3724          0      13648          0          0 EEM ED RPC    

              285   0     260084        316     265556          0          0 CEF: IPv4 proces

              286   0        216       1544       7140          0          0 ADJ background

              287   0        444          0      10368          0          0 IP Background 

              288   0      29676      29996       8924          0          0 crypto sw pk pro

              289   0          0          0      12924          0          0 hulc cfg mgr mas

              290   0          0  234851240      12924          0          0 DHCP Client   

              291   0        548        180       7292          0          0 NTP           

              292   0          0          0       6940          0          0 SNMP Timers   

              293   0  951421580  951221680      13864          0          0 IP SNMP       

              294   0 1322966628          0      12924          0          0 PDU DISPATCHER

              295   0 2421475764 3746699316      12924          0          0 SNMP ENGINE   

              296   0       1056        180      13800          0          0 IP SNMPV6     

              PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

              297   0          0          0      12924          0          0 SNMP ConfCopyPro

              298   0    4387548    4579616      13264          0          0 SNMP Traps    

              299   0  120504948  119979056     532752          0          0 hulc running con

              300   0      48300      12792      15616          0          0 BGP Router    

              301   0          0          0       9924          0          0 BGP I/O       

              302   0       1808          0      11732          0          0 BGP Scanner   

              303   0          0          0       6924          0          0 TCP Listener  

              304   1      45124      40336      16992          0          0 Virtual Exec  

                                               45396908 Total

               

               

               

               

              SWITCH A (Juniper)

               

              Physical interface: ge-0/0/23, Enabled, Physical link is Up

                Interface index: 135, SNMP ifIndex: 519, Generation: 138

                Link-level type: Ethernet, MTU: 2016, Speed: 100mbps, Duplex: Full-Duplex,

                BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled,

                Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Disabled,

                Remote fault: Online

                Device flags   : Present Running

                Interface flags: SNMP-Traps Internal: 0x0

                Link flags     : None

                CoS queues     : 8 supported, 8 maximum usable queues

                Hold-times     : Up 0 ms, Down 0 ms

                Current address: 2c:6b:f5:9d:24:57, Hardware address: 2c:6b:f5:9d:24:57

                Last flapped   : 2011-05-06 07:04:45 UTC (9w3d 14:50 ago)

                Statistics last cleared: Never

                Traffic statistics:

                 Input  bytes  :       12895205568003             47963272 bps

                 Output bytes  :        2212441898958              7674648 bps

                 Input  packets:          13471319952                 5793 pps

                 Output packets:           9886031166                 4033 pps

                 IPv6 transit statistics:

                  Input  bytes  :                   0

                  Output bytes  :                   0

                  Input  packets:                   0

                  Output packets:                   0

                Input errors:

                  Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0,

                  L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0,

                  Resource errors: 0

                Output errors:

                  Carrier transitions: 1, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0,

                  FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0

                Egress queues: 8 supported, 6 in use

                Queue counters:       Queued packets  Transmitted packets      Dropped packets

                  0 BE                             0           8066283234                    0

                  1 AF                             0              1487535                    0

                  2 NC                             0                 1200                    0

                  3 EF                             0           1814717671                    0

                  5 expedited-fo                   0                    0                    0

                  7 network-cont                   0              3545506                    0

                Queue number:         Mapped forwarding classes

                  0                   BE        

                  1                   AF        

                  2                   NC        

                  3                   EF            

                  5                   expedited-forwarding

                  7                   network-control

                Active alarms  : None

                Active defects : None

                MAC statistics:                      Receive         Transmit

                  Total octets                12895205568003    2212441898958

                  Total packets                  13471319952       9886031166

                  Unicast packets                13370352444       9880518610

                  Broadcast packets                  1858096          1967038

                  Multicast packets                 99109412          3545518

                  CRC/Align errors                         0                0

                  FIFO errors                              0                0

                  MAC control frames                       0                0

                  MAC pause frames                         0                0

                  Oversized frames                         0

                  Jabber frames                            0

                  Fragment frames                          0

                  Code violations                          0

                Autonegotiation information:

                  Negotiation status: No-autonegotiation

                Packet Forwarding Engine configuration:

                  Destination slot: 0

                CoS information:

                  Direction : Output

                  CoS transmit queue               Bandwidth               Buffer Priority   Limit

                                            %            bps     %           usec

                  0 BE                     50       50000000     r             NA      low    none

                  1 AF                     30       30000000    15             NA      low    none

                  2 NC                      5        5000000     5             NA      low    none

                  3 EF                      r              r    15             NA strict-high    none

               

                Logical interface ge-0/0/23.0 (Index 78) (SNMP ifIndex 520) (Generation 143)

                  Flags: SNMP-Traps 0x0 Encapsulation: ENET2

                  Traffic statistics:

                   Input  bytes  :           1745225971

                   Output bytes  :            473832736

                   Input  packets:             23633334

                   Output packets:              3529966

                  Local statistics:

                   Input  bytes  :           1745225971

                   Output bytes  :            473832736

                   Input  packets:             23633334

                   Output packets:              3529966

                  Transit statistics:

                   Input  bytes  :                    0                    0 bps

                   Output bytes  :                    0                    0 bps

                   Input  packets:                    0                    0 pps

                   Output packets:                    0                    0 pps

                  Protocol eth-switch, Generation: 159, Route table: 0

                    Flags: Trunk-Mode

              • 4. Re: Total Output Drops
                StuO

                Now I have other links that I'm looking into and the discards are always located on Switch B

                • 5. Re: Total Output Drops
                  smsnaqvi

                  Hi dude

                   

                  Please read below. This could be related to the QOS parameters.

                   

                  http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a0080883f9e.shtml#topic3

                  • 6. Re: Total Output Drops
                    smsnaqvi

                    Please also correct me if I am wrong but wont this

                    priority-queue out command prioritize the traffic for the 4th queue thus causing congestion issues????

                    • 7. Re: Total Output Drops
                      Brian

                      I think smsnaqvi may be on to something.  You may have a problem with the QoS settings.  Have you changed any of the default parameters other than what is on the interface?

                       

                      Here is a picture showing the default CoS and DSCP mappings regarding which queue services that particular CoS or DSCP value.

                       

                      cat3750-qos-default-queue-map.gif

                       

                      Also, I would check the current CoS to Egress queue mappins

                       

                      sh mls qos maps cos-output-q

                       

                      and the DSCP to Egress queue mappings

                       

                      sh mls qos maps dscp-output-q

                       

                      HTH

                      • 8. Re: Total Output Drops
                        StuO

                        Hi Brian,

                        #show mls qos map cos-output-q

                           Cos-outputq-threshold map:

                                      cos:  0   1   2   3   4   5   6   7 

                                      ------------------------------------

                          queue-threshold: 4-3 4-2 3-3 2-3 3-3 1-3 2-3 2-3

                         

                        #

                        show mls qos maps dscp-output-q

                           Dscp-outputq-threshold map:

                             d1 :d2    0     1     2     3     4     5     6     7     8     9

                             ------------------------------------------------------------

                              0 :    04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-01 04-02

                              1 :    04-02 04-02 04-02 04-02 04-02 04-02 03-03 03-03 03-03 03-03

                              2 :    03-03 03-03 03-03 03-03 02-03 02-03 02-03 02-03 02-03 02-03

                              3 :    02-03 02-03 03-03 03-03 03-03 03-03 03-03 03-03 03-03 03-03

                              4 :    01-03 01-03 01-03 01-03 01-03 01-03 01-03 01-03 02-03 02-03

                              5 :    02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03

                              6 :    02-03 02-03 02-03 02-03

                         

                        QoS setting are the same across all switches.

                         

                        Regards,

                         

                         

                        Stu

                        • 9. Re: Total Output Drops
                          Brian

                          Hey StuO,

                           

                          Sorry for the late reply.  I see you have changed these from the defaults.  Is it possible to post the switch configs  you can hide any proprietary information.

                           

                          Brian

                           

                          • 10. Re: Total Output Drops
                            StuO

                            Hi Brian and thanks for the HELP!!

                            Please see attachment.

                             

                            Regards,

                             

                             

                            Stu

                            • 11. Re: Total Output Drops
                              StuO

                              Just wondering if anyone has any further ideas into this issue?

                               

                              thanks