High Process Counts

In this example, we report an unusually high number of system processes.

The HighProcessCount script might send an alert message like the following:

                                PIKT ALERT
                         Thu Mar 27 10:15:29 2003
                                  naples

EMERGENCY:
    HighProcessCount
        Report extremely high number of system processes

        Unusually high process count (219): Tasks:  219 total,   1 running,
                                                    216 sleeping,   0 stopped,
                                                      2 zombie

        USER    PID %CPU %MEM   VSZ  RSS TTY  STAT START   TIME COMMAND
        root      1  0.0  0.0   620   72 ?    S    Mar25   0:05 init [5]    
        root      2  0.0  0.0     0    0 ?    SW   Mar25   0:00 [keventd]
        root      3  0.0  0.0     0    0 ?    SW   Mar25   0:00 [kapmd]
        root      4  0.0  0.0     0    0 ?    SWN  Mar25   0:00 [ksoftirqd_CPU0]
        root      5  0.0  0.0     0    0 ?    SW   Mar25   0:00 [kswapd]
        root      6  0.0  0.0     0    0 ?    SW   Mar25   0:00 [bdflush]
        root      7  0.0  0.0     0    0 ?    SW   Mar25   0:00 [kupdated]
        ...

The script follows.

HighProcessCount

        init
                status =piktstatus
                level =piktlevel
                task "Report extremely high number of system processes"
                input proc "=top -b -d 1 -n 1 | egrep -i '^tasks:'"
                dat "([[:digit:]]+)[[:space:]]total"

        rule    // set the process count
                set #procct = #val($1)

#ifdef debug
        rule
                output "\$inlin is $inlin"
                output "\#procct is $text(#procct)"
#endifdef

        rule    // for diagnostic purposes
                =output_alarm_log($inlin)

        // if we ever need to check this on a per-machine (or per-
        // hostgroup) basis, we really should set up a new objects file,
        // ProcessCounts.obj, with fields like so:
        //
        // //host      //processcount    //pcincr
        //
        // then read the data in using =readvals() and process in the usual
        // manner

        rule    // report unusually high process count
                if #procct >= 200 && =increased(procct, 200, 20)
                        output mail "Unusually high process count
                                     ($text(#procct)): $inlin"
                        output mail =newline
                        =outputproc(mail, "=psall")
                fi

For more examples, see Samples.