Swap File Utilization
In this example, we report excessive swap file utilization, exhaustion of system virtual memory.
The SwapExhausted script might send an alert message like the following:
PIKT ALERT
Wed Dec 12 21:13:21 2001
kiev2
EMERGENCY:
SwapExhausted
Report when the system swapfile is exhausted
swap utilization is 100%:
swapfile dev swaplo blocks free
/dev/dsk/c0t1d0s1 32,9 8 1025112 0
swap 662992 655952 7040 99% /tmp
655652 /tmp/SAS_worka0000268D
4 /tmp/screens
contents of /tmp:
total 624
drwx------ 2 droulser perf 473 Dec 12 17:41 SAS_worka0000268D
drwxr-xr-x 2 root other 69 Dec 12 08:14 screens
-rw-rw-r-- 1 root sys 301828 Dec 12 19:32 ups_data
last pid: 11924; load averages: 0.27, 0.07, 0.05 21:13:24
34 processes: 33 sleeping, 1 on cpu
Memory: 224M real, 4320K free, 501M swap in use
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
11924 root 1 0 0 1580K 1004K cpu0 0:01 1.27% top
1 root 1 58 0 708K 164K sleep 0:08 0.09% init
3524 droulser 4 51 0 24M 20M run 2:10 19.27% sas
...
In the SwapExhausted Pikt script following, note how we go beyond reporting just the fact that system swap space is exhausted. We also provide diagnostic information to suggest possible causes.
#if solaris
SwapExhausted // note: we should seriously consider having this
// script auto-installing auxiliary swap files, at
// least in some situations; we might then have
// a system startup script auto-delete these auxiliary
// swap files on system reboot
init
status =piktstatus
level =piktlevel
task "Report when the system swapfile is exhausted"
input proc "=swap -l | =behead(1)"
dat $swapfile 1
dat #blocks $-1
dat #free $
// keys $swapfile
begin
set #blksum = 0
set #fresum = 0
set #hr = #hour()
rule
set $line[#innum()] = $inlin
set #blksum += #blocks
set #fresum += #free
end
set #use = (#blksum-#fresum)/#blksum
if #use >= 100%
output mail "swap utilization is
$text(100*#use,0)%:=newline"
output mail "swapfile dev swaplo blocks free"
for #i=1 #i<=#innum() #i+=1
output mail $line[#i]
endfor
output mail =newline
output mail $command("=dfk /tmp | =behead(1)")
=dutop(10, /tmp)
output mail "contents of /tmp:=newline"
=outputproc(mail, "=ll /tmp")
output mail =newline
=toptop(20)
endif
#endif // solaris
This is just one program example. You could add rules, or write new scripts, for example to: report when system virtual memory runs low, create additional swap files on the fly, log memory usage over time, log and report per-process or per-user memory usage, etc.
For more examples, see Samples.
