alerts

Here are some sample PIKT alerts.  These are only highlights.  Examples of more routine and mundane problem reports and information messages are not shown.

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Sep 27 12:33:15 2002
                                trondheim2

CRITICAL:
    AuthLogScanCritical
        Scan the authlog for critical authorization incidents

        Sep 27 12:24:40 trondheim2 statd[174]: [ID 462824 auth.error] statd:
        attempt to create "/var/statmon/sm/^D...^D...^E...^F...^F...^G...^G...
        %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %0242x
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Oct  2 15:49:25 2002
                                 athens4

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct  2 15:40:21 athens4 unix: WARNING:
          /[email protected],0/QLGC,[email protected],10000/[email protected],0 (sd11):
        Oct  2 15:40:21 athens4 SCSI transport failed:
          reason 'reset': retrying command
        Oct  2 15:41:12 athens4 unix: WARNING:
          /[email protected],0/QLGC,[email protected],10000/[email protected],0 (sd11):
        Oct  2 15:41:12 athens4 SCSI transport failed:
          reason 'reset': retrying command
        ...


-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct  3 04:30:06 2002
                                 athens4

CRITICAL:
    DiskCapCritical
        Report critical filesystem full or near-full situations

        Filesystem /ckp on /dev/md/dsk/d10 is 100% full, 0 Kb left
        
        17370930        /ckp/ingres
        8       /ckp/lost+found
        
-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct  3 07:08:06 2002
                                 athens2

URGENT:
    ProcessSystemDeadUrgent
        Report or restart 'dead' crucial system processes

        The process 'syslogd' is not running

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Jul 28 02:00:02 2002
                                  davao

WARNING:
    MessagesScanWarning
        Scan the system messages log for perhaps worrisome entries

        Jul 27 14:19:37 handel sshd[5973]: Accepted publickey for root from
          ::ffff:11.22.33.44 port 42351 ssh2
        Jul 27 21:14:52 handel sshd[16487]:
          Illegal user test from ::ffff:12.34.56.78
        Jul 27 21:14:52 handel sshd[16487]: input_userauth_request:
          illegal user test
        Jul 27 21:14:55 handel sshd[16487]:
          Address 12.34.56.78 [gecko.blechcough.org.]
          maps to gecko.blechcough.org, but this does not map back to the address
          - POSSIBLE BREAKIN ATTEMPT!
        Jul 27 21:14:55 handel sshd[16487]:
          Failed password for illegal user test from
          ::ffff:12.34.56.78 port 33851 ssh2
        Jul 27 21:14:55 handel sshd[16487]:
          Received disconnect from ::ffff:12.34.56.78:11: Bye Bye
        Jul 27 21:14:55 handel sshd[16488]:
          Illegal user guest from ::ffff:12.34.56.78
        Jul 27 21:14:55 handel sshd[16488]:
          input_userauth_request: illegal user guest
        Jul 27 21:14:55 handel sshd[16488]:
          Address 12.34.56.78 [gecko.blechcough.org.]
          maps to gecko.blechcough.org, but this does not map back to the address
          - POSSIBLE BREAKIN ATTEMPT!
        Jul 27 21:14:55 handel sshd[16488]:
          Failed password for illegal user guest
          from ::ffff:12.34.56.78 port 33974 ssh2
        Jul 27 21:14:55 handel sshd[16488]:
          Received disconnect from ::ffff:12.34.56.78:11: Bye Bye
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Apr 16 23:54:44 2003
                                  vienna

EMERGENCY:
    NetworkDownEmergency
        Report if network is down

        pikt.org is down

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct  3 13:51:14 2002
                                  madrid

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct  3 13:26:15 madrid su: [ID 8191 auth.crit] 'su root' failed for cokaka
          on /dev/pts/6

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Oct  2 14:29:17 2002
                                  moscow

CRITICAL:
    ProcZombieTotalCountsCritical
        Report unusually high number of zombie and other processes

        Unusually high process count (458): 458 processes: 454 sleeping,
          2 zombie, 2 on cpu
        
             UID   PID  PPID  C    STIME TTY      TIME CMD
            root     0     0  0   Oct 01 ?        0:13 sched
            root     1     0  0   Oct 01 ?        0:38 /etc/init -
        ...
         rdupqah  4398  4397  0 14:27:51 ?        0:00 imapd
          mckym3  2501   291  0 12:41:12 ?        0:00 imapd
        mjcoltrn  4385  4382  0 14:27:50 ?        0:00 imapd
         pilson0 22436   291  0 14:22:17 ?        0:00 imapd
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Nov 12 22:18:17 2002
                                  kiev

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Nov 12 21:40:44 kiev bsd-gw[29187]: Error reading from connection:
          Bad file number
        Nov 12 21:44:04 kiev bsd-gw[29193]:
          Invalid protocol request (66): BBBXXXXXXX%
          .156u%300$n%.21u%301$nsecurity%302$n%.192un1ECf]fE'MECC1?A^u1FEMU/bin/sh
          .232u%300$n%.199u%301$nsecurity.i%302$n%.192un1]fE'MECC1?A^u1FEMU/bin/sh
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Oct  4 11:27:17 2002
                                 moscow

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct  4 11:16:30 moscow nfs:
          [ID 174370 kern.notice] NFS write error on host sun:
          No space left on device.
        Oct  4 11:16:36 moscow nfs:
          [ID 174370 kern.notice] NFS write error on host sun:
          No space left on device.
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Fri Oct  5 10:32:21 2002
                                 moscow

URGENT:
    MailQueueLengthyUrgent
        Report worrisomely long mail queue

        223 messages, 3885 lines in mail queue

URGENT:
    NewSystemStartupFileUrgent
        Report new system startup files

        new system startup file: -rwxr--r--   1 root     sys     1471 Jan  5  2000
                                   /etc/init.d/sendmail.011005

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sat Oct  6 14:31:05 2002
                                 cologne

CRITICAL:
    CksumDifferenceCritical
        Report checksum differences for certain critical system files

        /usr/bin/login checksums differ!
            auth:   63605c254a6a9928b8075963c951dbbf 29144 /usr/bin/login
            actual: 938a4ca4a6dd7b8665faa4cde6a83f73 29356 /usr/bin/login

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Oct  7 02:49:22 2002
                                 cologne

WARNING:
    FileCtimeChangeWarning
        Report ctime-changed files/dirs in file systems that should be stationary

        /usr/bin/login: ELF 32-bit MSB executable SPARC 1, dynamically linked
        -r-sr-xr-x   1 root     bin        29144 Dec 17 07:08 /usr/bin/login
        

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct 24 02:38:09 2002
                                 paris6

WARNING:
    DumpDatesProblemsWarning
        Report backup problems as revealed by dumpdates

        LAST RECORDED INCR BACKUP 16 DAYS OLD FOR /PUB/DISK40 (/DEV/DSK/C0T2D0S4)
        last recorded full backup 15 days old for /pub/disk41 (/dev/dsk/c0t2d0s5)
        LAST RECORDED INCR BACKUP 16 DAYS OLD FOR /PUB/DISK41 (/DEV/DSK/C0T2D0S5)

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Oct  7 08:15:49 2002
                                  prague

URGENT:
    RootCoreFileExistUrgent
        Deal with /core files

        /core:          ELF 32-bit MSB core file SPARC Version 1, from 'syslogd'
        -rw-------   1 root     other    1078508 Oct  7 03:10 /core
        moved /core file to /tmp, after the move:
        /dev/dsk/c0t0d0s0    1984230 1426880  497824    75%    /

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Apr 20 15:37:01 2005
                                  ottawa

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Apr 20 14:43:07 ottawa named[1374]: zone earlymusichicago.com/IN:
          refresh: failure trying master 192.168.5.16 []#53: timed out
        Apr 20 14:43:52 ottawa named[1374]: zone earlymusichicago.com/IN:
          refresh: failure trying master 192.168.5.16 []#53: timed out
        Apr 20 14:53:29 ottawa named[1374]: zone earlymusicchicago.org/IN:
          refresh: failure trying master 192.168.5.16 []#53: timed out
        Apr 20 14:54:14 ottawa named[1374]: zone earlymusicchicago.org/IN:
          refresh: failure trying master 192.168.5.16 []#53: timed out
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Oct  7 08:19:23 2002
                                 paris6

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct  7 00:25:04 paris6 automountd[175]:
          berlin2:/home server not responding:  RPC: Timed out

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct  8 10:23:20 2002
                                  moscow

EMERGENCY:
    PerUserProcessCountsEmergency
        Report unusually high counts of per-user procs.

         150  esmith5 imapd
        killed all esmith5 imapd processes

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Jan 31 10:21:07 2002
                                 nantes

URGENT:
    YPPasswdFileProblemsUrgent
        Report problems with NIS passwd file requiring immediate attention

        User boinha has no password!

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Oct 21 14:26:10 2002
                                  warsaw

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct 21 14:07:39 warsaw sshd[23866]: log: ROOT LOGIN as 'root'
          from server1.underworld.org

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Jul 29 12:30:16 2002
                                  moscow

CRITICAL:
    AuthLogChkCritical
        Report noteworthy authorization incidents

        Jul 29 11:56:20 moscow imapd[4534]: [ID 210418 auth.alert] Login
          SYSTEM BREAK-IN ATTEMPT
          user=+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++auth=
          ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
          host=murmansk.uppity.edu [123.111.130.35]

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Fri Nov  9 02:41:50 2002
                                  kiev

WARNING:
    SuLogScanWarning
        Scan the sulog for noteworthy su incidents

        SU-TO-ROOT FAILURE: SU 11/08 18:06 - pts/2 fjring-root
        SU-TO-ROOT FAILURE: SU 11/08 18:19 - pts/2 fjring-root

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct 22 08:17:48 2002
                                 paris6

URGENT:
    RunawayProcUrgent
        Report runaway processes, and cpu-hogs

            USER TT     PID  PPID        TIME COMMAND
        picoopal ?    17958     1  3-07:44:21 /opt/local/matlabr12/bin/sol2/matlab

URGENT:
    CPUUsageUrgent
        Report unusually high CPU percentage usage figures

        17958 picooalr   4 -15    0   50M 4464K sleep  79.7H 63.76% matlab

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct 22 02:31:58 2002
                                  padua

WARNING:
    PasswdFileNISProblemsWarning
        Report /etc/passwd file problems with NIS

        User trylim not found in NIS passwd
        User pywong doesn't match the NIS uname ywong for uid 673
        User dregurt doesn't match the NIS uname gregorio for uid 5134
        User andre6 doesn't match the NIS uname lrezez for uid 5135
        ...

WARNING:
    ShadowFileNISProblemsWarning
        Report /etc/shadow file problems with NIS

        User trylim not found in NIS passwd

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Oct 23 18:20:07 2002
                                  moscow

URGENT:
    LogUpdatesUrgent
        Check to see if various log files are being updated

        The last /var/log/popauth.watcher.log line is at least
          600 seconds out-of-date:
          Oct 23 17:28:27 mkorman authenticating relaying for 172.144.122.230

CRITICAL:
    FileUpdatesCritical
        Check to see if various files are being updated

        /var/log/popauth.watcher.log is outdated, is 3136 seconds old

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct 24 04:00:06 2002
                                  moscow

NOTICE:
    MailFileProblemsNotice
        Check several aspects of mail spool files

        claffm is 15 MB long
        cpoul is 13 MB long
        bad file ownership: -rw-rw---- 1 sibelius mail 2 Oct 11 16:39 sibelius09mO
        rfarmed is 12 MB long
        SCBASSO IS 22 MB LONG
        SJVEERMA IS 21 MB LONG
        VLI IS 22 MB LONG

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct 24 06:12:31 2002
                                 athens2

URGENT:
    NISNoBindingUrgent
        Report failures in NIS binding (as reported by ypwhich)

        Domain egbdf not bound on athens2.

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Oct 24 06:25:11 2002
                                 berlin2

CRITICAL:
    MetastatErrorsCritical
        Report DiskSuite metastat errors

        Submirror 0: d21, State: Resyncing
        Submirror 1: d22, State: Resyncing
        d21: Submirror of d20, State: Resyncing
        d22: Submirror of d20, State: Resyncing

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Oct 25 12:26:37 2002
                                 paris7

URGENT:
    SwapLowUrgent
        Report when swap use is high

        swap utilization is 96%:
        
        swapfile             dev  swaplo blocks   free
        /dev/dsk/c0t3d0s1   32,25      8 524712      0
        /swap1/swap1          -        8 3870712 179888
        /swap2/swap2          -        8 3870712 185520
        
        swap                 4236660 4026108  210552    96%    /tmp
        
        4025808 /tmp/SAS_worka00006EE4
        296     /tmp/ups_data
        8       /tmp/ps_data
        4       /tmp/screens
        
        contents of /tmp:
        
        total 624
        drwx------   2 pkfitro perf          336 Oct 25 08:23 SAS_worka00006EE4
        -rw-rw-r--   1 root     sys         6088 Oct 25 03:58 ps_data
        drwxr-xr-x   2 root     other         69 Oct 18 08:35 screens
        -rw-rw-r--   1 root     sys       302884 Oct 21 19:40 ups_data
        
        last pid:  3178;  load averages:  4.66,  4.70,  4.53    12:26:47
        40 processes:  36 sleeping, 2 running, 1 zombie, 1 on cpu
        
        Memory: 224M real, 4264K free, 3859M swap in use, 177M swap free
        
        
          PID USERNAME THR PRI NICE  SIZE   RES STATE   TIME    CPU COMMAND
        28388 pkfitro    4   0    0   23M   18M run   200:49 33.94% sas
         6981 jartigoo   1  50    0 1676K 1056K run    29.3H 21.60% top
         3178 root       1  50    0 1580K  996K cpu     0:01  4.52% top
         2755 root       1  58    0 6240K 3888K sleep   0:12  1.70% pikt
        ...
        
-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Oct 25 15:11:12 2002
                                  moscow

EMERGENCY:
    PerUserProcessCountsEmergency
        Report unusually high counts of per-user procs.

         329     root /usr/lib/sendmail

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Oct 25 23:54:44 2002
                                  vienna

URGENT:
    SysDownUrgent
        Report systems down or off the network

        paris6 is sick, possibly down, or off the network (rpc failure)

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Fri Oct 26 14:21:28 2002
                                  moscow

URGENT:
    NewSystemStartupFileUrgent
        Report new system startup files

        new startup file:
          -rwxr--r-- 1 root sys 5577 Aug 20 13:10 /etc/init.d/mdemon
        new startup file:
          -rwxr--r-- 1 root sys 7333 Aug 20 13:10 /etc/init.d/rdacct

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Fri Oct 26 14:26:01 2002
                                  moscow

EMERGENCY:
    PerUserProcessCountsEmergency
        Report unusually high counts of per-user procs.

         683     root /usr/lib/sendmail
        killed all root /usr/lib/sendmail processes
         317   nobody /opt/local/bin/python
        killed all nobody /opt/local/bin/python processes

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sun Oct 28 14:18:07 2002
                                trondheim2

URGENT:
    SysRebootUrgent
        Scan the 'last' command output for signs of recent system reboots

        reboot    system boot                   Sun Oct 28 13:55 

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct 29 07:22:00 2002
                                  moscow

DEBUG:
    ForwardFileExistDebug
        Check for existence of vital .forward files

        /home/ives/.forward not found!  recreating from backup

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct 29 10:51:04 2002
                                  moscow

URGENT:
    MessagesScanUrgent
        Scan the system messages log for urgent entries

        Oct 29 10:48:42 moscow scsi: [ID 107833 kern.warning] WARNING:
          /[email protected],4000/[email protected],1/[email protected],0 (sd100):
        Oct 29 10:48:42 moscow  corrupt label - wrong magic number

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Oct 29 07:19:22 2002
                                  moscow

URGENT:
    FileExistWarnUrgent
        Warn about, or possibly do something with, certain files.

        /etc/nologin found!

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Fri Nov  2 16:50:56 2002
                                  madrid

URGENT:
    DiskCapUrgent
        Report urgent filesystem full or near-full situations

        Filesystem /pub/comp_disk_1 on /dev/dsk/c1t2d0s0 is 100% full,
          37243 Kb left
        
        51115   /pub/comp_disk_1/fstrelk
        26975   /pub/comp_disk_1/kchelis
        26327   /pub/comp_disk_1/dturnky5
        ...
        
-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Nov  6 02:30:19 2002
                                trondheim2

WARNING:
    PasswdFileProblemsWarning
        Report /etc/passwd file problems

        User webown not in /etc/shadow file

WARNING:
    PasswdShadowCrosscheckWarning
        Report /etc/passwd entries not in /etc/shadow and vice-versa

        webown in /etc/passwd, not in /etc/shadow
        webown not found in NIS passwd

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Wed Nov 28 12:56:33 2002
                                  vienna

EMERGENCY:
    DirSystemNotExistEmergency
        Report system directory disappearances

        /opt/lib not found!
        /opt/libexec not found!
        /opt/man not found!
        /opt/sbin not found!

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sat Dec  1 10:25:17 2002
                                  nantes

URGENT:
    YPPasswdFileProblemsUrgent
        Report problems with NIS passwd file requiring immediate attention

        Users nextuid and nbarlow3 have duplicate uids: 52634

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Sat Dec  8 11:23:04 2002
                                  munich

URGENT:
    LpHungUrgent
        Report problems with printing

        For the c216ps queue, 10 entries, possibly jammed

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Mon Dec 17 19:18:23 2002
                                trondheim2

CRITICAL:
    ShadowFileProblemsUrgent
        Report urgent /etc/shadow file problems

        User mailsrv has NO PASSWORD!

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Dec 18 02:53:41 2002
                                  milan

WARNING:
    FileCtimeChangeWarning
        Report ctime-changed files/dirs in file systems that should be stationary

        /usr/bin/login: ELF 32-bit MSB executable SPARC 1, dynamically linked
        -r-sr-xr-x   1 root     bin        29144 Dec 17 07:08 /usr/bin/login
        

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Tue Feb 26 16:54:09 2002
                                 murmansk

EMERGENCY:
    LoadAverageEmergency
        Report perilously high system load averages

        4:54pm  up 13 min,  0 users,  load average: 131.92, 99.22, 50.60
        
        USER       PID %CPU %MEM   VSZ  RSS TTY STAT START TIME COMMAND
        root         1  1.4  0.0  1020  460 ?   S    16:40 0:11 init [2]
        ...
        www-data   269  1.9  0.6  8548 6140 ?   R    16:41 0:14 /usr/sbin/apache
        mysql      271  0.0  0.8 33764 8608 ?   S    16:41 0:00 /usr/sbin/mysqld
        ...

-------------------------------------------------------------------------------

                                PIKT ALERT
                         Thu Mar  7 10:07:07 2002
                                  milan

EMERGENCY:
    PerUserProcessCountsEmergency
        Report unusually high counts of per-user procs.

         158   webrun menu.cgi

-------------------------------------------------------------------------------

For more examples, see Samples.

 
Home | FAQ | News | Intro | Samples | Tutorial | Reference | Software
Developer's Notes | Licensing | Authors | Pikt-Users | Pikt-Workers | Related Projects | Site Index | Privacy Policy | Contact Us
Page best viewed at 1024x768 or greater.   Page last updated 2019-01-12.   This site is PIKT® powered.
Copyright © 1998-2019 Robert Osterlund. All rights reserved.
Home FAQ News Intro Samples Tutorial Reference Software
PIKT Logo
PIKT Page Title
View sample
script abort
macros