Report Recent Google Mediabot Visits

In this example, we report recent Google Adsense mediabot (ad server crawler) visits.  (The ad server agent identifies itself as "Mediapartners-Google".)  Why would you want to know this?  Maybe you have introduced a new page.  Or maybe you are employing section targeting or using other on-page tweaks to attract more relevant advertisements.  Any additions or changes you make won't take effect unless and until the ad server spider comes crawling.  If the spider doesn't come crawling, maybe you have a problem that needs fixing.

The GoogleMediabotVisit script might send an alert message like the following:

                                PIKT ALERT
                         Fri Sep  9 05:40:06 2005
                                 calgary

INFO:
    GoogleMediabotVisit
        Report recent Google mediabot visits

        devnotes/devnotes_1.7.x_developing_alarm_scripts.html: 09/Sep/2005:02:26:37
        devnotes/devnotes_1.9.x_system_files_backup.html: 09/Sep/2005:04:44:39
        samples/files_system_root_core_file_alarms.cfg.html: 09/Sep/2005:04:12:31
        samples/main_cf_files.cfg.html: 09/Sep/2005:03:51:55

In the GoogleMediabotVisit script following, note how it uses a Perl script, googlebot.pl, for its input.

GoogleMediabotVisit

        init
                status =piktstatus
                level =piktlevel
                task "Report recent Google mediabot visits"
                input proc "=httpd_cgibin_root/googlebot.pl pikt"
                seps "|"
                dat $page [1]
                dat $date [3]
                keys $page

        rule
                if $date ne %date
                        output mail "$page: $date"
                fi

A companion script, GooglebotVisit, reports recent visits by the Google googlebot (search engine spider).  (Note the similarity between the two scripts, the only significant difference being the change in the date field position.)

For more examples, see Samples.