Broken Link Checker (internal links)
The =broken_links_internal link checker is a Pikt script macro to check the validity of internal (same-site) URL references within a collection of HTML documents.
In the 'input proc' statement below, we use the Unix find command to find all .htm files within the web documents tree.
For every .htm or .html file, we invoke the find_urls.pl script to find all URLs within the file. (Since the documents are local, entirely accessible on disk, we avoid any complicated, recursive spidering schemes of our own directly accessible website.)
We bypass ftp: and other references. (Include these if you wish.) We also bypass all "foreign" (off-site) URLs, since we check these using a different Pikt script (see CheckBrokenLinksExternal).
In the http: pattern match, we determine the $site name and file $relpath (relative path). Next, we have to determine the file absolute path, according to several categories (for example, references to pages in the parent directory, cgi-bin files, relative path references, absolute path references, and so on). Note that we simply ignore the complication of named anchors. If the file is there, good enough (even if the named anchor within that file is missing or misnamed).
Finally, we do a simple '-e' file test to test for the file's existence. If not found, we report the broken URL (incorrect file reference).
#if emcwebsys
broken_links_internal(site, website, webdir)
init
status =piktstatus
level =piktlevel
task "Check for broken internal links"
input proc "=find =httpd_doc_root/(site)
-name \\*.htm -o -name \\*.html -print"
rule
set $filename = $inlin
rule // bypass /tmp and other pages
if $filename =~~ "(/tmp/)"
next
fi
rule
if #popen(URLS, "/usr/local/bin/find_urls.pl $filename", "r") != #err()
while #read(URLS) > 0
#ifdef debug
output $rdlin
#endifdef
// ignore ftp: and others
if $rdlin !~ "^http:\/\/"
next
fi
// ignore off-site URLs, since we check these elsewhere
if $rdlin =~~ "http.*:\/\/([^/]+)(.+)"
set $site = $1
set $relpath = $2
if $site !~~ "(website)"
cont
fi
else
set $relpath = $rdlin
fi
#ifdef debug
output $relpath
#endifdef
// set the absolute path
if $relpath =~~ "(\\.\\./)(.+)"
set $abspath = "(webdir)" . "/$2"
elsif $relpath =~~ "^/cgi-bin/"
set $abspath = "/var/www" . $relpath
if $abspath =~~ "^(/var/www/cgi-bin/yabb_emc/YaBB\\.cgi).*"
set $abspath = $1
fi
elsif $relpath =~~ "^/var/www"
set $abspath = $relpath
elsif $left($relpath,1) eq "/"
set $abspath = "(webdir)" . "$relpath"
else
set $abspath = "(webdir)" . "/$relpath"
fi
// ignore named anchors
if $abspath =~~ "^(.+\\.htm)#.+$"
|| $abspath =~~ "^(.+\\.html)#.+$"
set $abspath = $1
fi
#ifdef debug
output $abspath
#endifdef
if ! -e $abspath
output mail "in $filename: $abspath not found"
fi
endwhile
do #pclose(URLS)
else
output mail "Can't open '/usr/local/bin/find_urls.pl $filename'
for processing!"
quit
fi
#endif // emcwebsys
For both the PIKT and Early MusiChicago websites, we invoke the =broken_links_internal() macro from within alarms.cfg (or one of its #include files):
/////////////////////////////////////////////////////////////////////////////// BrokenLinksInternalPIKT =broken_links_internal(pikt, pikt\\\\.org, =httpd_doc_root) /////////////////////////////////////////////////////////////////////////////// BrokenLinksInternalEMC =broken_links_internal(emc, earlymusichicago\\\\.org, =httpd_doc_root/emc) ///////////////////////////////////////////////////////////////////////////////
Here is some example BrokenLinksInternalEMC script output:
in /var/www/html/emc/events_january_2004.htm:
/var/www/html/emc/soloists_instrumental_keyboard.htm not found
in /var/www/html/emc/events_november_2004.htm:
/var/www/html/emc/soloists_instrumental_strings.htm not found
...
For more examples, see Samples.
