Other Scripting Languages
[posted 2000/12/01]
I've written before that the Pikt script language is not the be-all and end-all of script languages. Far from it!
Case in point: The acctmgr suite of programs that we use to manage our thousands and thousands of user accounts. This is a fairly complex undertaking. Our acctmgr suite consists of nearly three dozen Perl and shell scripts scattered across different systems in our domain. Perl is admirably suited for the purpose.
That is not to say that PIKT has no part to play. What follows is an extended discussion of still another way you can leverage the power of PIKT to ease your programming with other script languages (in this case Perl).
In the PIKT distribution lib/configs_samples/files directory, we show how we can "publish" both our HTML Web documentation and our plain-text distribution docs from a common doc base, piktdoc_{intro,quickref}_files.cfg. For example, to install our Web pages, we just
# piktc -iv +F all +H warsaw
And to install the plain-text version to our development machine, we just
# piktc -iv +F all +H vienna
Gone are the days when we had to synchronize edits between two different documentation sources.
I have recently applied similar PIKT tricks to our acctmgr suite. Why was this necessary?
Whenever we needed to troubleshoot or extend our old acctmgr setup, we had to take down the acctmgr system and run tests affecting actual production files, for example, NIS passwd, our LDAP database, etc. This would cause an interruption in service, sometimes we would forget to back out our test data, and worse, sometimes we would forget to restart the acctmgr. Until a PIKT alert would signal us, that is. :-)
In the new-and-improved acctmgr, we now support two different code and data environments: production (prod) and testing (test). We have set up a parallel directory structure, with parallel sets of data and script code. We can safely extend and debug in the test environment, then when all is ready, install the extended and debugged code in our production environment with full confidence that it will work. We no longer have to take the production acctmgr manager down for testing. All around, things are working much nicer than before.
Now for the implementation details.
(In the examples that follow, pardon me if my Perl is not cutting-edge, or conformant to the latest accepted standards. Larry Wall himself has written, "You can program in Perl Baby-Talk... Any level of language proficiency is acceptable in Perl culture.")
We have created a macro #include file, invoked from within macros.cfg with the statement
# if acctmgr # include <macros/acctmgr_macros.cfg> # endif
"acctmgr" here is a hostgroup (defined in systems.cfg) including the systems where various pieces of the acctmgr script and data suite are located (e.g., our NIS master server). Here is the acctmgr stanza from our systems.cfg:
acctmgr // systems where acctmgr components (programs and
// data) reside
members acctmgrmaster munich moscow milan
Here is the acctmgr_macros.cfg file:
///////////////////////////////////////////////////////////////////////////////
//
// PIKT acctmgr macros configuration file
//
// $Header$
//
///////////////////////////////////////////////////////////////////////////////
#ifndef generic
acctmgrdir
# ifndef test
=usrlocal/etc/acctmgr
# elsedef
=usrlocal/etc/acctmgr/test
# endifdef
// make this /bin always after switchover to new scheme
//# ifndef test
//acctmgrbindir =acctmgrdir
//# elsedef
acctmgrbindir =acctmgrdir/bin
//# endifdef
acctmgrindir =acctmgrdir/.in
acctmgroutdir =acctmgrdir/.out
acctmgrtmpdir =acctmgrdir/tmp
# ifndef test
acctmgrntindir /var/acctmgr/.nt
acctmgrwebappindir /egbdfdata/username/prod
acctmgrnisdir /etc/NIS
acctmgrrembindir /etc/acctmgr/bin
# elsedef
acctmgrntindir /var/acctmgr/test/.nt
acctmgrwebappindir /egbdfdata/username/test
acctmgrnisdir /etc/NIS/test
acctmgrrembindir /etc/acctmgr/test/bin
# endifdef
acctmgrnoedit =acctmgrnisdir/noedit
acctmgrnomake =acctmgrnisdir/nomake
acctmgrskeldir /usr/local/etc/skel
acctmgrldapbindir /opt/local/ldap/bin
acctmgroperator systems\ alexander.borodin\
bedrich.smetana\
acctmgrtestoperator johannes.brahms\
acctmgrnismaster =nismaster
acctmgrmmserver moscow
acctmgrammaster vienna
acctmgrencode =acctmgrbindir/encode
#endifdef // generic
///////////////////////////////////////////////////////////////////////////////
In all the files that follow, we refer to these macros constantly. Observe the heavy use of macro defs referencing other macros.
Note the '# ifndef test ... # endifdef' wrappers. By using the '+D test' or '-D test' when installing files with piktc, our code uses the environment- appropriate directories (and other settings) automatically.
We've also created a short defines #include file, invoked from defines.cfg with
#include <defines/acctmgr_defines.cfg>
Here is that acctmgr_defines.cfg file:
///////////////////////////////////////////////////////////////////////////////
//
// PIKT acctmgr defines configuration file
//
// $Header$
//
///////////////////////////////////////////////////////////////////////////////
acctmgrmake FALSE // are we having acctmgr invoke NIS make as
// necessary via the command-line -m switch,
// or are we doing it independently (via, e.g.,
// a cron'ed nismake.pl)?
///////////////////////////////////////////////////////////////////////////////
Our programs.cfg file #include's the acctmgr_programs.cfg file, also the acctmgr_classparms_programs.cfg file, with this:
/////////////////////////////////////////////////////////////////////////////// #ifndef generic // acctmgr is a suite of a couple dozen Perl and shell scripts that we use // to handle our (very!) complicated account management # if acctmgr # include <programs/acctmgr_programs.cfg> # include <programs/acctmgr_classparms_programs.cfg> # endif #endifdef // generic ///////////////////////////////////////////////////////////////////////////////
The acctmgr_programs.cfg file starts this way:
///////////////////////////////////////////////////////////////////////////////
//
// PIKT acctmgr programs configuration file
//
// $Header$
//
///////////////////////////////////////////////////////////////////////////////
#ifndef generic // wrap the entire file this way
///////////////////////////////////////////////////////////////////////////////
//#define test
//#undefine test
//or use 'piktc -iv -D test' or 'piktc -iv +D test' at the command line
///////////////////////////////////////////////////////////////////////////////
#if acctmgr // to prevent multiple reinstalls on nonacctmgr systems
///////////////////////////////////////////////////////////////////////////////
#if acctmgrmaster
acctmgr.pl path "=acctmgrbindir/acctmgr.pl"
mode 750 uid 0 gid 1
#!=perl
#include <programs/acctmgr_common_programs.cfg>
die "Usage: acctmgr.pl [-h] [-d] [-t] [-D &]\n" if $HELP ;
$hostname = `hostname` ;
chop $hostname ;
$MAIL = "/usr/ucb/mail" ;
#ifdef test
$OPERATOR = "=acctmgrtestoperator" ;
#elsedef
if ($DEBUG || $TEST) {
$OPERATOR = "=acctmgrtestoperator" ;
} else {
$OPERATOR = "=acctmgroperator" ;
}
#endifdef
$PAUSE = 5 ; # seconds between NTREQ file checks
#ifdef test
$NAP = 5 ; # seconds between request sweeps
#elsedef
if ($TEST) {
$NAP = 5 ; # seconds between request sweeps
} else {
$NAP = 60 ;
}
#endifdef
#ifdef acctmgrmake
$GLOBALMAKE = 1 ; # do we pass the -m argument to the various
# acctmgr programs?
$MAKEOPT = " -m " ;
#elsedef
$GLOBALMAKE = 1 ;
$MAKEOPT = " " ;
#endifdef
= ("l", "n", "r", "c", "a", "ts", "s", "tm", "m", "p", "f") ;
for ($i=0; $i<; $i++) {
$classrank{$acctclasses[$i]} = $i ;
}
require "=acctmgrbindir/classparms.pl" ;
...
Note the macro reference "=acctmgrbindir" in
acctmgr.pl path "=acctmgrbindir/acctmgr.pl"
mode 750 uid 0 gid 1
and similar macro references that follow. Simply by changing a line or two in the acctmgr_macros.cfg file, we can shift around our prod or test environment quite easily.
By means of the '#ifdef test' and '#ifdef acctmgrmake' statements, we can make appropriate changes throughout our code just by setting or unsetting the test and/or acctmgrmake logical #define in defines.cfg (or acctmgr_defines.cfg).
We want to enforce a consistent style across all acctmgr programs, and we want to have all scripts refer to a common code base as much as possible. So, every acctmgr Perl program begins with the statement
#include <programs/acctmgr_common_programs.cfg>
Here is that acctmgr_common_programs.cfg #include file:
///////////////////////////////////////////////////////////////////////////////
//
// acctmgr_common_programs configuration include file
//
// $Header$
//
///////////////////////////////////////////////////////////////////////////////
use Getopt::Std ;
getopts('hdtmD') ;
$HELP = $opt_h ;
$DEBUG = $opt_d ;
$TEST = $opt_t ;
$MAKE = $opt_m ;
$DAEMON = $opt_D ;
sub docmd {
if ($TEST || $DEBUG) {
print "$_[0]\n" ;
}
if (! $DEBUG) {
system($_[0]) ;
}
}
sub print_mail {
if ($TEST || $DEBUG) {
print "$_[0]\n" ;
}
if (! $DEBUG) {
print MAIL "$_[0]\n" ;
}
}
# this may go unused in this particular script
sub getid {
local($key, $database, $ypmatch) = ;
local($match) ;
#ifndef acctmgrmake
if (! $ypmatch) {
$cmd = sprintf "=ssh %s -n grep '^$key:'
=acctmgrnisdir/$database",
"=nismaster" ;
open(GREP, "$cmd 2> /dev/null |") ;
$match = <GREP> ;
chomp $match ;
close(GREP) ;
} else {
#endifdef
open(YPMATCH, "=ypmatch $key $database 2> /dev/null |") ;
$match = <YPMATCH> ;
chomp $match ;
close(YPMATCH) ;
#ifndef acctmgrmake
}
#endifdef
(split(/:/, $match))[2] ;
}
///////////////////////////////////////////////////////////////////////////////
Every acctmgr Perl program should support a common set of command-line options--h, d, t--and sometimes m or D, as in:
die "Usage: acctmgr.pl [-h] [-d] [-t] [-D &]\n" if $HELP ;
The getopt code at the beginning of acctmgr_common_programs.cfg sets us up for that.
What's the different between DEBUG (-d) and TEST (-t) mode? And what if neither mode is in effect?
In DEBUG mode, our code just prints to stdout command strings, for example,
/bin/ssh munich -n /usr/local/etc/acctmgr/test/bin/add_yppasswd.pl tsfoobar g0r3b00sh 49308 65 Teresita_Foobar /home/tsfoobar /local/bin/mbash
In TEST mode, the command strings are both printed to stdout and executed.
In normal mode, with neither -d nor -t specified, the commands are just executed.
We don't send out feedback e-mail in either TEST or DEBUG mode.
These three different operational modes apply to both the test and the prod environments. That is, we can do both
# /opt/local/etc/acctmgr/bin/acctmgr.pl -d
and
# /opt/local/etc/acctmgr/test/bin/acctmgr.pl -d
Here is one of the many component scripts in the acctmgr suite:
///////////////////////////////////////////////////////////////////////////////
#if nismaster
add_autohome.pl path "=acctmgrbindir/add_autohome.pl"
mode 750 uid 0 gid 1
#!=perl
#include <programs/acctmgr_common_programs.cfg>
die "Usage: add_autohome.pl [-h] [-d] [-t] [-m] <acct> <home>\n"
if $HELP || ($#ARGV != 1) ;
$AHM = "=acctmgrnisdir/auto.home" ;
$BAK = "$AHM.bak" ;
$NAP = 60 ; # 60 seconds wait
while (1) {
last if ! -e "=acctmgrnoedit" ;
sleep $NAP ;
}
&docmd("echo add_autohome.pl > =acctmgrnoedit") ;
#ifdef acctmgrmake
if ($MAKE && ! -e "=acctmgrnomake") {
$DOMAKE = 1 ;
}
#endifdef
&docmd("=cp -p $AHM $BAK") ;
if ($TEST || $DEBUG) {
printf "%s\t\t%s/%s >> %s\n", $ARGV[0], $ARGV[1], $ARGV[0], $AHM ;
}
if (! $DEBUG) {
open(AHM, ">>$AHM") ;
printf AHM "%s\t\t%s/%s\n", $ARGV[0], $ARGV[1], $ARGV[0] ;
close(AHM) ;
}
#ifdef acctmgrmake
if ($DOMAKE == 1) {
&docmd("(cd /var/yp; make auto.home)") ;
}
#endifdef
&docmd("=rm =acctmgrnoedit 2>/dev/null") ;
#endif // nismaster
///////////////////////////////////////////////////////////////////////////////
Ordinarily, we wouldn't call this program directly from the command line. Rather, it is invoked by the master acctmgr.pl program.
I don't want to go into much more detail or explain any further the internal logic of the acctmgr system. I think from the examples already provided, you can begin to get a sense for the flexibility and ease in managing the setup.
To install the test version of the acctmgr suite (and all other programs), we would
# piktc -iv +D test +P all +H acctmgr
To install the production version, we would
# piktc -iv -D test +P all +H acctmgr
or more simply
# piktc -iv +P all +H acctmgr
leaving out the '-D test' because 'test FALSE' is the default in defines.cfg.
We could refresh just a single program file as in
# piktc -iv +D test +P acctmgr.pl +H acctmgrmaster
This might seem like over-engineering to you, but to us, the whole PIKT-ized acctmgr suite is a joy to work with. For the first time, acctmgr seems less like a loose aggregation of far-flung scripts and more like an integrated whole. Given, too, that most of the acctmgr code resides in a single PIKT .cfg file, programs/acctmgr_programs.cfg, it is really so easy to apply changes uniformly across all scripts. Just apply a vi 's1,$ //\/g' across the entire acctmgr_programs.cfg file, or in our case, we use the Emacs Query-replace function or a quicky macro to do the job.
There's no going back to the old way.
We are likely to include the revised, cleaned-up version of acctmgr in the next PIKT release (either a bug-fix 1.12.1 coming early next year else in the next major release). The version in 1.12.0's lib/configs_samples/programs/test was a rough work-in-progress.
The point here is not to suggest a better way of doing account management (or demonstrating my poor Perl technique) but rather to demonstrate as a whole how PIKT can help you program in other scripting languages.
Note that this discussion had nothing at all to do with systems monitoring or using the Pikt script language. This is just one more example of: The uses of PIKT are limited only by your imagination!
For more examples, see Developer's Notes.
