Setting Environment Variables
[posted 2004/09/27]
[Notes issued with the release of PIKT 1.18.0pre1, which introduced the new piktc +E option to set environment variables, also -E to unset environment variables.]
[WARNING: Advanced PIKT discussion follows.]
In the pikt-1.17.0pre* release notes, we mentioned that we manage hundreds of Web pages across several different Web sites. For that reason, our PIKT configuration is huge, amounting to hundreds of .cfg files in dozens of subdirectories. As the configuration has grown, piktc operations, especially piktc +/-F (files) operations, were taking longer and longer to the point of near unbearability.
piktc is selective about which parts of the PIKT configuration it preprocesses. piktc always looks at systems.cfg, and usually (but not always) at defines.cfg and macros.cfg. As for the other configuration files, it depends on the operation. For instance, for a 'piktc -i +A ... +H ...' operation, only alerts.cfg and alarms.cfg are additionally preprocessed. For a 'piktc -i +O ... +H ...' operation, only objects.cfg and its #include files are preprocessed; alerts.cfg, alarms.cfg, programs.cfg, and files.cfg are all bypassed.
So, when we manage our Web pages, using 'piktc -iv +F ... +H webserver', much time is saved by ignoring the irrelevant .cfg files (alerts.cfg, alarms.cfg, etc.). But what if we only need to install (or diff) a single Web page among hundreds of such pages? It can still take piktc a very long time to churn through files.cfg and its hundreds of #include files. There appears to be no way to direct piktc to look at only certain of the files.cfg #include files and ignore all the rest.
Ah, but there is a way!
In our top-level files.cfg, we might include our PIKT Web pages by means of
#include <files/pikt/doc_pikt_files.cfg>
Here is one way to do our doc_pikt_files.cfg:
... #include <files/pikt/doc_index_files.cfg> #include <files/pikt/doc_intro_files.cfg> #include <files/pikt/doc_licensing_files.cfg> ...
Now, it just so happens that there is no effective difference between doing this
#include <files/pikt/doc_index_files.cfg>
and instead doing this
#include [/bin/cat /pikt/lib/configs/files/pikt/doc_index_files.cfg]
Whether we directly #include a file or #include a process cat'ing the file makes no difference to piktc.
What if we now do this?
#include [test "$pages" = "index" &&
/bin/cat /pikt/lib/configs/files/pikt/doc_index_files.cfg]
If the environment variable $pages is set to "index", we #include the index.html #include file, doc_index_files.cfg. Otherwise, if $pages is set to something else or is not set at all, the test fails, and the command following the '&&' is not run--so we effectively #include nothing.
Suppose we redo our doc_pikt_files.cfg this way:
...
#include [test "$pages" = "index" &&
/bin/cat /pikt/lib/configs/files/pikt/doc_index_files.cfg]
#include [test "$pages" = "intro" &&
/bin/cat /pikt/lib/configs/files/pikt/doc_intro_files.cfg]
#include [test "$pages" = "licensing" &&
/bin/cat /pikt/lib/configs/files/pikt/doc_licensing_files.cfg]
...
Now, what if we do this?
# export pages=index; piktc -iv +F index.html +H webserver
Because $pages is set to "index", this line includes content
#include [test "$pages" = "index" &&
/bin/cat /pikt/lib/configs/files/pikt/doc_index_files.cfg]
while all the rest do not. By this trick, we have instructed piktc to #include only a certain file and bypass all the rest.
One problem is that, after the above operation, $pages is still set to "index". We would like to set $pages for the duration of the piktc command only, and to automatically unset it after the command.
The new piktc +/-E option, introduced in pikt-1.18.0pre1, will do just that. (Note: The old '-E' option, for listing all config elements, has been renamed to '-Z'. This list config elements options was seldom if ever used and was only added before for development purposes.)
So now we can do this:
# piktc -iv +E pages=index +F index.html +H webserver
which will tell piktc to look only at the doc_index_files.cfg file and ignore the #include files for all other pages.
Actually, our setup is more complicated than this. We manage several different websites, each with hundreds of pages scattered across many different categories, not to mention the usual set of system configuration files.
In fact, our files.cfg looks like this:
... #include [test $adm && /bin/cat /pikt/lib/configs/files/adm/adm_files.cfg] #include [test $pikt && /bin/cat /pikt/lib/configs/files/pikt/doc_pikt_files.cfg] #include [test $emc && /bin/cat /pikt/lib/configs/files/emc/doc_emc_files.cfg] ...
And, for example, our doc_pikt_files.cfg looks like this:
...
#include [(test "$pages" = "index" || test "$pages" = "all") &&
/bin/cat /pikt/lib/configs/files/pikt/doc_index_files.cfg]
#include [(test "$pages" = "intro" || test "$pages" = "all") &&
/bin/cat /pikt/lib/configs/files/pikt/doc_intro_files.cfg]
#include [(test "$pages" = "licensing" || test "$pages" = "all") &&
/bin/cat /pikt/lib/configs/files/pikt/doc_licensing_files.cfg]
...
So, if we only want to install the pikt index.html file, we would issue the command
# piktc -iv +E pikt pages=index +D pikt +F index.html +H webserver
Because the $pikt environment variable is set (by default to "1", when we don't specify a value), we effectively #include doc_pikt_files.cfg (but not adm_files.cfg, doc_emc_files.cfg, ...). And because $pages is set to "index", we effectively #include doc_index_files.cfg (and piktc ignores all the other files/pikt/doc_*_files.cfg files).
(The '+D pikt' option tells piktc to customize macros and .cfg file lines for pikt, for example, '#ifdef pikt <do this> #elsedef <do that> #endifdef. The +/-D option operates on the per-line level, unlike this proposed use for the +/-E option, which operates on the per-file level.)
It used to take many minutes for piktc to churn through hundreds of #include files (some of which themselves might #include files or process output, etc.). Now, by means of the techniques described above, we can preprocess and install the pikt index.html file in seconds.
Here is a command to install all of the pikt web pages:
# piktc -iv +E pikt pages=all +D pikt +F =piktdocs +H webserver
Here is a command to diff just the intro.3.html page:
# piktc -fv +E pikt pages=intro +D pikt +F intro.3.html +H webserver
This tells piktc to consider all the pikt intro-related .cfg files and ignore all the rest; and among all the considered intro files, to actually install only the intro.3.html file.
Here is a command to install all the emc business-related Web pages:
# piktc -iv +E emc pages=businesses +D emc +F =emcdocs_businesses +H webserver
Here is a command to diff all PIKT-managed files except for our pikt and emc Web pages:
# piktc -fv +E adm -F =piktdocs =emcdocs +H ...
With that last command, only the adm #include files are looked at; the hundreds of Web page .cfg files are ignored.
Just for the record:
# piktc ... +E foo ...
sets the $foo environment variable (to the default value "1").
# piktc ... +E foo=bar ...
sets $foo equal to "bar".
# piktc ... +E foo = "candy bar" ...
sets $foo to the multi-word string "candy bar". (In the case of multi-word strings, the quote marks, also the spaces around the '=' sign, are required.)
# piktc ... -E foo ...
unsets $foo.
You may unset and set environment variables in the same piktc command:
# piktc ... -E fee fie ... +E fo fum foo=bar ...
You may set environment variables in this way to #include (or not) files (i.e., processes cat'ing the file(s)), or to access environment variable values using the #echo command, or to do some other clever thing.
Again, and admittedly, this is very advanced stuff. If your PIKT configuration is much smaller and simpler, you would probably not need to use any of this. But if your PIKT configuration is massive and complicated like ours is, the new piktc '+/-E' option solves a significant scaling problem and potentially saves you time. *Much* time!
Note that for +/-A, +/-O and +/-P piktc operations, we don't need to mess with this +/-E setting environment variables, testing environment variables, and cat'ing #include files complexity. This is because our alerts.cfg, alarms.cfg, objects.cfg, and programs.cfg, and all of their #include files, are not unusually large or numerous. It's only our massive collection of files.cfg #include files and +/-F piktc operations that require this special treatment.
All the piktc options (E, D, and so on), the many different preprocessor directives (#if, #ifdef, #include, #set, #echo, and so on), the standard Unix/Linux tools (shell commands, scripts, and so on)--when joined together make for an extremely powerful combination.
Have fun playing with all of this!
For more examples, see Developer's Notes.
