Macromania User Documentation

Doug Neuhauser, UC Berkeley Seismological Laboratory Last revised: 1999.162

Tables of Contents

Overview of Macromania
Macromania command line syntax
Macromania operation
Details of Macromania operation.
File formats
Perl version of macromania

Overview

The purpose of macromania is to create a series of files that are dependent on input template files, key files, and keymacro files. The goal is to be able to dynamically configure the startup of the MSHEAR software and the aqcfg file for multiple stations in a network with the smallest amount of site-dependent information isolated in a site-specific key file.

Macromania is used to dynamically construct:

aqcfg file, which control data acquisition and data disposition.
network configuration files and network startup scripts.
process configuration files and startup scripts.

Syntax

macromania filelist keyfile [-u] [-d] [-r] [-p] [-f] [-k=keysize [-x-expansize]
where:

filelist is a file containing a list of files to created by macromania.
keyfile is an optional primary keyfile to be processed.
-u is an option to unlink the "macroimage" data module that contains a previously constructed keyword table.
-d is an option to dump the state to the file ./expansion.
-r is an option to force the recreation of the "macroimage" data module that contains the constructed keyword table.
-f is an option to run the program in "fussy" mode.
-k=keysize is an option to specify the of the key space (minimum 32000 bytes).
-x=expansize is an option to specify the size of the expansion space (minimum 100,000 bytes).

The first directive in the keyfile should be a keyword directive:

	grp	N

which is used by macromania to determine which set of input files to process. For each output file listed in the filelist, the input file is determined to be filename_%grp%, where %grp% is replaced by the value of the grp keyword. For example, If the filelist contains the filename

	aqcfg

and the keyfile contains the line

	grp	1

the file aqcfg will be created from them template file aqcfg_1. This allows the key file to select which group of input template files should be used to create the requested output file.

Operation

Macromania performs two distinct operations.

Keyword table construction:
Macromania first creates a keyword table consisting of keywords and keyword values which is constructed by processing the keyfile and the default keyword macro definition file keymacro. Keywords are single token unquoted strings, and are case insensitive. Keyword values may be either blank (empty) or non-blank strings. Keywords are case insensitive: the keywords joe, Joe, and JOE are all the same keyword.
Only the FIRST definition of the keyword is used -- subsequent definitions of the same keyword are ignored. This means that system-specific values can be defined early in the key file, and later definitions (often defined in an include file or defined in keymacros) can supply "default" values for keywords that have not been previously defined.
Macromania stores the keyword table in a memory module which can be used by subsequent infocations of macromania in order to save the time needed to re-create the keyword table. If the keymacro file is not specified, macromania will use a previously constructed keyword table in the memory module to process the template files.
Template file expansion:
Macromania reads the templatelist file which contains a list of files to be created by macromania. For each file in the list, it determines the name of the input template file and processes that input template file to create the specified output file. Processing a template file consists of processing template macro definitions and template macro calls, performing keyword replacement, processing conditional directives, and writing the resulting lines to the output file.

Details

Macromania constructs a keyword and keyvalue list by reading the key file, optional user-specified key include files, optional user-specified keymacro files, and the default keymacro file. It then processes a template file with this keyword and keyvalue table in order to create the specified output file.

Keyword table construction:

Macromania processes the key file and default keymacro file keymacro in the following manner to create the keyword table.

Key file pass 1:
The key file is opened and each line is read and processed in the following manner:
1. If the line is a comment, it is discarded.
2. If the line is a keyword definition, the keyword and value are inserted into the keyword table ONLY IF the keyword has not previously been defined. Even if a keyword was previously defined without a value (or with a value of the empty string), the keywork is still considered to be previously defined.
3. If the line is a $INCLUDE line, the include file is opened and processed at this time. It logically inserts the contents of the include file at this point in the key file.
4. If the line is a $MACROFILE line, the filename of the macrofile is added to a list of user-defined keymacro files that should be processed BEFORE the default keymacro file.
5. If the line is any other macro call (eg the line begins with a $), it is ignored at this time.
User keymacro files:
If any keymacro files were specified by $MACROFILE lines in the key file, these keymacro files are now processed in the order in which they were specified.
1. Lines of the form:
  [$macroname] formal_argument_list
  define the start of a keymacro. The format argument list is an optional comment that documents the expected arguments to the macro. If the keymacro name has not previously been defined in the keymacro table, the keymacro name and body of the keymacro are inserted into the keymacro table in memory. The body of the macro starts on the line following the macro definition line, and go to the next macro definition line or end of file. As with keywords, only the FIRST occurance of each keymacro is inserted into the keymacro table in memory, i.e. a keymacro is defined ONLY if that keymacro name was not previously defined. (?) This allows users to override the value of keymacros in the default keymacro file with their own version of the keymacro.
2. Any line that ocurrs outside of a keymacro definition (ie before the forst keymacro definition in the file) is ignored.
Default keymacro file:
The default keymacro file keymacro is now processed, using the same processing rules that were used for the user keymacro files.
Key file pass 2:
The key file is reopened and each line is read and processed in the following manner:
1. If the line is a comment, it is ignored.
2. If the line is a keyword definition, the keyword and value are inserted into the keyword table ONLY IF the keyword has not previously been defined. [Note that keys defined in the key file and include files were already processed in the first pass through the files.]
3. If the line is a $INCLUDE line, the include file is opened and processed at this time. It logically inserts the contents of the include file at this point in the key file.
4. If the line is a $MACROFILE line, it is ignored.
5. If the line is any other macro call (eg the line begins with a $), it is processed by:
  1. Substituting the formal arguments in the keymacro body with the actual arguments from the keymacro call line. When a keyword macro is called, the formal arguments $1$ through $Z$ in the body of the keymacro are replaced by the actual arguments in the keymacro call. The value of any formal argument not specifed in the keymacro call is set to the empty string. If an actual argument is a quoted string, the quotes are removed from the string when it is assigned to the corresponding formal argument.
  2. Recursively processing the body of the keymacro with the same key file pass 2 algorithm. The body of the keymacro call can contain keyword definitions and keywork macro calls.
  Note that lines within keymacros may have formal arguments embedded with keywords and the keyvalue strings. Since arguments may be embedded in the keyword, the actual arguments to keymacros may not contain references to other keys, because keyword replacement is not performed until later, and a keyname may NOT have a keyword reference in it.

Template file processing:

Macromania reads each target filename from the templatelist file, determines the name of the input filename by appending the string _%grp% to the filename (where %grp% is replaced by the value of the GRP keyword. The input file is then processed in the following manner in order to create the specified output file. The file may contain template macro directives (which begin with a $. All template macro directives are case insensitive, eg $MDEFINE, $Mdefine, and $mdefine are all equivalent. Since the template file contains directives for conditional processing, macromania maintain a stack of conditional processing states as well as the current conditional processing state. At startup, the initial processing state is TRUE, and the previous processing state on the stack is initialized to TRUE.

Macromania reads each line from the template file, and processes it in the following manner:

It processes the following template directive lines:
- $MEND
  If the line is a $MEND line and a template macro is currently being defined, it terminates the definition of the $MDEFINE template macro, and adds the template macro name and macro body to the template macro table. The line is then discarded.
- $ENDIF
  If the line is a $ENDIF line, it pops the conditional processing stack, setting the current processing state to the most recent processing state on the stack. The line is then discarded.
- $ELSE
  If the line is a $ELSE, it computes a new processing state whose value is computed by the expression:
  ((!current process state) && previous processing state on stack) The line is then discarded.
It substitutes keyword references in the line with the corresponding keyword value in a left to right order. A keyword reference is denoted by a keyword bracketed with the keyword escape character "%", e.g. the substring %comlink% is replaced by the value of the keyword comlink. Since the value of a keyword may contain additional keywords, the string must be reexamined after each keywork replacement. It is an error if a keyword is encountered that is not defined in the keywork table. Before each keyword is replaced, and after all keywords have been replaces, the line is examined to see if begins with the characters *#. Any line found that begins with this character sequence is discarded.
It evaluates all arithmetic expressions in the line of the form {@arithmetic_expression}@, and replaces the expression with the value of the expression. The following operators may be used within the expression:
- + (addition)
- - (subtraction)
- * (multiplication)
- / (division)
- & (logical AND)
- | (logical OR)
- ^ (exponentiation)
- ( ) (parenthetical expressions for explicit grouping)
Contants may be specified in either fixed point notation (eg 5 or 5.23) or exponential notation (eg 1.23E2, 1.23e2, 1.23D2, or 1.23d2).
It processes all other template directives lines of the format:
- $MDEFINE $template_macro
  This lines starts the definition of the specified template macro. The $MDEFINE directive may also contain additional comment tokens which may document the possible parameters that may be passed to the macro. The body of the macro consists of all following lines up to a $MEND line. The body of the macro (including conditional directives and keyword substitution) is NOT evaluated during macro definition. The macro body may contain formal arguments $1$ through $Z$ which will be replaced with the corresponding actual arguments when macro is called.
- $IF arith_expression
  The arithmetic expression is evaluated, and if the resulting value (rounded to an integer?) is non-zero, the condition is TRUE; otherwise the condition is FALSE. Since conditionals may appear within other conditional blocks, the new processing state is set to (conditional && current_state). The current state is pushed on to the processing stack, and the current processing state is set to the new processing state. If the new processing state is TRUE, all lines up to the matching $ELSE or $ENDIF will be processed (or ignored if the condition is FALSE).
- $IFEQ arith_expression
  The arithmetic expression is evaluated, and if the resulting value (rounded to an integer?) is zero, the condition is TRUE; otherwise the condition is FALSE. Since conditionals may appear within other conditional blocks, the new processing state is set to (conditional && current_state). The current state is pushed on to the processing stack, and the current processing state is set to the new processing state. If the new processing state is TRUE, all lines up to the matching $ELSE or $ENDIF will be processed (or ignored if the condition is FALSE).
- $IFDEF %keyword%
  By this time, the keyword has already been evaluated and replaced by its value. If the value is blank (or the empty string), the condition is FALSE; otherwise the condition is TRUE. Since conditionals may appear within other conditional blocks, the new processing state is set to (conditional && current_state). The current state is pushed on to the processing stack, and the current processing state is set to the new processing state. If the new processing state is TRUE, all lines up to the matching $ELSE or $ENDIF will be processed (or ignored if the condition is FALSE). Note that this directive would be more correctly name $IFNBLANK.
- $IFNDEF %keyword%
  By this time, the keyword has already been evaluated and replaced by its value. If the value is blank (or the empty string), the condition is TRUE; otherwise the condition is FALSE. Since conditionals may appear within other conditional blocks, the new processing state is set to (conditional && current_state). The current state is pushed on to the processing stack, and the current processing state is set to the new processing state. If the new processing state is TRUE, all lines up to the matching $ELSE or $ENDIF will be processed (or ignored if the condition is FALSE). Note that this directive would be more correctly name $IFBLANK. This line is then disarded.
- $ESCAPE new_escape_char
- $ESCAPE
  This directive defines a new escape character to be used for delimiting keywords. If no character is specified, the default keyword delimeter will be used.
- $INCLUDE filename If filename is not blank or the empty string, process the contents of the specified file at this point. If the filename is blank, ignore the directive.
- $macro argument_list
  If the line begins with a $ and, it is assumed to be a call to a previously defined template. When a template macro is called, the formal arguments $1$ through $Z$ in the body of the template macro are replaced by the actual arguments in the macro call. The value of any formal argument not specifed in the keymacro call is set to the empty string. If an actual argument is a quoted string, the quotes are removed from the string when it is assigned to the corresponding formal argument. The body of the macro is processed at this time.
If the line is not template macro call or directive and was not previously discarded, the resulting line is written to the output file.

File Formats

Key file:
A key file can contain the following types of lines:
1. keyword definitions of the form:
```
keyword 	value
```
  The keyword must begin in column 1. The keyword is a single token not containing whitespaces. The keyword and value are separated by whitespace. The value is either a single whitespace-delimited token, a quoted string (quoted by double quotes), or an empty string. If additional tokens are present on the line, they are considered to be comments and are ignored by macromania. Keywords are case insensitive, ie the keywords "hello", "Hello", and "HELLO" are considered to be the same keyword. The following are examples of valid keyword definition lines:
```
ip_net_1	192.108.32	Network number of network 1
ip_addr_1	208
ip_net_2
BANNER		"BDSN Station BOZO"	Station login banner
```
  If the string is quoted, the value of the string is taken to be the characters BETWEEN the quotes, ie the quotes are NOT considered part of the keyword value.
2. Comments:
  Any line beginning with the character "*", or any blank or empty line is considered to be a comment and is ignored by macromania.
3. keyword macro calls lines of the form:
```
$macroname	arglist
```
  The arglist is an optional argument list which is separated by whitespace from the macroname. The arguments in the arglist are separated from each other by a ";" (semicolon). An argument may be a single unquoted token, a quoted string or empty. Any additional tokens on the line are considered to be comments (?). Keyword macro names are case insensitive -- the keyword macro name "hello", "Hello", and "HELLO" are all identical.
  There are 2 special keyword macro calls that are processed by macromania:
```
$INCLUDE keyfile
$MACROFILE keymacrofile
```
  The $INCLUDE macro will include the contents of the specified keyfile at this point in the file. The $MACROFILE macro will include the contents of the specified keymacrofile at this point in the file.
  Examples of valid keyword macro calls are:
```
$Q4128
$DEST_IP	PRI;SEC;AUX
$DEST_IP	PRI;;AUX
```
Keymacro file:
A keymacro file contains key macro definitions. Each macro definition is of the form:
```
[$macroname] formal_argument_list
keyword	value
....
```
The formal_argument_list is currently considered interpreted as a comment, and is provided as an example of values might be used as arguments. A keymacro can accept up to 36 positional arguments which it refers to as $1$, $2$, ..., $9$, $A$, $B$, ..., $Z$. When the keymacro is called, the first argument is assigned to $1$, the second argument to $2$, and soforth. The body of the keymacro consists of keyword definition lines. Arguments can appear within a keyword or within a keyword value token. The body of the macro starts with the line following the keymacro definition line and continues to to the next keymacro definition line or the end of file. When a keyword macro is called, the formal arguments $1$ through $Z$ are replaced by the actual arguments in the keymacro call. The value of any formal argument not specifed in the keymacro call is set to the empty string. If an actual argument is a quoted string, the quotes are removed from the string when it is assigned to the corresponding formal argument. Within a body of a keymacro, you may refer to a substring of an actual argument by using
$arg:start:length$
where:
- arg is the name of the formal argument (1 through 9, A through Z).
- start is the index of the first character of the actual argument (starting at 1).
- length is the length of the desired substring.
Template file:
A template file may contain the following types of lines:
1. Template Macro definitions:
  A template macro definition consists of 2 or more line of the format
```
$MDEFINE template_macroname  optional_comments_on_argument_list
... body of macro ...
$MEND
```
  The template_macroname is the name of the template macro, and the body of the macro consists up to (but not including) the $MEND line. A template_macro can accept up to 36 positional arguments which it refers to as $1$, $2$, ..., $9$, $A$, $B$, ..., $Z$. When the template_macro is called, the first argument is assigned to $1$, the second argument to $2$, and soforth. The body of the template_macro consists of any valid template file line OTHER than $MDEFINE and $MEND, i.e. template_macros definitions can not be nested.
2. Conditional line, consisting of:
  - $IF expression
  - $IFEQ expression
  - $IFDEF %keyword%
  - $IFNDEF %keyword%
  - $ELSE
  - $ENDIF
  - $ESCAPE optional_escape_char
3. Lines, which may evaluate to comment lines that begin with *#, which are discarded as soon as they are recognized as comment lines.
4. Lines, which after evaluation, are written to the output file.

Perl version of macromania

As an aid to developing and debugging key files, keymacro files, and template files in a Unix environment, UC Berkeley developed a perl version of macromania. The perl version should conform to Quanterra's macromania version 23 with the following exceptions:

The perl version of macromania does not save key macros in a memory module between invocations. Therefore, a key file must be provided to macromania every time it is run.
The perl version of macromania uses dynamic arrays and hash tables, and does not require the user to define the size of the key and expansion tables.
The perl verions has different debugging outputs, and therefore supports different command line options. Type macromania -h for details.