Sometimes, while working on a project, a simple templating system is necessary either for code or documentation generation. While you can get quite far with sed and awk, their nifty one-liners will eventually become monstrous, especially as your templates get more complex.

How do you deal with this growing complexity? For a lot of use-cases the simple m4 macro processor is exactly what you want. This command is probably installed on your machine without you even realizing it. It comes standard on Macs. On Linux it is part of the build-essential package and therefore probably installed on your distribution.

In this article we’ll briefly explore m4 and some of the interesting things it can do. If you’re looking for a dead simple templating system that is available on most Linux/BSD platforms then this might be just what you’re looking for.

m4 Basics

m4 is a macro processor, it takes simple text files, expands all of the macros defined in them, and produces a new text file. A simple m4 script that does nothing is the following:

Just print some text

If you run this through m4 you’ll see something like the following:

$ m4 example_1.m4
Just print some text

We can make this a bit more interesting my forcing m4 to evaluate something:

format(`Result is %d', eval(`2**15'))

This will evaluate to something like the following:

$ m4 example_2.m4
Result is 32768

Now if we want to define a variable and have it passed in via the command-line we can do this as follows:

Hello xNAME, welcome to m4!

We can then cause the variable NAME to be replaced by arbitrary text by specializing our invocation of m4:

$ m4 -D xNAME="Eric" example_3.m4
Hello Eric, welcome to m4!

This also allows us to pass in the output of evaluating other commands as inputs to templates, like the following:

$ m4 -D xNAME="$(date)" example_3.m4
Hello Tue Jul 17 08:33:23 PDT 2018, welcome to m4!

You can also inline define static variables to avoid duplication as follows:

define(`xTEST', `my value')dnl
The value of xTEST is "xTEST".

The output of this script is the following:

$ m4 example_4.m4
The value of my value is "my value".

In this example we use the define macro to create a new static value, though it can also be used to define much more complex values as well. You’ll also notice the value “dnl” at the end of the line – this tells m4 to ignore the newline that immediately follows. If we removed the “dnl” we’d get the following output:

$ m4 example_4.m4

The value of my value is "my value".

Notice the extra line before our output. This is because m4 does not replace the newlines after a “define” but instead simply echos them.

m4 For Templates

For most of the simplest templates, the above examples will likely be enough. That being said, there are times when you want a little more functionality from your templates. In what follows we’ll explain a few of the additional features of m4 using the following example:

divert(-1)
define(`xNL', '
')
define(`xPROSE', `But Iron - Cold Iron -')
define(`xDATE', translit(esyscmd(`date'), xNL))
divert(0)dnl
GENERATED AT: xDATE
GENERATED BY: xGENERATOR

GOLD is for the mistress - silver for the maid" -
Copper for the craftsman cunning at his trade! "
" Good! " said the Baron, sitting in his hall,
xPROSE is master of them all."

So he made rebellion 'gainst the King his liege,
Camped before his citadel and summoned it to siege.
" Nay! " said the cannoneer on the castle wall,
" xPROSE shall be master of you all! "

If you run this through m4 you’ll get the following output containing a fragment of Rudyard Kipling’s Cold Iron:

$ m4 -D xGENERATOR=escrivner test.m4
GENERATED AT: Tue Jul 17 08:50:18 PDT 2018
GENERATED BY: escrivner

GOLD is for the mistress - silver for the maid" -
Copper for the craftsman cunning at his trade! "
" Good! " said the Baron, sitting in his hall,
But Iron - Cold Iron - is master of them all."

So he made rebellion 'gainst the King his liege,
Camped before his citadel and summoned it to siege.
" Nay! " said the cannoneer on the castle wall,
" But Iron - Cold Iron - shall be master of you all! "

So there are a few things happening here that will be interesting to unpack.

Diversions

The first thing you might notice is the presence of the divert macro, this macro essentially does two things:

• When given a positive integer, it places all output generated by the following lines into the output buffer with that number.
• When given a negative integer, it throws away any output that would be generated by the lines that follow it. Until a diversion to a positive integer buffer is encountered.

When m4 reaches the bottom of your file, it will output all text diverted to positive number buffers in numerical buffer number order. To really get a sense for what this means run the following example:

divert(-1)
define(`xNAME', `Eric')
divert(1)dnl
This is the middle
divert(2)dnl
This is the end
divert(0)dnl
CREATED BY: xNAME
This is the beginning.

Here we first divert to a negative number buffer, effectively getting rid of all the output that might result from spurious newlines and the define statement. We then divert to buffer number 1 the text we actually want  in the middle section of the output. We then divert to buffer number 2 the text that we want at the end of the output. Finally, we divert to buffer number 0 the text that we want at the beginning of the output. Once m4 reaches the bottom it then outputs the contents of these buffers starting with buffer 0, then buffer 1, then buffer2.

This results in the following output:

$ m4 example_6.m4
CREATED BY: Eric
This is the beginning.
This is the middle
This is the end

As you can see, we can specify the output in any order that makes sense in our m4 file and guarantee that the output happens in the order that we expect. Removing the “divert(-1)” is also instructive, when removed from the above script you get the following output:

$ m4 example_6.m4

CREATED BY: Eric
This is the beginning.
This is the middle
This is the end

Notice the extra newline echoed following our “define” statement.

From this you should be able to see that the divert method is a nice way to do the following:

1. Avoid spurious output from macro evaluations.
2. Control the order in which the final output is displayed.

System Commands

The second major feature of the above script that you’ll notice is that we can ask m4 to grab the output of a system command and assign it to a variable, namely via the following line:

define(`xDATE', translit(esyscmd(`date'), xNL))

In this line we use the esyscmd (expand to output of system command) macro to capture the output of the date command. We then use the translit macro to remove the newline character that typically follows the output of date so that we don’t get unexpected newlines in our resulting output.

This could also be used to run a custom script and insert its output into a template. For example, let’s say we had the following ksh script in a file “test.sh”:

#!/usr/bin/env ksh
integer counter=0
until [[ ${counter} -gt 10 ]]
do
    echo ${counter}
    ((counter++))
done

We could then insert the output of this script into our m4 template as follows:

define(`xTEST', esyscmd(`ksh test.sh'))dnl
xTEST

Running this would yield the following output:

$ m4 example_7.m4
0
1
2
3
4
5
6
7
8
9
10

Conclusion

From the above I hope you’ve seen that m4 is a simple, powerful, and widely available tool for creating templates. I also think it is wildly underutilized given how simple and ubiquitous it is.

If you’re interested in learning more I highly recommend checking out the GNU m4 Manual.