Configuration Templates with Python and Mako

This post is part of the series “How to build your own Network Configuration Generator”. You find the overview about the entire series here. The last state of the code is available at the Network Configuration Generator GitHub repository.

The Network Configuration Generator has no real functionality until today. I’ll like to discuss the Mako Template Engine and the integration to the Network Configuration Generator in this post. The initial Use Case for the Web service includes the following requirements: dynamically detection of configuration variables, control structures within the templates and the bulk generation of configuration data.

I pushed today the entire Config Generator implementation to the Network Configuration Generator Github repository. The Web service works now as expected. It looks a little bit ugly, because all pages are plain HTML without any additional styling.

Mako based Config Generator

I already wrote about the config generation with Python and Jinja2 and from a conceptual perspective, this post discusses a similar topic. Jinja2 is a very popular template language within the Python ecosystem and is used for example in the Flask microframework. The configurations for network devices are in many cases just plaintext files, therefore these types of template engines are perfect for this Use Case.

So, why use the Mako Template Engine? The Mako Template Engine is from my point of view more intuitive compared to Jinja2. The template syntax is better readable (less curly brackets 😉) and offers some interesting features, e.g. the possibility to work with pure Python code inside the templates.

These are some reasons, why I decided to use the Mako Template Engine within the Network Configuration Generator.

Mako Templates 101

Now lets dive into some examples how to work with the Mako Template Engine. It’s installed as always using Python pip:

$ pip install mako

The Network Configuration Generator doesn’t store any templates or configuration result on the harddisk, therefore we can use the following code snippet to trigger the Mako Template Engine using a string variable:

from mako.template import Template

def render_config(template_string, template_variable_dict):
    return Template(self.template_string).render(**template_variable_dict)

The previous code example requires two parameters: template_string that contains the Mako Template and the template_variable_dict dictionary that contains the key-value pairs for the rendering process. Every time you get a configuration result, the config generator is triggered.

If you work with control structures, the result will contain a lot of whitespace and empty lines. For this reason, we extend our code example to drop any blank lines from the rendering result. This should improve the quality of the output and make it more readable for the user.

from mako.template import Template

def render_config(template_string, template_variable_dict):
    result = Template(self.template_string).render(**template_variable_dict)

    # remove empty lines
    lines = result.splitlines()
    clean_result = ""
    counter = 1
    for line in lines:
        if line != "":
            clean_result += line
            if len(lines) != counter:
                 clean_result += "\n"
        counter += 1

    return clean_result

Now lets have a look at the template syntax itself. I’ll discuss only the relevant syntax elements in this post, that I’ve used within the Network Configuration Generator. The entire syntax and structure of the Mako Template Engine is available in the official Mako documentation. To define a template with variables, you need to use the following syntax:

The is a valid Mako ${variable} definition.
This is also a valid Mako ${ variable } definition.

You can also use comments within the templates, either single line or multiline. Comments are used to annotate the templates and they are not part of the rendering result. Single line comments uses ## as the first characters on a line. Multi-line comments are enclosed with tags. Mako defines some of these tags that are similar to XML tags except that the first character of the tag name is always a % character.

The following example shows both types of comments.

## Single line comment
<%doc>
mutliline
comment
<%doc>

Mako also supports control structures like conditionals (if-else) and loops (for). A % character following a space introduces them. The % can appear anywhere in a line as long as no text should be processed within it. You can check for example if a variable is defined:

% if var:
var is defined.
% else:
var is not defined.
% endif

This should be enough to start working with the Mako Template Engine. It provides a lot more capabilities that I’ve discussed in this section. If you like to learn more about the engine, please take a look at the official Mako documentation.

Integration to the Network Configuration Generator

At the beginning of the Project, we defined the following requirements that are somehow associated to the template engine and the processing of the results within the Network Configuration Generator:

  • Upload a configuration template with any number of variables (variables are dynamically detected)
  • the template language must support if-then-else constructs

Within the Network Configuration Generator code, you’ll find a new (utility) class with the name MakoConfigGenerator in the app.util module. This class is in fact the entire Config Generator implementation. It is more or less the code from our Mako introduction that I’ve discussed earlier in this post and implements the dynamic detection of the template variables. The dynamic detection is based on the variable syntax that is used within Mako. Our data model within the Web service is already capable to work with a flexible amount of variables for each template. This utility class is used as the source to populate the database. The dynamic detection of the variables is currently accomplished using a regular expression. Every part of the template that matches the following regular expression is added to the database as a variable:

_variable_name_regex = r"(\$\{[ ]*(?P[a-zA-Z0-9_]+)[ ]*\})"

There are some limitations associated to this approach: As you can see, we only parse the use of the variables in the regular ${ var_name } notation (with or without spaces). If you have variables that are only used within control structures, you must add at least one statement using the expression above, to allow the parser to detect the variable. This variable can be used for example within a comment so that it doesn’t affect the template processing at all.

Another limitation is the naming convention of the variable: only the signs a-z, A-Z, 0-9 and the underline sign (_) are allowed and detected by the parser. If you use a filter on the variable (e.g. ${var|trim}), it won’t be discovered. I think that these limitations are acceptable for the first version of the Web service.

I added this implementation to the Flask views. If the content of the template has a valid syntax and format, the variables are parsed and stored in the database. After then, you can add a description to the variables. There is one additional limitation associated to the dynamic variable handling at time of this writing: If you need to change the configuration template, all Template Value Sets that are associated to it are removed.

Conclusion

After the last update, the Network Configuration Generator works as expected. **All features (generate configurations based on templates, single/ZIP download, bulk change of variables using CSV etc.) **except the Excel import are now implemented, but the entire site looks a little bit ugly. In the next post, we’ll take a look at the UIkit CSS framework and the overall refresh of the look and feel of the Web service.

That’s it for today, thanks for reading.

Links within this post