Custom filters for a Jinja2 based Config Generator

This week, I’ll show you how to integrate custom filters into your Jinja2 based config generator. The use case from this post is based on the idea of a feature-centric and vendor independent parameter set for the configuration of network devices. We will see how custom filters can be helpful to generate configurations for multiple vendors based on a common parameter set. There will be two custom filters: the first filter will convert a prefix length to a dotted decimal representation. The second one will convert a string to a format, which is always useable as a VLAN name.

The python script that is used within this post is included in my python script examples repository on GitHub. This is based on the JSON example from my earlier post about configuration generation with python and Jinja2. I’ll highly recommend reading this post first before continuing.

Config generator for multiple vendors

In this example, we will create configurations for multiple vendors based on a common vendor independent parameter set. This parameter set should include anything that is required to configure a certain feature, for example IP interfaces. The following diagram illustrates this concept.

For today, I will keep it simple and create my individual parameter set based on a JSON model. There are other modeling languages out there like XML and YANG. One definition which is more related to network parameters is the Open Network Configuration specification from Google. The target of this specification is to create a simple, open and complete format to describe multiple network configurations (Wi-Fi, Ethernet, cellular etc.). It goes further than the idea of this post, but I recommend to have at leas a look at this when diving more into the network modeling topic.

Within the following example, we will create two configurations for IP interfaces for a Cisco and a Juniper device. The JSON model for Jinja2 will look similar to the following:

{
  "interfaces": [
    {
      "vlan_id": 10,
      "ip_address": "10.0.10.1",
      "prefix_length": 24,
      "name": "Data VLAN"
    }
  ]
}

Within the python script, we add a parameter that will be used to identify the vendor (Cisco IOS or Juniper), which should be used for the configuration. As described in the introduction, we need to convert two values within the templates:

  • The network name is converted into a valid format for a VLAN name (slug)
  • We need to convert the prefix_length to a dotted-decimal representation for Cisco IOS

As you can see in the parameter definition, the interface name can contain almost anything. Within the template we will create a slug from this string to match the formatting requirement. A slug is basically a lowercase string without blanks and special characters and is normally used within a URL. For simplicity, we will skip the implementation of a maximum length for the network name. Just keep in mind that the recommended length should not exceed 20 characters.

python script example

Adding a custom filter into your Jinja2 environment is quite simple. You just need to define the python function and add it to the filters dictionary on the Jinja2 environment. For the slugify_string function, we will use the library python-slugify that is now part of the requirements within the python example repository. You can easily install it using pip using the following command on your terminal/shell:

$ pip install python-slugify

The following snippet shows the relevant part of the python code example, which defines the custom filters for the Jinja2 environment:

def dotted_decimal(prefix_length):
    """
    converts the given prefix to a IPv4 dotted decimal representation
    :param prefix_length:
    :return:
    """
    try:
        ip = IPv4Network("0.0.0.0/" + str(prefix_length))
        return ip.netmask
    except Exception:
        return "[INVALID VALUE(" + str(prefix_length) + ")]"


def slugify_string(text):
    """
    convert the given string to a slug
    :param text:
    :return:
    """
    return slugify(text)


if __name__ == "__main__":
    # create Jinja2 template environment with the link to the current directory
    env = jinja2.Environment(loader=jinja2.FileSystemLoader(searchpath="."),
                             trim_blocks=True,
                             lstrip_blocks=True)
    # register custom filters on the jinja2 environment
    env.filters["dotted_decimal"] = dotted_decimal
    env.filters["slugify_string"] = slugify_string

Now you can work with the custom filters within the templates. If you like to see the script in action, just clone the python example, install the requirements to your python3 environment and start the python script config-generator-with-custom-filters.py. You can find this script in the directory config-generator-with-custom-filters.

If you like to run the python example in an isolated linux based environment, have a look at my blog post about the use of Vagrant to run the python examples.

After the execution of the script, you will find a _output directory in the template repository, which contains the two interface configuration scripts based on the parameter set defined in the parameters.json file.

Use custom filters in Jinja2 templates

You can use a filter within the Jinja2 template language by adding a function name after the variable name separated by a pipe (|) character. There are some build-in filters within Jinja2, for example to lowercase or uppercase a string. For our custom slugify_string filter, we can use the following syntax within the template file:

vlan {{ interface.vlan_id }}
 name {{ interface.name|slugify_string }}

As already mentioned, you can also chain filters within the templates. If you like to have the VLAN name in upper cases within you configuration, you can simply add the build-in filter upper to the previous interface.name statement, like the following example:

name {{ interface.name|slugify_string|upper }}

The use of the dotted_decimal filter is quite similar, but it expects a value between 0 and 32, otherwise the result will be INVALID VALUE. The template for this post contains also another new statement: the set function. Following the DRY principle (“don’t repreat yourself”), the template will contain the following statement to store the slug version of the VLAN name in a separate variable:

{% set vlan_name = interface.name|slugify_string %}

The new vlan_name variable can now be reused within the rest of the script and we will avoid mistakes because of an invalid filter statement when executing the template. Within the JUNOS configuration we will reuse this parameter at many points.

You can find the entire script in the file ip-interface-config.jinja2 in my python script examples repository within the directory config-generation-with-custom-filters.

Conclusion

Okay, compared to my first post about this topic, it was this time only a single use case and a little bit more than 80 lines of code. I think you learned some new ways to structure and extend your Jinja2 templates when generating configurations using this library. If you read my first post about this topic, you should now know what I meant with “it gives you more capabilities out of the box”. I get my head around multi vendor config generation for quite some time and I liked to share this idea with you.

That’s it for this week. Thanks for reading.