Skip to main content

Command Palette

Search for a command to run...

Configuration Validation and Testing – Safe Network Changes in a Multi-Vendor Environment

Published
9 min read
Configuration Validation and Testing – Safe Network Changes in a Multi-Vendor Environment

Generating configurations with Jinja is powerful. Deploying them safely is engineering.

We explored how to use Jinja2 templates to standardize configurations across vendors. But generating configuration is only half the job.

The real question is:

How do you make sure your generated configuration won’t break production?

Before you push any configuration to production, you need to know if it's correct. One typo can bring down a network. One misconfigured route can create a routing loop. One wrong ACL can block critical traffic.

The golden rule of network automation: Never deploy untested configurations.

Why Configuration Validation Matters ?

Network outages can be extremely costly for enterprises. Most unplanned outages result from human errors during configuration changes rather than hardware failures. Implementing a disciplined pre- and post-change validation workflow significantly reduces this risk by automating the comparison of network states before and after each change window.

In a multi-vendor environment, syntax and behavior differ:

Vendor Interface Syntax Commit Model Validation Behavior
Cisco IOS-XR interface GigabitEthernet0/0/0/0 Commit-based Fails at commit
Junos set interfaces ge-0/0/0 Candidate + Commit Commit check available
Nokia SROS /configure router interface <name> port 1/1/c1/1 Candidate + Commit Validate check available
💡
Key insight: You cannot know whether a change went wrong unless you know exactly what the network looked like before the change was applied.

Real Case 1: Interface Description Template Gone Wrong

Scenario

You generate this Jinja template:

interface {{ interface_name }}
 description {{ description }}
 ip address {{ ip_address }} {{ mask }}

It works for:

  • Cisco IOS-XR

But your Junos device expects:

set interfaces ge-0/0/0 description "Uplink to Core"
set interfaces ge-0/0/0 unit 0 family inet address 10.1.1.1/24

Real Case 2: BGP Policy Mismatch Across Vendors

Scenario

You template BGP policies for 50 devices.

Your data model:

local_as: 65001
neighbor: 10.0.0.2
remote_as: 65002 

Problem

On IOS-XR:

router bgp 65001
 neighbor 10.0.0.2
  remote-as 65002

On Junos:

set protocols bgp group EBGP neighbor 10.0.0.2 peer-as 65002

But your automation mistakenly renders:

remote-as 65001

Result?

  • Session never comes up

  • No routing exchange

  • Silent failure

Types of Validation

  1. Template Validation

Before pushing configurations, validate:

  • YAML variables

  • Required fields

  • Data structure integrity

Example Python Validation

import yaml
from jinja2 import Template, TemplateError

errors = []

def validate(data_text, template_text):
    try:
        yaml.safe_load(data_text)
    except yaml.YAMLError as e:
        errors.append(f"YAML error: {e}")

    try:
        Template(template_text)
    except TemplateError as e:
        errors.append(f"Template error: {e}")

    return errors
  1. Syntax Validation

Each vendor supports some form of pre-check.

Nokia SROS Model Driven - Validate

validate

Junos – Commit Check

commit check 

Cisco IOS-XR – Commit Replace (Validate Before Apply)

commit replace
  1. Logical Validation

Syntax may pass. But logic may be wrong.

Examples:

  • Local AS equals Remote AS in eBGP

  • IP address overlaps existing subnet

  • Duplicate loopback

  • MTU mismatch across link

Example Logical Check in Python

if data["local_as"] == data["remote_as"]:
    raise ValueError("Local AS and Remote AS cannot be equal in eBGP")
  1. Pre- and Post-Change Checks

Before applying config:

  • Is interface already configured?

  • Does BGP session already exist?

  • Is policy already attached?

  • Is ISIS adjacency healthy?

This ensures:

One must avoid implementing new changes on an already broken network.

After applying config:

  • Check BGP state = Established

  • Check ISIS adjacency = Up

  • Check route present in RIB

  • Ping test

Safe Change Workflow in Multi-Vendor Networks

Here's the workflow:

          Inventory Data (YAML)
                    │
            Jinja Rendering
                    │
         Template + YAML Validation
                    │
          Vendor Syntax Check
                    │
          Logical Validation
                    │
          Pre-State Snapshot
                    │
              Deployment
                    │
          Post-State Verification
                    │
            Auto Rollback (if fail)

Config Builder Tool

About this Project

I began my network automation journey in 2018, seeking more efficient ways to perform my job. Initially, everything was manual—logging into devices individually, typing show commands, copying configurations, and hoping nothing would break during a 2 AM change window. Then, I discovered a course by David Bombal "Python Network Programming for Network Engineers", which transformed my approach, and I haven't looked back since.

The Config Builder Tool was one of my first complete project, allowing me to demonstrate to my colleagues the power of network automation. This project is designed to automate the generation of network configurations using Python and Jinja2. The tool has been tested with Nokia SROS routers running releases 22.10.R1 and 24.10.R5, operating in model-driven mode. It utilizes NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) to automate interactions with network devices via NETCONF.

NAPALM offers a configuration method called compare_config, which compares the candidate and running configurations on the SROS target. This is useful for pre-checking before deploying any configurations. This tool is interactive the user will be given a prompt to apply the configurations if required.

Prerequisites

Step 1

Enable MD-CLI on the Nokia SR OS network Device

A:R1# /configure system management-interface cli md-cli auto-config-save
A:R1# /configure system management-interface configuration-mode model-driven

Step 2

Enable NETCONF on the Nokia SR OS network Device

(gl)[configure system management-interface]
A:admin@R1#
    netconf {
        admin-state enable
        auto-config-save true
    }

Select YANG models to use on the Nokia SR OS network Device

(gl)[configure system management-interface]
A:admin@R1#
    yang-modules {
       nokia-modules false
       nokia-combined-modules true
    }

Select NETCONF user and permissions

(gl)[configure local-user system security user-params]
A:admin@R1#
 {
        user “admin" {
            password “admin"
            access {
                netconf true
            }
            console {
                member ["administrative"]
            }
        }
    }

Step 3

Before you begin, ensure you have the following installed:

  1. Python 3.x

  2. NAPALM library

  3. PyYAML and Jinja2 libraries

Repository Structure

├── vars.yml
├── builder.py
└── servicevpls.xml.j2

Usage

  1. File: vars.yml
vpls:
  - { name: demo_vpls1, id: 80, saps: [1/1/c3/2:15 , 1/1/c3/3:16] }
  - { name: demo_vpls2, id: 90, saps: [] }
  - { name: demo_vpls3, id: 100, saps: ["1/1/c3/4:17", "1/1/c4/2:18", "1/1/c4/2:19"] }
auto:
  start: 2000000000
  end: 2147483647
  1. File: servicevpls.xmls.j2
<configure xmlns="urn:nokia.com:sros:ns:yang:sr:conf">
        <service>
            <customer>
                <customer-name>demo</customer-name>
                <description>NETCONF L2VPN demo Nokia SR OS</description>
                <contact>DEVOps Team</contact>
            </customer>
            <md-auto-id>
                <service-id-range>
                    <start>{{ auto.start }}</start>
                    <end>{{ auto.end }}</end>
                </service-id-range>
                <customer-id-range>
                    <start>{{ auto.start }}</start>
                    <end>{{ auto.end }}</end>
                </customer-id-range>
            </md-auto-id>
{% for svc in vpls %}
          <vpls>
            <service-name>{{ svc.name }}</service-name>
            <customer>demo</customer>
            <admin-state>enable</admin-state>
{% for sap in svc.saps %}
            <sap>
              <sap-id>{{ sap }}</sap-id>
              <admin-state>enable</admin-state>
            </sap>
{% endfor %}
          </vpls>
{% endfor %}
        </service>
</configure>
  1. Main Script
import sys
import json
from jinja2 import Environment, FileSystemLoader
from napalm import get_network_driver

#Import YAML from PyYAML
import yaml

def configbuilder ():
    print('''

****************************************************
CONFIG BUILDER TOOL
****************************************************
    ''')

configbuilder()

if len(sys.argv) == 3:
    #Load data from YAML file into Python dictionary
    config = yaml.load(open(sys.argv[1]), Loader=yaml.FullLoader)

    #Load Jinja2 template
    env = Environment(loader = FileSystemLoader('./'), trim_blocks=True, lstrip_blocks=True)
    template = env.get_template(sys.argv[2])

    #Render template using data and print the output
    print('''

****************************************************
PRE-LOADED CONFIGS
****************************************************
    
    ''')
    print(template.render(config))
    
else:

    print("Usage: python3 builder.py <data_yml_file> <jinja_template_file>")
    print()
    print()
    sys.exit();
    
# Return template with data and store it into variable

response = template.render(config)

# Connect to the router and merge config

hostname = '172.20.20.13'
username = 'username'
password = 'password'

def connect ():
   """ This function is used to connect into the network device and prompt a user to commit or discard changes """
   driver = get_network_driver('sros')
   device = driver(hostname=hostname, username=username, password=password)
   device.open()
   device.load_merge_candidate(config=response)

   print()
   print()
   print('''

***************************************************************************
COMPARE AND LOAD CONFIGS
***************************************************************************
         ''')

   print("Results will be displayed when the is a DIFFERENCE between running and candidate configurations ")
   print()
   print("No Results will be displayed if running and candidate have the same configurations")
   print()
   print(device.compare_config())
   print()
   # for json format use the below option
   #print(device.compare_config('json_format':True))
     
   try:
       choice = input("\nWould you like to commit these changes? [y or N]: ")
   except NameError:
       choice = input("\nWould you like to commit these changes? [y or N]: ")
    
   if choice == "y":
       print()
       print(f"Committing the configurations on router {hostname} as {username}")
       print()
       device.commit_config()
   else:
       print()
       print(f"Discarding and not applying the configurations on router {hostname} as {username}")
       print()
       device.discard_config()
       
connect()
  1. Expected Output

Rendered configuration template

****************************************************
PRE-LOADED CONFIGS
****************************************************


<configure xmlns="urn:nokia.com:sros:ns:yang:sr:conf">
        <service>
            <customer>
                <customer-name>demo</customer-name>
                <description>NETCONF L2VPN demo Nokia SR OS</description>
                <contact>DEVOps Team</contact>
            </customer>
            <md-auto-id>
                <service-id-range>
                    <start>2000000000</start>
                    <end>2147483647</end>
                </service-id-range>
                <customer-id-range>
                    <start>2000000000</start>
                    <end>2147483647</end>
                </customer-id-range>
            </md-auto-id>
          <vpls>
            <service-name>demo_vpls1</service-name>
            <customer>demo</customer>
            <admin-state>enable</admin-state>
            <sap>
              <sap-id>1/1/c1/2:15</sap-id>
              <admin-state>enable</admin-state>
            </sap>
            <sap>
              <sap-id>1/1/c1/3:16</sap-id>
              <admin-state>enable</admin-state>
            </sap>
          </vpls>
          <vpls>
            <service-name>demo_vpls2</service-name>
            <customer>demo</customer>
            <admin-state>enable</admin-state>
          </vpls>
          <vpls>
            <service-name>demo_vpls3</service-name>
            <customer>demo</customer>
            <admin-state>enable</admin-state>
            <sap>
              <sap-id>1/1/c1/4:17</sap-id>
              <admin-state>enable</admin-state>
            </sap>
            <sap>
              <sap-id>1/1/c2/2:18</sap-id>
              <admin-state>enable</admin-state>
            </sap>
            <sap>
              <sap-id>1/1/c2/3:19</sap-id>
              <admin-state>enable</admin-state>
            </sap>
          </vpls>
        </service>
</configure>

Comparison candidate vs running configuration

💡
The is no L2VPN (VPLS) services on this router therefore you will see only the new addition
***************************************************************************
COMPARE AND LOAD CONFIGS

***************************************************************************

Results will be displayed when the is a DIFFERENCE between running and candidate configurations

No Results will be displayed if running and candidate have the same configurations

[
    "add",
    "configure.service",
    [
        [
            "customer",
            {
                "contact": "DEVOps Team",
                "customer-name": "demo",
                "description": "NETCONF L2VPN demo Nokia SR OS"
            }
        ],
        [
            "md-auto-id",
            {
                "customer-id-range": {
                    "end": "2147483647",
                    "start": "2000000000"
                },
                "service-id-range": {
                    "end": "2147483647",
                    "start": "2000000000"
                }
            }
        ],
        [
            "vpls",
            [
                {
                    "admin-state": "enable",
                    "customer": "demo",
                    "sap": [
                        {
                            "admin-state": "enable",
                            "sap-id": "1/1/c1/2:15"
                        },
                        {
                            "admin-state": "enable",
                            "sap-id": "1/1/c1/3:16"
                        }
                    ],
                    "service-name": "demo_vpls1"
                },
                {
                    "admin-state": "enable",
                    "customer": "demo",
                    "service-name": "demo_vpls2"
                },
                {
                    "admin-state": "enable",
                    "customer": "demo",
                    "sap": [
                        {
                            "admin-state": "enable",
                            "sap-id": "1/1/c1/4:17"
                        },
                        {
                            "admin-state": "enable",
                            "sap-id": "1/1/c2/2:18"
                        },
                        {
                            "admin-state": "enable",
                            "sap-id": "1/1/c2/3:19"
                        }
                    ],
                    "service-name": "demo_vpls3"
                }
            ]
        ]
    ]
]

Device commit prompt

Would you like to commit these changes? [y or N]:

Would you like to commit these changes? [y or N]: N

Discarding and not applying the configurations on router 172.20.20.13 as admin

Key Takeaways

  • Never deploy untested configurations

  • Use multiple validation layers

  • Always take pre-deployment backups

  • Capture pre/post state

  • Have rollback procedures ready

  • Use vendor-native commit checks

Download the Code

All templates and script: configbuildertool

Final Thoughts

If Jinja gives you power…

Validation gives you safety.