Enterprise NetDevOps Architecture: Ecosystem, Pipelines, and Implementation with Nautobot, Batfish, and PLG

Network infrastructure management has fundamentally shifted from manual, error-prone CLI operations to software-driven Infrastructure as Code (IaC) principles. Modern network operations—collectively known as NetDevOps—require programmatic control, automated pre-flight testing, transactional deployment, and real-time operational feedback loops.

This article explores the entire NetDevOps landscape: from the broader tooling ecosystem to a detailed blueprint implementing Nautobot, GitLab, Nornir, Batfish, Ansible, and the Prometheus/Loki/Grafana (PLG) monitoring stack across Cisco, Juniper, and Linux environments.

1. The NetDevOps Tooling Ecosystem

Building a modern network automation pipeline requires integrating specialized tools across five key functional domains:

2. Comprehensive NetDevOps Architecture Diagram

The end-to-end framework operates on a strict declarative lifecycle: intent is modeled in the Source of Truth, controlled in Git, rendered and validated in CI, pushed during CD, and continuously monitored.

+---------------------------------------------------------------------------------------------------+
| 1. SOURCE OF TRUTH (SoT)                                                                          |
|   • Nautobot / NetBox (IPAM, VLANs, Cabling, Device Inventory)                                    |
|   • Structured Data Models (GraphQL / REST API)                                                   |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                                  v
+---------------------------------------------------------------------------------------------------+
| 2. VERSION CONTROL & CI/CD ORCHESTRATION                                                          |
|   • GitLab / GitHub (Git Repositories & Feature Branches)                                         |
|   • GitLab CI Runners (Merge Requests & Webhook Triggers)                                         |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                                  v
+---------------------------------------------------------------------------------------------------+
| 3. CI PIPELINE: RENDER & PRE-DEPLOYMENT VALIDATION                                                |
|   ├── A. Config Generation : Jinja2 Templates + Nornir Python Framework                           |
|   └── B. Pre-Flight Test   : Batfish Static Analysis (ACLs, Reachability, BGP Policy Verification)   |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                    [ Passed Pre-Flight Checks? ]
                                             /          \
                                          (Yes)         (No)
                                           /              \
                                          v                v
+---------------------------------------------------+    +------------------------------------------+
| 4. CD PIPELINE: DEPLOYMENT ENGINE                 |    | [ Pipeline Failed ]                      |
|   • Ansible Playbooks (Transaction Commits)       |    | • Notify Team (Slack / GitLab MR Comment)|
|   • Transport: SSH, NETCONF, RESTCONF, gNMI       |    +------------------------------------------+
+---------------------------------------------------+
                          |
                          v
+---------------------------------------------------------------------------------------------------+
| 5. INFRASTRUCTURE LAYER                                                                           |
|   +---------------------------+    +---------------------------+    +-------------------------+   |
|   | Cisco IOS-XE / NX-OS      |    | Juniper Junos Routers     |    | Linux Compute Servers   |   |
|   +---------------------------+    +---------------------------+    +-------------------------+   |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                                  v (Streaming Telemetry / Syslogs / Metrics)
+---------------------------------------------------------------------------------------------------+
| 6. OBSERVABILITY & CLOSED-LOOP FEEDBACK                                                           |
|   • Prometheus (gNMI / SNMP Metrics)  +  Loki (Syslog Aggregation)  -->  Grafana Dashboards       |
|   • Post-Validation: pyATS / Suzieq Operational State Verification                                |
+---------------------------------------------------------------------------------------------------+

3. Detailed Architectural Stages Explained

Stage 1: Intended State Definition

Engineers never modify production devices directly via CLI. Desired IP allocations, interface descriptions, VLANs, and BGP peering relationships are modeled as structured intent within Nautobot. Nautobot exposes this state programmatically via a GraphQL API.

Stage 2: Version Control & Merge Request Workflows

All Jinja2 configuration templates, Python scripts, and Ansible playbooks reside in a GitLab repository. Any proposed network change begins by creating a Git feature branch and opening a Merge Request (MR).

Stage 3: Continuous Integration (CI Rendering & Pre-Flight Validation)

Upon committing code or updating Nautobot data, GitLab CI triggers an ephemeral runner:

1. Rendering: Nornir executes parallel Python tasks that pull data from Nautobot’s GraphQL endpoint and populate Jinja2 templates to construct complete dry-run vendor configurations (Cisco IOS-XE/NX-OS, Juniper Junos, Linux Netplan).
2. Validation: Batfish ingests the rendered configurations offline, builds a virtual model of the network topology, and evaluates routing table consistency, dangling ACL entries, and end-to-end reachability. If Batfish finds security leaks or loops, the pipeline immediately halts.

Stage 4: Continuous Deployment (CD Execution)

Once peer code review passes and the MR is merged, GitLab CD invokes Ansible:

• Cisco Devices: Applied via cisco.ios / cisco.nxos collections using rollback checkpoints.
• Juniper Devices: Applied via juniper.device collection using atomic transaction loads (commit confirm).
• Linux Compute Servers: Applied via standard SSH to update system interfaces and daemon routing policies.

Stage 5: Observability & Closed-Loop Verification

After deployment, the Prometheus, Loki, and Grafana (PLG) stack monitors actual operational state:

• Prometheus scrapes interface metrics, link utilization, and BGP session health.
• Loki ingests log events (e.g., Cisco %LINK-3-UPDOWN, Juniper LINK_UPDOWN, Linux systemd messages).
• Grafana provides single-pane-of-glass dashboards, alerting the operations team if operational state diverges from intended state.

4. Implementation Code Reference

A. Rendering Configurations with Nornir & Jinja2 (render_configs.py)

from nornir import InitNornir
from nornir_jinja2.plugins.tasks.templating import template_file
from nornir_utils.plugins.tasks.files import write_file

def generate_device_configs(task):
    # Render Jinja2 template using parameters fetched from Nautobot GraphQL
    rendered = task.run(
        task=template_file,
        template=f"{task.host.platform}.j2",
        path="templates/"
    )
    # Save rendered config file for Batfish static analysis
    task.run(
        task=write_file,
        filename=f"rendered_configs/{task.host.name}.cfg",
        content=rendered.result
    )

if __name__ == "__main__":
    nr = InitNornir(config_file="nornir_config.yaml")
    results = nr.run(task=generate_device_configs)

B. Pre-Flight Validation with Batfish (validate_batfish.py)

from pybatfish.client.session import Session
import sys

bf = Session(host="batfish-server.internal")
bf.set_network("Production_DC")
bf.init_snapshot("rendered_configs/", overwrite=True)

# Test 1: Validate vendor syntax and parsing status
parsed_status = bf.q.fileParseStatus().answer().frame()
unparsed = parsed_status[parsed_status["Status"] != "PASSED"]
if not unparsed.empty:
    print(f"[X] Syntax / Parsing Failures Detected:\n{unparsed}")
    sys.exit(1)

# Test 2: Check for dangling references (e.g., referenced ACLs that don't exist)
unused = bf.q.unusedStructures().answer().frame()
if not unused.empty:
    print(f"[X] Unused or Dangling Structures Found:\n{unused}")
    sys.exit(1)

# Test 3: Verify path reachability across the virtual topology
reachability = bf.q.reachability().answer().frame()
print("[✓] Batfish Validation Successful: Zero critical policy violations.")

C. Multi-Vendor Deployment with Ansible (deploy_playbook.yml)

---
- name: Deploy Cisco IOS/NX-OS Infrastructure
  hosts: cisco_devices
  gather_facts: no
  tasks:
    - name: Push Rendered Config via IOS CLI/API
      cisco.ios.ios_config:
        src: "rendered_configs/{{ inventory_hostname }}.cfg"
        save_when: modified

- name: Deploy Juniper Junos Infrastructure
  hosts: juniper_devices
  gather_facts: no
  tasks:
    - name: Commit Junos Config with Auto-Rollback Guard
      juniper.device.config:
        config_file: "rendered_configs/{{ inventory_hostname }}.cfg"
        load: "override"
        comment: "GitLab Pipeline ID: {{ lookup('env', 'CI_PIPELINE_ID') }}"

- name: Deploy Linux Compute Host Network Settings
  hosts: linux_servers
  become: yes
  tasks:
    - name: Copy Netplan Interface Configuration
      ansible.builtin.copy:
        src: "rendered_configs/{{ inventory_hostname }}.cfg"
        dest: "/etc/netplan/01-netdevops.yaml"
        mode: '0644'
    - name: Apply Network Settings
      ansible.builtin.command: netplan apply

5. Key Advantages of this Architecture

1. Elimination of Configuration Drift: Because Nautobot serves as the single source of truth, any manual CLI changes are detected and overwritten during scheduled CI/CD runs.
2. Shift-Left Security & Testing: Running Batfish inside GitLab CI catches routing loops, security leaks, and invalid syntax prior to hitting production hardware.
3. Multi-Vendor Policy Uniformity: Jinja2 templating abstracts hardware differences, maintaining consistent security and operational policies across Cisco, Juniper, and Linux endpoints.
4. Transactional Safety & Rollback: Ansible utilizes Junos commit confirm and Cisco configuration checkpoints, ensuring automatic rollback if management reachability is compromised.
5. End-to-End Audit Trail & Observability: Every network alteration is tied to a GitLab commit and Merge Request, while Grafana, Prometheus, and Loki provide instant feedback on real-world health.

Conclusion

Combining Nautobot, GitLab, Nornir, Batfish, Ansible, and PLG Observability converts manual network tasks into a reliable, software-defined discipline. By implementing automated pre-flight testing and centralized intent management, network teams can deploy continuous changes rapidly without sacrificing stability or compliance.