Case Study

Eliminating Network Downtime with Golden Configuration Across 5,000+ Devices

How a major U.S. airline ended 47 hours of annual production downtime – and the $1M-per-hour business impact behind it – by standardizing configurations across 3,500 routers and 1,500 devices.

Challenge

Configuration drift across 3,500 routers and 1,500 devices spanning 100+ airports produced 47 hours of production downtime in one year – costing up to $1M an hour and creating a $47M opportunity.

Solution

Implemented Golden Configuration per location – by region, airport, and data center – with orchestrated workflows that deliver configuration changes end-to-end, from ticket creation through ticket closure.

Why Itential

One platform managed CLI and API devices across any vendor and OS, with flexible Golden Configuration templates and remediation in minutes – reusing existing Ansible and Python work instead of replacing it.

The Challenge

Network Transformation Hindered by Bad Configuration

A major U.S. domestic airline responsible for over 134 million passengers a year set out to transform its network. The environment was full of bad configurations – producing 47 hours of production downtime in a single year, grounding flights and blocking ticket sales. With business impact running up to $1M per hour, manual configuration challenges had become a $47M opportunity, and growing.

The root cause was time and staff limitations. The team had to work within short change windows, and was not staffed to keep up with the growing volume of work as the network continued to expand. The configuration backlog grew, security issues mounted, and network incidents traced back to configurations that were not in compliance. The network and security teams needed a new way to improve visibility and make change windows more efficient.

Three Forces Behind the Configuration Backlog

Each one compounded the others – and made it impossible for the existing operating model to catch up.

Short Change Windows, Growing Backlog

Network change had to happen inside narrow windows, but the volume of needed work kept expanding faster than the windows could accommodate. The backlog grew, and devices increasingly fell out of compliance.

Manual Work, Limited Staff

The team was not staffed to keep up with the growth of the network. Manual change processes consumed engineering time that should have gone to higher-value work, while configuration drift accumulated in the background.

Drift That Cost Real Money

Bad configurations led to 47 hours of production downtime in a single year – passengers unable to buy tickets, planes unable to take off – at up to $1M per hour of impact.

The configuration drift started to pile up more and more each year. We didn’t have the right tools to keep up, so without something new we would have fallen farther and farther behind.

Network Engineer

Major U.S. Airline

The team didn’t need more scripts. They needed an orchestration platform that could standardize configurations across 100+ airports and turn change windows into engineered, repeatable processes.

Why Itential

Why They Chose Itential

The airline needed to automate and standardize complex configurations across the entire network infrastructure – and improve security posture through audit compliance and automated remediation. Itential delivered six capabilities that anchored the decision.

One Platform for Multi-Vendor Configuration & Compliance

Six capabilities sat at the center of the decision – together giving the team a foundation for end-to-end configuration management across CLI and API devices, any vendor, any OS.

Explore the Platform

One Application for Every Device

A single platform managed device backups, inventory, and changes across CLI-based physical devices and API-based cloud networking services – any vendor, any OS – eliminating the need to context-switch between per-domain tools.

Flexible Golden Configuration Templates

Reusable Golden Configuration templates for managing any device and service across the network, with hierarchies that broke configuration down by region, airport, and data center.

Remediation in Minutes, Not Hours

Automated remediation of out-of-compliance devices in minutes instead of hours or days, closing the loop on configuration drift before it could grow into the next outage.

Reuse Existing Ansible & Python Work

The platform leveraged existing automation in Ansible and Python when building orchestrated workflows – no rip-and-replace of the work the team had already invested in.

Open Source Integration Marketplace

The Itential Marketplace offered hundreds of pre-built integrations and workflows – kickstarting oautomat without starting from zero.

Auto-Generated Integrations for In-House Systems

The ability to auto-generate integrations for in-house developed systems was essential for orchestrating processes end-to-end through the airline’s existing ecosystem.

The Solution

Golden Configuration Per Location, Orchestrated End-to-End

With Itential, the airline executed proactive change management across Cisco IOS routers throughout all 100+ airports – and built orchestrated workflows that delivered configuration changes from ticket creation through ticket closure as a single end-to-end outcome.

Golden Configuration Per Location

Standardized configurations implemented by region, airport, and data center – using Golden Configuration trees to manage 3,500 routers and 1,500 devices with a single hierarchical model.

Audit Compliance & Automated Remediation

Continuous compliance audits across the network plus automated remediation of out-of-compliance devices – improving security posture and closing drift before it grew.

Ticket-to-Ticket Orchestration

Orchestrated workflows that incorporated change management activities, delivering configuration changes as end-to-end outcomes from ticket creation through ticket closure.

Cisco IOS, Ansible, ServiceNow & Beyond

Proactive change management across Cisco IOS routers throughout 100+ airports, integrating with Ansible, ServiceNow, Bitbucket, and the airline’s existing toolchain.

If we setup two routers for this project and they’re not the same, can you imagine what the field would look like? Itential has made our jobs so much easier by implementing Golden Configuration.

Manager, Network Automation

Major U.S. Airline

The Outcome

Eliminated Network Outages Through Automation

The airline eliminated configuration-based outages across the entire country – and rebuilt the operating model behind them. The standardization that made it possible scales as the network continues to expand.

47 Hrs

Annual Downtime Eliminated

Production downtime caused by bad configuration – completely eliminated across the entire country.

83%

Reduction in Ticket Time & Effort

Time spent filling out and closing tickets was completely eliminated through Itential’s orchestration and integration capabilities.

5,000+

Devices Standardized

Golden Configuration trees standardized configurations across 3,500 routers and 1,500 devices.

100+

Airports Automated

Proactive change management across Cisco IOS routers throughout every airport in the network.

A $47M Opportunity, Solved at the Source

With downtime running up to $1M per hour, manual configuration challenges represented a $47M opportunity. Standardized configurations and automated remediation turned that opportunity into a scalable, audit-ready operating model.