AI incident response skill decay: the aviation fix that works

TL;DR

AI incident response skill decay is the gap left behind when an AI agent starts resolving most of your routine incidents: average MTTR drops, but the engineers who used to build judgment on those routine cases stop getting reps, and the incidents automation can’t handle get slower to resolve, not faster. Aviation hit this exact problem decades ago and fixed it with forced recurrent practice, not less of it — commercial pilots retrain on simulators on a fixed schedule no matter how rarely engines actually fail. On-call teams need the same discipline, or the next incident nobody has seen before takes longer than it should.

What is AI incident response skill decay?

AI incident response skill decay is the gradual loss of an engineer’s ability to diagnose and fix system failures, caused by an AI agent absorbing the routine incidents that used to be how that skill got built and maintained. It’s not a hypothetical. SRE and DevRel writer Sylvain Kalache — a former LinkedIn SRE and co-founder of Holberton School — laid the mechanism out clearly: as AI-assisted tools get better at resolving routine incidents, the humans nominally responsible for the system get fewer chances to actually touch it.

The name for this comes from a 1983 paper by cognitive scientist Lisanne Bainbridge, “Ironies of Automation”. Bainbridge’s argument, written about industrial process control four decades before LLM-based on-call agents existed, still lands exactly: automating the routine part of a job doesn’t remove the human from the loop, it just leaves them responsible for the abnormal cases while stripping away the practice that used to make them competent at handling anything.

Why AI incident autoresolution makes novel outages worse

Here’s the part that doesn’t show up on a dashboard. If an AI agent auto-resolves 80% of your incidents, your on-call engineers get 80% fewer chances per quarter to read a stack trace under pressure, correlate a metric spike with a deploy, or trace a cascading failure back to its root cause. Those reps don’t come back. They were how the skill got built in the first place.

Kalache’s prediction is directional, not a measured statistic, but it’s sharp: average MTTR keeps falling as the routine cases get automated, while resolution time for the remaining novel, complex incidents climbs, because the responders who’d normally handle them have lost touch with the system. The two metrics move in opposite directions on the same team, and only one of them shows up in a quarterly incident report.

What aviation already solved: recurrent training over rare failures

Commercial aviation ran into this exact shape of problem long before software did, and it didn’t solve it by hoping pilots would stay sharp on their own. Engine failures on modern airliners are genuinely rare — well under one per 100,000 flight hours — which means a working pilot could fly an entire career without a real one. Airlines don’t leave that to chance. Under 14 CFR 121.427, regulators require recurrent simulator training on a fixed interval — commonly every six to twelve months depending on the airline and aircraft type — specifically so pilots stay current on failures they may never see live.

The cost of skipping that discipline is on the public record. On February 4, 2015, TransAsia Airways Flight 235 lost its right engine to an auto-feather fault shortly after takeoff from Taipei — a known failure mode with a documented procedure. The crew misidentified which engine had failed, throttled back the working one, and then shut it down too. The aircraft, now without any functioning engine, clipped an overpass and crashed into the Keelung River 117 seconds after the first warning, killing 43 of the 58 people aboard. The official investigation found defects in the airline’s training program among the contributing causes — the exact gap recurrent simulator training exists to close.

That’s the whole lesson, and it transfers directly: rare, high-stakes failures need more forced practice as they get rarer, not less. AI incident response is making outages rarer for your team the same way better engineering made engine failures rarer for airlines. The fix aviation found isn’t “trust the automation and move on” — it’s recurrent, mandatory, simulator-grade practice on exactly the failures automation has made rare.

Manual on-call vs full AI autoresolution vs simulation-augmented on-call

Model	Routine-incident MTTR	Novel-incident MTTR	Skill retention	Cost
Manual on-call, no AI	Slow — every incident works a human	Moderate — engineers stay in practice by default	High, by accident	High engineer-hours on repetitive noise
Full AI autoresolution	Fastest on paper	Rises over time as reps disappear	Decays silently	Cheapest until a novel incident hits
Simulation-augmented on-call	Fast — AI still handles routine cases	Stays flat or improves	Maintained deliberately	AI savings minus a fixed training budget

The middle row is the trap. It’s the cheapest option quarter over quarter, right up until the incident the model can’t classify, and by then the team that used to be able to handle it has forgotten how.

How to build an on-call rotation that resists skill decay

You don’t have to give up the efficiency AI incident response buys you. You have to spend a fixed slice of it on staying sharp, deliberately, instead of letting the savings compound into an unpracticed team.

1. Set a human-handled floor. Route a fixed percentage of incidents — even ones the AI could resolve — to a human with the assist turned off. Pick the number your team can sustain, and don’t let it drift to zero because the dashboard looks good.

2. Run scheduled failure simulations. Borrow aviation’s cadence: recurring, calendared game days that inject a failure nobody has seen recently, using your real observability stack, not a slide deck.

3. Rotate ownership of AI-resolved runbooks back to humans periodically. If an agent has owned a class of incident for two quarters, have a human work the next one manually before automating it again. Muscle memory needs refreshing even for cases you’ve already automated.

4. Track a system-familiarity metric, not just MTTR. Time since an engineer last manually diagnosed each major subsystem is a leading indicator MTTR can’t show you — MTTR looks great right up until the quarter it doesn’t.

5. Require a human postmortem on every AI-resolved incident above a severity threshold. Reading the AI’s diagnosis and confirming it, in writing, is a cheaper rep than a live incident and still builds the mental model a human will need later.

When should you trust full incident automation?

Full automation is fine for the failure modes you’ve already characterized well enough to trust a runbook: restart-and-recover patterns, known noisy alerts, capacity blips with an established remediation. It’s a bad idea for anything novel by definition, because “novel” is exactly the category no runbook covers yet — and that’s the category your team’s practiced judgment exists to handle. Trust automation for the incidents you’ve stopped learning anything new from. Keep humans on everything else.

What breaks if AI handles 100% of your incidents?

The failure mode isn’t a single dramatic outage — it’s a slow-motion one. Each quarter the team’s baseline familiarity with the system erodes a little further, invisibly, because the dashboard that matters to leadership (aggregate MTTR) keeps improving. Then a genuinely novel incident arrives — a dependency nobody flagged, a cascading failure across services that were never tested together — and the people paged to fix it haven’t manually debugged anything in months. The resolution takes hours instead of the twenty minutes it would have taken a team that stayed in practice, and nobody can point to the exact day the skill went missing, because it didn’t go missing on any one day.

If your team is already fighting the version of this problem where AI agents make changes nobody signed off on, AI agent permissions and approval fatigue covers the other half of the human-in-the-loop tradeoff. And if you’re building the detection layer that decides what counts as an incident in the first place, distinguishing a traffic anomaly from a real outage is the baseline-method problem one layer upstream of everything in this post.

For teams still deciding how much of the response loop to hand an agent at all, agent-to-human delegation and the vibe-to-live production gap are the two posts to read next — and an enterprise security model for agentic AI is the governance layer that has to exist before any of this is safe to automate at scale. If skill-building is the part you’re optimizing for beyond incidents, developer productivity tools for senior engineers is the adjacent read.

FAQ

Does AI incident response actually cause skill decay?

Not the AI itself — the mechanism is what it removes. Every routine incident an AI agent resolves is a rep a human engineer doesn’t get, and reps are how on-call skill is built and kept. The decay shows up later, not on the dashboard that tracks routine MTTR, but in how long it takes a team to diagnose the rare incident nothing has seen before.

What is the “ironies of automation” problem?

It’s a term from Lisanne Bainbridge’s 1983 paper of the same name: automating the routine parts of a job leaves the human responsible for exactly the abnormal cases the automation can’t handle, while giving them far less practice at handling anything at all. The irony is that the better the automation gets, the less prepared the remaining human operator becomes for the moment they’re actually needed.

How often should engineers practice manual incident response?

Commercial pilots retrain on simulators on a fixed schedule regardless of how rarely engines actually fail, because currency has to be manufactured once real practice becomes too infrequent to rely on. An on-call team should apply the same logic: a standing cadence of game days and failure simulations, sized to the gap between how often AI resolves incidents and how often humans need to stay sharp, not to how few real incidents are left over.

What happened in the TransAsia Flight 235 crash?

In February 2015, an ATR72’s engine 2 propeller auto-feathered on climbout, triggering a routine warning. The crew misidentified which engine had failed and throttled back, then shut down engine 1 — the one still working. With both engines out, the aircraft crashed into Taipei’s Keelung River just 117 seconds after the first warning, killing 43 of the 58 people aboard.

Will AI incident automation make MTTR go down or up?

Both, split by incident type. Average MTTR falls because AI resolves the routine majority of incidents faster than any human rotation could. Resolution time for the remaining novel incidents rises, because the humans who used to build pattern-matching instinct on the routine cases no longer get those reps, and novel incidents are exactly where that instinct used to save time.

What’s the minimum viable fix for AI incident response skill decay?

Set a floor: a fixed percentage of incidents, or a scheduled game day, that must be worked by a human with the AI assist turned off, and track a system-familiarity metric alongside MTTR so the gap becomes visible before a real outage exposes it. It costs some of the efficiency gain AI bought you. That cost is the insurance premium against the incident automation can’t touch.

Sources

• Sylvain Kalache, “AI handles incidents, engineers lose touch with their systems”
• Wikipedia, “Ironies of Automation” (Lisanne Bainbridge, Automatica, 1983)
• Wikipedia, “TransAsia Airways Flight 235”