Small AI Errors Can Drift Into Important Decisions

The risky error is not always a conspicuous hallucination in an external report. A small, plausible drift can pass through an internal workpaper, become the next forecast’s baseline, and later support a decision no one intended it to support. The operational question is whether routine controls catch that drift as it travels.

A drift path

Hypothetical: A tool summarizes regional figures using a consistent rounding convention. Each output is individually immaterial, but a rounded value is copied into a working model and then into a forecast. A reviewer later assumes the baseline is the exact original figure. No single step looks obviously wrong, so the difference can pass through routine review.

Why routine review misses cumulative change

Reviewers often assess one output at a time. A small deviation may stay below a materiality threshold in each report while repeated reuse lets it become part of a later baseline. Automation increases the number of outputs and the speed of reuse, so controls need to compare recurring results against an independent source over time.

Controls for drift

Telling reviewers to “just be more careful” doesn’t work against an error type specifically built to not look wrong. What’s actually helped in practice:

Periodic reconciliation against an independent source, on a schedule, rather than only when something feels off. If a tool is producing recurring outputs, spot-check a sample against the original source data on a fixed cadence, specifically looking for small, consistent drift, beyond obvious errors.

Treating “this has been running fine for months” as a reason to check harder, not a reason to relax. The failure mode described here gets more dangerous the longer it goes unnoticed, because more downstream work builds on the quietly-wrong number. Comfort with a tool’s track record should trigger a deeper check on the assumptions underneath it, not a lighter one.

Naming an owner for drift, specifically, separate from the owner for obvious errors. Most control frameworks are built to catch things that look wrong. This failure mode requires someone whose job is explicitly to ask “has anything subtly changed” on things that still look fine.