Data teams watch for data drift. Few teams watch for automation drift. Your bots degrade quietly, and a weekly review catches it.
The drift you already know
Every data team knows data drift. A model learns from one picture of the world. Then the world changes.
Customer behavior shifts. A source system adds a field. A product line goes away. The model keeps scoring, but its answers get worse.
Nobody announces this. The model does not throw an error. It just becomes a little less right each week.
That is why good teams monitor models. They track inputs, outputs, and accuracy over time. They retrain when the numbers move.
The drift nobody watches
I run a team of bots. They draft, watch inboxes, run routines, and log their own work.
This week I found a second kind of drift. I call it automation drift.
Automation drift is what happens when bots and automations stop updating. Their processes and access fall behind the best possible setup. The bots keep running, but they run an older version of the job.
It looks a lot like data drift:
- The setup was right on the day you built it.
- The world around it changed.
- Nothing failed loudly.
- The output got quietly worse.
The difference is that nobody has a dashboard for it. Model drift has tools and a vocabulary. Automation drift mostly has silence.
What drift looked like this week
I did not find this in theory. I found it in my own setup, in one week.
Thin time log
The time log was too thin. Each bot is supposed to log its time after every job.
When I looked at the weekly log, I could not trust it. About 40% of the rows were estimates. Most quiet runs were not logged.
A log full of guesses is not a measurement. A log that skips quiet runs hides the work that costs the most to keep.
Silent watchers
Some watchers never spoke. A watcher checks for an event and reports when it sees one. One watcher was quiet on 10 of 10 runs.
A quiet watcher can mean two things. Either nothing happened, or the watcher cannot see what happens. From the outside, those two look the same.
Stale routines
Stale routines were still on. Some routines served a need that had passed. Nobody switched them off. They kept running on schedule and used time and attention.
Rules that did not spread
New rules did not reach every bot. I changed some rules during the week. Some bots got them. Other bots kept working from the old rules.
None of these problems broke anything that day. Each one made the system a little less true to what I wanted.
Why automations drift
I see three causes in what I found.
- The builder moves on. You build a bot, it works, and you start on the next one. The first bot keeps its first setup.
- Change does not spread by itself. A new rule lives where you wrote it. Each bot must pick it up, and some do not.
- Quiet looks like success. A routine that reports nothing seems healthy. A log with no gaps seems complete. Silence hides both drift and real calm.
This is the same trap as data drift. The system keeps producing output, so you assume it still works.
What I check for
You cannot fix drift you cannot see. So I turned each problem into a question.
- Is the work log measured, or mostly estimated?
- Does every run leave a row, including quiet runs?
- Which watchers have been quiet on every run?
- Which routines serve a need that has passed?
- Did the latest rule change reach every bot?
- Does each bot still have the access its current job needs?
The last question matters more than it looks. Access drifts too. A bot can keep access it no longer needs, or miss access a new job requires.
A watcher that is always quiet needs a test. I would feed it a known event and check that it reports. If it stays quiet, it is blind, not calm.
The fix: a weekly drift review
My answer was a weekly Friday drift review. Data teams retrain on a schedule. Bots need a review on a schedule too.
Here is how I would run it:
- Pull the week's work log and mark the estimated rows.
- List every watcher and routine with its run count and its report count.
- Test or retire any watcher that was quiet on every run.
- Switch off routines whose need has passed.
- Check that each new rule reached every bot.
- Write down what changed, so next Friday starts from a known setup.
Keep the review short. The goal is not a perfect system. The goal is to stop small drift before it adds up.
I expect the first few reviews to find the most. After that, the review should get faster each week.
What changes for leaders
Most leaders now have automations in their teams. Some are scripts. Some are agents. Many have no clear owner.
The question to ask is not only whether the bots work. Ask when someone last checked that they still match the best setup.
I expect three changes when a team reviews drift each week:
- Logs become numbers you can trust, not guesses.
- Quiet watchers either prove they work or get retired.
- New rules reach every bot, not only the ones you remember.
You already treat models as systems that age. Treat your automations the same way.
Check your drift this Friday
You've heard of data drift. Automation drift is the same problem in a new place.
Pick one Friday. Pull the log, list the quiet watchers, and check your rules. Then do it again next week.