The Automation Failure Nobody Catches: The Workflow That Runs Green and Does Nothing

The automation that crashes is not your problem. It sends an alert, somebody swears, somebody fixes it, and by lunchtime the thing is running again. Total damage: a morning.

Your problem is the one that runs every night, finishes in four seconds, reports success, and does nothing at all. Nobody investigates a green tick. So it keeps not working — for a week, a quarter, until somebody asks a question the automation was supposed to have been answering all along and discovers the answer has been empty since March.

I have shipped this bug. More than once, on the automation that runs the site you are reading. Every safeguard I describe below exists because something ran green and did nothing first, and I only found out later than I would like to admit.

What a silent failure actually is

A silent failure is a job that completes without error while producing no effect. The distinction that matters is between the job ran and the work happened — and almost every automation tool reports the first while you are reading it as the second.

Put plainly:

A crash tells you it failed. A silent failure tells you it succeeded. Only one of those needs finding.

It is worth being precise about why this class of bug is so much more expensive than a crash, because the instinct is to treat them as roughly equal problems.

That last row is the whole argument. A crash costs you the work that did not happen. A silent failure costs you the work that did not happen plus everything you decided while believing it had.

Why “success” in your automation tool does not mean what you think

Your automation tool reports on itself, not on your business. It is telling you that it executed the steps you defined without throwing an exception. That is a much smaller claim than “the thing you wanted is now true”, and the gap between those two statements is where every bug in this article lives.

The tool cannot know your intent. If you asked it to send an email to everyone in a list, and the list came back empty, it sent zero emails perfectly. That is not a bug from the tool’s point of view. It is a flawless execution of an instruction whose input disappeared.

This is the same reasoning error people make with tracking systems. A dashboard showing zero conversions is reporting one of two completely different situations — nobody converted, or nothing recorded — and it renders them identically. I have written about how affiliate tracking breaks quietly and how to catch it before payout day, and the failure shape is identical: a system that is confidently reporting a number it did not actually measure.

Once you see it in one place you see it everywhere. Zero is the most dangerous value in any automated system, because it is simultaneously a perfectly valid answer and the signature of a broken pipe.

The five shapes silent failure takes

Almost every instance of this bug I have found reduces to one of five shapes. Learning to recognise them is most of the work, because once you can name the shape the fix is usually obvious.

1. Empty input treated as valid. The trigger fires, the query returns nothing, and every downstream step processes the nothing correctly. Zero records fetched, zero records transformed, zero records written, status: success. This is the single most common shape, and it is the one built into the default behaviour of essentially every tool on the market.

2. The branch that just ends. You built a condition. The true path does the work and posts to Slack. The false path does… nothing, because you never built it. Every run since the data shape changed has taken the false path and terminated in silence. The workflow is not broken. It is doing exactly what you drew.

3. The vacuous check. A validation step that passes because it had nothing to validate. “Check that every record has a valid email address” returns true for an empty set — correctly, mathematically, uselessly. This one deserves its own section below, because it is the shape that fools careful people.

4. The swallowed error. Somebody wrapped a step in error handling to stop a nuisance alert, and the handler catches the failure, logs it somewhere nobody reads, and continues. The step that mattered is now permanently optional. This is nearly always introduced by a well-intentioned person fixing a different problem.

5. The job that stopped being scheduled. The most complete silence of all. The workflow did not fail; it did not run. A credential expired, a trigger got deauthorised, an account hit a limit, somebody paused it to test something in March. There is no failed run in the history because there is no run in the history — and every monitor that watches for failed runs sees a clean record.

Shape five is the one your monitoring is least likely to catch, because most monitoring is built to notice bad events rather than the absence of good ones.

The check that passes because it had nothing to check

A validation step that runs against an empty set will pass, and it will pass loudly and reassuringly. This is the failure I have personally shipped most often, and it is worse than having no check at all, because a check that cannot fail actively manufactures confidence.

I hit exactly this on this site’s own build. The build has a step that verifies every post references a cover image that exists on disk. It worked. Then a change upstream altered how the post list was assembled, the checker started receiving an empty list, and it began passing instantly on every build — reporting, quite truthfully, that all zero posts had valid covers.

The fix is one line, and it belongs in every check you write:

Assert the count before you assert the condition. “All records are valid” is only meaningful alongside “and there were N records, where N is greater than zero.”

Better still, assert a floor that reflects reality. This site has dozens of posts; a build that finds fewer than ten of them has not discovered a small blog, it has discovered a bug. A check written as “at least 10 posts, all with covers on disk” cannot pass vacuously. A check written as “all posts have covers” can, forever.

The same applies to the second half of the pipeline. A search index that builds successfully but indexes zero pages is a broken search box that reports itself as healthy. I know because I broke it: I added a Content-Security-Policy to the site, everything built green, every page rendered perfectly, and site search silently returned nothing for every query — the index needed a capability the new policy no longer permitted. No error, anywhere, at any stage. Now the build runs a real query against the finished index and refuses to ship unless it comes back with results. That check exists because the polite version did not catch it.

How to tell whether your automation is still working right now

Compare the automation’s output against a source it does not control. This is the only test that actually settles it, and it takes about ten minutes.

Do not read the run history. The run history is the system’s opinion of itself, and in every failure mode above the opinion is “fine”.

Instead, pick the thing the automation is supposed to produce and count it somewhere else:

• If it is supposed to create records, count the records created this week directly in the destination.
• If it is supposed to send messages, look in the outbox — or better, ask a recipient.
• If it is supposed to update a field, sort by last-modified and look at the top of the list.
• If it is supposed to move money or reconcile numbers, compare against the statement, not the report.

Then run one item through end to end yourself. Trigger it with real input, watch it land in every system that is supposed to receive it, and confirm it looks right at the destination rather than in the log. This is the same discipline that saves you on tracking, and it is tedious in exactly the way that things which work tend to be.

Do this on a schedule — monthly is enough for most things — and write down the number you found. The number is the point. “It looked fine” decays into nothing by next quarter; “412 rows on 12 August” is a baseline you can compare against.

How to design a job that fails loudly

Build the job so that doing nothing is an error state. Everything below is a variation on that single idea.

Report a number, not a status. The most valuable change you can make to any automated job is to have it end by saying what it did — “synced 240 records, skipped 3, updated 12” — rather than “completed”. A number is falsifiable at a glance. A green tick is not. When the notification one morning says “synced 0 records”, you have caught the bug in the two seconds it takes to read a line you already read every day.

Make zero explicit. Decide, for each job, whether zero is a legitimate outcome. Usually it is not: if the nightly import has never legitimately imported nothing, then importing nothing should raise. If zero genuinely is legitimate sometimes — a quiet weekend, a holiday — set the threshold at the level where you would actually be surprised, not at zero.

Add a heartbeat for the jobs that must not stop. For shape five, the only defence is inverting the check: instead of alerting when the job fails, alert when the job has not reported success within its expected window. The job pings on completion; a separate watcher raises the alarm when the ping does not arrive. This is the only monitoring that can detect a job that has ceased to exist, and it is the piece almost nobody builds.

Never let a step be optional by accident. If you add error handling, the handler must do something a human will see. Catching an error and continuing is a legitimate design choice for a nice-to-have step and a catastrophe for a load-bearing one. Write down which kind each step is; the answer changes what “handle the error” means.

Build the false branch. Every condition needs a defined path for both outcomes, even if that path is only “log that we took this route and why”. A branch that terminates in silence will eventually become the branch every run takes.

Give every automation one named owner. Not a team — a person. Unowned automations are where this bug lives, because everybody assumes somebody else would have noticed. When the answer to “who would find out if this stopped?” is a shrug, you already have your answer about how long it would take.

The alert nobody reads is the same bug wearing a different hat

An alert that fires constantly is functionally identical to no alert at all. If you fix everything above by making the system noisy, you have moved the silence from the machine into the person, and people are worse at this than machines.

The rule I hold to: an alert must be rare enough that its arrival is itself information. If a channel produces messages nobody opens, it is not monitoring — it is a log with a notification badge.

Practically, that means routing on consequence rather than on volume. A job producing zero records when it has never produced zero should interrupt someone. A job that took eleven seconds instead of nine should go in a log to be looked at never, unless somebody is investigating something. Most teams have these exactly backwards, and end up with a channel full of latency warnings and no notification at all for the import that stopped three weeks ago.

When the automation is an AI agent, add one more failure mode

An AI step introduces a failure that traditional automation cannot have: producing confident, well-formatted, entirely wrong output. It does not run green and do nothing. It runs green and does something plausible, which is worse, because a plausible wrong answer defeats a glance in a way that an empty result does not.

Everything above still applies, plus one thing: an AI step needs a check on the shape and substance of what it returned, not just that it returned. Did the summary mention the document, or did it summarise an empty input into a fluent paragraph about nothing? Does the extracted field match the format it must match? Does the output reference something that actually exists?

The tendency of these systems to produce fluent output regardless of whether they had anything to work with is a property worth understanding properly — it is the same underlying behaviour I described in why ChatGPT gives wrong answers, applied to a pipeline where no human reads the output before it acts. If you are putting a model inside a workflow that runs unattended, the validation step is not optional, and “the model said it was done” is not validation.

The safest structure I have found: let the model do the judgement, and let deterministic code do the checking. The model drafts, extracts or decides; a plain rule verifies the result is well-formed, non-empty, and within expected bounds before anything downstream acts on it. Judgement is what models are for. Verification is not.

Retrofitting this onto automations you already have

Start with the automations whose failure would cost you most, not the ones that are easiest to fix. You are triaging, and the honest first pass takes about half an hour.

1. List every automation you have running. Most people discover during this step that they cannot. That is itself a finding — an automation nobody can name is an automation nobody is watching.
2. For each one, ask: if this silently stopped today, when would I find out? Sort by the answer. Anything where the answer is “when a customer complains” or “I genuinely don’t know” goes to the top.
3. Check the top three against an external source, using the ten-minute method above. Assume they are broken and try to prove otherwise; it is a more productive posture than the reverse.
4. Add the count to the notification. Cheapest change on this list, biggest single improvement.
5. Add a heartbeat to anything that must not stop.
6. Write the owner’s name next to each one.

You will not get through all of them, and you do not need to. The distribution here is severe: a small number of automations carry almost all of the risk, and the rest can fail for a month without anyone caring. Spend your attention accordingly — which is the same logic that governs deciding what is worth automating in the first place, where the maintenance cost of a workflow, not the build cost, is what usually decides whether it was ever worth having.

The mistakes I see most

Trusting the run history. It is the system’s account of itself, and in every failure mode above that account reads “fine”. Check the destination, not the log.

Building the check but not the floor. “All records valid” passes on zero records. Assert the count first, every time, or you have built a check that cannot fail.

Adding error handling to silence an alert. The alert was the feature. If a step fails often enough to be annoying, fix the step or decide out loud that it is optional — do not quietly catch it and carry on.

Monitoring for failures only. Failure monitoring cannot see a job that stopped running. You need an expectation of success within a window, not just an absence of errors.

Automating a process nobody owns. Automation does not remove the need for an owner; it removes the daily reminder that there is one. The process used to fail visibly because a person was doing it. Now it fails invisibly because nobody is.

Testing only the happy path. The happy path is the one case you can be sure someone will notice is broken. Test the empty input, the malformed record, the expired credential — those are the paths that will run in production without telling you.

FAQs

How often should I check that my automations are actually working?

Monthly for anything load-bearing, and immediately after any change to the systems it touches. The second half matters more than the first: most silent failures start on the day something adjacent changed — a permission, a field name, a policy, an integration. If you changed something near an automation, verify the automation, even when the change looks unrelated. Mine broke because of a security header, which is about as unrelated as it gets.

Isn’t this what monitoring tools are for?

Partly, and they help, but most monitoring watches for error events and this bug does not produce one. A tool that alerts on failed runs will report perfect health for a job returning zero rows or one that has not run since March. What you need is an assertion about expected output — a count, a floor, a heartbeat window — and that has to come from you, because only you know what “working” means for your process.

What if zero is a legitimate result for my workflow sometimes?

Then set the threshold where the surprise actually starts, and give it a time window. “Zero today” might be normal; “zero every day this week” almost certainly is not. Alert on the streak rather than the instance. And write down which case is legitimate, because in six months neither you nor anyone else will remember whether an empty Tuesday is expected.

Should I fix these one at a time or redesign everything?

One at a time, in order of what failure would cost. A full redesign is the response that feels proportionate and never gets finished. Adding an output count to your three most important jobs this afternoon is worth more than a monitoring architecture you will start next quarter, and it will tell you within a week whether anything is currently broken.

How does this apply to simple marketing automations rather than technical pipelines?

Identically, and with less excuse, because the checks are easier. A welcome sequence that stopped sending, a lead form that stopped writing to the CRM, a tag that stopped applying — none of these announce themselves, and all of them are visible the moment you count records at the destination instead of trusting the platform’s dashboard. If you are building these, the marketing automation guide covers what to build first; this article is what to do so you find out when one of them quietly stops.

Key takeaways

• The dangerous automation bug is not the crash. It is the run that reports success and produces nothing.
• Your tool reports that it executed, not that the work happened. Those are different claims.
• Zero is the most dangerous value in any automated system: a valid answer and a broken pipe look identical.
• Silent failure takes five shapes — empty input, the unbuilt branch, the vacuous check, the swallowed error, and the job that stopped being scheduled.
• A check that runs against an empty set passes. Assert the count and a realistic floor before you assert the condition.
• Verify against a source the automation does not control. Never against its own run history.
• Have every job report a number rather than a status, and treat an unexpected zero as an error.
• Failure monitoring cannot detect a job that never ran. Use a heartbeat with an expected window.
• With an AI step, add a check on the substance of the output — models produce fluent text whether or not they had any input.
• Alerts must stay rare enough to be information. A channel nobody reads is the same silence, relocated.