Error handling and retries in a media automation

Before you start

You need a flow that already runs end to end on the happy path — if you have not built one, start with building a first media automation flow from blocks and come back. You also need two decisions made before you touch a retry block: which downstream call in the flow can actually fail (an external API, an add-on, a webhook you own), and where a human will look when it does. The second is not optional. Every step below assumes a failure has somewhere to land.

The mechanics of adding, branching and wiring blocks are covered in Cloudinary’s guide to building a MediaFlows flow; this page is about what the branches should mean.

Steps

1. Sort the failures before you add a single retry.

   Go through each block that can fail and write down, for each error it can return, whether running the same input again could plausibly succeed. A rate limit, a timeout, a 5xx from a service you call: repeating those is worth something, because the condition that produced them is time-bound. A malformed asset, an unsupported format, a missing required field, a file over the plan’s size limit: those will fail identically every time, and a retry only burns credits and delays the moment somebody notices.

   Put that distinction in the flow itself, as a condition on the error branch — not in your head, and not in a note. The flow that treats every failure the same way makes one of two mistakes: it retries the terminal ones forever, or it gives up on the transient ones after one attempt. Both are avoidable with one branch: retryable errors go left, everything else goes straight to step 4.

2. Space the retries, and vary the spacing.

   A retry that fires immediately after the failure is asking the same question in the same instant and getting the same answer. Worse, when many runs fail at once — see the next step — retries without spacing all synchronise onto the same moment and reproduce the very burst that caused the failure. You have not added resilience; you have added a second wave.

   The pattern that works is exponential backoff with jitter: wait, then wait longer, then longer again, and add a random offset to each wait so that a hundred failed runs do not all come back at the same second. AWS’s Builders’ Library write-up on timeouts, retries and backoff with jitter is the reference for why — it shows how the naive version turns a brief overload into a sustained one, and why the randomisation matters as much as the growth. Cap the attempts. Three to five with growing gaps is a common shape; the point is that the count is finite and the last attempt hands off to the dead-letter path rather than looping.

   If the failing dependency is something an agent also calls, the same reasoning applies on that side; retry and backoff behaviour for agent-driven MCP calls covers it from the caller’s perspective.

3. Size the flow for the day it runs a thousand times, not the day it runs once.

   A flow triggered per upload runs at the rate uploads arrive. On a normal day that is a trickle. On the day someone runs a bulk ingestion from a spreadsheet it is thousands of runs in minutes, and a downstream failure that would have been one error becomes thousands of identical errors — every one of them then retrying on the schedule you set in step 2. This is why the jitter matters and why the attempt cap matters: without them the import does not just fail, it takes the dependency down for everything else that uses it.

   Two practical consequences. First, decide whether the per-upload trigger is the right trigger for bulk work at all; the trigger types available to a hosted media automation differ in exactly this respect, and a batch of a thousand assets is often better handled by a flow that runs once over the batch than by a thousand independent runs. Second, whichever trigger you keep, make sure the dead-letter path in the next step can absorb a thousand entries without becoming useless — a folder or a tag scales; a thousand emails does not.

4. Build the dead-letter path.

   When the retry cap is reached, or when step 1 classified the failure as terminal, the asset has to go somewhere that a person will look. Three forms, in rough order of how well they scale:

   • A tag — apply something like flow-failed and, if you can, a second tag naming the failure. Anyone can then search for the tag, and the set is trivially bulk-processed later.
   • A folder — move the asset to a holding folder. Same benefit, and it also keeps half-processed assets out of the paths that serve production.
   • A notification — for the cases where a human needs to know now, not next Monday. Cloudinary’s notifications documentation describes how event notifications reach an endpoint you own, which is the hook for pushing failures into whatever channel your team actually reads.

   Use a tag or folder for the record and a notification for the alert; one without the other either wakes people up for nothing or lets the queue grow unwatched. What all three have in common is the thing that matters: they turn a silent failure into a queue somebody can work through. A failure that is only in a run log is, for practical purposes, not recorded at all.

   Pair this with a scheduled reconciliation flow that periodically sweeps the dead-letter tag or folder — it catches whatever the per-event path dropped and gives the queue a second reader that does not depend on a person remembering.

5. Refuse to let a branch report success it did not earn.

   This is the failure that costs most, and it is the one the previous four steps make possible if you get it wrong. A flow with an error branch that catches the failure, does nothing useful, and continues — on to the next block, on to the end, marked complete — has not handled the error. It has swallowed it. The asset ends the run in a state nothing will revisit: not tagged, not moved, not reported, and not distinguishable from an asset that succeeded. Nobody searches for those, because nobody knows they exist.

   So audit every error branch for how it terminates. Each one should end in exactly one of: a retry that eventually reaches the dead-letter path, or the dead-letter path directly. If a branch rejoins the main flow after a failure, it must first do something that marks the asset as not-done — a tag at minimum. A flow that reports “succeeded” for a run whose alt text was never written, or whose moderation call was never answered, is worse than one that reports failure, because the failing one gets fixed.

   The same rule applies to human-in-the-loop branches. If a flow routes assets through approval inside a media automation, a rejected or timed-out approval is a terminal outcome for that asset and needs the same tag-or-folder landing, not a quiet fall-through.

6. Break it on purpose before it breaks on its own.

   Run the flow against one deliberately malformed asset and one asset that will trip a rate limit or timeout, and check that each lands where steps 1 to 5 say it should. Testing a no-code automation covers the general method; the specific things to confirm here are that the terminal case reaches the dead-letter path without retrying, that the transient case retries with visibly growing gaps and then dead-letters, and that in neither case does the run end marked as a success.

What done looks like

Every error branch in the flow ends in a retry with capped, jittered backoff or in a dead-letter tag, folder or notification — never in a silent rejoin. Terminal errors skip the retry entirely. A bulk import that hits a failing dependency produces one dead-letter queue and a spread-out retry pattern, not a synchronised burst. And a search on the failure tag returns exactly the assets that still need a human, because nothing that failed was ever reported as done.