Automatic tagging through an MCP server, kept usable

Before you start

You need three things in place, and the order matters because the last two shape what the first one is allowed to write.

• A product environment with the tagging add-on enabled. Automatic tagging is delivered by the AI Content Analysis add-on, which is a separate purchase from the Image & Video API plan and is billed on its own invoice line, not out of base-plan credits. The Free plan reaches only the free tier of each add-on, so check the add-on’s own quota before you point it at a library rather than at a test folder.
• An MCP client connected to the Analysis MCP server. The remote endpoint is https://analysis.mcp.cloudinary.com/sse and it authenticates over OAuth; the MCP server documentation lists all five servers and how each connects. What the Analysis server exposes and how it differs from the other four is covered in the Analysis MCP server for tagging, moderation and detection. If your project only needs tagging, you may not need the others — choosing which MCP servers a project actually needs works through that.
• A written list of the tags your application actually filters on. This is the allowlist in step 3, and it is the whole reason the output stays usable. If the product’s filter UI has twelve facets, those twelve (and their accepted values) are the list. If nobody can produce it, stop here: tagging a library against no taxonomy produces a tag cloud, not a filter.

Steps

1. Decide what the tags are for, and write down the threshold that follows from it.

   Automatic tagging does not return “the tags”. It returns candidate tags, each with a confidence score, and the candidate list for one image is routinely much longer than anything a person would type. Writing all of it back is the default failure: after a few thousand assets the tag list contains hundreds of near-synonyms and one-off guesses, and a search over it returns everything or nothing. The confidence threshold is the first cut. Its right value depends on what a wrong tag costs downstream — a tag that drives an editorial filter can afford to be permissive; a tag that decides which assets a legal-review workflow skips cannot. Pick the number deliberately and record it, because step 6 depends on knowing what it was.

2. Define the structured metadata fields the tags will land in.

   Do not write automation output into the free-form tags array that people also edit. Use structured metadata: fields defined once per product environment with a fixed type and, for the enumerated kinds, a fixed set of allowed values. You want at least three fields:

   Field	Type	Holds
   auto_tags	multi-select (enumerated)	tags that survived filtering, restricted to allowlist values
   auto_tag_source	string	model or add-on identifier and the threshold used
   auto_tagged_at	date	when this run wrote the field

   Keep human-authored subject tags in a separate field (or in the ordinary tag array) rather than merging them. The two need different trust levels: a person’s tag on an asset is a decision someone can be asked about; an automation’s tag is a probability that cleared a bar on a given day. Anything that later depends on tags — a rights check, a takedown filter, a “safe to publish” gate — has to be able to tell which kind it is reading. Mixing them in one field destroys that distinction permanently. Defining these fields is itself an MCP-driven operation; defining a structured metadata schema through an MCP server walks through it, and the Structured Metadata MCP server is the one that does the writing.

3. Load the allowlist into the enumerated field.

   Set the multi-select field’s allowed values to exactly the tags your taxonomy recognises — the list from the prerequisites. This is the second cut, and it does more work than the threshold. A confidence score tells you the model is sure; it does not tell you the tag is one your filter has a facet for. “Golden retriever” at 0.97 is a fine tag and useless if the product only filters on “animal”. Threshold plus allowlist together turn raw model output into something a filter can use: the threshold removes guesses, the allowlist removes truths nobody asked for. Because the field is enumerated, a write of an off-list value is rejected rather than silently accepted, so the constraint holds even when a future run is scripted by someone who never read this page.

4. Budget the run before starting it.

   Because the add-on bills separately from base credits, a backfill over an existing library is a budgeted operation, not a background one you kick off and forget. Count the assets in scope, check the add-on tier’s included quota and what an overage costs on your plan, and decide whether to tag everything, a category, or only assets uploaded after a date. Base-plan transformation and bandwidth usage ages out over a rolling 30-day window; that does not help here, because this spend lands on the add-on line. If the number is uncomfortable, narrow the scope rather than lowering the threshold to “make the run worth it” — a cheaper run that writes junk costs more later.

5. Run tagging through the Analysis MCP server and filter before writing.

   Have the agent request tagging asset by asset (or in whatever batch shape the server accepts) and, for each result, apply the threshold, then intersect with the allowlist, then write the survivors to auto_tags along with auto_tag_source and auto_tagged_at. The filtering happens in the agent’s step between reading the analysis result and calling the metadata write, not afterwards in a cleanup pass. Two reasons: the enumerated field will reject off-list values anyway, so writing first and pruning later just generates failed calls; and a cleanup pass that never runs is how unfiltered tags end up in production. Do not let the agent write to the human tag field or any field outside the three above — the write surface is the schema, and it should be that narrow. If you are choosing between doing this over an MCP server or against the REST API directly, an MCP server against a direct REST integration sets out the conditions under which each is the better fit; for a one-off backfill driven by an agent that already speaks MCP, the server is the shorter path.

6. Record the run date, and treat every re-run as a new run.

   Re-running tagging after the underlying model updates produces different tags for unchanged assets. That is not a bug to file; it is what a probabilistic classifier does when its weights change. It means a tag without a date is unreadable later — you cannot tell whether two assets differ because their content differs or because they were tagged six months apart. auto_tagged_at and auto_tag_source exist so that a later job can select “everything tagged before the model change” and re-tag exactly that set, and so that a person looking at one asset can see how old its machine tags are. When you re-run, overwrite the machine fields wholesale rather than appending; appending across model versions produces the tag cloud from step 1 by another route.

What done looks like

Every asset in scope carries a auto_tags value drawn only from the allowlist, a auto_tag_source naming the threshold and add-on used, and a auto_tagged_at date. Human-authored tags sit in a separate field and were not touched by the run. A filter over auto_tags returns a bounded, predictable set of values because the field cannot hold anything else. The add-on spend for the run matches the estimate from step 4, and a query on auto_tagged_at identifies exactly which assets a future model update will need to revisit.