Content publishing is a good example of a workflow that looks simple from the outside (fetch a source, format it, post it) but accumulates real complexity once you account for deduplication, formatting rules, categorization, image handling, and failure recovery. It’s also one of the most naturally automatable workflows a content-driven business runs.
The work is repetitive, rules-based, and happens on a schedule. Those are exactly the conditions where n8n earns its place over manual, per-post effort. This guide covers how a publishing pipeline is usually shaped, where it gets harder than expected, and what separates a pipeline that runs for a year from one that quietly breaks in its second week.
The Basic Shape of a Publishing Pipeline
A typical n8n content pipeline starts with a trigger (a scheduled poll of an RSS feed, an incoming webhook, or a call to a source API) followed by a filtering step that checks whether an item is new, relevant, and not a duplicate of something already published. From there, the content gets transformed: reformatted, categorized, optionally summarized or rewritten, and matched with an image before it’s pushed to the CMS through its REST API.
Choosing a trigger
- Scheduled poll. Best for RSS feeds and APIs that do not push updates. Simple and predictable, but there is always a delay between something being published at the source and your pipeline seeing it.
- Webhook. Best when the source can notify you. Faster, but you need to secure the endpoint and handle the case where the sender retries and delivers the same item twice.
- API call. Useful when you need richer data than a feed provides, or need to filter at the source. Usually comes with authentication and rate limits to manage.
The stages, in order
- Trigger the run.
- Fetch items and validate that the response has the shape you expect.
- Filter for new, relevant, non-duplicate items.
- Transform: clean up formatting, categorize, optionally summarize or rewrite.
- Attach an image you have the right to use.
- Validate the final post against what the CMS expects.
- Publish through the CMS API.
- Log the outcome of every item.
Where This Gets Harder Than It Looks
Deduplication is the first real problem. The same story often appears from multiple sources with slightly different wording, and a naive “has this URL been seen” check misses that. Formatting is the second: a CMS expects specific field structures, and a pipeline that doesn’t validate before publishing will eventually push malformed posts live. Rate limits, both on the source APIs and the destination CMS, need explicit handling rather than assumption that every call succeeds.
Deduplication beyond URLs
Practical approaches layer several checks. Keep a record of source IDs or URLs you have already processed, so exact repeats are dropped cheaply. Then compare normalized titles (lower-cased, punctuation removed) against recent posts. For harder cases, compare text similarity between the new item and a recent window of published content. Each layer catches something the previous one missed. The trade-off to tune is how similar two items must be before one is dropped.
Validating before publishing
Before anything reaches the CMS, check that required fields exist and are sensible: a title within a reasonable length, a body that is not empty, a category that actually exists in the CMS, an image URL that resolves. Items that fail go to a review or error branch instead of being published. This one step prevents most of the embarrassing failures people associate with automated content.
Handling rate limits and errors
Treat a rejected request as information, not a crash. Space requests out, process in small batches, and retry with a delay on temporary errors. Separate those from permanent errors (bad data, missing permissions), which should be logged and skipped rather than retried forever.
Image handling
Images are easy to forget and easy to get wrong. A pipeline needs a clear rule for where images come from, confirmation that you have the right to use them, a fallback image when none is available, and a check that the image actually uploaded before the post goes live.
Categorization and Tagging Without a Human in the Loop
Rule-based categorization (keyword matching, source mapping) is reliable but rigid. An AI step in the pipeline (classifying an item into a category, generating tags, or flagging content that needs human review) adds flexibility, but should be treated like any AI step in a workflow: validated before it drives a publishing decision, with a fallback category when the model’s confidence is low.
In practice, many pipelines combine the two. Source mapping handles the obvious cases (everything from a technology feed goes to Tech). An AI classifier handles the ambiguous ones. The classifier’s answer is checked against the list of categories that actually exist, so it cannot invent a new one. Anything it is unsure about either goes to a default category or is held for review.
Tagging deserves the same discipline. Letting a model create new tags freely tends to produce a long list of tags with one post each, which helps nobody. A better approach is to give the model a fixed list of allowed tags and let it choose from that list.
Monitoring a Pipeline That Runs Without Anyone Watching
The pipelines that hold up long-term have logging on every run (what was fetched, what was filtered out and why, what got published, what failed) and an alert path when the failure rate on a source spikes or a destination API starts rejecting requests. A publishing pipeline that fails silently for a week is a worse outcome than one that never existed.
Useful alerts are specific: a source that returned nothing for longer than usual, a publish step that failed several times in a row, or a sudden jump in items being filtered out. Generic “something went wrong” alerts get ignored quickly.
Common Mistakes
- Deduplicating on URL alone.
- Publishing without validating fields first.
- Letting an AI step create categories or tags that do not exist.
- No plan for images, or using images without checking rights.
- No logging of why items were filtered out.
- Alerts that fire too often to be taken seriously, or not at all.
A Live Example
We’ve built exactly this kind of pipeline for a live news site. The full breakdown, including how deduplication and categorization are handled, is in our case study on building an automated news publishing pipeline with n8n for Pentagon Defence News.
How CodeixLab Approaches This
We begin with the destination: what fields the CMS needs, what the taxonomy is, and what a good post looks like. Then we work backwards to sources, deduplication rules, and categorization. Validation, error branches, and monitoring are scoped from the start rather than added after the first failure. This is a core part of our AI automation and integration work, and when a pipeline needs dedicated hosting or a self-managed n8n instance, our DevOps and cloud team handles that side.
Frequently asked questions
Can n8n publish to CMS platforms other than WordPress?
Yes. Any CMS with an API for creating content can be a destination. n8n’s HTTP Request node can call most REST APIs, and it has built-in nodes for a number of common platforms.
Should an AI step rewrite every article?
Not necessarily. Rewriting and summarizing are optional steps. Where they are used, the output should be validated like any other AI output before it is published.
How do I stop the same story being published twice?
Layer your checks: source ID or URL first, then normalized titles, then text similarity against recent posts for harder cases.
What happens if a source changes its format?
A well-built pipeline validates the response shape, skips items that do not match, and alerts you. If you want a pipeline like this for your own content, contact us.