Pentagon Defence News is a live news site covering World, Business, Tech, and Lifestyle stories, publishing on a rolling basis throughout the day. Most of its posts are created and published automatically. CodeixLab built and runs the n8n pipeline behind it, and the site is a real example of what a production content-automation workflow looks like once it’s past the demo stage.
This post walks through the problem the pipeline solves, how it is structured, where the real engineering effort went, and how the same pattern applies to other businesses. Where we explain techniques in more depth, we describe how those techniques generally work, so you can judge whether the approach fits your own content operation.
The Problem We Were Solving
A news site that wants continuous, multi-category coverage across World, Business, Tech, and Lifestyle either needs a large editorial team publishing around the clock, or a pipeline that can reliably source, structure, and publish content without a human touching every article. Pentagon Defence News needed the second option: a system that could run unattended and still produce a coherent, correctly categorized site.
“Unattended” is the important word. A workflow that needs someone to check it every hour is not really automation, it is a different kind of manual work. The goal was a pipeline whose normal state is running quietly, and whose abnormal states are visible quickly.
How The Pipeline Is Structured
The workflow polls source content on a schedule, checks each item against what’s already been published to avoid duplicates and near-duplicates, and routes it through category-matching logic that maps incoming stories to the site’s taxonomy (World, Business, Tech, Life Style, and their subcategories like Games and Travel). Each item is formatted to match the site’s expected post structure (title, body, featured image, category, and byline) before being pushed into WordPress through its REST API.
Published posts on the site are attributed to the automation account, which is a deliberate choice: it keeps automated and human-authored content clearly distinguishable in the WordPress admin.
Broken into stages, the shape looks like this:
- Schedule. A timed trigger starts each run, so publishing happens on a predictable rhythm rather than depending on someone pressing a button.
- Fetch. Source content is pulled in for the run.
- Deduplicate. Each item is compared against what is already on the site, catching both exact duplicates and near-duplicates.
- Categorize. Category-matching logic decides where the story belongs in the site’s taxonomy.
- Structure. The item is shaped into the post format WordPress expects.
- Publish. The post is created through the WordPress REST API under the automation account.
- Log and alert. The run records what happened and raises an alert if something is wrong.
Why separate stages matter
Keeping each stage distinct is what makes a pipeline maintainable. When a story ends up in the wrong category, you can look at the categorization step alone. When a post appears twice, you look at deduplication. In n8n, this separation is visible on the workflow canvas, which makes the logic readable to someone who did not build it.
Why the REST API rather than scraping the admin
The WordPress REST API is the supported, documented way to create and update posts from outside WordPress. It accepts structured fields, returns clear success or error responses, and works with application-level authentication. That makes it far more reliable than any approach that tries to imitate a person using the admin screens.
Where The Real Engineering Work Was
The visible part of a pipeline like this (fetch, format, publish) is the easy 20%. The work that actually makes it production-grade is everything around that: handling source feeds that go stale or change format without warning, catching near-duplicate stories (the same event covered with different headlines) that a simple URL check would miss, respecting rate limits on both the source and the WordPress API, and making sure a single malformed item doesn’t stall the entire run. Each of these needed explicit handling in the workflow rather than being left to fail silently.
Here is what each of those problems generally involves, and how they are typically handled in content pipelines.
Stale or changing source feeds
Sources do not announce when they change. A feed can stop updating, start returning an empty response, or rename a field that the pipeline relied on. The general defence is to validate the shape of what comes back before using it, and to treat “no new items for an unusually long time” as a signal worth checking, not as a quiet day.
Near-duplicate detection
A URL check only catches the exact same link. The same event is often covered by different outlets, or updated under a new headline, and each version arrives with a different URL. Common techniques for catching these include normalizing titles (lower-casing, stripping punctuation and filler words) and comparing them, measuring text similarity between the new item and recent posts, and limiting comparisons to a recent time window so the check stays fast. The trade-off is sensitivity: too strict and duplicates slip through, too loose and genuinely different stories get dropped.
Rate limits on both ends
Both the source and the destination can refuse requests if they arrive too quickly. Typical handling includes spacing requests out, processing items in small batches, and retrying with a delay when a request is rejected for rate reasons, rather than treating that as a permanent failure.
Isolating malformed items
If one item is missing a title or has a broken image, it should be skipped and logged, not allowed to crash the run and block every item behind it. In n8n, this generally means validating each item before the publish step and routing failures down a separate branch so the rest of the batch carries on.
Monitoring an Unattended Pipeline
Because nobody is watching every run in real time, the pipeline logs what it fetched, what it filtered out and why, and what it published on every execution, with alerting if a source stops returning data or the publish step starts failing. That visibility is what turns “automation that mostly works” into automation you can actually rely on.
Logging the reason an item was filtered out is especially useful. Without it, a drop in published volume could mean a quiet news day, an over-aggressive duplicate check, or a broken source, and there is no way to tell which. With it, the answer is in the log.
Common Mistakes In Content Pipelines
- Relying on URL checks alone for deduplication, which lets the same story through under different headlines.
- Publishing under a human author’s name, which blurs the line between automated and editorial content in the admin.
- Letting one bad item stop the run instead of isolating it.
- Assuming every API call succeeds, with no handling for rate limits or temporary errors.
- Having no alerting, so a broken source is discovered days later by a reader rather than by the team.
The Broader Pattern
Nothing about this pipeline is specific to news. The same shape (source, dedupe, structure, publish, monitor) applies to any business that needs a steady stream of structured content going into a CMS, whether that’s a content site, a product blog, or an internal knowledge base. If a similar pipeline would save your team real editorial time, that’s a conversation worth having with our AI automation and integration team.
How CodeixLab Approaches This
We start by mapping the content flow end to end: where it comes from, what the destination expects, how items should be categorized, and what counts as a duplicate for that particular business. The happy path gets built quickly; most of the effort goes into the failure cases and the monitoring around them. If the destination needs more than a standard CMS can offer, that part becomes custom software development rather than a workflow. You can see more of our delivered work on the case studies page.
Frequently asked questions
Does this kind of pipeline only work with WordPress?
No. WordPress is the destination here because the site runs on it, but the same pattern works with any CMS that offers an API for creating content.
Why are automated posts attributed to a separate account?
It keeps automated and human-authored content clearly distinguishable in the WordPress admin, which makes review, auditing, and editorial decisions easier.
What is the hardest part of building a publishing pipeline?
Not the publishing itself. It is the surrounding work: catching near-duplicates, coping with sources that change or go stale, respecting rate limits, and making sure one bad item does not stop the run.
Can a pipeline like this be built for an internal knowledge base?
Yes. The source, dedupe, structure, publish, and monitor shape applies to internal content as well. If you have a use case in mind, get in touch and we can talk it through.