Judgement as infrastructure, part two: the system still needs editors

Capturing judgement is the easy half. Keeping it from hardening into a house style is the part that needs a person.

Part one argued that most editorial judgement is used once and then thrown away — corrections applied to one document, in one workflow, at one moment, while the same principle gets explained again to the next writer. OpenAI’s content marketing team has started treating those recurring comments as a dataset instead, and I found I had been building a version of the same thing by accident, having assumed I was only improving prompts.

This part is about what it takes to do that deliberately, and what it costs.

---

The feedback completes the chain

This is the connection Sandhya’s post made visible to me.

I had been uncertain what to do with all the feedback generated by editor, subject-matter expert and customer review. Applying it to the current story was obvious. Preserving every comment was not useful. Turning every request into another prompt instruction would eventually create a contradictory mess.

But the feedback can become evidence.

The system can ask:

• What changed?
• Who requested it?
• Was the issue factual, structural, narrative, stylistic or simply a preference?
• Which stage should have caught it?
• Has it occurred in another story?
• Is it specific and checkable?
• Did changing the workflow prevent it next time?

That creates a controlled learning loop:

brief → human decision → draft → editorial QA → human, SME and customer review → classified feedback → validated system change → next story

The system does not automatically learn from everything. That would be dangerous. A one-off customer preference should not govern every later story. An SME correction is not the same as an editorial principle. A stakeholder request may reflect internal politics rather than reader value.

Human judgement controls what the system is allowed to learn.

This is what makes the project different from turning a prompt into an agent for convenience. Previously, an agent would mostly have been another interface around the same instructions. Easier to find, perhaps easier to share, but not a more intelligent editorial method.

A validated workflow connected to a feedback loop gives the agent a reason to exist.

The agent becomes the operational surface of the latest approved editorial knowledge. The value sits underneath it: the workflow, examples, failure modes, review history, validation rules and human decision points.

That also makes the workflow model-agnostic. I may develop the personal version through ChatGPT and Claude and implement a Sage version through Microsoft Copilot, but the durable asset should belong to neither. The workflow and accumulated judgement should be able to move as models, employers and approved platforms change. Portable Intelligence is the design principle underneath that portability.

Customer stories are the achievable first implementation

A company-wide editorial agent system is an ambitious thing to propose inside any large organisation. It crosses teams, permissions, governance, approved tools and ownership, and it tends to try to solve several different editorial problems at once.

A single recurring format is different, which is why customer stories are the version of this I would attempt first. They provide comparable source material, observable review patterns and a workflow whose stages have already been tested together — along with a body of completed work and a catalogue of what has repeatedly gone wrong with it.

So the useful next step isn’t building anything organisation-wide. It’s running the existing workflow through the next few stories, recording what the AI stages catch and miss, comparing that against human review, and seeing whether the recurring corrections begin to fall.

That also produces something that can eventually be handed over — not a claim that an agent contains anybody’s judgement, which it doesn’t, but the lessons already paid for: how the workflow developed, which failures recur, why the checks exist, what evidence supports them, what stays human and how the thing should continue learning. A prompt tells the next person what to ask. A learning editorial system explains why the workflow works and how to keep improving it.

This is more than a prompt library

There is a temptation to describe all of this as prompt engineering, which undersells it. A prompt library stores useful instructions. An editorial system has to preserve the knowledge underneath those instructions: examples, decisions, source material, standards, relationships and the record of what changed.

“Write a report with a clear argument” is not especially helpful on its own. What does this organisation consider a clear argument? Which reports achieved it? Where did they fail? What objections did the editor raise? What evidence was considered strong enough? Which audience was the report for? What should remain consistent, and what should change with the subject?

The useful asset isn’t the sentence instructing the model. It’s the accumulated editorial context that makes the instruction specific.

That is why the repository matters. As I argued in Why every company needs a GitHub for knowledge, storage is only the beginning. A working knowledge system also needs history, connection, review and a canonical version that can keep changing without losing where it came from.

The agent is the interface.

The judgement behind it is the asset.

The system still needs editors

There is an obvious risk in formalising judgement this way: a system that makes established standards easier to apply also makes them harder to question. Yesterday’s successful example can become tomorrow’s formula. A recurring comment can be turned into a universal rule even though it only made sense for one format or audience. An organisation can encode its own habits and then mistake consistency for quality.

AI is particularly capable of reinforcing whatever it is given. A repository full of safe, conventional work will make it easier to produce more safe, conventional work.

So editorial infrastructure needs maintenance rather than reverence. Which examples are still strong? Which standards are protecting quality, and which are protecting habit? What has the audience changed its mind about? Where should a writer deliberately break the house pattern? Which previous decision no longer holds?

This is why the human role doesn’t disappear once the agents exist. Somebody has to curate the examples, interpret the feedback, distinguish a principle from a preference and decide when the system itself needs editing.

The better the system becomes at enforcing yesterday’s judgement, the more important it becomes to have someone capable of changing that judgement tomorrow.

A practical version of the upstream shift

I recently argued that editorial jobs are moving upstream: as production becomes cheaper, more value moves towards evidence, narrative, standards, orchestration and the decisions about what should be produced at all. Sandhya’s post is a practical implementation of that shift. The editorial team is not simply making every sentence better at the end. It is building a system that allows more of its judgement to operate earlier and across more work.

That changes the editor from the final person correcting the draft into someone who helps design the environment in which drafts are made.

It also gives me a clearer way to describe what I have been doing with Editorial Intelligence OS.

I have not simply used ChatGPT and Claude to help me publish a large volume of work. I have been capturing the thinking, standards, connections and revision history that allow each piece to begin with more than the last one did.

Without AI, I could have maintained a website and written the articles. I could not realistically have kept interrogating, connecting and revising the whole body of work at this speed while preserving the arguments underneath it.

That is the more significant change, and it’s the part that doesn’t show. The output is visible; the compounding editorial infrastructure underneath it is the real experiment.

And Sandhya’s post suggests that this is not merely an eccentric way for one writer to work. It may be an early version of how editorial teams turn what they already know into a system that keeps making the next piece better.