AI can make work look finished before it works

That is not an argument against AI. It is a reason to get much better at deciding what good means.

I came across a new piece of research in Harvard Business Review that captures something I have been noticing in my own work. Ipsos and researchers at Syracuse University tested 20 video adverts with 3,000 US consumers. Half were existing human-made adverts. The other half were generated entirely with AI from briefs reverse-engineered from the originals.

Most people could not reliably tell which was which. The AI adverts looked like adverts: credible, polished and good enough to sit alongside the human work without immediately giving themselves away. But they did not work as well. According to Ipsos, the human-made adverts were 14% stronger on its measure of predicted short-term sales effectiveness and 17% stronger on predicted long-term brand equity.

The obvious reading is that human creativity still matters. I think it does, but there is something more useful here. The AI work passed the visible test: it looked finished. The difference appeared when somebody measured whether it was likely to do the job. That feels much bigger than advertising.

Good enough has become much easier

AI is extremely good at producing the recognisable shape of professional work. An article has a strong heading, a clean introduction and a logical structure. A presentation has a polished visual language. A report sounds confident and includes the sections a report is meant to include. Nothing looks obviously broken, which is useful because getting from a blank page to something workable used to consume a lot of time.

But appearance used to tell us more than it does now. A polished output usually suggested that somebody had spent time working through the decisions underneath it. Not always, but often enough that the finished work operated as a rough signal of the thinking behind it.

AI weakens that connection because it can produce the surface before the difficult decisions have been made. The argument may still be familiar. The evidence may not support the confidence of the language. The presentation may answer the wrong question beautifully. The advert may look exactly like an advert while doing less for the brand.

I wrote about a related problem in Raising the floor is not building a ladder. That piece asked whether a strong output still proves that the person producing it has developed the judgement behind it. This is the other side of the same break: a finished-looking output no longer proves that the work itself is effective.

Approval is not the same as evaluation

Most content workflows end with somebody approving the asset. They check the facts, tone, brand and presentation, make a few changes and decide that the work looks ready to move forward.

AI can make that final decision harder because it removes many of the signals that used to tell an experienced person to slow down. Weak ideas arrive in strong sentences. Gaps are covered by smooth transitions. Familiar thinking can be presented with enough confidence to feel complete. The reviewer sees fewer obvious mistakes, but that does not necessarily mean there are fewer important problems.

There is also much more to review. If a team can create ten versions in the time it once took to create one, it has not removed the need for judgement. It has created ten things that may need judgement.

This is why “human in the loop” is not much of an answer by itself. A person glancing at the end of an AI workflow can easily become a rubber stamp for work that already looks convincing. Meaningful review needs the right person, enough context and a clear idea of what failure looks like. It may need a subject expert who can spot an overclaim, an editor who can see that the argument does not deserve the confidence of the prose, or performance evidence that arrives after publication.

Approval asks whether the work can go out. Evaluation asks whether it should exist, whether it worked and what should change next time. Those are different jobs.

This is not an argument that AI makes work worse

The wider evidence is not that simple. In a separate experiment published in Science, professionals given access to ChatGPT completed writing tasks 40% faster and produced work assessed as 18% higher in quality. The people with weaker initial skills benefited most.

That is a substantial improvement, and it fits what I have seen myself. AI can raise the quality of a first draft, help someone organise an argument and remove a lot of low-value production work. The difference is the kind of task and the meaning of quality.

A contained writing task can be judged fairly quickly. An advert is meant to influence behaviour and build a brand over time. A thought-leadership article might need to earn trust, change how somebody understands a problem and remain useful months after publication. The further the real purpose sits from the finished asset, the easier it becomes to celebrate the wrong gain.

Time saved is visible immediately and more outputs are easy to count. Whether the work was remembered, believed or useful is slower and harder to see. An organisation can therefore become more efficient at production without becoming more effective at communication.

I can feel this happening in my own work

AI has been straightforwardly useful for Editorial Intelligence. It helps me connect new observations to things I have already written, test arguments, find relevant evidence and turn rough thinking into a draft. I can move much faster without every new piece starting from an empty page.

But the draft can start looking like an article before I have decided whether the argument deserves one. Once the heading works and the sections fit together, I naturally begin editing the piece as if publishing it is the remaining task. The fluency creates momentum and can make the idea feel more settled than it really is.

Sometimes the right decision is to narrow the claim. Sometimes the idea belongs inside an existing article. Sometimes the draft is perfectly competent and still does not add enough to justify another page on the internet.

That is one reason I have started thinking that prompting is becoming routing. The important decision is not only how to get a better response from a model. It is whether this task needs AI, which evidence should travel with it, whether another perspective would help and whether the output is worth producing at all.

The difficult part is no longer getting something that looks finished. It is retaining the willingness to reject it once it does.

The feedback has to survive the asset

The practical answer is not a heavier approval process for everything. I have tried over-engineering AI workflows and found that the checking can easily become more work than the article. The amount of scrutiny should match the stakes. A routine transformation does not need the same process as a financial claim, a major brand argument or a piece of advice somebody may act on.

But every workflow needs some definition of success beyond completion. For content, that might mean asking whether the audience found it, whether it answered a real question, whether it changed understanding, whether the evidence held up and whether it strengthened a larger narrative. Different work needs different signals. The important thing is that the signal comes from outside the generative system and then influences what happens next.

In Judgement as infrastructure, I argued that useful editorial feedback should not disappear after one edit. The recurring problem, the strong example and the reason a claim was rejected can all improve the conditions under which the next piece is produced.

The HBR research adds something important to that idea. The system should not only remember what an editor approved. It should remember what happened after approval: whether the audience responded, whether the work helped, whether a credible claim turned out to be wrong and whether something everyone liked failed to perform.

Without that connection, AI can become very good at reproducing the kind of work an organisation has previously approved without learning whether any of it worked.

The next AI problem is evaluation

Most organisations are understandably focused on AI production: tools, prompts, workflows and ways to create more quickly. The next question is whether their evaluation systems can keep up.

The real opportunity is not to build a machine that produces more professional-looking content. The internet already has enough of that. It is to connect faster production with evidence, judgement and outcomes so the system becomes more intelligent over time.

AI makes it easier to cross the line from nothing to credible. That is genuinely valuable because it can raise the floor and remove a great deal of avoidable effort. But credible was never the final goal.

The work still has to be accurate enough to trust, distinctive enough to remember and useful enough to change something. We can no longer infer any of that from how finished it looks.

Before scaling AI production, there is a less exciting question worth answering: how will we know whether this worked?

If there is no answer, the organisation may not be scaling intelligence yet. It may just be scaling things that look like work.