Editorial Intelligence vs FPL, part four: Luck has a nasty habit of disguising itself as intelligence

Part of Editorial Intelligence vs FPL, a running experiment in evidence, judgement, football and occasionally making terrible decisions.

My Fantasy Football team went from 38 points in Gameweek 1 to 103 points in Gameweek 2, against an average of 79. You might reasonably conclude that something improved. It did. But the single biggest reason for the improvement was that I forgot to do what ChatGPT and I had decided. Which makes this considerably more interesting.

I was supposed to captain João Pedro

After the first Gameweek, I was trying very hard not to overreact. Editorial xG had scored 38 against an average of 48. Manchester United had been terrible, my bench had scored 17 points, and Riccardo Calafiori — the cheaper Arsenal defender I’d considered and rejected — had outscored Gabriel.

There were suddenly lots of attractive stories available. The team was wrong. I needed transfers. Calafiori was essential. United were a mistake. Cole Palmer had looked good enough to start another argument about whether Bruno Fernandes should still be in the team.

This is where AI was surprisingly useful. It didn’t find me a magical replacement team. It mostly told me to do nothing — or, more accurately, it kept forcing me to answer a question: what has actually changed enough to justify changing the decision? One Gameweek wasn’t enough.

So I rolled the transfer. Haaland stayed. Gabriel stayed. Bruno stayed. Mbeumo stayed. João Pedro stayed. No chip, no reactive rebuild.

Then we discussed captaincy. Bruno had Ipswich at home and was initially the recommendation. I wasn’t convinced — I already had three Manchester United players, and I wasn’t keen to compound that bet by putting the captaincy on Bruno as well. João Pedro had scored 11 points in Gameweek 1, Chelsea looked good, and he looked good. I challenged ChatGPT. It reconsidered the evidence and changed its recommendation: captain João Pedro. I agreed.

Then I forgot to actually change it.

Bruno scored 23 points

Which meant my accidental captain scored 46. João Pedro scored nine.

This is where the experiment becomes useful, because I can tell an extremely convincing story about what happened. I held my nerve after a poor opening week. I recognised that United’s opening defeat was noise. I trusted the favourable fixture. I resisted chasing João Pedro’s points. I backed Bruno. I was rewarded.

There is only one problem. None of that happened. I forgot to press a button. The scoreboard doesn’t know that — it gives me exactly the same 46 points it would have given someone who spent hours analysing the decision and deliberately concluded that Bruno was the best captain.

And that’s a problem much bigger than Fantasy Football.

Outcomes are terrible auditors of decisions

We do this constantly in business. A campaign performs well, so the strategy must have been good. An article gets huge traffic, so the editorial judgement must have been right. A LinkedIn post goes viral, so we reverse-engineer a methodology for producing another one. A launch succeeds, so the positioning was obviously correct.

Sometimes it was. Sometimes something worked despite the decision-making behind it. And sometimes excellent decisions produce terrible outcomes.

Fantasy Football has now given me both examples in two weeks. In Gameweek 1, I had lots of information, used AI to analyse it, challenged some of that analysis with another model, recorded the reasoning, and still made decisions that produced a poor result: reasonable process, bad outcome. In Gameweek 2, I agreed to change my captain and failed to implement the decision: bad execution, excellent outcome.

If I judge the process entirely by points, I learn the wrong lesson from both. That is why writing the decisions down matters. Without the record, luck becomes very easy to rewrite as intelligence.

I was missing a stage

I’ve been describing the Editorial xG loop roughly like this: information → signals → interpretation → decision → outcome → learning. Bruno has exposed something missing: execution.

The loop should actually be: information → signals → interpretation → decision → execution → outcome → learning.

That sounds almost laughably obvious, but there’s a meaningful distinction here. The decision existed in the conversation. The execution existed in the actual FPL team. They weren’t the same, and ultimately the world responds to what gets executed, not what somebody intended.

That’s true in editorial work too. You can have a brilliant narrative strategy that never makes it into the published work. A good brief can be badly executed. An excellent article can be badly distributed. An agreed editorial decision can be changed by a stakeholder five minutes before publication. What we decided is not necessarily what the audience receives.

So if we’re building AI into decision systems, preserving the reasoning isn’t enough. We also need to know what actually happened next. That feels like a small addition to the loop. I suspect it’s a fairly important one.

AI was still useful

This would be a strange article if the conclusion were that forgetting to follow AI produces better results. That’s obviously not what happened — another part of Gameweek 2 makes a stronger case for the system.

AI stopped me fiddling. After 38 points, I wanted action. Instead, we mostly held the team. Haaland returned 13, Mbeumo returned 11, Gabriel returned eight, João Pedro returned another nine. Bobby Thomas, who came into the starting XI based on fixture and defensive-contribution reasoning rather than because he’d scored three points the previous week, returned eight.

There were failures too. Tzolis blanked. John Egan scored nine points while sitting on my bench. Tyrick Mitchell gave me zero. Fantasy Football has an especially cruel way of displaying the counterfactual immediately underneath your actual team.

But the broader lesson from holding remains interesting. AI wasn’t useful because it knew what was going to happen — it didn’t. It was useful because it acted as a decision brake. It forced me to distinguish I don’t like what happened from the evidence underpinning the decision has materially changed. Those are very different things.

The obvious AI story is usually about acceleration — faster research, faster drafting, faster analysis, faster decisions. There may be just as much value in the opposite function: introducing enough friction to stop a human reacting to the latest emotionally salient result.

Gabriel still doesn’t become the right original decision

This distinction matters enough to make explicit.

Gabriel scored eight points this week. Calafiori’s nine-point Gameweek 1 had already exposed a weakness in the original recommendation to spend £8m on Gabriel rather than £5.5m on Calafiori. Gabriel returning eight now does not erase that — the original reasoning can still have underweighted price and opportunity cost even if the expensive player subsequently scores well.

That’s the mirror image of Bruno. Bruno’s 46 captaincy points do not turn my failure to change the captain into brilliant judgement. Gabriel’s eight do not turn the original marginal choice into perfect reasoning. Results update the evidence. They do not rewrite the process that produced the earlier decision. That may be one of the hardest disciplines in any learning system.

But I can influence the AI too

There’s another uncomfortable part of the Bruno story. ChatGPT initially preferred Bruno. I talked it out of the recommendation. My argument wasn’t ridiculous — there was concentration risk, United had disappointed, and João Pedro and Chelsea had supplied positive evidence. The model reconsidered and switched.

That’s normally presented as one of the strengths of working interactively with AI: challenge it, ask it to reconsider, give it more context, don’t blindly accept the first answer. I still believe all of that. But there’s another possibility. What if I persuade the AI away from a perfectly reasonable answer?

Human judgement doesn’t sit outside the AI system correcting it. It becomes part of the system. My biases can improve the model’s reasoning. They can also contaminate it. The model can persuade me. I can persuade the model. Neither participant necessarily knows when that persuasion has improved the decision.

That makes the idea of human-in-the-loop more complicated than it sounds. A human in the loop is not automatically a quality-control layer. Sometimes the human is the source of the distortion.

Three failure modes in two Gameweeks

We’re only two weeks into this and I can already see three different kinds of failure.

The first is bad weighting — Gabriel versus Calafiori. The information was largely there. The model didn’t hallucinate. It simply gave historical security and established output more weight than price, role and opportunity cost. Correct information and coherent reasoning do not guarantee correct judgement.

The second is human overreaction. A 38-point opening week created an emotional incentive to rebuild. The AI-assisted process helped resist it. A bad outcome does not automatically require immediate action.

The third is execution failure. The final captaincy decision was João Pedro. The actual FPL team still had Bruno captain. A good outcome does not prove good process.

Those are very different problems, and treating all AI failure as hallucination misses most of what’s interesting here.

And now the stories are starting

This is another reason I wanted to run the experiment across a season rather than ask AI to pick a team once. Pre-season football gave us information. The actual season is beginning to give us narratives.

Manchester United looked disastrous in the opening Gameweek. Now Bruno has produced 23 and Mbeumo 11. Does that mean United are suddenly good? Of course not. But the story has changed.

Chelsea are more interesting too. João Pedro has now returned in consecutive Gameweeks, and Cole Palmer has looked dangerous. The question of whether Palmer should eventually replace Bruno is becoming a genuine decision rather than a pre-season guess.

Haaland has his own story. After Manchester City lost 3–0 to Arsenal before the season, my instinct was to get rid of a £15.5m striker around whom we’d constructed the entire team. AI argued that one match wasn’t enough evidence to dismantle the structure. He’s just returned 13. That doesn’t prove keeping Haaland will ultimately be right — it tells me the evidence threshold for reversing a structural decision probably should be higher than for changing a marginal one.

And then there’s Arsenal. Gabriel returned eight this week, Tzolis returned zero, and Calafiori remains the obvious comparison because of the price difference. But if Arsenal really are going to be one of the dominant teams this season, the bigger question might eventually become: how much Arsenal do I want? Gabriel, Calafiori, Saka, Ødegaard — defence or attack? Suddenly we’re moving from individual player selection towards portfolio allocation. And that’s where narrative becomes useful.

Stories help us think. They can also fool us.

“United are terrible.” “Chelsea are back.” “Haaland isn’t worth £15.5m.” “Calafiori is essential.” “Arsenal are going to dominate.”

These are incredibly efficient ways of compressing lots of information. Humans think this way constantly, and so does editorial work — we turn messy reality into narratives because otherwise there’s too much information to process. AI is extraordinarily good at doing the same thing, and that’s both its strength and its danger. Give a language model enough football evidence and it can construct an extremely persuasive explanation for why almost any plausible transfer makes sense.

The Editorial Intelligence job isn’t therefore simply to find the story. It’s to keep track of the evidence underneath it. What strengthened the story? What weakened it? What would make us abandon it? What did we actually know when the decision was made, and what are we only claiming was obvious because we know the result?

That distinction matters whether you’re selecting an FPL captain or constructing a corporate narrative. It also makes this experiment more relevant to the work I’m actually interested in. Narrative isn’t separate from analysis — it’s one of the ways humans make analysis usable. The danger begins when the story becomes more persuasive than the evidence supporting it.

This is deliberately over-engineered

Some people will reasonably look at the amount of analysis going into a Fantasy Football team and think this is a spectacular waste of time. It is definitely more analysis than Fantasy Football requires. That’s partly why it works — the stakes are low enough that I can deliberately slow decisions down, see what ChatGPT notices, bring in another model to disagree, compare that with my own instinct, record what I thought, and then watch reality do something completely different a few days later.

I’m not trying to outsource the team to AI. I’m trying to understand what happens when model reasoning and my own judgement coexist: when the model catches something I missed, when I resist it, when it sounds persuasive but weights the evidence badly, when I persuade it, when it stops me reacting too quickly. And now, apparently, when the decision we reached isn’t even the one I execute.

Fantasy Football is useful because it gives that whole system a scoreboard without making the consequences particularly important.

What I’ve done for Gameweek 3

The next test is whether the system can recognise when the evidence threshold has been crossed rather than using patience as an excuse to do nothing forever.

This time I made the changes early for an extremely human reason: I didn’t want to forget. The two rolled transfers are now executed: Gabriel → Calafiori and Tzolis → Cherki.

They are not the same kind of decision. Moving Tzolis is a response to the question around his role and minutes becoming more important, not simply to a zero-point week. Moving Gabriel is an opportunity-cost decision: retain Arsenal defensive exposure at a lower price and use the money to strengthen another part of the team. Those reasons now exist in the record before Gameweek 3 tells me whether the moves score well or badly.

The team currently saved in the app is Kinsky; Calafiori, Maguire, Mitchell; Bruno Fernandes, Mbeumo, Cherki, Le Fée; Haaland, João Pedro, Igor Jesus. Egan, Slater and Bobby Thomas are on the outfield bench.

And the aggressive decision is already visible on the team screen: Haaland is captain with Triple Captain active for Manchester City at home to Coventry. Bruno is vice-captain.

That chip is active, but I still don’t regard it as intellectually final. The deadline is Friday evening. I have a scheduled check before it closes to revisit team news, fitness and rotation risk and then confirm something deliberately mundane: that Haaland is still captain and the Triple Captain chip is actually saved.

That’s the execution lesson from Gameweek 2 turned into a procedure rather than a paragraph. The decision can still change before the deadline. What I don’t want again is for the decision in my head, the decision in the AI conversation and the decision saved in FPL to quietly become three different things.

There is another useful shift here. Gameweek 2 was about resisting action because one bad result had not crossed the evidence threshold. Gameweek 3 begins with the opposite test: recognising when the evidence has changed enough that action is justified. If the system only ever tells me to wait, it isn’t judgement. It’s inertia.

So did the system work?

I don’t know. Two Gameweeks is nowhere near enough evidence — a 103-point week against an average of 79 doesn’t prove anything any more than 38 points did last week.

But the experiment has already become considerably more useful than “let ChatGPT pick my Fantasy Football team.” In two Gameweeks: AI has made a persuasive recommendation I think was poorly weighted. It has stopped me reacting emotionally to a bad outcome. I’ve challenged its reasoning, and it has changed its mind because of my challenge. And an execution mistake has then massively outperformed the decision we jointly reached.

That’s quite a lot of mess. But perhaps that’s the point. The goal isn’t to remove human judgement from the system. It’s to make the interaction between information, narrative, AI reasoning, human judgement, execution and outcomes visible enough that we can learn from it.

Because luck has a nasty habit of disguising itself as intelligence. AI doesn’t solve that problem. But a system that records what we knew, what we thought, what we decided, what we actually did and what happened afterwards might make the disguise a little easier to spot.

And apparently sometimes forgetting to change your Fantasy Football captain helps too.

Editorial Intelligence 1. Fantasy Football 1.