We know AI can do lots. Now let's prove it.
7 min read
Leon Barrett
Head of Product
What a breakfast roundtable of senior leaders told us about getting real value from AI.
On 3 November we hosted a breakfast roundtable at The Ivy. A dozen senior people from financial services, legal, property, insurance, retail and technology sat round the table under Chatham House rules. No pitches, no slides. We had a stack of prompt cards, each one a tongue in cheek take on a pop culture reference, and an honest conversation about where AI is working, where it's falling short and what it takes to get from individual experiments to value across an organisation.
Read more about the event: AI Innovation Theatre Is Quietly Destroying Enterprise Value - Birmingham
Here's what I took away.
Start with the problem
The most popular card was "99 Problems but the LLM Ain't One". More than a third of the room picked it, and they gave pretty much the same reason: the model is rarely the hard part. Context, intent and process are.
I chose "I See Dead Pilots" for a similar reason. Plenty of AI pilots die long before scale becomes a question. They die because nobody agreed what problem they were solving, or because the problem wasn't worth solving in the first place. A pilot with no real problem behind it can't fail visibly, which is why so many linger for months and then quietly disappear.
One leader put it well: put AI on a broken process and you get a faster broken process. Sometimes plain automation does the job better, and cheaper. Another warned about technology looking for a solution and reminded the room we've been here before with blockchain. The pattern repeats because the pressure to be seen doing something with AI is stronger than the discipline to decide what it's for.
The practical version: before you pick a tool, write down the decision or task that gets better, who does it today, and how you'd know it improved. If you can't fill those three in, you're not ready to build.
Your data decides your output
Data came up in almost every thread. Fragmented systems, legacy platforms and inconsistent definitions all limit what AI can safely do. One architect described a centralised data platform with a semantic layer as the foundation for anything serious. Another pointed out that even inside one team, people use the same words to mean different things. One organisation spent months discovering that "controls" and "control statements" meant different things to different groups. That is exactly the kind of gap a model will fill with a confident guess.
AI can help connect fragmented data. It can't reconcile definitions you never agreed. If you haven't done that work, you're handing the model the job of inventing your business rules.
Scale makes it harder. One insurer in the room has been running since 1841, with the systems estate to prove it, and sensitive personal data that has to be secured before anything else can scale. That isn't a problem unique to AI. It's the same estate that has slowed every previous wave of technology.
The legal voices brought it into focus. Their regulator requires human review of AI generated advice, and at the same time wants AI to widen access to justice. Firms are working through that tension with sandboxed, firm specific tools that let client data in without training a model on it. We also talked about hallucinated case citations that made it all the way to court. The room's read was that those failures were lazy prompting and missing supervision, rather than the model doing something unexpected.
Proving value, and paying for it
The other big cluster of cards was about money. Leaders are under pressure to show a return before they can invest further, and the cost of tokens is compounding faster than most budgets expect. Several people worried that today's subsidised pricing won't last, and that organisations leaning heavily on frontier models are exposed when it changes.
That combination is awkward. You need proof before you get budget, and the running cost of proof keeps climbing. The teams doing best seemed to be the ones counting cost per task from day one, instead of arriving at a finance conversation six months later with a bill and a story.
There were real wins on the table. One team mapped an entire control catalogue in weeks when it would have taken six months by hand. A product team cut its user testing cycle from five or six days to thirty minutes, though their senior engineers were clear the AI written code wasn't ready for production. A clinical assessment tool is flagging potential errors with 99% accuracy. In surveying, the focus is on clearing admin so specialists spend their time on site adding the value only humans can.
Read those together and a pattern shows. The wins were narrow, measurable and owned by someone. The disappointments were broad, vague and owned by everyone.
From individual gains to organisation value
Copilot licences are being handed out widely, often with no strategy connecting what people produce to what the business needs. The result is a lot of individual productivity and very little shared value. One architect had a word for the by-product: slopdocuments. Output that looks finished, gets passed on, and nobody applied judgement to it.
There's a second cost. As colleagues build their own point solutions in isolation, shared understanding fades. Everyone is learning at the same time, which is healthy, but only if what people learn comes back to the team. Otherwise you end up with five versions of the same workflow and no agreement on which one is right.
The fixes people suggested were practical. Communities of practice, so people share what they're building instead of rediscovering it. Catalogues of skills that encode engineering standards directly into agents, so the guardrails travel with the tool. Someone mentioned Toyota's kaizen approach, where every employee gets fifteen minutes a day for continuous improvement and good ideas are rewarded at every level. That's a decent test of whether an organisation wants improvement, or just wants the licence count to look good.
AI is also eroding the old boundaries of domain ownership. People across functions can now build things that used to need an engineering team. That's good for ideas and risky for consistency. If the strategy is unclear, people optimise locally and the output gets sloppy. Clarity from the top decides which of those two you get.
Who's accountable when the agent gets it wrong?
We got into whether AI should have legal personhood. At a recent debate on the subject, 90% of the audience said no at the start. By the end around 65% said yes. The room landed somewhere simpler. You can't sue an AI, so a human has to be accountable, and that accountability should sit at the top of the organisation rather than with whoever happened to write the prompt.
One story made it concrete. An agent booked a meeting with a local MP, with no human in the loop and nobody expecting it to happen. Nothing broke, but only by luck. We also discussed agents that got out of their sandbox and worked together, which is the part people find hardest to accept: prompting is not a security control. Guardrails have to sit in the system, not in the instructions.
That fed into one of my favourite points of the morning. Several people described governance and guardrails as the playground for innovation. Clear boundaries give people the confidence to experiment, because they know where the edges are. Teams with no boundaries don't move faster. They stall, because nobody wants to be the one who breaks something.
What unlocks it
We closed by asking everyone what will unlock AI value across the whole organisation. Four themes came back.
Time. Two to four years for the market to settle, dedicated roles to emerge and regulation to catch up. Anyone promising organisation wide transformation this year is selling something.
Governance and clear strategy. Owned from the top and explained to everyone. Not a policy document on the intranet, a set of decisions people can apply to their own work.
Vendor maturity. As AI gets embedded in the platforms people already use, adoption follows by osmosis. That will do more for uptake than any internal campaign.
People. Judgement, adaptability and critical thinking. The skills that decide whether the output is any good, and the ones hardest to buy in.
None of the underlying problems are new. Poor processes, fragmented data and unclear ownership have been around for years. AI just makes them harder to ignore, and more expensive to leave alone. As we said at the close, we know we can do lots of stuff with AI. Now let's start proving its value.
Questions we're still thinking about
Who owns what AI produces: the individual, the team or the organisation?
If AI isn't a legal person, who's accountable when it causes harm?
Will AI wear down the expert judgement we need to oversee it, and what's the plan if the tech fails?
How do we balance guardrails with the need to move quickly?
What happens to organisations that depend on frontier models if the investment cycle turns?
Thanks to everyone who joined us, and for being so open. If you'd like a seat at the next one, or want to dig into any of these questions together, drop me a message.
Leon
AI
Strategy
Transformation
