Why did Apple’s AI fail so badly, and how can others avoid this? While Apple hasn’t shared details and is unlikely to do so, there are ways of getting ahead of these kinds of foreseeable AI failures. This is called AI Assurance—a rigorous process of testing, evaluation, and risk management to catch issues before they harm users or reputations.
To prevent failures like this, we believe every AI system should meet three core criteria:
- Testing aligned with the specific use case.
- Broad stakeholder agreement on the sufficiency of those tests.
- Clear, actionable presentation of test results for all audiences.
Here’s how these principles could have helped Apple avoid its AI misstep.
1. Align Tests with the Use Case
LLMs are versatile, but their effectiveness in practice depends heavily on how they’re tested before they’re deployed. Summarising news articles, for example, requires tests tailored specifically to news summarisation. Unfortunately, many organisations rely on prebuilt static evaluation datasets designed for generic tasks, like measuring an AI’s toxicity, general knowledge, or mathematical reasoning. While useful, these datasets don’t address the unique challenges of specific applications.
Summarising news is not the same as summarising legal documents or medical papers. News stories cover diverse topics and involve subtle nuances, and sensitive topics that require careful handling. If Apple didn’t develop tests that reflected this complexity—such as verifying that summaries stayed true to the original news story across a wide range of article types—errors were always more possible. This is made doubly hard because news is constantly evolving – how can you test for the unknown future?
It is important that you start with testing requirements, rather than the test data. Instead of forcing the AI to fit to pre-existing datasets, you need to curate “aligned” evaluation data to reflect the specific issues the AI will face in production. This ensures that you are testing the most relevant things, and that the AI is more likely to be effective for longer when deployed.
When tests aren’t aligned, AI systems are essentially flying blind—and that’s no way to launch a public-facing tool.
2. Get Stakeholder Buy-In
AI systems impact a broad range of stakeholders, and failing to include them in the testing process is an own goal. For a news summariser, stakeholders might include product managers, legal teams, PR professionals, journalists, and the AI engineers building the tool. Each group has unique concerns and priorities: legal experts might worry about defamatory or false content; PR teams will focus on avoiding brand damage from embarrassing outputs; and AI engineers need to address technical vulnerabilities like hallucinations.
AI failures often arise from overlooked risks. To address this, the stakeholders should work together to identify the major risks—like hallucinations, factual inaccuracies, or bias—and reach agreement on the tests needed to address them. For example, a PR team might prioritise testing for offensive language, while the AI Engineer might focus on robustness testing against edge cases.
Getting broad stakeholder buy-in ensures the AI system is evaluated from every angle. Once all parties agree on the testing plan, you can confidently say, “If the AI passes these tests, it’s ready to deploy.” This doesn’t guarantee perfection, but it significantly reduces the risk of failure.
3. Present Results Clearly
Even the most robust tests are useless if stakeholders can’t understand the results. Decision-makers need clear, actionable insights to evaluate whether an AI system is ready for deployment. But clarity means different things to different people. Non-technical leaders need high-level summaries, such as pass/fail outcomes or traffic-light indicators (green, yellow, red) for key metrics, while AI engineers require detailed reports showing metrics, methodologies, and specific areas where the AI succeeded or failed.
A one-size-fits-all report won’t work. Imagine a PR executive receiving a jargon-filled, data-heavy report—they wouldn’t feel confident signing off on the system. Similarly, a vague email saying, “The AI passed” wouldn’t satisfy an engineer’s need for technical details.
The solution is a dynamic dashboard that caters to different audiences. Non-technical stakeholders get the big picture, while technical teams can drill down into the specifics. This transparency builds trust and ensures everyone understands the AI’s strengths and limitations before launch.
Without clear results, no one feels comfortable approving a system—especially one as public-facing as a news summariser.
Putting It All Together
Looking back at Apple’s AI news debacle, it’s likely they missed one or more of these critical steps. Perhaps their tests weren’t well-aligned to the challenges of news summarisation. Maybe they didn’t involve enough stakeholders to identify all potential risks. Or perhaps their evaluation results weren’t clear enough to make informed decisions.
Whatever the cause, the incident underscores a crucial point: LLMs will continue to hallucinate for the foreseeable future. But with proper testing, broad collaboration, and clear communication, organisations can minimise these risks and avoid public embarrassment.
Why AI Assurance Matters
AI is evolving rapidly, unlocking new possibilities every day. But with this progress comes responsibility. If the industry wants to avoid costly failures and maintain public trust, we must co-evolve our testing and assurance practices alongside AI technologies.
AI Assurance helps organisations deploy AI systems that are safe, reliable, and aligned with their goals. Whether it’s summarising news or tackling more complex challenges, robust evaluation is the key to success.
