Assurance-Engineered AI Adoption Playbook
An assurance-first strategy and implementation guide for safe, secure and trustworthy artificial intelligence systems.
For product owners or idea champions who want to successfully adopt AI by preventing failure.
Our Playbook will guide you in the development of an AI use-case with built-in best practice assurance, so you can deploy AI responsibly.
Each stage has been designed to help you turn a good idea into a trustworthy reality.
It will provide the necessary frameworks and best practices for adopting AI, maximising success and mitigating failures.
The Value of our Playbook
Stages of the Playbook
There are five broad stages of the AI Adoption Playbook, which group several key steps within each stage.
Drawing on our three years of cutting edge AI Safety experience, we've mapped the stakeholders, assets and 'Business State' information required to effectively progress through each critical step of each the five stages in our AI Adoption Playbook.
We guide you through the 'Analysis' and 'Actionable Tasks' you must perform to successfully complete each stage.
Stage 1
UNDERSTAND USE CASE
Understanding the model’s purpose, its job and what its value is for the business.
Stage 2
MEASURE RISK.
Allocating risks is essential because The AI Act’s primary concern is the level of risk posed to the public.
Stage 3
RESOURCES.
Understand if you have the data and skills to feasibly and efficiently achieve use-case objectives.
Stage 4
DEVELOP AND TEST THE MODEL.
Ensuring development is undertaken according to the compliance requirements for the model’s use.
Stage 5
DEPLOY AND MONITOR THE MODEL.
Monitoring continuing operation to prevent lapses or deviations from intended behaviour or use.
Understanding the model’s purpose, its job and what its value is for the business.
This stage will guide you in answering the following questions:
- Is the use-case viable?
- Are we ready for the use-case?
- How do we measure success?
- Do we understand the context of the model use, who it impacts and if there any resulting ethical concerns?
This stage guides you through the consideration of risk.
Allocating risks is essential because risk is the main reason AI projects don't reach production. Further, The AI Act’s primary concern is the level of risk posed to the public.
This stage will guide you in answering the following questions:
- Do we understand the risks involved?
- Have we performed a risk assessment?
- Do we understand the context and impact of the use-case?
- Do we understand the relevant implications, such as Legal, Data and Security?
This stage will help you map the data, skills and technology you need to feasibly and efficiently achieve use-case objectives.
This stage will guide you in answering the following questions:
- Do we have or need Suppliers?
- Do we have the expert knowledge?
- Is data fit for use case? Is it labelled?
- Do we have the required IT Skills and resources?
Ensuring development is undertaken according to the compliance requirements for the model’s use.
This stage will guide you in answering the following questions:
- What models are possible with our data?
- Which model is most appropriate?
- Which model aligns best with company principles?
- How can I train and test effectively?
- Are the models up to the performance, robustness, and transparency standards we would need, and do we have risk-based metrics to track them against?
This stage will help you deploy and monitor the model.
Monitoring continuing operation to prevent lapses or deviations from intended behaviour or use.
- Is the model running as expected?
- Is the model suitably governed?
- Can we identify when the model is failing?
- Do we have Governance/Approval process in place?
The Five Playbook Stages
What to expect from our AI Adoption Playbook.
Below, we will run through the first stage to show how the result of a each stage is a 'card'.
In this case, 'Stage 1: Awareness of the Use-Case' results in the creation of a 'Use-Case Card'.
Note that you would create one of these cards for each new use-case you consider.
This enables a methodical and fair comparison of all use-cases ultimately helping you decide which to proceed with.
Instructional Card Overview
Each stage has an instructional card. Its purpose is to guide you in the creation of the stage deliverable. To the right, you see our Stage 1 instructional card: Awareness of the Use-Case.
At the top left, 'Required Stakeholders' shows which 'roles' are required. Precise job titles may differ, but somebody from each of these disciplines will be needed to share their perspective on the use-case.
'Assets Needed' lists the information and policies required to complete this stage.
'Business State' refers to a living read-write document, which holds knowledge you will need about the business more broadly, required to complete this stage. Note that you are also likely to discover new knowledge while completing each stage to 'write' back to the Business State.
On the right, you'll see the incremental steps of the stage, the stage 'Purpose', and the 'Analysis' and 'Actionable Tasks' you will perform in completing this stage.
The 'Use-Case Cards'
To the left, you'll see a 'Use-Case Card', which is the Playbook's output for Stage 1 'Understand Use-Case and Scope'.
The 'Stage Card' essentially guides your to ask the right questions and perform tasks needed to complete these Cards.
When exploring use-cases, you may find that you hit a barrier that prevents you from progressing. That's ok because, over time, you will grow a collection of these physical/digital cards.
This will help compare the value of use-cases in a standardised way and re-visit previously blocked use-cases when your Business State changes.
The cards for stages 2 through 5 are as follows:
Step 2: Risk Assessment Card
Step 3: Resources Card
Step 4: System Card
Step 5: Monitoring Card
Perhaps the most important function of the AI Playbook is to coordinate the right roles at the right time throughout the AI adoption process.
