Managing AI risk and responsible AI
The risks specific to AI projects and how to manage them: an AI risk register, scoring likelihood and impact, responsible AI principles (fairness, transparency, accountability, privacy, safety), approvals proportional to risk, and incident response when AI goes wrong.
🎯 By the end of this lesson you will be able to:
- Identify the main categories of AI risk and add them to a risk register
- Apply responsible AI principles in day-to-day delivery decisions
- Prepare for AI incidents with clear response steps
In this lesson
The risks AI adds
AI projects carry the usual risks of scope, schedule and budget, plus some new ones. A delivery manager doesn't need to solve them all personally, but must make sure each is identified, owned and managed.
| Risk category | Example |
|---|---|
| Quality | Confident wrong answers mislead customers or staff |
| Fairness and bias | Worse results for some languages, regions or groups |
| Privacy and data | Personal data sent to the wrong place or kept too long |
| Security | Prompt injection causes data leaks or unwanted actions |
| Legal and compliance | Use falls under regulation without required controls |
| Dependency | Provider outage, price change or model retirement |
| Cost | Usage grows faster than budget |
| Adoption and people | Users over-trust or ignore the AI; roles change without support |
| Reputation | A harmful output becomes public |
An AI risk register
Risk register entries
R1 · Quality · Assistant gives wrong refund amounts · Likelihood: medium · Impact: high Controls: amounts come from the order system, not the model; evaluation case set for refunds; human approval for refunds > $100 Owner: Product owner · Review: every release R2 · Dependency · Primary model retired with 6 months' notice · Likelihood: high · Impact: medium Controls: gateway with fallback model; evaluation set ready for migration; deprecation notices monitored Owner: Tech lead · Review: quarterly
Each risk has a cause, a score, specific controls, an owner and a review date.
Responsible AI in practice
| Principle | What it means for delivery |
|---|---|
| Fairness | Evaluate quality per group, language and region; fix gaps before launch |
| Transparency | Tell users when AI is involved and what its limits are |
| Accountability | A named person owns each AI system and its decisions |
| Privacy | Minimise data, respect retention, honour user rights |
| Safety and security | Red team, guardrails, limits on actions, human approval where needed |
| Human oversight | People can review, override and appeal AI-influenced decisions |
Approvals proportional to risk
- Low risk (internal drafting, summarising public content): standard checks, approved tools, light documentation.
- Medium risk (customer-facing answers, internal decisions support): evaluation evidence, guardrails, security and privacy review, monitoring plan.
- High risk (affects people's jobs, credit, health, safety or rights): full risk assessment, fairness testing, human oversight, legal review, formal sign-off and regular audits.
When AI goes wrong: incident response
- Detect: monitoring alerts, user reports, the "report a problem" button.
- Contain: switch off the feature, roll back the prompt or model, or tighten guardrails. Have this "kill switch" ready before launch.
- Assess: who was affected, what data or decisions were involved, and whether legal or regulatory notification is needed.
- Fix: root cause, add the failure to the evaluation set, verify the fix.
- Learn: blameless review, update the risk register and controls.
What this means for your role
Developer
Build kill switches, feature flags, versioned prompts and action limits so risks can be controlled and incidents contained quickly.
Tester / QA
Turn each risk into tests: fairness slices, adversarial cases and checks that controls such as approvals actually work.
Business Analyst / PO
Identify affected people and decisions for each AI feature; that sets the risk level and required oversight.
Tech Lead / Architect
Design technical controls for each major risk and make sure they are testable and monitored.
Project / Delivery Manager
Own the AI risk register, get each risk an owner and review date, scale approvals to risk, and make sure an incident plan exists before launch.
Scrum Master / Agile
Run blameless incident reviews and help the team bring risk items into normal backlog work rather than a separate checklist.
Try it: start an AI risk register
- Pick an AI feature and list one risk for at least five categories from the table.
- Score each for likelihood and impact, and write one testable control and an owner for each.
- Write the three steps you would take in the first hour if it produced a harmful output.
Key takeaways
- AI adds quality, fairness, privacy, security, compliance, dependency, cost, adoption and reputation risks.
- Keep an AI risk register with scores, testable controls, owners and review dates.
- Apply responsible AI principles as concrete delivery practices, and scale approvals to risk.
- Prepare incident response, including a kill switch and rollback, before launch.
Key terms
- Risk register
- A list of risks with likelihood, impact, controls, owner and review date.
- Responsible AI
- Developing and using AI in ways that are fair, transparent, accountable, private, safe and overseen by people.
- Kill switch
- A quick way to turn off or roll back an AI feature when it misbehaves.
- Blameless review
- An incident review focused on improving systems rather than blaming individuals.
Check your understanding
Answer all 5 questions. Score 3 or more to complete the lesson.
1. Which is a testable risk control?
2. An AI feature affects who gets shortlisted for jobs. What level of approval does it need?
3. What is the first priority when an AI feature starts producing harmful outputs?
4. How do you check fairness before launch?
5. Why add the failing case to the evaluation set after an incident?