Nishaglobal Education logo

Managing AI risk and responsible AI

The risks specific to AI projects and how to manage them: an AI risk register, scoring likelihood and impact, responsible AI principles (fairness, transparency, accountability, privacy, safety), approvals proportional to risk, and incident response when AI goes wrong.

Lesson 30 of 4417 min readQuiz: 5 questions

🎯 By the end of this lesson you will be able to:

  • Identify the main categories of AI risk and add them to a risk register
  • Apply responsible AI principles in day-to-day delivery decisions
  • Prepare for AI incidents with clear response steps
In this lesson
  1. The risks AI adds
  2. An AI risk register
  3. Responsible AI in practice
  4. Approvals proportional to risk
  5. When AI goes wrong: incident response
  6. What this means for your role
  7. Key takeaways
  8. Quiz

The risks AI adds

AI projects carry the usual risks of scope, schedule and budget, plus some new ones. A delivery manager doesn't need to solve them all personally, but must make sure each is identified, owned and managed.

Risk categoryExample
QualityConfident wrong answers mislead customers or staff
Fairness and biasWorse results for some languages, regions or groups
Privacy and dataPersonal data sent to the wrong place or kept too long
SecurityPrompt injection causes data leaks or unwanted actions
Legal and complianceUse falls under regulation without required controls
DependencyProvider outage, price change or model retirement
CostUsage grows faster than budget
Adoption and peopleUsers over-trust or ignore the AI; roles change without support
ReputationA harmful output becomes public

An AI risk register

HighMediumLowLowMediumHighLikelihoodImpactR1 wrong refundsR2 model retiredunfair resultsslow first reply
Score each risk by likelihood and impact. High-impact risks need strong controls even when they seem unlikely.

Risk register entries

R1 · Quality · Assistant gives wrong refund amounts · Likelihood: medium · Impact: high
  Controls: amounts come from the order system, not the model; evaluation case set for refunds; human approval for refunds > $100
  Owner: Product owner · Review: every release

R2 · Dependency · Primary model retired with 6 months' notice · Likelihood: high · Impact: medium
  Controls: gateway with fallback model; evaluation set ready for migration; deprecation notices monitored
  Owner: Tech lead · Review: quarterly

Each risk has a cause, a score, specific controls, an owner and a review date.

Responsible AI in practice

PrincipleWhat it means for delivery
FairnessEvaluate quality per group, language and region; fix gaps before launch
TransparencyTell users when AI is involved and what its limits are
AccountabilityA named person owns each AI system and its decisions
PrivacyMinimise data, respect retention, honour user rights
Safety and securityRed team, guardrails, limits on actions, human approval where needed
Human oversightPeople can review, override and appeal AI-influenced decisions

Approvals proportional to risk

  • Low risk (internal drafting, summarising public content): standard checks, approved tools, light documentation.
  • Medium risk (customer-facing answers, internal decisions support): evaluation evidence, guardrails, security and privacy review, monitoring plan.
  • High risk (affects people's jobs, credit, health, safety or rights): full risk assessment, fairness testing, human oversight, legal review, formal sign-off and regular audits.

When AI goes wrong: incident response

  1. Detect: monitoring alerts, user reports, the "report a problem" button.
  2. Contain: switch off the feature, roll back the prompt or model, or tighten guardrails. Have this "kill switch" ready before launch.
  3. Assess: who was affected, what data or decisions were involved, and whether legal or regulatory notification is needed.
  4. Fix: root cause, add the failure to the evaluation set, verify the fix.
  5. Learn: blameless review, update the risk register and controls.

What this means for your role

Developer

Build kill switches, feature flags, versioned prompts and action limits so risks can be controlled and incidents contained quickly.

Tester / QA

Turn each risk into tests: fairness slices, adversarial cases and checks that controls such as approvals actually work.

Business Analyst / PO

Identify affected people and decisions for each AI feature; that sets the risk level and required oversight.

Tech Lead / Architect

Design technical controls for each major risk and make sure they are testable and monitored.

Project / Delivery Manager

Own the AI risk register, get each risk an owner and review date, scale approvals to risk, and make sure an incident plan exists before launch.

Scrum Master / Agile

Run blameless incident reviews and help the team bring risk items into normal backlog work rather than a separate checklist.

Try it: start an AI risk register

  1. Pick an AI feature and list one risk for at least five categories from the table.
  2. Score each for likelihood and impact, and write one testable control and an owner for each.
  3. Write the three steps you would take in the first hour if it produced a harmful output.

Key takeaways

  • AI adds quality, fairness, privacy, security, compliance, dependency, cost, adoption and reputation risks.
  • Keep an AI risk register with scores, testable controls, owners and review dates.
  • Apply responsible AI principles as concrete delivery practices, and scale approvals to risk.
  • Prepare incident response, including a kill switch and rollback, before launch.

Key terms

Risk register
A list of risks with likelihood, impact, controls, owner and review date.
Responsible AI
Developing and using AI in ways that are fair, transparent, accountable, private, safe and overseen by people.
Kill switch
A quick way to turn off or roll back an AI feature when it misbehaves.
Blameless review
An incident review focused on improving systems rather than blaming individuals.

Check your understanding

Answer all 5 questions. Score 3 or more to complete the lesson.

  1. 1. Which is a testable risk control?

  2. 2. An AI feature affects who gets shortlisted for jobs. What level of approval does it need?

  3. 3. What is the first priority when an AI feature starts producing harmful outputs?

  4. 4. How do you check fairness before launch?

  5. 5. Why add the failing case to the evaluation set after an incident?

Answer all 5 questions to check.

Practise more