AI feature readiness checklist

Checklist progress is saved in this browser on this device.

Use this checklist when evaluating a generative or predictive AI feature. It is informed by the NIST AI Risk Management Framework and human-AI interaction guidance. Risk depends on context, data, affected people, and the consequences of an error.

Define the job and alternatives

Describe the user problem and the outcome the feature should improve.
Compare AI with a simpler interface, search, rules, or a manual workflow.
Define where AI output is advisory and where a person remains responsible.
Identify affected users, foreseeable misuse, and potential harms.
Set limits for the feature, including tasks it must refuse or hand off.

Data and provider review

Identify data sent to the model, provider, logs, and downstream systems.
Confirm rights, permissions, retention, residency, and training-use terms.
Minimize personal or confidential data in prompts and stored outputs.
Review access controls, secrets, prompt injection, and tool permissions.
Document the model/provider version and the fallback behavior if unavailable.

Evaluation and user experience

Create representative evaluation cases, including edge cases and adversarial inputs.
Define quality, safety, latency, and cost thresholds before release.
Test factual errors, bias, privacy leakage, unsafe output, and overreliance.
Tell users when output is AI-generated and explain uncertainty and limitations.
Provide edit, retry, correction, report, or human escalation paths.
Test with the people who will use the feature and review accessibility.

Operations and governance

Assign owners for monitoring, incident response, model changes, and user reports.
Log enough to investigate quality without retaining unnecessary sensitive data.
Re-run evaluations when prompts, models, data, tools, or user populations change.
Define rollback and disable controls and communicate material changes.
Record remaining risks, approvals, and the date for the next review.

References