AI testing evaluation checklist ContentsDefine the jobBound executionRun a controlled evaluationReview the evidenceMake the operating decisionDefine the job Identify whether AI is the testing method, the assessment target, or both. Name the application, build, roles, workflows, and expected security rules. Separate documented capabilities from behavior demonstrated in the trial. Bound execution Record authorized hosts, ports, identities, time window, and action types. Identify the controls that enforce scope outside model instructions. Check how redirects and links to unapproved hosts are handled. Restrict credentials and confirm the operator can revoke them. Define request limits, stop conditions, and the escalation contact. Preserve rejected actions and reasons in the audit record. Check that target content cannot authorize a scope change. Run a controlled evaluation Use an isolated or authorized staging environment with synthetic data. Record known permitted and forbidden actions in a separate answer key. Include an expired-session case and an unreachable route. Preserve configuration, target build, timestamps, and available version information. Repeat selected checks and explain changed conditions. Separate not tested, tool failure, denied action, and confirmed application behavior. Review the evidence Trace a claim from hypothesis to executed action to target response. Identify the account role and the protected object without exposing secrets. Confirm returned content rather than relying on a status code alone. Reproduce accepted findings within the agreed scope. Preserve observed impact separately from unverified wider consequences. Record review effort and unsupported claims as well as run time. Make the operating decision Assign unassessed workflows to another method or an explicit follow-up. Have the engineering owner define repair acceptance criteria. Retest the allowed and denied behaviors after the change. Record the accepted use case, exclusions, operating owner, and next review date.