# AI testing evaluation checklist

## Define the job

- [ ] Identify whether AI is the testing method, the assessment target, or both.
- [ ] Name the application, build, roles, workflows, and expected security rules.
- [ ] Separate documented capabilities from behavior demonstrated in the trial.

## Bound execution

- [ ] Record authorized hosts, ports, identities, time window, and action types.
- [ ] Identify the controls that enforce scope outside model instructions.
- [ ] Check how redirects and links to unapproved hosts are handled.
- [ ] Restrict credentials and confirm the operator can revoke them.
- [ ] Define request limits, stop conditions, and the escalation contact.
- [ ] Preserve rejected actions and reasons in the audit record.
- [ ] Check that target content cannot authorize a scope change.

## Run a controlled evaluation

- [ ] Use an isolated or authorized staging environment with synthetic data.
- [ ] Record known permitted and forbidden actions in a separate answer key.
- [ ] Include an expired-session case and an unreachable route.
- [ ] Preserve configuration, target build, timestamps, and available version information.
- [ ] Repeat selected checks and explain changed conditions.
- [ ] Separate not tested, tool failure, denied action, and confirmed application behavior.

## Review the evidence

- [ ] Trace a claim from hypothesis to executed action to target response.
- [ ] Identify the account role and the protected object without exposing secrets.
- [ ] Confirm returned content rather than relying on a status code alone.
- [ ] Reproduce accepted findings within the agreed scope.
- [ ] Preserve observed impact separately from unverified wider consequences.
- [ ] Record review effort and unsupported claims as well as run time.

## Make the operating decision

- [ ] Assign unassessed workflows to another method or an explicit follow-up.
- [ ] Have the engineering owner define repair acceptance criteria.
- [ ] Retest the allowed and denied behaviors after the change.
- [ ] Record the accepted use case, exclusions, operating owner, and next review date.