agentic-browser-testing

Dieser Auszug wurde bei der Prüfung gespeichert. Die externe Quelle enthält die vollständige und aktuelle Version.

---
name: agentic-browser-testing
description: >-
  Goal-driven E2E testing where a browser agent (Playwright MCP / computer-use) reads a
  natural-language goal and explores the app via the accessibility tree to assert outcomes —
  no pre-written script. Covers when intent-driven beats scripted, making agent runs
  deterministic (pinned model, temperature 0, seeded data, bounded steps, explicit success
  assertion, snapshot-not-pixel), cost/latency control, the accessibility-tree-first
  interaction model, CI gating, and graduating a stable run into a scripted Playwright test.
  Use when: "agentic browser test," "goal-driven browser test," "let an agent explore the app,"
  "natural-language E2E," "browser agent smoke test," "Playwright MCP test."
  Not for: Writing/maintaining deterministic scripted Playwright tests — that is
  playwright-automation. Testing your product's OWN LLM features — that is ai-system-testing.
  Related: playwright-automation, ai-system-testing, exploratory-testing, test-reliability, qa-project-context.
license: MIT
metadata:
  author: kindlmann
  version: "1.0"
  category: ai-qa
---

<objective>
A scripted Playwright test breaks the moment a button moves or a class renames; writing one
for a dashboard that changes weekly is a maintenance treadmill. This skill stands up a
goal-driven browser agent instead: it reads a natural-language goal, explores the app via the
accessibility tree (Playwright MCP `browser_snapshot`), and asserts the outcome against an
explicit oracle. The failure mode it prevents is the one that makes teams distrust agents — an
agent that reports "success" while stuck on the login page because nothing forced it to prove
where it landed. You leave with a deterministic, CI-gated agent run and a graduation path to a
durable scripted test once the flow stabilizes.
</objective>

## Quick Route

| Situation | Go to |
|-----------|-------|
| Stand up a goal-driven run from scratch | Discovery + `references/setup.md` |
| Decide agentic vs scripted for a given flow | Fit: Intent-Driven vs Scripted |
| Agent passes one run, fails the next | Determinism |
| "How does it click without screenshots?" | Interaction Model |
| Runs are slow / burning tokens | Cost and Latency |
| Agent reports false success | Success Assertion (the Oracle) |
| Flow is stable — make it permanent | Graduation → `references/graduation-and-ci.md` |
| Block a merge on the goal | CI Gating → `references/graduation-and-ci.md` |
| Canvas / no accessibility tree | Canvas Fallback → `references/graduation-and-ci.md` |

## Discovery Questions

First, check `.agents/qa-project-context.md` in the project root and skip anything it already
answers (stack, environments, seed/reset tooling, model access).

1. **Which flow, and how often does its UI change?** Fast-changing/experimental UI favors
   intent-driven; a stable critical path (login) favors scripted. This decides the whole approach.
2. **Is there a seeded fixture and a way to reset state?** Determinism is impossible without
   seeded data and a per-run reset. If neither exists, that is step zero.
3. **Can you deep-link past auth to a seeded entry point?** Re-driving login every run is the
   biggest avoidable cost; a seeded entry URL scopes the goal and cuts steps.
4. **What is the unambiguous success oracle?** Specific account text, a `/dashboard` URL, an
   order number — plus a forbidden state. "No error" is not an oracle.
5. **Does the target render to canvas / WebGL?** No accessibility tree means snapshot-first
   won't work; plan the vision fallback or instrument the canvas with ARIA.
6. **Which model and budget?** Pin a model id and a step budget up front; tier cheap steps to
   Haiku 4.5 / Sonnet 4.6 and reserve Opus 4.8 for genuinely ambiguous flows.

---

## Core Principles

1. **Intent, not instructions — but only where churn earns it.** The agent reads a goal and
   finds its own path through the accessibility tree, so it survives a moved button or renamed
   cl