Playwright end-to-end tests agents don't make flaky

# AGENTS.md: Playwright end-to-end tests

End-to-end tests with Playwright Test (`@playwright/test`). These rules apply when you write tests by hand and when
you use Playwright's own test agents (planner, generator, healer from `npx playwright init-agents`).

## Commands
- Run: `npx playwright test`. One file: `npx playwright test tests/todo.spec.ts`. List without running:
  `npx playwright test --list`.
- Debug a failure: `npx playwright show-trace test-results/<test>/trace.zip`. Do not guess from the error alone.
- Before finishing, run the `write-e2e-test` skill's linter (`scripts/lint_specs.mjs` in the skill folder) over the
  test folder. It must report no findings.

## Writing tests
- Locate by what the user sees: `getByRole` with a name first, then `getByLabel`, `getByPlaceholder`, `getByText`.
  `getByTestId` when there is nothing better. No CSS chains, XPath or `nth-child`.
- Web-first assertions only: `await expect(locator).toBeVisible()`, `toHaveText`, `toHaveCount`, `toHaveURL`.
  Never `expect(await locator.isVisible()).toBe(true)`: it does not retry.
- No fixed waits: no `page.waitForTimeout`, no `waitForLoadState("networkidle")`. Wait for the thing you need.
- No `force: true` on clicks. If an element is covered, that is a bug or a wrong locator.
- Each test sets up its own state and runs alone and in any order. Share setup with fixtures or `beforeEach`, and
  authentication with a setup project and `storageState`.
- Never commit `test.only`. Use `test.skip` with a reason, or `test.fixme` and an issue link.
- Test data and accounts are test-only. Never use production credentials; read secrets from the environment.

## Healing failing tests
- First decide whether the test or the app is wrong. Read the trace. If the app regressed, report it; do not change
  the test to match a bug.
- Never make a test pass by deleting assertions, loosening them to `toBeTruthy`, adding timeouts or retries, or
  skipping it without saying so.
- Locator changed because the UI changed on purpose: update the locator to a role-based one and say what changed.

## Configuration
- `retries` only in CI (`process.env.CI ? 2 : 0`), with `trace: "on-first-retry"`.
- `webServer` starts the app; do not assume a server is already running.
- Ask before adding browsers, projects or dependencies. Pin `@playwright/test` to an exact version.

# CLAUDE.md

@AGENTS.md

## Claude Code notes
- Skills from this loadout live in `.claude/skills/`: `write-e2e-test`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From the Agent Loadout "Playwright end-to-end tests agents don't make flaky" v1.0.0 — https://loadout.jnw-it.com/loadouts/playwright-e2e-tests/ -->

# GEMINI.md

@./AGENTS.md

## Gemini CLI notes
- Skills from this loadout live in `.agents/skills/`: `write-e2e-test`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From the Agent Loadout "Playwright end-to-end tests agents don't make flaky" v1.0.0 — https://loadout.jnw-it.com/loadouts/playwright-e2e-tests/ -->

---
name: write-e2e-test
description: Write or repair a Playwright end-to-end test with role-based locators, web-first assertions and no fixed waits, then lint it and run it. Use for new user flows and for failing or flaky tests.
---

# Write or repair an end-to-end test

Example: `references/todo.spec.ts`.

## New test
1. Write the flow as steps a user takes and what they should see. One behaviour per test.
2. Open the page and find each element the way a user would: `page.getByRole("button", { name: "Add" })`.
   If you cannot find a good role or label, ask whether the app should have one before reaching for a test id.
3. Assert with web-first assertions after every meaningful step (`toBeVisible`, `toHaveText`, `toHaveCount`,
   `toHaveURL`).
4. Set up state inside the test or a fixture. Do not depend on another test having run.
5. Lint: `node scripts/lint_specs.mjs tests`. Fix every finding.
6. Run it three times in a row (`npx playwright test <file> --repeat-each 3`). Flaky means wrong, not unlucky.

## Failing or flaky test
1. Run it once with tracing: `npx playwright test <file> --trace on`, then open the trace.
2. Decide: is the app wrong, or the test? Write down which and why.
3. App wrong: stop and report the regression with the trace path. Do not edit the test.
4. Test wrong: fix the locator or the setup. Never add sleeps, never loosen assertions, never add `force: true`.
5. Lint and run three times as above.

import { expect, test, type Page } from "@playwright/test";

async function addTodo(page: Page, text: string) {
  await page.getByRole("textbox", { name: "New todo" }).fill(text);
  await page.getByRole("button", { name: "Add" }).click();
}

test.beforeEach(async ({ page }) => {
  await page.goto("/");
  await expect(page.getByRole("heading", { name: "Todos" })).toBeVisible();
});

test("adds a todo and updates the counter", async ({ page }) => {
  await addTodo(page, "Buy milk");
  await expect(page.getByRole("listitem")).toHaveText(["Buy milk"]);
  await expect(page.getByRole("status")).toHaveText("1 item left");
});

test("completing a todo moves it out of the active count", async ({ page }) => {
  await addTodo(page, "Write tests");
  await addTodo(page, "Ship it");
  await page.getByRole("checkbox", { name: "Write tests" }).check();
  await expect(page.getByRole("status")).toHaveText("1 item left");
  await expect(page.getByRole("checkbox", { name: "Write tests" })).toBeChecked();
});

test("rejects an empty todo with a visible message", async ({ page }) => {
  await page.getByRole("button", { name: "Add" }).click();
  await expect(page.getByRole("alert")).toHaveText("Write something first.");
  await expect(page.getByRole("listitem")).toHaveCount(0);
});

#!/usr/bin/env node
// Flags Playwright test anti-patterns: fixed waits, non-retrying assertions, brittle selectors, force clicks, test.only.
// Usage: node lint_specs.mjs <dir-or-file> [...]. Exit 1 on any finding. Reads files only.
import { readdirSync, readFileSync, statSync } from "node:fs";
import { join } from "node:path";

const RULES = [
  [/\bwaitForTimeout\s*\(/, "fixed wait: wait for a locator or assertion instead"],
  [/waitForLoadState\(\s*["']networkidle["']/, "networkidle is discouraged: wait for the element you need"],
  [/expect\(\s*await\s+.*\.(?:isVisible|isHidden|isChecked|isEnabled|textContent|innerText|count)\(\)\s*\)/, "non-retrying assertion: use a web-first assertion such as toBeVisible() or toHaveText()"],
  [/\bforce\s*:\s*true/, "force: true hides real problems: fix the locator or the app"],
  [/\btest\.only\(|\bdescribe\.only\(/, "test.only must not be committed"],
  [/\bpage\.\$\$?\(|\bpage\.\$eval\(/, "page.$ is the old API: use page.locator or getByRole"],
  [/locator\(\s*["'`][^"'`]*(?::nth-child|:nth-of-type|\/\/|>>|\.[a-z][\w-]*\s+\.[a-z])/i, "brittle selector: prefer getByRole, getByLabel or getByTestId"],
  [/\bsetTimeout\s*\(/, "setTimeout in a test: wait for state instead"],
  [/\.toBeTruthy\(\)/, "toBeTruthy proves little: assert the actual value or state"],
];

function files(p, out = []) {
  if (statSync(p).isDirectory()) {
    for (const n of readdirSync(p)) if (n !== "node_modules") files(join(p, n), out);
  } else if (/\.(?:spec|test)\.[cm]?[jt]sx?$/.test(p)) out.push(p);
  return out;
}

const targets = process.argv.slice(2);
if (!targets.length) {
  console.error("usage: node lint_specs.mjs <dir-or-file> [...]");
  process.exit(2);
}
const findings = [];
const all = targets.flatMap((t) => files(t));
for (const f of all) {
  readFileSync(f, "utf8").split("\n").forEach((line, i) => {
    for (const [re, msg] of RULES) if (re.test(line)) findings.push(`${f}:${i + 1}: ${msg}`);
  });
}
for (const x of findings) console.log(x);
console.log(findings.length ? `${findings.length} finding(s) in ${all.length} file(s)` : `ok: ${all.length} spec file(s), no findings`);
process.exit(findings.length ? 1 : 0);