CI/CD

Playwright in GitHub Actions: PR gating that holds

A Playwright job that runs on every pull request is easy. One that merges fast, fails only for real reasons, and hands you a debuggable trace when it does — that takes deliberate configuration. Here is the setup: a baseline workflow, caching, sharding, failure artifacts, and the path to checks that genuinely block merges.

Pratik Rana 10 min read

01The baseline workflow

Start with one workflow that runs on pull requests and on pushes to main. The version below already includes the three decisions most teams retrofit later: matrix sharding, blob-report merging, and artifact upload scoped to failures. We’ll unpack each in the following sections.

name: e2e
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm

      - run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run tests (shard)
        run: npx playwright test --shard=${{ matrix.shard }}/4 --reporter=blob

      - name: Upload report
        uses: actions/upload-artifact@v4
        if: ${{ always() }}
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report/
          retention-days: 7

  merge-reports:
    if: ${{ always() }}
    needs: [test]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-reports
          merge-multiple: true
      - run: npx playwright merge-reports --reporter=html ./all-reports
      - uses: actions/upload-artifact@v4
        if: ${{ always() }}
        with:
          name: html-report
          path: playwright-report/
          retention-days: 7

Note fail-fast: false: with it unset, one flaky shard cancels the others mid-run and you lose half your evidence. The tradeoff is paying for full execution even after an early failure — worth it, because parallel evidence from the other shards often contains the trace that explains the first one.

02Cache what’s slow

Two things dominate cold-start time: dependency installation and browser binaries. The cache: npm input above handles dependencies by hashing the lockfile. Browsers need explicit handling — roughly 130MB per browser plus system deps, typically 30–60 seconds of install per run:

- uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: pw-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

Keying on the lockfile works because Playwright versions are pinned there; bump the version, the key changes, the cache rebuilds once, and every subsequent run restores instantly. Keep npx playwright install --with-deps chromium in the workflow regardless — cache hits make it near-instant, misses make it correct. Only install browsers you actually use; installing all three engines triples download time for zero benefit.

03Shard until PRs wait minutes, not quarters of an hour

Sharding splits the suite across matrix jobs. Pick the target wall-clock time first, then derive shard count: a 36-minute serial suite across 4 shards lands around 9–10 minutes including overhead. Playwright balances shards by test count; if durations vary wildly, sort specs into size buckets or split known-slow files so no single shard dominates. Watch for two scaling ceilings: shared backend contention (parallel shards hammering one database turn into a different kind of red), and per-shard fixed costs (checkout, install, boot) that set a floor around 3–5 minutes no matter how few tests each shard holds.

If sharding exposes order dependence — tests failing only under some shard arrangements — fix the dependence, not the arrangement. That class of bug is covered in the isolation section of our E2E practices guide.

04Traces and artifacts on failure

Configure capture in playwright.config.ts so every CI failure arrives debuggable:

export default defineConfig({
  retries: process.env.CI ? 1 : 0,
  use: {
    trace: 'retain-on-failure',
    video: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
});

With retries: 1, a transient environmental hiccup absorbs itself while the trace records both attempts — you keep velocity without losing signal, provided someone reviews what needed retrying (see the flake policy below). Open any captured trace with npx playwright show-trace trace.zip or upload it as an artifact as the baseline workflow does. What separates a two-minute diagnosis from an hour-long one is usually just this: the DOM, network waterfall, and console at the moment of failure. Molar extends the same idea with deterministic replay — DOM, network, console, and clone state side by side — but the principle is identical; our Trace comparison covers the differences. Keep retention-days short (7) on high-churn PR artifacts; storage costs compound quietly.

05Make the check required — carefully

A green-required gate starts with naming. Branch protection references check names exactly, and a matrix job produces one check per shard named like e2e / test (1), (2), and so on. Require either each shard explicitly or add a final aggregate job whose success means “all shards passed” — the latter survives future shard-count changes without editing protection rules. The exact configuration steps are in our required status checks guide, and the GitHub Actions integration page shows how Guard reports its own verdicts as native check runs.

Why gating discipline matters more every year: GitHub reported that 46% of code in enabled repositories is now AI-generated (GitHub, 2026). When humans write less of the diff by hand, machine-verified gates carry proportionally more of the safety burden — a gate that can be trained-around is a liability at exactly the moment usage patterns are drifting that way.

Concurrency saves real money on busy repos. Add concurrency: { group: e2e-${{ github.ref }}, cancel-in-progress: true } so new pushes cancel superseded runs instead of queueing behind them.

06Keep the gate honest as the suite grows

CI quality decays through retries, not failures. Cap retries at one in CI, review which tests consume them weekly, and quarantine repeat offenders out of the blocking set with an owner and expiry — the full policy lives in our flaky-tests guide. Run the complete suite nightly rather than forcing every edge case onto PRs; a fast required smoke on every PR plus a thorough scheduled run catches more regressions per minute of engineer attention than a monolithic blocking suite ever will. This tiered shape is the core of continuous testing in the CI/CD pipeline.

The end state to aim for: every pull request gets sub-10-minute E2E feedback it can trust, every failure arrives with enough evidence to diagnose without a local reproduction, and the merge gate stays green for exactly one reason — the app actually works.

Gates that hold, evidence that ships

Molar runs your scenarios in hermetic clones, reports pass, fail, and flaky as distinct check states, and attaches full replay evidence to every failure.