# `Playwright.Test`
[🔗](https://github.com/ftes/playwright_ex/blob/v1.65.0-rc.3/lib/playwright/test.ex#L1)

Playwright Test's per-test machinery on ExUnit: each test's result, its artifacts (traces,
screenshots, videos, `error-context.md`), Playwright Test's context defaults, soft errors,
snapshot and output paths, and steps. The browser comes from `Playwright.Test.BrowserPool`,
contexts and pages from the plain API; `Playwright.Case` puts them together.

    # config/test.exs
    config :playwright, :test,
      trace: :on_first_retry,       # traced under `mix test --failed`
      screenshot: :only_on_failure,
      base_url: "http://localhost:4000"

    defmodule MyApp.CheckoutTest do
      use ExUnit.Case, async: true
      use Playwright.Test

      alias Playwright.{Browser, BrowserContext, Page}

      setup_all do
        %{browser: Playwright.Test.BrowserPool.browser()}
      end

      setup %{browser: browser} do
        %{page: BrowserContext.new_page!(Browser.new_context!(browser))}
      end

      test "pays", %{page: page} do
        ...
      end
    end

The runner is native, not a port of Playwright Test: it follows the runners of the official
non-JS clients (pytest-playwright, Playwright.NUnit, Java's `@UsePlaywright`) on ExUnit, with
Playwright Test's names and defaults where they exist (`packages/playwright/src/index.ts`).
What the ported matchers and traces need of Playwright Test's worker is ported: testInfo's
snapshot paths and errors (`testInfo.ts`), `step/3` (`test.step`) and the `test.trace` of
`trace.zip` (`testTracing.ts`).

## Registration

`use Playwright.Test` comes after `use ExUnit.Case` (or in a case template's `using`). It
imports a `test` macro, which hands the test to `ExUnit.Case.test/3` with the body's end
marked, so the test's `on_exit` learns whether it passed, and `assert/1,2` and `refute/1,2`,
which are also steps (see "Assertions"). And it adds a `setup` that
registers each test: its recording modes, its output dir and an `on_exit` (the first, so it
runs last) that keeps or drops the artifacts by the result, then releases the test's
`BrowserPool.browser/1` drivers. Tests of other macros (doctests, a `property`) count as
failed for their artifacts, and a failure only in an `on_exit` callback counts as passed.

In a case template, `use Playwright.Test` may also be the first line of the template's
body: a template's own `setup` callbacks run before those its `using` brings in, so the test
is registered before them, and their contexts are recorded too. That `use` takes no options
(they go in `using`); a module's later `use`s (a template's `using` and the module's own) only
add their options (see "Artifacts"). With the `use` only in the template's body, the
`setup_all` timeout (see "Timeouts") covers only the template's `setup_all` callbacks. A
context that a test process creates, or a `BrowserPool.browser/1` call it makes, before its
registration (a template's `setup` with no `use` in the body) makes the registration raise,
with that fix. It registers the module's `setup_all` process too, as upstream runs its
`beforeAll` hooks with its `all-hooks-included` fixtures: see "Contexts of `setup_all`". A
`setup_all` runs in that process, not a test's, so one that creates a context or calls
`BrowserPool.browser/1` before the registration (a template's own `setup_all`, or one above
the module's `use`) makes the module's `setup_all` raise, with the fix, and fails all its tests. A context created outside
both (in an `on_exit` callback, in a module without `use Playwright.Test`, in a process with
neither in its `$callers`, on a driver you started yourself) gets no defaults and no recording.

The rest is called qualified, as upstream's `test.info()` and `test.step()`: `info/0`
(`info/1` in `on_exit`, with the test's result), `output_path/1`, `attach/2`, `snapshot_path/2`, `step/2,3` and `step_skip/2,3`. They work in the test's
process and in processes with it in `$callers` (a `Task`, and the client's callback
processes, see "Route handlers and listeners" in the Testing with ExUnit guide); outside a
test they raise. `info/0`, `step/2,3` and `step_skip/2,3` also work in the module's
`setup_all`: `info/0` with the module's fields (see "Contexts of `setup_all`"), the steps
unrecorded (see "Steps").

An error a listener or route handler of the test raises fails the test and stops it at once,
as upstream's worker does (`unhandledError`): the test's process exits with the error, and
later such errors are dropped. A test that traps exits isn't stopped: ExUnit may report it
passed, while the error is in `error-context.md` and `Playwright.Test.info/1`. A test
defined with `ExUnit.Case.test` isn't a Playwright test: its listener errors are only logged.
Two deviations: the stopped test's failure screenshots are taken at teardown, as its contexts
close, after its `on_exit` callbacks may have run, where upstream takes them first
(`_didFinishTestFunction`); and once the body ended (in `on_exit`), such an error is logged,
where upstream still fails the test during its `afterEach` hooks and teardown.

## Artifacts

Every context the test creates on a `BrowserPool` browser, in the test or a process with it
in `$callers`, is recorded. The recording starts with the test's first context (or at its
registration when it traces, for its steps); a test without contexts records nothing.
Contexts left open are closed after the test, last in its `on_exit`, and request contexts of
`Playwright.APIRequest.new_context/2` left are disposed; a context closed earlier keeps its
artifacts. Each test writes to its own dir under `output_dir`, named as upstream's
`testInfo.outputDir` (`checkout-pays-1a2b3` for the test "pays" in `test/checkout_test.exs`,
the suffix a hash of the module and its `parameterize:` parameters):

  * `trace.zip` - one trace of all its contexts, their `request`s and its request contexts
    (`Playwright.APIRequest.new_context/2`), recorded with screenshots, snapshots and
    sources unless the trace's options say otherwise. A failed test that keeps one prints
    `mix playwright.cli show-trace <path>` after the run, below ExUnit's summary.
  * `test-finished-N.png`, or `test-failed-N.png` for a failed test - a screenshot of every
    page open at the end of the test's body (before its `on_exit` callbacks), then of those
    open when their context closes (`timeout: 5000, caret: :initial`, as upstream). A test
    that a listener's or route handler's error stopped (above) has no end of its body: its
    pages' screenshots are taken as its contexts close, after its `on_exit` callbacks
    (upstream takes them before the `afterEach` hooks).
  * `video.webm`, `video-1.webm`, ... - every page's video. A context created with its own
    `record_video` keeps it, and its video is copied here too.
  * `error-context.md` - a failed test's errors, source and an aria snapshot (`mode: :ai`) of
    the first page of the first context closed after the failure (upstream's `didFinishTest`
    and `buildErrorContext`), whatever the modes.

The modes are config, the project's over it (`parameters/1`), the module's `use` options over
those, and `use/1` in tags over those (a test's `@tag` over its `@describetag` over the
module's `@moduletag`): `trace` and `video` `:off`
(default), `:on`, `:retain_on_failure` or a retry mode, `screenshot` `:off`, `:on`,
`:only_on_failure` or `:on_first_failure` (upstream's deprecated `:retry_with_trace` and
`:retry_with_video` are `:on_first_retry`; another value records nothing, as upstream's). Each
is also the `:mode` of a keyword list (or map) of options, as upstream's object forms, `:off`
without one: `trace:
[mode: :on, snapshots: false]`, `screenshot: [mode: :on, full_page: true]`, `video: [mode:
:on, size: %{width: 640, height: 480}]`. The trace's `:snapshots` (a boolean or `%{dom: _, aria: _, screen: _}`),
`:screenshots` and `:sources` are `Playwright.Tracing.start/2`'s, `true` by default, as is
`:attachments` (those of `attach/2` in `trace.zip`); the
screenshot's `:full_page` and `:omit_background` are `Playwright.Page.screenshot/2`'s; the
video's `:size` and `:fps` are `record_video`'s, and its `:show` is upstream's map:
`%{actions: _}` is `record_video`'s `showActions`, `%{test: %{level: _, position: _, fontSize:
_}}` overlays the test's title and its steps' (`level` `"file"`, `"title"` or `"step"`) on
each page of its contexts. With it, don't call `step/2,3` in a route handler that holds a
page's navigation (a document request): the step's overlay waits for that navigation, so the
test hangs until its timeout, as upstream's does.

`use Playwright.Test, trace: :on` is upstream's file-level `test.use({ trace: 'on' })` (the
modes are worker-scoped options there), with the values the config takes; an unknown option (or a
context option the facade rejects) raises at compile time. A module's `use`s merge per key, the later winning
(see "Context defaults").
A template's own `use Playwright.Test, ...` lines go before the forwarded options, which win per
option, as a test file's `test.use` beats a project's device.
ExUnit.Case's keys (`:async`, ...) are ignored, so a template forwards its options from `using
opts` as is: `use Playwright.Test, unquote(opts)`, as `Playwright.Case` does. A template
with options of its own pops them first (`{sandbox, opts} = Keyword.pop(opts, :sandbox)`),
as they raise here. Tags named `trace`, `screenshot` or `video` set no mode: only `use/1`'s
keys are read.

The `*_on_failure` modes record every test and keep the files of failed ones, as upstream
(files you write to `output_path/1` stay). A failing artifact step is logged; the test's
result stands.

The retry modes go by the test's `retry` (see below), as upstream's: `:on_first_retry`
records retry 1, `:on_all_retries` every retry, `:retain_on_first_failure` records the first
run and keeps a failure's, `:retain_on_failure_and_retries` records every run and keeps a
failure's and every retry's; the screenshot's `:on_first_failure` is `:only_on_failure` of
the first run.

A test's dir is deleted before it runs, as upstream's, so no file of an earlier run is left
in it. ExUnit has no retries in a run: a test under `mix test --failed` is upstream's retry
(`info/0`'s `retry`, 1), and writes to `<dir>-retry1`, next to the dir of its first run. So
with `trace: :on_first_retry`,
`mix test || { [ $? -eq 2 ] && mix test --failed; }` traces the failures' rerun and keeps the
first run's files (exit status 2 means tests failed; anything else isn't retried). A run
without `--failed` deletes the test's `-retry1` dir too, where
upstream deletes all of `output_dir` at the start of a run: here the dirs of tests that didn't
run stay. Config `preserve_output` is upstream's `preserveOutput`: `:always` (default) keeps
the files at the end of each test, `:failures_only` only a failed test's, `:never` none.

## Context defaults

Every context and request context a test (or its module's `setup_all`) creates on a
`BrowserPool` browser (`Playwright.Browser.new_context/2`,
`Playwright.BrowserType.launch_persistent_context/3`, `Playwright.APIRequest.new_context/2`) gets Playwright Test's context defaults (`locale:
"en-US"`, a 1280x720 viewport, `color_scheme: :light`, ...), then the config's context options
over them, then the project's, the module's, the describe's and the test's (below), for each
option the call doesn't give, as upstream's `_combinedContextOptions`. `viewport: :null` is
none. An option replaces the one under it as a whole (`extra_http_headers` too, as upstream's
shallow spread).

The context options are options of their own, upstream's option fixtures, with the names
`Playwright.Browser.new_context/2` takes them by: `accept_downloads`, `base_url`,
`bypass_csp`, `client_certificates`, `color_scheme`, `contrast`, `device_scale_factor`,
`extra_http_headers`, `forced_colors`, `geolocation`, `has_touch`, `http_credentials`,
`ignore_https_errors`, `is_mobile`, `java_script_enabled`, `locale`, `offline`, `permissions`,
`proxy`, `reduced_motion`, `screen`, `service_workers`, `storage_state`, `timezone_id`,
`user_agent` and `viewport`. `use Playwright.Test, locale: "de-DE"` (or `use Playwright.Case,
...`) is upstream's file-level `test.use({ locale: 'de-DE' })`: the module's options,
evaluated and validated at compile time (the values as `new_context/2` checks them). A
function raises there: upstream's function-valued options, its option fixtures' functions,
aren't ported. A module's `use`s apply in order, each over those before it per option.

A device is a keyword list of these options and its `default_browser_type` (see "Browser"),
spread into the others as upstream's `...devices['iPhone 13']`: `use Playwright.Test,
Playwright.device("iPhone 13")` after `use Playwright.Case, async: true`, or in one `use`,
`use Playwright.Case, [async: true] ++ Playwright.device("iPhone 13")`. A later key wins, as in
JS's spread, so upstream's `{ ...devices['iPhone 13'], viewport }` is
`Playwright.device("iPhone 13") ++ [viewport: %{width: 800, height: 600}]`.
The options may be a map too.

`context_options` is upstream's `contextOptions`: `new_context/2`'s options, for those that
are no option of their own (`record_har`, `strict_selectors`). It is one option, as upstream's
one fixture: the highest level that sets it replaces the others' as a whole. And it is under
every option of its own, whatever level sets either, as upstream's fixture of an option reads
`contextOptions` only when no level sets the option. `context_options: nil` falls back as any
option's `nil` (below), so does a misspelled `Playwright.devices()["typo"]`, silently: use
`Playwright.device/1`, which raises for an unknown name.

Below the module, `use/1` gives the options to ExUnit as a tag, for a describe block
(`@describetag Test.use(locale: "de-DE")`, upstream's `test.use` in a `test.describe`) or a
test (`@tag`); `@moduletag` works too, the worker options included, as upstream's top-level
`test.use`. A project's
are `parameters/1` in the module's `parameterize:` (upstream's `project.use`).

The layers, each over the one before per option: Playwright Test's defaults, the config's,
the project's, the module's (`use`, then `@moduletag`), the describe's, the test's, then the
call's own. Any option a layer (the config, a project, the module's `use`, a tag) gives as
`nil` is unset there, as upstream's `undefined` option override, so the layer under it gives
it, in the end the default: the config's `nil` falls back to the default, the project's to
the config's, and a module's or a tag's to the project's or the config's, past the module's
own `use`s and tags, as upstream's `test.use` of `undefined` goes back to the config's.
`context_options: nil` is the bag of the layer under. A call's own options are no layer: a
context option's `nil` there is passed as given, over the layers, as upstream's
`newContext({ locale: undefined })` spreads it over the defaults, so
`Browser.new_context!(browser, locale: nil)` gets the client's default, not the layers'.
`BrowserPool.browser(connect_options: nil)` launches; its `browser_name` and
`launch_options` take no `nil`. The config's, the project's and the
module's reach every context the module creates: a test's, those of its `Task`s, and those of
`setup_all`, as upstream's `beforeAll` gets the `use` options. A describe's and a test's reach
the test's contexts only: ExUnit runs `setup_all` without them. The project's options are read
from the parameters ExUnit runs the module with, the others from the test's tags.

## Browser

`BrowserPool.browser/1` without a `browser_name`, as `Playwright.Case` calls it, launches the
module's browser, upstream's `browserName`: the `browser_name` of the module's `use`
(`use Playwright.Case, browser_name: :webkit`), else its project's (`parameters/1`), else the
config's if it sets one. Else their `default_browser_type`, in the same order, upstream's
worker option `defaultBrowserType`, which a device has (`Playwright.device("iPhone 13")` is
WebKit's) and upstream's `browserName` defaults to; else `:chromium`. The module's
`launch_options` (its `use`'s, else its project's, else the config's, each as a whole) and
`connect_options` are that call's defaults too, upstream's `launchOptions` and
`connectOptions`: a project with `launch_options: [channel: "chrome"]` gets a worker of its
own. So are its `headless` and `channel`, upstream's options of their own, set over the launch
options' when given (`headless: false` launches headed whatever `launch_options` say); unset,
they are the launch options' (`headless` `true` when those have none). These six are the
worker's, as upstream's: the module's (its `use`, then all of its
`@moduletag Test.use(...)`s merged, the later per option, upstream's top-level `test.use`, for
every test of the module, also those above such a line) or its project's. Its other options
don't reach a test above the line, as ExUnit gives a test the `@moduletag`s set before it:
under `@moduletag Test.use(Playwright.device("iPhone 13"))`, such a test gets the device's
`default_browser_type` but not its viewport or user agent, so put the line above the tests.
In a `@describetag` or a `@tag` of a test of the `test` macro they raise when the module compiles, as upstream's `test.use` in a describe
when the file loads, a device's `default_browser_type` too; other tests (a doctest, a bare
`ExUnit.Case.test`) get the module's worker regardless. For a browser or a device per run,
make each a project (`parameters/1`) in the module's `parameterize:`.

Where each option can be set:

| Option | Config | Project (`parameters/1`) | Module (`use`, then its `@moduletag use/1`s, merged) | Describe, test (`@describetag`, `@tag use/1`) |
|---|---|---|---|---|
| the context options, `context_options`, `action_timeout`, `navigation_timeout` | yes | yes | yes | yes |
| `trace`, `screenshot`, `video` | yes | yes | yes | yes (an addition: upstream's are the worker's; they work fully per test, as the recorder records per test and touches no browser) |
| `browser_name`, `headless`, `channel`, `launch_options`, `connect_options` | yes | yes | yes | raises when the module compiles a `test` under it, as Playwright Test's `test.use` in a describe |
| `default_browser_type` (a device's) | picks the browser | picks the browser | picks the browser | raises, as the browser keys |
| `test_id_attribute` | yes | no | no | no (a deviation: upstream's is an option of every level, global to a worker, which runs one test at a time; here it is global to the VM, which the tests running at once share) |
| `output_dir`, `preserve_output`, `workers`, `snapshot_dir`, `snapshot_path_template`, `driver`, `on_start` | yes | no | no | no |

## Contexts of `setup_all`

`use Playwright.Test` adds a `setup_all` (where the `use` is, so before the module's own) that
registers the module's `setup_all` process: every context and request context it creates, or
a process with it in `$callers` creates, gets the context defaults, the config's
context options (so `base_url`), the project's and the module's (its `use` and `@moduletag`,
see "Context defaults"), merged as a test's, and the timeouts of those same layers (see
"Timeouts"): a context's `action_timeout` and `navigation_timeout`, a request context's
`timeout:` the `action_timeout` (`0`, no limit, for `:infinity`) and no navigation timeout. Not
the video, as upstream's comes with the `context` fixture.

`info/0` there returns the module's `file`, `timeout` and `use` (`Playwright.Case` reads the
`browser_name` from it): see `info/0`, also for reading the config outside these scopes.

Only processes with `$callers` count: a `Task`, the client's callback processes (route
handlers, listeners, exposed bindings registered in `setup_all` carry its `$callers`, see
"Route handlers and listeners" in the Testing with ExUnit guide), or a process that sets them
itself (`Process.put(:"$callers", callers)`). A `spawn`, a `GenServer` or `Agent` start, and
`start_supervised` don't set them, so their contexts get no defaults. Deviation: a long-lived
process or handler of `setup_all`'s with them gets the module's defaults for every context it
creates later, during a test too, unrecorded. Upstream's `runBeforeCreateBrowserContext`
resolves the current test, so such a context gets that test's options and is recorded in it.

Not ported yet: a `setup_all` context isn't recorded per test (its actions aren't in a test's
trace, and it gets no screenshots or `error-context.md`), where upstream's `ArtifactsRecorder`
records every open context per test, `beforeAll` ones included: create the context per test,
as `Playwright.Case` does. And after a failed test, `setup_all` state (its pages too) carries
on, where upstream starts a fresh worker: bound calls on shared pages with `action_timeout`.

## Timeouts

As in Playwright Test, a test's contexts have no action or navigation timeout by default: the
test's timeout bounds them. `action_timeout` and `navigation_timeout` (the config's, or an
option of any level over it; upstream's
`actionTimeout`, `navigationTimeout`; milliseconds, `:infinity` by default) set each context's
`Playwright.BrowserContext.set_default_timeout/2` and `set_default_navigation_timeout/2`
(`navigation_timeout` alone governs navigations), and the action timeout is a request
context's `timeout`, when the call gives none. `0` is no limit too, not an immediate failure,
as `timeout: 0` is everywhere in Playwright (see [API conventions](api-conventions.md)):
write `:infinity` to say so.

Shortly before ExUnit's timeout (1 s before, at most 5% of the timeout), the test's open
contexts are closed with upstream's reason, "Test timeout of 60000ms exceeded.": a call that
hangs until then fails with that message and its call log, after the close kept the artifacts.
A test stuck elsewhere (`Process.sleep/1`, a `receive`) still fails with ExUnit's
`ExUnit.TimeoutError`. The timeout is ExUnit's: the `@tag timeout:` (`:infinity` for none) or
`mix test --timeout`; `mix test --trace` has none. As ExUnit's (and upstream's, which counts
`beforeEach` hooks), the timeout counts the test's `setup` callbacks: the clock starts at the
module's first one, also one defined before `use Playwright.Test` or in its case template.
What ExUnit does before them (`@tag :tmp_dir`'s cleanup, `capture_log`) is not counted, and
comes out of the margin.

A module's `setup_all` callbacks get the same timeout, the module's (`@moduletag timeout:`, else
ExUnit's `timeout`: `ExUnit.start(timeout:)` or `mix test --timeout`), as upstream's `beforeAll`
hook gets the test timeout: past it, the `setup_all` fails with upstream's message with
`beforeAll` renamed, `"setup_all" hook timeout of 60000ms exceeded.`, and the stack it was at.
ExUnit has no timeout there, so the runner exits the `setup_all` process, which fails the
module as a crash does; its `on_exit` callbacks still run. The failure is ExUnit's for an exit
(`** (EXIT from #PID<...>)`), with the stack unpruned, and a hung call fails without its call
log: unlike a test's, its contexts aren't closed with a reason before the exit. Its pooled
browsers are reset at the module's end instead, as a failed test's, which closes its contexts:
a failed `setup_all`'s are, as upstream stops a worker whose job failed. A `setup_all` process
that traps exits is killed 1 s after the timeout instead: it fails with `:killed`, without the
message. One timeout counts all of the module's `setup_all` callbacks, where upstream counts
each hook on its own. `BrowserPool.browser/1` (the wait for a worker, its start) is not counted,
as upstream's queue has no timeout and its worker fixtures have a timeout of their own: there,
nothing bounds them. Its contexts get the `action_timeout` and
`navigation_timeout` of the config, the project and the module (`:infinity` by default, not the client's 30 s), as upstream's `beforeAll`
contexts get `actionTimeout || 0`.

## Snapshots

In a test, `Playwright.Expect.to_have_screenshot/2` and
`Playwright.Expect.to_match_aria_snapshot/3` take upstream's `name:` and resolve it as
`testInfo` does, by upstream's default templates:

  * screenshots: `{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-snapshotSuffix}{ext}`,
    `test/checkout_test.exs-snapshots/home-darwin.png` for `name: "home.png"` in
    `test/checkout_test.exs`.
  * aria snapshots: `{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{ext}`.

Without a name, a snapshot is named after the test and its count in the test:
`pays-1-<platform>.png` for a screenshot, `pays-1.aria.yml` for an aria snapshot.
A name is sanitized for the file system, a list of path segments is not. `snapshot_dir` is
`test/` by default; `snapshot_path_template` (or the matcher's `path_template` in `config
:playwright, :expect`) takes upstream's tokens: `{testDir}`, `{snapshotDir}`,
`{testFileDir}`, `{testFileName}`, `{testFileBaseName}`, `{testFilePath}`, `{testName}`,
`{arg}`, `{ext}`, `{platform}` and `{snapshotSuffix}` (the platform, as upstream's);
`{projectName}` is the `name` of the module's project (`parameters/1`), empty without one. `snapshot_path/2`
returns a name's path. A failed comparison writes its `-actual`, `-expected`, `-previous` and
`-diff` images to the test's output dir.

A snapshot the matcher writes (a missing one, under the default `update_snapshots`) fails the
test at the end of its body, as upstream's soft error: the rest of the body runs, and ExUnit
reports all of the test's errors, also when another failure follows. `path:` in place of
`name:` is the snapshot file itself, also outside a test, where a written snapshot is printed.

## Steps

`step/3` is upstream's `test.step`: `Playwright.Test.step "log in" do ... end`, or
`step(title, fun, options)` for wrappers, with upstream's `box`, `location`, `params` and
`subtitle` (`step_skip/3` is `test.step.skip`). Each matcher call in a test is a step too
(`category: 'expect'`), and a soft one's failure (`Playwright.Expect.soft/1`, upstream's
`expect.soft`) fails the test at the end of its body, as a written snapshot does. So is each
driver call (`pw:api`, titled as upstream's: `Click getByRole('button')`), in the step or
`Playwright.Tracing.group/3` it runs in; a matcher's driver call is its step. The trace's
`test.trace` entry (upstream's `TestTracing`) has the steps, nested by the step they run in,
and the test's errors, so `trace: :on` gives a `trace.zip` also for a test without contexts.
`attach/2` adds a `test.attach` step with the attachment (the trace's `attachments: false`
leaves the attachments out). Not ported: the test's stdout and stderr in the trace, hooks and
fixtures as steps, the step's `TestStepInfo` (the body takes no argument, so no
`step.attach`) and its `timeout` (the test's timeout bounds a step).

A step also works in the module's `setup_all` (and processes with it in `$callers`), as
upstream's in a `beforeAll` hook: it returns its body's value, nests, and raises its error on,
a boxed step's where its helper is called. Deviation: it is recorded nowhere (no trace, no
video overlay), also in a long-lived process or handler of `setup_all`'s during a test, where
upstream records a `beforeAll` step in the first test (its trace's "Before Hooks") and a
handler's in the current test: ExUnit runs `setup_all` before any test, and hooks aren't
steps here.

## Assertions

`use Playwright.Test` imports `Playwright.Test.Assertions`' `assert/1,2` and `refute/1,2` in
place of `ExUnit.Assertions`' (it imports that module again without them). Each is ExUnit's, also a
step of the running test, as upstream's `expect(value).toBe(...)` is: `category: 'expect'`,
titled `Expect "assert"` or `Expect "refute"`, with the expression as its subtitle and its
`expression` param, located at the call. Its `expected` param, as upstream's
`toEqual(expected)`'s, is the right side ExUnit's expansion binds (`left == right` and the other
operators, `pattern = right`, `match?(pattern, right)`; a failed one's from its error), when it
is JS-truthy (`if (args[0])`: not `nil`, `:null`, `false`, `0`, `0.0` or `""`), previewed as upstream's
`generatePreview` but cut to 100 list elements and 1000 characters (`…`), as upstream's isn't
(Elixir only: bound trace size); a truthy `assert x` has none. A message (`assert x, "msg"`)
is the title, as upstream's `expect(value, message)`.

A passed one expands to ExUnit's `assert` as it is, then records its step, so it binds as
ExUnit's (`assert {:ok, user} = create()`, `assert (n = count()) > 0`), and its driver calls
come before it, as upstream's. A failed one raises ExUnit's error, with its message and
stacktrace; only a function that ends in an assertion keeps its frame in the stacktrace, as
the assertion is no longer its tail call (as JS keeps every frame). A failed `assert/1` or
`refute/1` is recorded once, with that error, where it ends: in a `Playwright.Expect.to_pass/2`
attempt, else at the end of the test's body, as a step of the test's (not of the `step/3` it
failed in, which fails with the same error). `assert/2` and `refute/2` are functions in
ExUnit, so only their call is wrapped in a `try`, which records the failure there, in its
step, also when the test rescues it.
Outside a test, and in the module's `setup_all` (as a step there), nothing is recorded. The
other assertions (`assert_receive`, `assert_raise`, ...) are ExUnit's, no steps.

They are on by default. `use Playwright.Test, assert_steps: false` keeps ExUnit's own
`assert` and `refute` (its plain `import ExUnit.Assertions`) and records none in its tests,
passed or failed, a helper module's either; `config :playwright, :test, assert_steps: false` makes that the default,
which a module's `assert_steps: true` overrides. As it decides the import, it is read when the
module compiles: set it in `config/config.exs` or `config/test.exs`, not `config/runtime.exs`
(Mix recompiles the modules when it changes). The option takes a literal boolean.

Limits:

  * A passed assertion is recorded only where the import is: a helper module opts in with
    `import Playwright.Test.Assertions`, and, if it imports `ExUnit.Assertions` too,
    `import ExUnit.Assertions, except: [assert: 1, assert: 2, refute: 1, refute: 2]`. A library
    with its own `assert/2` excludes the arities it defines (`except: [assert: 2]`); its own
    `refute/2` also takes the `refute(value, expr: ...)` that ExUnit's `refute match?` calls
    (see `Playwright.Test.Assertions`). A failed `assert/1` or `refute/1` is recorded wherever it was
    written.
  * A test module that imports `ExUnit.Assertions` again after `use Playwright.Test` fails to
    compile, with an ambiguous call of `assert`: drop that import, or add the same `except:`.
  * A case template passes the import on with `use Playwright.Test` in its `using`; the one in
    its body (see "Registration") doesn't.
  * A failed `assert/1` or `refute/1` is a step of the test's, not nested in the `step/3` it
    failed in. One that the test rescues itself (a retry loop without steps) leaves no step,
    and so does one in the test's body under `mix test --breakpoints`, which runs the body
    without the `test` macro's `catch`. One rescued and raised again with `raise error` (not
    `reraise`) is located at that `raise`.
  * A failed `assert/1` or `refute/1` in a process of its own (a `Task`, a listener, a route
    handler) or in a `setup` callback leaves no step: the test body's catch doesn't see it.
  * Each attempt of `Playwright.Expect.poll/3` is its assertion's own steps, under the poll's
    step (as upstream's attempts are its matcher's); a poll with a message has the matcher's,
    titled by it, as upstream's. An attempt without one of these `assert`s (`assert_receive`,
    a raise of its own) leaves no step when it passes, the matcher's when it fails.
  * A failed assertion's expression shows a capture's argument as ExUnit's error has it
    (`_& == 2` for `&assert(&1 == 2)`); a passed one's is `&1 == 2`. On Elixir 1.18 both show
    the argument as `capture` (`capture == 2`).

## Config

    config :playwright, :test,
      browser_name: :chromium,      # BrowserPool: :chromium | :firefox | :webkit, or "chromium", ...
      headless: nil,                # BrowserPool: over launch_options' (nil: launch_options' headless, else true)
      channel: nil,                 # BrowserPool: over launch_options' (nil: launch_options' channel)
      launch_options: [],           # BrowserPool: Playwright.BrowserType.launch/2
      connect_options: nil,         # BrowserPool: [ws_endpoint: _, ...] to connect instead of launching
      driver: [],                   # BrowserPool: [env: _], Playwright.Supervisor's (the driver is config :playwright, :driver's)
      on_start: nil,                # BrowserPool
      workers: max(div(System.schedulers_online(), 2), 1),
      context_options: [],          # Playwright.Browser.new_context/2's options, as upstream's contextOptions
      # base_url: "http://localhost:4000", locale: "de-DE", ...: a context option, unset by default
      default_browser_type: nil,    # a device's: the browser, without a browser_name
      action_timeout: :infinity,    # ms, each context's default (and navigation_timeout's); :infinity or 0: none
      navigation_timeout: :infinity,
      trace: :off,                  # :off | :on | :retain_on_failure | :on_first_retry | ..., or [mode: _, ...]
      screenshot: :off,             # :off | :on | :only_on_failure | :on_first_failure, or [mode: _, ...]
      video: :off,                  # as trace, or [mode: _, size: _]
      output_dir: "test-results",
      preserve_output: :always,     # :always | :failures_only | :never, or "failures-only", ..., see Artifacts
      snapshot_dir: nil,            # upstream's testDir: test/
      snapshot_path_template: nil,  # upstream's defaults, see Snapshots
      test_id_attribute: "data-testid",
      report_slow_tests: [max: 5, threshold: 300_000], # Playwright.Test.GitHubFormatter's, ms; false: none
      assert_steps: true            # assert/refute as steps, at compile time (see Assertions)

Each context option (`base_url`, `locale`, `viewport`, `storage_state`, ...) is a config key of
its own, over the `context_options` bag, as under every other level, and in `config/0` only
when set. A test reads the resolved values with `info().use`.

The artifact modes are `off` by default and `output_dir` is `test-results`, as upstream's.
`workers` is half the schedulers, as upstream's default `workers` is half the cores (`'50%'`).
`test_id_attribute` is upstream's `testIdAttribute`, set on each pooled driver: the client's
value is global to the VM, so set it here, not per module. `config/0` returns the config
resolved. An unknown key raises at application start (and when the runner reads a config
changed since), and so does a bad value of the keys upstream checks: `preserve_output`,
`output_dir`, `browser_name`, `report_slow_tests` and the context options (as
`Browser.new_context/2` checks them), and of `workers` (a positive integer: upstream's also
takes `"50%"` or a numeric string, which aren't supported), `on_start` and Elixir's
`assert_steps` (a boolean). `browser_name` and `preserve_output` take an atom (`:webkit`,
`:failures_only`) or upstream's string (`"webkit"`, `"failures-only"`): an unknown atom raises
as any bad value, an unknown string with upstream's error. Other values pass as given, as
upstream's: an unknown artifact mode records nothing.

# `attach`

```elixir
@spec attach(String.t(), keyword()) :: :ok
```

Upstream's `testInfo.attach(name, options)`: attaches a file or a binary to the running test,
as a `test.attach` step of its trace, which shows it. Options:

  * `:path` - a file, copied to `attachments/<name>-<hash><ext>` in the test's output dir,
    which keeps it also when the test passes.
  * `:body` - the content itself.
  * `:content_type` - by default the path's MIME type, or `text/plain` for a UTF-8 `body`
    (`application/octet-stream` otherwise).

One of `path` and `body`, or neither for an attachment of just a name. Raises outside a test.

    Playwright.Test.attach("screenshot", body: Page.screenshot!(page), content_type: "image/png")

# `config`

```elixir
@spec config() :: map()
```

The resolved `config :playwright, :test` (see "Config" in the moduledoc), for libraries
that build on the runner: a map of the listed keys, each with its default filled in and its
value validated, as the runner reads it, and the context options that are set. The recording
modes are keyword lists with `:mode` first (`trace: [mode: :off, snapshots: true, ...]`), a
context option is as configured (none of Playwright Test's context defaults; one set only in
`context_options` is there only), and an unset `action_timeout` or `navigation_timeout` is
`:infinity`. A test's resolved options, over every level, are `info().use`. `expect: %{timeout: ms}` is the matchers' timeout of `config :playwright,
:expect` (5000 by default, `0` for none). Works outside a test too (at startup, in a module
without `use Playwright.Test`); raises `ArgumentError` for an invalid config.

    Playwright.Test.config().workers

# `in_registered_scope?`

```elixir
@spec in_registered_scope?() :: boolean()
```

Whether `info/0` works in the calling process: a registered test or module `setup_all`, or a
process with either in its `$callers`. `info/0` has `test_pid` only in a test. See `info/0` for
reading the config elsewhere.

# `info`

```elixir
@spec info() ::
  %{
    test_pid: pid(),
    file: Path.t(),
    test_name: String.t(),
    output_dir: Path.t(),
    retry: non_neg_integer(),
    timeout: timeout(),
    deadline: integer() | :infinity,
    use: map()
  }
  | %{file: Path.t(), timeout: timeout(), use: map()}
```

The running test of the calling process or a process in its `$callers` (a `Task`), as
upstream's `test.info()`: `test_pid`, `file` (absolute), `test_name` (without the macro's
`test ` prefix), `output_dir` (absolute, with its `-retry1` suffix under `mix test --failed`,
see "Artifacts" in the moduledoc), `retry` (upstream's `testInfo.retry`: 0, or 1
under `mix test --failed`) and `timeout` (upstream's `testInfo.timeout`: ExUnit's timeout of the test in
ms, the `@tag timeout:` or `mix test --timeout`, `:infinity` for none, as under `mix test
--trace`) and `deadline` (upstream's internal `testInfo._deadline()`: monotonic milliseconds
(compare with `System.monotonic_time(:millisecond)`) by which the test must be done: after
it, the test's contexts are closed and Playwright calls fail; `:infinity` without a test
timeout; see "Timeouts"), from the test's setup to the end of its body. `use` is the test's
options as resolved from every level (the config's, its project's, its module's, its
describe's and its own), what upstream's option fixtures give a fixture: a map of the context
options every context of the test starts from (`base_url`, `locale`, `extra_http_headers`,
...; those set, and Playwright Test's defaults), `action_timeout`, `navigation_timeout`,
`trace`, `screenshot`, `video`, and the worker options of its `BrowserPool.browser/1` calls
(`browser_name`, `headless`, `channel`, `launch_options`, `connect_options`). A library that passes options in a
call merges into it, as the call's replace the resolved ones as a whole:
`Browser.new_context!(browser, extra_http_headers: Map.merge(Test.info().use[:extra_http_headers]
|| %{}, headers))`. A library that waits or
retries on its own (a server-side backstop) should stop by `deadline`, as the matchers
stop 250 ms before it, as upstream's stop 250 ms before its test deadline.

In the module's `setup_all` (and processes with it in `$callers`), it returns the fields that
apply to the module, with the same meaning: `file`, `timeout` (ExUnit's timeout of the module,
`@moduletag timeout:` or ExUnit's, the one "Timeouts" in the moduledoc applies to its
`setup_all` callbacks) and `use`, the module's options as resolved from the config, its
project and the module itself (its `use` and `@moduletag`): a describe's and a test's options
never reach `setup_all`. The test's fields (`test_pid`, `test_name`, `output_dir`, `retry`,
`deadline`) are absent. Deviations: upstream runs `beforeAll` hooks with the
first test's `testInfo`, which ExUnit's `setup_all`, run before any test, has none of; and a
long-lived process or handler of `setup_all`'s (with it in `$callers`) gets the module's fields
during a test too, as its contexts get the module's defaults (see "Contexts of `setup_all`"),
until the runner cleans up after it, where upstream's `test.info()` is the current test's. A
`setup_all` that runs before `use Playwright.Test` registers the module (a case template's
own, or one above the module's `use`) isn't registered: there it raises.

A library that also runs outside these scopes reads the config there:

    alias Playwright.Test
    use = if Test.in_registered_scope?(), do: Test.info().use, else: Test.config()

`config/0` has another shape: the config alone, with the context options set as keys of their
own (none of Playwright Test's defaults, nor those only in its `context_options` bag) and its
config-only keys (`workers`, `output_dir`, ...). Read the keys both have with Access,
`use[:base_url]` (`nil` when unset), not `use.base_url`, which raises for an unset key.

Raises elsewhere (see `in_registered_scope?/0`), and in `on_exit` callbacks (ExUnit runs them
in a process of their own, without `$callers`, `setup_all`'s too): there, use `info/1` for a
test.

    Playwright.Test.info().test_pid

# `info`

```elixir
@spec info(pid()) :: map()
```

`info/0` of the test `test_pid` (the context's `test_pid`), from any process, from its setup to
the end of its `on_exit` callbacks. Once the test's process has exited, so in `on_exit`, it has
upstream's result fields too, as `testInfo` has in teardown:

  * `status` - `:passed`, `:failed`, `:timed_out` (past ExUnit's timeout), or `nil`
    for a test of another macro (a doctest).
  * `errors` - upstream's `TestInfoError`s: `%{message: _, stack: _}` of an exception,
    `%{value: _}` of an exit or throw.
  * `expected_status` - `:passed` (there is no `test.fail()`).
  * `duration` - ms from the test's setup to its process's exit.

A failure in an `on_exit` callback doesn't change them.

    setup %{test_pid: test_pid} do
      on_exit(fn ->
        if Playwright.Test.info(test_pid).status != :passed, do: ...
      end)
    end

# `output_path`

```elixir
@spec output_path(String.t() | [String.t()]) :: Path.t()
```

The path of `path_segments` (a string or a list of them) in the test's output dir (see
"Artifacts" in the moduledoc), as upstream's `testInfo.outputPath(...pathSegments)`: creates the
dir, and raises for a path outside it. Files written
there stay when the test passes, under the default `preserve_output`.

# `parameters`

```elixir
@spec parameters(keyword() | map()) :: %{
  optional(atom()) =&gt; Playwright.Test.Project.t()
}
```

A project for `parameterize:`, upstream's `projects: [{ name, use: { ... } }]`: `name:` (a
string, upstream's `project.name`), `use/1`'s options and the worker's (`browser_name`,
`default_browser_type`, `headless`, `channel`, `launch_options`, `connect_options`), validated
as it does. A map of
one private key, `:"playwright.project"`, whose value is a `Playwright.Test.Project`, which
ExUnit takes there (a tag takes `use/1`'s keyword list).

    use Playwright.Case,
      async: true,
      parameterize:
        for device <- ["Desktop Chrome", "Desktop Firefox", "Desktop Safari"],
            do: Playwright.Test.parameters([name: device] ++ Playwright.device(device))

The project's options are under the module's `use` and over the config's, for every context
of the module's run, `setup_all`'s included. Its browser is its `browser_name`, else the
config's, else its `default_browser_type` (a device's), see "Browser" in the moduledoc. A failed
test prints the project as its name (`Parameters: %{"playwright.project":
#Playwright.Test.Project<"Desktop Safari">}`, `[Desktop Safari] ›` in
`Playwright.Test.GitHubFormatter`); the name is also the snapshot templates' `{projectName}`.
An unnamed project, as upstream's, has no label: it prints as its options, so name each one.
For parameters of your own next to it: `Map.put(Test.parameters(...), :role, :admin)`.

# `snapshot_path`

```elixir
@spec snapshot_path(String.t() | [String.t()], keyword()) :: Path.t()
```

The snapshot path of `name` for the running test, as upstream's `testInfo.snapshotPath`:
where `Playwright.Expect.to_have_screenshot/2` (`kind: :screenshot`) and
`Playwright.Expect.to_match_aria_snapshot/3` (`kind: :aria`) with `name:` read and write it
(see "Snapshots" in the moduledoc). `name` is a string or a list of path segments
(`["home", "hero.png"]`, upstream's `snapshotPath(...name)`), which are not sanitized. Option
`:kind`: `:snapshot` (default, upstream's), `:screenshot` or `:aria`.

# `step`
*macro* 

Upstream's `test.step(title, body, options)`: runs the body as a step of the running test
and returns its value, also from a process with the test in its `$callers` (a `Task`); steps
nest. A step's error is raised on after the step records it. In the module's `setup_all`
(and processes with it in `$callers`) it runs too, recorded nowhere (see "Steps" in the
moduledoc); elsewhere it raises.

    step "log in" do
      ...
    end

    step "log in", box: true do
      ...
    end

    step("log in", fn -> ... end, box: true)

Options:

  * `:box` - the step is located at the call of the function the step is in (a helper's
    caller), unless `:location` is given, and an error inside it is reported there: its
    stacktrace starts at that call, instead of the frames inside it. The steps inside keep
    their own locations. A boxed step outside a helper (in the test's body) has no location of
    its own; pass `location:`.
  * `:location` - `%{file: _, line: _, column: _}` of the step in the trace. The `do` form
    passes its own, unless boxed.
  * `:params`, `:subtitle` - shown with the step in the trace.

The body runs in the calling process. Upstream's `timeout` is not supported (it raises an
`ArgumentError`): a process can't be interrupted, and the test's timeout bounds the step.

The step, the matchers in it and the test's errors are in the test's trace (`test.trace` in
`trace.zip`). A step in a `Task` has no parent step: the `Task` doesn't carry the zone.

# `step_skip`
*macro* 

Upstream's `test.step.skip(title, body, options)`: a step that doesn't run its body, marked
`skip` in the trace. Returns `nil`. The forms and options are `step/3`'s.

# `test`
*macro* 

`ExUnit.Case.test/1`: a test that fails as not implemented.

# `test`
*macro* 

`ExUnit.Case.test/3`, with the end of the body marked: the test passed if its process then
exits as ExUnit ends a test, and fails there with the errors its matchers recorded (see
"Snapshots" in the moduledoc). A `catch` notes an error for `error-context.md` and re-raises it
unchanged, so ExUnit reports the failure as without this macro (with the recorded errors
before it, if any).

# `use`

```elixir
@spec use(keyword() | map()) :: keyword()
```

Options for a module's `@moduletag`, a `describe` block or a test: upstream's `test.use`. It
takes the options of `use Playwright.Test` (the context options, `context_options`,
`action_timeout`, `navigation_timeout`, `trace`, `screenshot`, `video`, and in `@moduletag`
the worker's), as a keyword list or a map, validates them as it does, and returns a keyword
list for an ExUnit tag. A device in `@moduletag` too; in a describe's or a test's tag only
without its `default_browser_type`.

    # test.describe(() => { test.use({ timezoneId: 'Europe/Berlin' }) ... })
    describe "in Berlin" do
      @describetag Test.use(timezone_id: "Europe/Berlin")
      ...
    end

    # One test. Upstream nests a describe around it; ExUnit's can't nest.
    @tag Test.use(trace: :on)
    test "pays", %{page: page} do
      ...
    end

    # A device's emulation without its browser (a device in a test's or a describe's tag
    # raises, as its `default_browser_type` is the worker's: make each device a project).
    @tag Test.use(Keyword.delete(Playwright.device("iPhone 13"), :default_browser_type))

Each option gets a private key of its own (`:"playwright.use.trace"`,
`:"playwright.use.context.locale"`), so ExUnit, which merges a test's `@moduletag`,
`@describetag` and `@tag` per key, layers them per option, the test's over the describe's
over the module's: a describe's `timezone_id` and a test's `trace` both apply, as do several
calls on one test, and an option given as `nil` resets it to the project's, the config's or
the default. A plain tag named like an option (`@tag trace: :on`) sets nothing.

An unknown option raises `ArgumentError`, when the test module compiles. So do
`browser_name`, `default_browser_type` (a device's), `headless`, `channel`, `launch_options` and
`connect_options` in a `@describetag` or a `@tag`, when the module compiles a test of the `test` macro under it,
with upstream's message ("Cannot use({
browser_name }) in a describe group, because it forces a new worker.
Make it top-level in the
test file or put in the configuration file.", "in a test" for a `@tag`): they are the
worker's (a test the check doesn't see gets the module's worker regardless). They are options
of the module's `use`, its `@moduletag` and `parameters/1`. For a
project in `parameterize:`, which takes maps, see `parameters/1`. See "Context defaults",
"Browser" and "Artifacts" in the moduledoc for the layers.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
