This library is the client, ExUnit is the test runner. Playwright.Test adds what Playwright
Test's worker gives each test: a browser, a fresh context with Playwright Test's defaults, and
traces, screenshots and videos of failures. This guide tests an app that runs somewhere and is
reached through a base URL, whatever it is written in. Playwright.Test's moduledoc is the
reference; the guides show how to use it:
- this one: setup, assertions, steps, handlers and the app under test;
- Options, devices and projects;
- Artifacts, retries and workers;
- Porting Playwright Test specs, and what isn't supported.
Snippets alias Playwright.{Page, Locator, ...}.
Setup
Configure the runner in config/test.exs:
# config/test.exs
config :playwright, :test,
base_url: "http://localhost:4000",
trace: :retain_on_failure,
screenshot: :only_on_failureA test module uses Playwright.Case, which gives each test a context and a page:
defmodule MyApp.SignUpTest do
use Playwright.Case, async: true
test "signs up", %{page: page} do
Page.goto!(page, "/sign-up")
Page.get_by_label(page, "Email") |> Locator.fill!("ada@example.com")
Page.get_by_role(page, :button, name: "Sign up") |> Locator.click!()
Expect.to_have_text!(Page.get_by_role(page, :status), "Welcome, ada@example.com")
end
endPlaywright.Case is a case template of three pieces, the same as:
use ExUnit.Case, async: true
use Playwright.Test
alias Playwright.{Browser, BrowserContext, Expect, Keyboard, Locator, Mouse, Page, Test}
alias Playwright.Test.BrowserPool
setup_all do
%{browser: BrowserPool.browser(), browser_name: Test.info().use.browser_name}
end
setup %{browser: browser} do
context = Browser.new_context!(browser)
%{context: context, page: BrowserContext.new_page!(context)}
enduse Playwright.Test goes after use ExUnit.Case. It registers each test, so the contexts it
creates get Playwright Test's defaults and are recorded, and imports a test macro that marks
the end of each test's body, so the recording knows whether the test passed (ExUnit reports
failures as before; see Registration).
Playwright.Test.BrowserPool.browser/1 is Playwright Test's browser fixture: called in
setup_all, it gives the module a driver and browser of its own while it runs, as Playwright Test
runs a file in one worker. A module that passed hands its browser on, warm, to the next; one that
failed has it replaced (Workers). A context is a fresh browser
profile with its own cookies, storage and cache, so tests don't share state. The drivers stop
after the suite, so test_helper.exs needs only ExUnit.start().
A module's tests run one after another (ExUnit balances whole modules, where Playwright Test's
fullyParallel balances single tests), so many small async: true modules parallelise better
than a few big ones: split a big spec into a module per describe. Modules that share state in
the app can stay async in one ExUnit group (use Playwright.Case, async: true, group: :tree,
Elixir 1.18+): no two tests of a group run at once, while other modules keep running, which beats
async: false. mix test --slowest N turns on --trace, which runs one module at a time, so it
can't time a parallel run.
Options go in as arguments, over the config: BrowserPool.browser(browser_name: :webkit),
Browser.new_context!(browser, locale: "de-DE"). Every context a test creates on a pooled
browser starts from Playwright Test's defaults (locale: "en-US", a 1280x720 viewport,
color_scheme: :light, ...), then the options of the config, the project, the module, the
describe block and the test (Options, devices and projects).
With base_url, goto and the URL matchers take paths. To set it from the shell, read an env
var in the config: base_url: System.get_env("BASE_URL", "http://localhost:4000").
As in Playwright Test, actions and navigations have no timeout of their own in a test: ExUnit's
test timeout (60 seconds) bounds them; Playwright Test's is 30 seconds, so for the same
behaviour, ExUnit.start(timeout: 30_000) in test_helper.exs. Just before it, the test's
contexts are closed, so a stuck call fails with "Test timeout of 60000ms exceeded." and its call
log, and the artifacts are kept. To fail sooner, set action_timeout and navigation_timeout in
the config (milliseconds). :infinity, the default, is no limit, and so is 0, as in every
timeout: option (API conventions).
setup_all's timeout and the rest: Timeouts.
Each driver call logs pw:api ... at :debug (Logging),
so give the test env a logger level (config :logger, level: :warning). ExUnit.start(capture_log: true) prints an in-VM app's logs only for failed tests, as Playwright Test hides the
webServer's output.
Assertions
Playwright.Expect has the web-first assertions: they retry until the page matches or the
timeout passes. The bang forms raise Playwright.ExpectError with the expected and received
values and the call log:
Expect.to_have_url!(page, ~r"/dashboard$")
Expect.to_be_visible!(Page.get_by_text(page, "Saved"))
Page.get_by_role(page, :listitem) |> Expect.to_have_count!(3)
Page.locator(page, ".error") |> Expect.not_() |> Expect.to_be_visible!()Prefer them to assert Locator.text_content!(...) == ..., which reads once and races the page.
Their timeout is 5 seconds; set it for the suite in config/test.exs:
config :playwright, :expect, timeout: 10_000A soft assertion (Playwright Test's expect.soft) doesn't stop the test: it fails the test once
its body has run, and ExUnit reports all of the test's errors:
Page.get_by_text(page, "Saved") |> Expect.soft() |> Expect.to_be_visible!()
Page.get_by_role(page, :listitem) |> Expect.soft() |> Expect.to_have_count!(3)Where an ExUnit.AssertionError is needed (a helper that rescues only that), wrap the matcher:
defmodule MyApp.Assertions do
@doc "Runs `fun`, re-raising a failed matcher as an `ExUnit.AssertionError`."
def assert_expect(fun) do
fun.()
rescue
e in Playwright.ExpectError ->
reraise ExUnit.AssertionError, [message: Exception.message(e)], __STACKTRACE__
end
endassert_expect(fn -> Expect.to_have_text!(Page.get_by_role(page, :status), "Saved") end)Snapshots
In a test, Expect.to_have_screenshot!/2 and Expect.to_match_aria_snapshot!/3 take a name:
that resolves next to the test file, as in Playwright Test:
Expect.to_have_screenshot!(page, name: "home.png")
# test/sign_up_test.exs-snapshots/home-linux.png (the suffix is the platform)
Expect.to_match_aria_snapshot!(Page.locator(page, "body"))
# test/sign_up_test.exs-snapshots/signs-up-1.aria.yml (no name: the test's name and count)A missing snapshot is written, and fails the test once its body has run, so CI catches a
snapshot nobody committed. update_snapshots in config :playwright, :expect (:all,
:changed, :missing, :none) rewrites them. A failed comparison leaves -actual,
-expected, -previous and -diff images in the test's dir under test-results/. Paths and
templates: Snapshots.
Steps
Playwright.Test.step/3 groups a test's actions under a title, as Playwright Test's test.step:
the trace shows the steps, nested, with each matcher call and each driver call as a step of its
own, and the test's errors. A step returns its body's value and works in a Task of the test too:
test "signs up in steps", %{page: page} do
Test.step "fill in the form" do
Page.goto!(page, "/sign-up")
Page.get_by_label(page, "Email") |> Locator.fill!("ada@example.com")
end
status =
Test.step("submit", fn ->
Page.get_by_role(page, :button, name: "Sign up") |> Locator.click!()
Page.get_by_role(page, :status)
end)
Expect.to_have_text!(status, "Welcome, ada@example.com")
endWith box: true, the step and an error inside it are reported where the helper is called,
which suits helpers, a library's too. The steps inside keep their own locations:
defp sign_up(page, email) do
Test.step "sign up", box: true do
Page.goto!(page, "/sign-up")
Page.get_by_label(page, "Email") |> Locator.fill!(email)
Page.get_by_role(page, :button, name: "Sign up") |> Locator.click!()
end
endA step runs in the calling process, so assert_receive works in it. It takes no timeout:, as a
running body can't be interrupted: the test's timeout bounds it. step_skip declares a step
without running it. In setup_all a step runs too, unrecorded
(Steps), so the same helpers serve both. use Playwright.Test
imports test and replaces ExUnit's assert and refute with Playwright.Test.Assertions' (the
rest of ExUnit.Assertions stays imported); the rest of Playwright.Test is called qualified (Test.step,
Test.info()), as Playwright Test's test.step(), and a wrapper library's own step doesn't
clash.
Each assert and refute is a step too, as Playwright Test's expect: Expect "assert", with
the expression next to it and its right side as expected, and a failed one with its error,
at the test's level. They are ExUnit's otherwise, with
the same bindings, messages and stacktraces (a function that ends in one keeps its frame).
Don't import ExUnit.Assertions again after use Playwright.Test, which makes assert
ambiguous. A helper module without use Playwright.Test needs import Playwright.Test.Assertions
(and, if it imports ExUnit.Assertions too, except: [assert: 1, assert: 2, refute: 1, refute: 2]
there), else its passed asserts silently leave no step. A library with its own assert/2 excludes
exactly the arities it defines (import Playwright.Test.Assertions, except: [assert: 2]); its own
refute/2 must also take the refute(value, expr: ...) that ExUnit's refute match? calls
(Assertions).
Assert steps are on by default: use Playwright.Test, assert_steps: false keeps ExUnit's own for a
module, and config :playwright, :test, assert_steps: false (in config/test.exs, as it is
read at compile time, not in runtime.exs) for all, which a module's assert_steps: true
overrides.
Route handlers and listeners
Route handlers (Page.route/3, route_web_socket/3 and their WebSocket on_message and
on_close), event listeners (Page.on/3), exposed functions, locator handlers and wait
predicates run in processes of their own, on behalf of the process that registered them: they
get its $callers, as a Task does. So the Ecto sandbox, Mox and Req.Test find the test in
them without an explicit allowance, and matchers and step count for the test.
An error a route handler, listener or locator handler raises fails the test and stops it at once,
as Playwright Test's worker does; once the body has ended (in your on_exit callbacks) it is only
logged (Registration has the details). In a :dialog
listener, accept or dismiss the dialog before asserting on it: a dialog left open blocks the page,
so the screenshot and the snapshot of error-context.md wait on it until they time out.
An on_exit callback is Playwright Test's afterEach hook. ExUnit runs it after the test
process has exited and its start_supervised!/1 children have stopped, but before the test's
contexts close, which is last: it can still use page and context, and their trace records its
calls. The test's listeners still run in your on_exit callbacks, but not as its contexts close.
State linked to the test (an Agent.start_link/1 of it, a start_supervised!/1 child) is gone in
on_exit, so a listener that uses it there logs "Uncaught in event listener: ... no process":
send to the test process (a late message is dropped with it), or keep the state unlinked
(Agent.start/1).
A check after the test
A fixture teardown that fails the test, such as LiveView's page fixture checking for page
errors, is an on_exit: an assertion there fails the test, reported by ExUnit as an on_exit
failure (the recording counts it as a pass). Its state must outlive the test process. Playwright
Test's option fixtures (test.extend with { option: true }, set with test.use) are plain
tags or parameter keys of your own: test.use({ ignoreJSErrors: true }) becomes
@tag :ignore_js_errors (or @describetag, @moduletag, a key of a project's
Test.parameters/1 map), which the setup reads:
setup %{page: page} = ctx do
{:ok, errors} = Agent.start(fn -> [] end)
Page.on(page, :page_error, fn error -> Agent.update(errors, &(&1 ++ [error])) end)
on_exit(fn ->
page_errors = Agent.get(errors, & &1)
Agent.stop(errors)
if !ctx[:ignore_js_errors], do: assert(page_errors == [], "unhandled page errors")
end)
endPolling and waiting
When the state shows in the page, assert on the page: Expect.* retries. For state elsewhere (an
email, a background job, a record behind an API), retry the assertion with Expect.to_pass!/2
(expect(fn).toPass()), or poll a value with Expect.poll!/3 (expect.poll(fn).toBe(x)), whose
matcher is an ExUnit assertion:
Expect.to_pass!(fn ->
response = APIRequestContext.get!(BrowserContext.request(context), "/api/exports/1")
assert APIResponse.json!(response)["status"] == "done"
end)
Expect.poll!(fn -> Repo.aggregate(Export, :count) end, &assert(&1 == 1), timeout: 10_000)As upstream's, to_pass retries until the test's timeout (timeout: for less) and poll for
the matchers' timeout, both at [100, 250, 500, 1000] ms intervals (intervals:). The callback
runs in the test process, so self() and its mailbox are the test's; a step or a soft receiver
(fun |> Expect.soft() |> Expect.to_pass!()) works as for any matcher.
As upstream's, a poll callback that raises fails at once. JS's (await rows())[0] is
undefined while the list is empty and polling goes on, where hd([]) raises: read with
Enum.at(rows, 0) and leave the check to the assertion.
Events forwarded to the test process arrive when the browser emits them, which can be well after
the action that led to them (a request, a timer). assert_receive waits only 100 ms by default,
so give it a timeout, or raise the default with ExUnit.start(assert_receive_timeout: 5_000):
test_pid = self()
Page.on(page, :console, &send(test_pid, {:console, ConsoleMessage.text(&1)}))
Page.get_by_role(page, :button, name: "Save") |> Locator.click!()
assert_receive {:console, "saved"}, 5_000To check forwarded events afterwards, as JS collects them into an array, drain the mailbox. An event is handled before the reply of the call that caused it (Concurrency), so once that call returned (or a matcher saw its effect), its message is there:
defp received(tag) do
receive do
{^tag, value} -> [value | received(tag)]
after
0 -> []
end
end
assert received(:console) == ["saved"]See Events and waiting for one-off events (downloads, popups, dialogs).
Logging in once
Log in once per module and start each context from the saved cookies and local storage. A
context in setup_all gets the defaults, base_url too, as a test's does, but no recording:
setup_all do
browser = BrowserPool.browser()
context = Browser.new_context!(browser)
page = BrowserContext.new_page!(context)
Page.goto!(page, "/login")
Page.get_by_label(page, "Email") |> Locator.fill!("ada@example.com")
Page.get_by_label(page, "Password") |> Locator.fill!("secret")
Page.get_by_role(page, :button, name: "Log in") |> Locator.click!()
Expect.to_have_url!(page, ~r"/dashboard$")
storage_state = BrowserContext.storage_state!(context)
BrowserContext.close!(context)
%{browser: browser, storage_state: storage_state}
end
setup %{browser: browser, storage_state: storage_state} do
context = Browser.new_context!(browser, storage_state: storage_state)
%{context: context, page: BrowserContext.new_page!(context)}
endA request context created in setup_all isn't disposed of by the runner either, as with
upstream's manual newContext in beforeAll. Dispose of it yourself inside the setup_all:
on_exit(fn -> APIRequestContext.dispose!(request) end).
To share the login between modules, BrowserContext.storage_state!(context, path: "tmp/auth.json") also writes it to a file, and storage_state: takes that path. An app that keeps
its tokens in IndexedDB (Firebase Authentication) needs storage_state!(context, indexed_db: true): IndexedDB is not in the state by default. See
Authentication.
To log in once per run, as Playwright Test's setup project or globalSetup does, log in from
test_helper.exs, which runs before any test, on a driver of its own (one more driver start)
that it stops before the tests start. Outside a test no defaults apply, so give the context its
base_url. The config's storage_state then starts every test's context from the file:
# test/test_helper.exs, after starting the app
{:ok, driver} = Playwright.Supervisor.start_link(name: MyApp.GlobalSetup)
browser = Playwright.instance(MyApp.GlobalSetup) |> Playwright.chromium() |> BrowserType.launch!()
context = Browser.new_context!(browser, base_url: Playwright.Test.config()[:base_url])
page = BrowserContext.new_page!(context)
Page.goto!(page, "/login")
# ... log in, as above
BrowserContext.storage_state!(context, path: "tmp/auth.json")
Supervisor.stop(driver)
ExUnit.start()# config/test.exs
config :playwright, :test, base_url: "http://localhost:4000", storage_state: "tmp/auth.json"Any other once-per-run setup (Playwright Test's project dependencies) goes there too.
Several users
Each user is a context, and a user on another browser takes it from another browser/1 call,
another worker that the module holds too (over workers, so it never waits for itself):
test "two users edit one document", %{browser: browser} do
ada = Browser.new_context!(browser)
bob = Browser.new_context!(browser)
eve = Browser.new_context!(BrowserPool.browser(browser_name: :webkit))
ada_page = BrowserContext.new_page!(ada)
bob_page = BrowserContext.new_page!(bob)
eve_page = BrowserContext.new_page!(eve)
# ...
endAll three contexts are recorded into the test's one trace, and each gets its own video.
The app under test
test_helper.exs starts what a run needs, as Playwright Test's webServer does. To start the app
server only when browser tests run (mix test --only e2e, also with --failed), check the
run's filters, which Mix sets before it loads the helper:
# test/test_helper.exs
if :e2e in ExUnit.configuration()[:include], do: MyApp.E2E.start_server!()
ExUnit.start(exclude: [:e2e])When the unit tests' helper doesn't suit a run against the real app (Mox globals, say), give the
browser tests a dir and an env of their own: test_paths: if(Mix.env() == :e2e, do: ["e2e"], else: ["test"]) in mix.exs, with e2e/test_helper.exs and config/e2e.exs, run by MIX_ENV=e2e mix test. A dir under test/ must also be kept out of the other envs with test_load_filters
(test_ignore_filters only silences the warning). Or make it a Mix project of its own that
reaches the app over HTTP.
An app running as an OS process of its own needs no start in the test VM: mix test --no-start
starts no application, so start this library's in test_helper.exs
(Application.ensure_all_started(:playwright)). A server the tests start themselves inherits
their environment, MIX_ENV included when the shell or CI job sets it: pass the child's
explicitly (env: [{~c"MIX_ENV", ~c"dev"}] in Port.open/2).
Async modules run at once against one app and one database. Seed data each module owns (names
with System.unique_integer([:positive])), or wrap seeding that can't be unique in
:global.trans/2. For an Ecto app in the test VM, each test's browser must reach its sandbox:
add Phoenix.Ecto.SQL.Sandbox to the endpoint and its LiveView hook (see its docs), and send the
test's metadata as the context's user agent:
setup %{browser: browser} do
owner = Ecto.Adapters.SQL.Sandbox.start_owner!(MyApp.Repo, shared: false)
metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(MyApp.Repo, owner)
context = Browser.new_context!(browser, user_agent: Phoenix.Ecto.SQL.Sandbox.encode_metadata(metadata))
on_exit(fn ->
BrowserContext.close(context)
Ecto.Adapters.SQL.Sandbox.stop_owner(owner)
end)
%{context: context, page: BrowserContext.new_page!(context)}
endShared mode ({:shared, self()}) would need async: false. The on_exit closes the context first:
the test's contexts close after your on_exit callbacks, by when the owner is gone and the app's
LiveViews log ownership errors.
Accessibility testing
@axe-core/playwright isn't ported: install axe-core with your package manager, evaluate its
script in the page (again after each navigation), as AxeBuilder does, and run it:
axe_path = "assets/node_modules/axe-core/axe.min.js"
Page.evaluate!(page, File.read!(axe_path))
result = Page.evaluate!(page, "() => axe.run()")
assert result["violations"] == []axe.run(context, options) takes what AxeBuilder's include, exclude and withTags set:
Page.evaluate!(page, "([c, o]) => axe.run(c, o)", [%{include: ["main"]}, %{runOnly: ["wcag2a"]}]).
Take the axe-core your JS suite's @axe-core/playwright resolves, for the same rules. It checks
the main frame only: AxeBuilder also runs axe in each iframe and merges the results.