Playwright Test's github reporter as an ExUnit formatter: GitHub Actions annotations for
failed tests, errors outside a test, slow test files and the run's summary. It prints only
workflow commands, so it runs next to ExUnit.CLIFormatter, on CI, as upstream's documented
reporter: process.env.CI ? 'github' : 'list':
mix test --formatter ExUnit.CLIFormatter --formatter Playwright.Test.GitHubFormatter--formatter replaces the configured formatters, so name ExUnit.CLIFormatter (and any of
your own) too. A CI that reruns failed tests (mix test || { [ $? -eq 2 ] && mix test --failed; }) passes it to both runs, as upstream annotates every attempt. Or add it in
test/test_helper.exs:
github = if System.get_env("GITHUB_ACTIONS"), do: [Playwright.Test.GitHubFormatter], else: []
ExUnit.start(formatters: [ExUnit.CLIFormatter | github])It prints:
- an
::errorfor each failure of a failed test, at the first frame of its stacktrace in your code (not Playwright's, ExUnit's, Elixir's, Erlang/OTP's or a dependency's underdeps/; a path dependency is yours), else on the test's file, titledfile:line › describe › test, with ExUnit's failure text, underRetry #1on the rerun. - an
::errorfor each failedsetup_all; its tests count as not run. - a
Slow Test::warningfor each of the 5 slowest files whose tests took more than 5 minutes (upstream'sreportSlowTests:config :playwright, :test, report_slow_tests: [max: 5, threshold: 300_000]in ms,falsefor none). - a
🎭 Playwright Run Summary::notice: the failed and flaky tests, the counts of skipped, not run and passed tests, and the run's duration.
ExUnit has no retries in a run: a test under mix test --failed is upstream's retry 1 (as
Playwright.Test.info/0's retry), so one that passes there is flaky. Each run prints its
own summary, and a flaky test keeps the first run's ::error, as upstream's flaky test does.
Unlike upstream, a test not run after a failed setup_all that passes on the rerun counts as
flaky too: the failures manifest doesn't record why a test is in it.
Paths are relative to GITHUB_WORKSPACE. It reads only ExUnit's events: any ExUnit suite
can use it, with or without Playwright.Test.
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.