Most of a spec ports call for call. This guide lists what ExUnit can't host, what isn't ported
yet, and what differs in the rest. Snippets alias Playwright.{Page, Locator, Expect, ...}.
Not supported
Playwright Test features ExUnit can't host, or that ExUnit and Elixir already cover (#210):
| Playwright Test | why | instead |
|---|---|---|
runtime test.skip(condition) | ExUnit can't skip a running test, and its filters don't see parameterize: values | a tag known before the run (@tag skip: ..., --exclude), or an if around the body (reported as passed) |
test.fixme, test.slow | ExUnit has tags for both | @tag :skip, @tag timeout: ... |
retries | ExUnit has no retries in a run | rerun the failures: a mix test --failed after a run whose tests failed (exit status 2), see Retries; the retry modes (trace: :on_first_retry, ...) go by it |
nested or duplicate-name describe | ExUnit's describe doesn't nest, and its names are unique in a module | a module per outer describe (its beforeEach a module setup, its beforeAll the module's setup_all, which a describe can't have); a describe repeated in a loop becomes one describe with the loop inside |
fullyParallel | ExUnit runs a module's tests one after another | a module per describe; for fullyParallel: false on shared state, a group: |
describe.serial, mode: 'serial' | ExUnit runs a module's tests in random order | one test, with a Test.step per test; modules sharing state in the app in one group: (see Setup) |
globalSetup, project dependencies | ExUnit has no projects | test_helper.exs (see Logging in once) |
asymmetric and value matchers (expect.objectContaining, expect.any(String), toMatchObject, toBeCloseTo, ...) | the jest matchers aren't ported | patterns: assert %{"id" => id} = hd(list), then assert is_binary(id); assert_in_delta/3 |
expect.configure({ timeout, soft }) | the options are the receiver's, not a configured expect | the matchers' timeout:, Expect.soft/1 |
json, junit reporters | ExUnit formatters cover them | an ExUnit formatter, such as the junit_formatter Hex package |
list, line, dot reporters | ExUnit's output covers them | ExUnit's CLI formatter |
| reporter plugins (monocart coverage, ...) | there are no reporters to plug into | none: Playwright.Coverage collects, your code reports |
webServer | test_helper.exs starts what a run needs | start the endpoint in test_helper.exs, or server: true in the test config (see The app under test) |
Not ported yet, the items a spec runs into most (What's ported has the full list; open an issue if you need one):
- the HTML reporter, with
blobandmerge-reportsfor partitioned runs; - rebaselining inline aria snapshots into the test source (
--update-source-method); _defaultLaunchOptions: aBrowserType.launchinside a test doesn't inherit the browser'slaunch_options;- recording
setup_allcontexts per test, and a fresh worker after a failed test: asetup_allcontext gets the defaults but no test records it, and its state carries on after a failure (Contexts ofsetup_all). Create the context per test, asPlaywright.Casedoes; - function-valued
useoptions (Playwright Test's option fixtures,test.use({ locale: async ({}, use) => ... })): the options are values; test.failand annotations (test.info().annotations, a trace'sannotationsare[]): they would be tags of the same kind asTest.use/1's,@tag Test.fail()and@tag Test.annotate(...)(#286);- a project's own
outputDir,snapshotDirandsnapshotPathTemplate:output_dir,snapshot_dirandsnapshot_path_templateare config only (#286); - the video's step overlay keeps the title of a process killed inside
step/2,3, and shows an IPv6 host with a zone id asfe80.
What differs
- JS
nullis:null(undefinedisnil), and objects fromevaluateare maps with string keys (result["entry"]["foo"]). A call that returnsX | nullreturnsnil(Locator.get_attribute!/2of a missing attribute). - A JS regex literal PCRE rejects (
/\patched/:\pispin JS withoutu) needsPlaywright.RegExp.new/1, which sends the JS source as it is. elementHandle.evaluate(...)isJSHandle.evaluate!(element_handle, ...): anElementHandletakes theJSHandlefunctions.- Module-level
letstate reset inbeforeEachbecomes per-test state, returned fromsetupor kept in anAgent(unlinked when anon_exitreads it, see A check after the test). expect(fn).toPass()isExpect.to_pass!/2, which returns the passing attempt's value: a callback that assigns an outerletreturns it instead (box = Expect.to_pass!(fn -> ...; box end)).expect.poll(fn).toBe(x)isExpect.poll!(fn, &assert(&1 == x)), and a call started now and awaited later (const p = locator.evaluate(...)) isPlaywright.Promise.async/1, andPromise.all([...])isPlaywright.Promise.all/1of those (see Concurrency).- The
browserNamefixture isbrowser_nameinPlaywright.Case's context (setup_all's and the test's):test "...", %{page: page, browser_name: name},:chromium,:firefoxor:webkit. - The
baseURLfixture isPlaywright.Test.info().use[:base_url]. Therequestfixture is a request context of its own,APIRequest.new_context!(Playwright.request(BrowserPool.playwright(browser))), which in a test gets the test's context options and is disposed of at its end; it is notPage.request/1(upstream'spage.request, which shares the page's cookies). addInitScript(fn)calls the function, but a string is sent as is, so"() => ..."only defines one: pass"(#{fun})()", or the function's body.expect(locator, 'should be logged in')islocator |> Expect.message("should be logged in") |> Expect.to_be_visible!().reporter: 'github'is thePlaywright.Test.GitHubFormatterformatter, which prints only annotations: list it next toExUnit.CLIFormatter(see CI).test('...', { tag: '@slow' })and--grep @sloware an ExUnit tag (@tag :slow, or@moduletag :slowfor the file) andmix test --only slow.- ExUnit runs the modules in a random (seeded) order, so a spec that relies on state another
file left behind (a user it created) fails now and then: set up the state per test, in
setup. - A selector with parentheses can't go in
~s(...): use~s[...]or a string. test.use({ locale: 'de-DE' })at a file's top isuse Playwright.Case, locale: "de-DE", in atest.describeit is@describetag Test.use(locale: "de-DE")(@tagfor one test);projectsareparameterize:with aTest.parameters/1map each (Options, devices and projects). When the module's context can't be configured enough (values only the module's ownsetupcomputes, as ExUnit runs the case template'ssetupfirst, or options that differ per test), create the context in the test or itssetup:Browser.new_context!(browser, storage_state: state). It still gets the defaults and the options of every level, and is recorded.- A spec that shares one page from
beforeAllwould still pay forPlaywright.Case's context and page in each test: write the few lines instead, asPlaywright.Case's docs show. No test records thatsetup_allcontext (see "Not ported yet" above).
Coming from Wallaby
Wallaby's queries become locators, and its assertions matchers that wait for the page, so the
:timer.sleep calls and retry loops go:
# assert_has(session, css(".alert", text: "Saved"))
Page.locator(page, ".alert") |> Locator.filter(has_text: "Saved") |> Expect.to_be_visible!()
# refute_has(session, css(".error"))
Page.locator(page, ".error") |> Expect.not_to_be_visible!()
# Enum.any?(all(session, css("li")), &(Element.text(&1) =~ "Apple"))
Page.get_by_role(page, :listitem) |> Locator.filter(has_text: "Apple") |> Expect.to_be_visible!()Locators are strict, as assert_has's default count of 1 is: a matcher of one element
(to_be_visible, to_have_text with a string) fails on a locator that matches several, and
assert_has's count: is Expect.to_have_count!/2. Element.text is the rendered text, so a text with CSS text-transform needs
Expect.to_have_text!(locator, "RED", use_inner_text: true); to_have_text! reads
textContent. fill_in types key by key, Locator.fill!/2 sets the value with one input
event: for handlers of key events, Locator.press_sequentially!/3. For the SQL sandbox, see
The app under test.