Cypress vs. Playwright for E2E Testing

Michael Lynch (@deliberatecoder)

https://decks.mtlynch.io/cypress-vs-playwright/

Experience

  • Intermediate Cypress user (four years experience)
  • Beginner Playwright user (four months experience)
  • Never written custom plugins for either
    • I’ve used some third-party Cypress plugins

Experience

  • PicoShare: (open-source) Minimalist web app for sharing files
  • Ported 10 nontrivial E2E tests from Cypress to Playwright

Playwright is significantly faster than Cypress

Task Cypress Playwright Difference
Run tests on CircleCI 127s 84s -34%
Run tests from development machine 40s 7s -83%

Cypress has redundant assertions

Cypress has redundant assertions

These two snippets are functionally equivalent

cy.get("#error-message").should("be.visible");

cy.get("#error-message").should(($el) => expect($el).to.be.visible);

Playwright has a single right way

expect(page.locator("#error-message")).toBeVisible();

Cypress assumes a desktop GUI

Playwright works headless

Cypress uses non-standard JavaScript

// Save the route to the guest link URL so that we can return to it later.
cy.get('.table td[test-data-id="guest-link-label"] a')
  .invoke("attr", "href")
  .then(($href) => {
    // Log out.
    cy.get("#navbar-log-out").click();
    cy.location("pathname").should("eq", "/");

    // Make sure we can still access the guest link after logging out.
    cy.visit($href);

    // Continue with the test
  });

Cypress uses non-standard JavaScript

Playwright uses real Promises

// Save the route to the guest link URL so that we can return to it later.
const guestLinkRouteValue = await page
  .locator('.table td[test-data-id="guest-link-label"] a')
  .getAttribute("href");
expect(guestLinkRouteValue).not.toBeNull();
const guestLinkRoute = String(guestLinkRouteValue);

// Log out.
await page.locator("#navbar-log-out").click();
await expect(page).toHaveURL("/");

// Make sure we can still access the guest link after logging out.
await page.goto(guestLinkRoute);

// Continue with the test.

Cypress makes text comparisons hard

<p data-test-id="github-instructions">
  Visit our
  <a href="https://github.com/mtlynch/picoshare">Github repo</a> to create your
  own PicoShare server.
</p>

Naive test:

cy.get("[data-test-id='github-instructions']").should(
  "have.text",
  "Visit our Github repo to create your own PicoShare server."
);

Timed out retrying after 10000ms
+ expected - actual

-'\n      Visit our\n      Github repo to create\n      your own PicoShare server.\n    '
+'Visit our Github repo to create your own PicoShare server.'

Cypress makes text comparisons hard

<p data-test-id="github-instructions">
  Visit our
  <a href="https://github.com/mtlynch/picoshare">Github repo</a> to create your
  own PicoShare server.
</p>

Correct test:

cy.get("[data-test-id='github-instructions']").should(($el) => {
  expect($el.get(0).innerText).to.eq(
    "Visit our Github repo to create your own PicoShare server."
  );
});

Unsurprising text comparisons in Playwright

<p data-test-id="github-instructions">
  Visit our
  <a href="https://github.com/mtlynch/picoshare">Github repo</a> to create your
  own PicoShare server.
</p>

await expect(page.locator("data-test-id=github-instructions")).toHaveText(
  "Visit our Github repo to create your own PicoShare server."
);

Cypress: “You figure out app/test orchestration”

Playwright launches your app for you

webServer: {
  command: "PS_SHARED_SECRET=dummypass PORT=6001 ./bin/picoshare",
  port: 6001,
},

Cypress can’t log to stdout

console.log("hello from Cypress"); // this does nothing

cy.log("hello from Cypress"); // this prints nothing to the terminal

console.log in Playwright just works

console.log("hello from Playwright");

[chromium] › auth.spec.ts:3:1 › logs in and logs out
hello from Playwright

Playwright team feels more responsive

  • I’ve had Cypress PRs sit ignored for months
  • Playwright triaged and responded to my bug in one business day
  • Open bug counts
    • Cypress: 2,782
    • Playwright: 603
Playwright integrates well with VS Code
  • Auto-complete helpful for finding right API

Playwright makes parallel tests easier

  • Parallel tests are one of the killer features of Cypress’ paid service
    • Can run parallel tests for free, not as convenient
  • Parallel tests are first-class citizens in Playwright
    • Testing in parallel against the same server is still a challenge even if Playwright makes it easier

Cypress’ syntax is more consistently fluent

cy.get(".navbar-item [data-test-id='log-in']").should("be.visible");

Playwright’s syntax feels less natural to me

How Playwright works:

await expect(
  page.locator(".navbar-item [data-test-id='log-in']")
).toBeVisible();

What I want:

// INVALID - not how Playwright actually behaves
await page
  .locator(".navbar-item [data-test-id='log-in']")
  .expect()
  .toBeVisible();

Summary

  1. Playwright is significantly faster than Cypress
  2. Cypress has redundant assertions
  3. Playwright works headless
  4. Playwright uses real Promises
  5. Cypress makes text comparisons hard
  6. Playwright launches your app for you
  7. Cypress can’t log to stdout
  8. Playwright team feels more responsive
  9. Playwright integrates well with VS Code
  10. Playwright makes parallel tests easier
  11. Cypress’ syntax is more consistently fluent