Implement E2E tests with Playwright to achieve user-centric testing including inter-system integration

Introduction

When thinking about improving system quality, the perspective of “Are we providing the service correctly to users?” becomes an important factor.

For frontend development tests, I have previously introduced unit tests using Jest and UI component tests using React Testing Library.

However, that alone is not enough to guarantee that the service is being correctly provided to users.

In reality, many systems are built on top of integrations with various backend services (APIs, external storage, etc.).

Therefore, this time we will implement tests for the entire system including those integrations using Playwright.

Goal for this article

We will implement test code using Playwright’s GUI tools.

Here is how it looks when performing test actions in a real browser.

After that, we will run the tests and confirm that they complete successfully.

We will also introduce a report that makes it easy to see which test failed and why, even when the number of test cases increases.

What is E2E testing?

E2E (end-to-end) testing is a method of testing the entire flow of an application from the user’s perspective.

It aims to reproduce how users operate the system and verify the behavior of the entire system in an environment close to actual usage.

Concretely, it tests a series of operations such as login, data input, page transitions, saving and loading data, and verifies whether they work as intended.

Benefits of E2E testing:

• Real user perspective
  You can test in a way that is close to the user experience, making it easier to detect user-facing issues in advance.
• System-wide verification
  You can verify the behavior of the entire system, including frontend, backend, and database.
• Early bug detection
  By detecting issues early in an integrated state of each feature, it contributes to improving quality before release.

However, E2E tests generally take longer to run and tend to have more complex environment setup.

Therefore, it is important to narrow down to the necessary cases and manage them appropriately.

What is Playwright?

Playwright is an open-source end-to-end (E2E) test automation tool developed by Microsoft.

It is specialized for web application testing and is designed to easily verify behavior across multiple browsers (Chromium, Firefox, WebKit, etc.).

https://playwright.dev/

Features of Playwright

• Multi-browser support
  Playwright supports major browsers such as Chromium (Chrome and Edge), Firefox, and WebKit (Safari’s engine). This makes it easy to verify whether your application works correctly in different browser environments.

• Parallel test execution
  Playwright can run tests in parallel, enabling fast execution even with a large number of test cases, which helps improve development speed.

• Cross-platform
  It runs on Windows, Mac, and Linux, so you can easily run tests on any OS. It also supports mobile browsers, allowing testing on both desktop and mobile.

• Rich APIs
  Playwright provides APIs that can finely control events such as page load completion, network requests, and DOM operations. This makes it easy to test complex user scenarios, page transitions, file uploads/downloads, and more.

• Screenshot and video capture
  It can capture screenshots and videos during tests, making it smooth to check test results and investigate bugs.

Benefits of Playwright

• Fast feedback
  Thanks to parallel tests and cross-browser support, you can quickly check test results, which contributes to faster development.
• High reliability
  Because it can control network and screen states, it allows testing under specific conditions and improves the reproducibility of errors.
• Easy setup
  It is relatively easy to set up and supports various browsers by default, so the introduction cost is lower compared to other tools.

Playwright is a convenient tool for frontend developers and QA engineers to perform E2E testing.

Introducing Playwright

The Playwright official documentation provides installation steps, so we will follow those.

https://playwright.dev/docs/intro

Run the following command in an appropriate directory.

npm init playwright@latest

You will then see several options, but the default settings are generally fine, so just press Enter to proceed with the installation.

• Choose between TypeScript or JavaScript (default is TypeScript)
  Choose either TypeScript or JavaScript (default is TypeScript)
• Name of your Tests folder (default is tests or e2e if you already have a tests folder in your project)
  Name of the Tests folder (default is tests, or e2e if your project already has a tests folder)
• Add a GitHub Actions workflow to easily run tests on CI
  Add a GitHub Actions workflow to easily run tests on CI (default is false)
• Install Playwright browsers (default is true)
  Install Playwright browsers (default is true)

E2E testing using sample code

If you check the test folder, you will find sample test code prepared.

import { test, expect } from '@playwright/test';

test('has title', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Playwright/);
});

test('get started link', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Click the get started link.
  await page.getByRole('link', { name: 'Get started' }).click();

  // Expects page to have a heading with the name of Installation.
  await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});

Two tests are prepared.

• has title
  When you open the first page of the official Playwright documentation, the title contains the text “Playwright”.

• get started link
  On the first page of the official Playwright documentation, there is a “Get started” link, and clicking it navigates to a page with the title “Installation”.

Since the test code already exists, let’s run the tests right away.

npx playwright test

If you see “6 passed” as shown on the screen, you are good.

Earlier I mentioned that there are two tests, but when we actually run them, six tests are executed.

If you open the playwright.config.ts file, you can see the following description.

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },

    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },

Playwright’s projects option is a feature that allows you to run the same tests under different test environments and settings.

For example, if you want to test behavior across different browsers (Chrome, Firefox, WebKit) or different devices (desktop and mobile) at once, you can easily switch and configure them using projects.

This enables efficient cross-browser and device testing across multiple environments.

This time, we ran two tests in three different browsers (Chrome, Firefox, WebKit), so the tests were executed six times.

Creating test code

You can write test code directly in .test.ts or .spec.ts files, but there is also a method similar to Excel macros where you operate the browser and generate code from those actions.

This time, we will create test code using the code generation method.

Running the following command will launch two windows.

npx playwright codegen

The browser you actually operate

The window that displays the generated code based on your browser operations

If you enter “ https://playwright.dev/ ” in the browser, the official Playwright documentation will be displayed, and your operations will appear as code.

Next, in the footer of the same top page, there is a link to Playwright’s “GitHub”, so we will use the eye icon “assert visibility” to record an operation that verifies its existence.

Your operations are recorded, and in the end, code like the following is generated.

test('test', async ({ page }) => {
  await page.goto('https://playwright.dev/');
  await expect(page.getByRole('link', { name: 'GitHub', exact: true })).toBeVisible();
});

With this, the test code “When you open the first page of the official Playwright documentation, a link to GitHub is displayed” is complete.

Let’s make the test name a bit clearer and paste it into the sample test code.

+ test('has github link', async ({ page }) => {
+   await page.goto('https://playwright.dev/');
+   await expect(page.getByRole('link', { name: 'GitHub', exact: true })).toBeVisible();
+ });

The browser we launched for creating test code is no longer needed, so close it with “Ctrl + C” in the terminal.

Finally, run the tests and confirm that they complete successfully.

npx playwright test

If all 9 tests complete successfully, you are good.

Test result reports

Up to now, we have been checking test results in the terminal, but reports are also being generated.

These become particularly useful when tests fail.

So we will first create a failing test case and then look at the report.

We slightly modified the test case we just created so that the test fails.

test('has github link', async ({ page }) => {
  await page.goto('https://playwright.dev/');
-   await expect(page.getByRole('link', { name: 'GitHub', exact: true })).toBeVisible();
+   await expect(page.getByRole('link', { name: 'GitHub1', exact: true })).toBeVisible();
});

By changing the link name from “GitHub” to “GitHub1”, we made it point to a non-existent link.

npx playwright test

When you run the tests, you can see the failure message.

(The terminal already shows enough information to identify the problem, but still…)

Six tests passed and three failed.

About the report server

The terminal shows “Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.”

If you actually access “http://localhost:9323” in your browser, you can see the test result report.

Previously, when all tests passed, the message “Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.” was not displayed.

This is Playwright’s default behavior: when tests fail, it immediately starts its own HTTP server locally so that you can quickly check the report.

https://playwright.dev/docs/test-reporters

So does that mean you cannot see the report when tests pass? Not at all—you can start the report server and view the report using the following command that appears at the end of the test run.

npx playwright show-report

That said, when everything passes, you probably won’t need to look at the test details that closely, so operating with the default behavior should be fine.

About the report contents

This time we only had 9 tests, so it was easy to see which ones failed, but when you have many tests, you can click the “Failed” tab in the upper right to filter down to only the failed test cases.

You can see the test name, the browser used, and the time taken for the test.

Let’s check the error details for chromium as an example.

The failure occurred on line 22, and the error is a timeout of 5000 ms (5 seconds).

In Playwright, the default Expect Timeout is 5 seconds.

There is an upper limit on how long expect assertions will wait for a condition to be satisfied. For example, when testing asynchronous processes or state changes, it defines how long to wait for a specific condition to be met.

Concretely, it is useful in tests that wait for a UI to load, or for data to be fetched asynchronously, or for a particular state to be reached.

https://playwright.dev/docs/test-timeouts

In simple terms, this error means it spent 5 seconds looking for a link named “GitHub1” but could not find it.

In addition to Playwright’s built-in reports, there are various third-party reporting tools.

If you only want to see the result of a single test run, the standard Playwright report is sufficient.

If you run tests daily, third-party tools can be useful for understanding trends over time or for integrating test results with GitHub or Microsoft Teams to reduce operational overhead.

https://playwright.dev/docs/test-reporters#third-party-reporter-showcase

Conclusion

In this article, we introduced Playwright and walked through everything from setup to running E2E tests.

As you saw with the code generation feature, even non-engineering roles who do not understand the internal implementation can easily create test code and run tests.

In the world of web services, systems change every day, so E2E tests also need to evolve accordingly.

By involving QA teams, PMs, directors, and others to achieve stable test operations, I believe we can also achieve stable system operations, and I am working on advocacy activities with that in mind.