Allure Report Integration with Playwright

Allure, Playwright, GitHub Actions and GitHub Pages

As you all may know, Playwright team has just released 1.13.0 version, with new Reporter API for Playwright Test runner. Moreover, which has been already used to create an Allure Playwright reporter.

So today I’ve prepared a detailed step-by-step guide of how to add Allure HTML reporter in your Playwright Test runner. Even more, I’ll show you how to generate it on each push with GitHub Actions and publish it on GitHub pages with history run. No more words let’s start.

Before we go, I just want to mention that all code examples you can find here. Besides, published Allure HTML report can be seen here.

Installation

I assume that you have already installed playwright and playwright test runner.

But if you haven’t here’s a command for you to install them:

npm i -D playwright @playwright/test

Now, let’s install experimental allure playwright reporter with the following command:

npm i -D experimental-allure-playwright

NOTE:
Once allure-framework/allure-js#297 lands and Allure team pushes the NPM, it’ll become allure-playwright instead of experimental-allure-playwright ©Pavel Feldman (Playwright team member)

And last but not least, we have to install allure-commandline tool to work with it in terminal:

npm i -D allure-commandline

NOTE: Allure command line tool requires Java 8 or higher

Add a test

Now it’s time to write our first test. Let’s create a new file and the tests directory. You can create a directory and file with the commands:

Create a folder:

mkdir tests

Create a file:

touch tests/github-search.test.ts

Write a test

Open up your favourite IDE and add the code below to github-search.test.ts test file:

github-search.test.ts

Run a test

To run it with the Allure reporter we have to use test command with additional reporter argument with line, experimenta-allure-playwright value:

npx playwright test --reporter=line,experimental-allure-playwright

After running it we will see message in the terminal:

passed test

Besides, we’re going to see new allure-results folder in our root directory, which contains few json files.

Generate a report

Using allure command line tool we can generate HTML report based on the json files that we have in the allure-results folder by running the command:

npx allure generate ./allure-results --clean

It will create allure-report folder.

Keep in mind that, you can specify target folder using ALLURE_RESULTS_DIR environment variable.

To open it we have another allure command line command and it’s open :

npx allure open ./allure-report

Woo hoo, it will open browser and show you the Allure HTML reporter:

Allure report

Add a failing test

Now let’s write our first failing test. Add another test below to github-search.test.ts test file in the describe block. So now your completed test suite should look like this:

test suite for github search

Add a configuration file

As you can notice we provided additional reporter argument with value to run our tests. Well, that’s not cool. Even more, we can omit it and include our reporter in the playwright.config.ts configuration file.

So let’s create playwright.config.ts file:

touch playwright.config.ts

Once we’ve created a config file we can add the code below to it:

playwright.config.ts

Run test without additional argument

Now we can run it with the shorter command:

npx playwright test

It should print the following message in the terminal and also the allure-results folder should be created:

playwright terminal message

Add a screenshot on failure

If you would like to add a screenshot on failure, you have to add screenshot property in the use object in the playwright.config.ts config file:

use: {
  screenshot: 'only-on-failure'
}

playwright.config.ts

Add custom npm scripts

One more thing that we can improve is creating custom npm scripts to generate and open the report. Open the package.json file and add the following scripts into the scripts object.

To generate report:

"allure:generate": "npx allure generate ./allure-results --clean"

To open report:

"allure:open": "npx allure open ./allure-report"

To serve report:

"allure:serve": "npx allure serve",

Also let’s add test script:

"test": "npx playwright test || :",

And finally we can add posttest script which will run automatically, after running npm test script. Let’s add generate script into the posttest script:

"posttest": "npm run allure:generate"

So our scripts in pacakge.json file should look like that:

Run npm test command

It should run tests and generate report.

The terminal should print the following message:

stack trace

and after that we can run npm run allure:open command to open the bowser and to see the generated report:

Allure report dashboard

That’s neat. We are almost done.

Create workflow in GitHub Actions

Let’s create folders for the GitHub Actions:

1. .github
2. .github/workflows

Now we can create a configuration yaml file with the steps in the workflows directory:

touch .github/workflows/ci.yml

Add the following code snippet to the ci.yml file:

ci.yml

It will run tests, generate report and publish it on the GitHub Pages, which you can access on the https://${YOUR_GITHUB_USERNAME}.github.io/${REPOSITORY_NAME}/1

Conclusion

I hope the above examples have shown that you can easily add pretty beautiful Allure HTML reporter into your Playwright project. Give Playwright, Playwright Test and Allure a shot and I’m convinced you won’t be disappointed.

Don’t forget to subscribe to Test Automation Weekly digest to keep up with latest news in test automation world.