API testing with Playwright & odottaa

9 min readApr 11, 2022

Quick summary: I’m pretty sure that you have heard that Playwright is a great tool for End-to-End and UI integration tests. But, have you ever heard that Playwright can be used as a tool for testing an API? In this article, I’m going to share a quick overview of how easy it can be to quickly set up your project to test API with Playwright and odottaa.

What is API testing?

Application Programming Interface is often called API. API testing is a type of software testing that analyzes an application program interface (API) to verify it fulfills its expected functionality, security, performance, and reliability.

Advantages of API testing

Time efficiency;
Language-independent;
Tests stability;
Reduced testing costs;
and many more…

Introduction to Playwright and odottaa

Playwright is a framework for Web Testing and Automation. Moreover, it can be used for UI and API testing. But, how do we test whether a GET request is returning the JSON file we are expecting, given a defined input? We need an assertion library, that is why Playwright is not enough.

So, here it is odottaa, the assertion library for the better testing feasibility

odottaa is a custom Playwright’s matchers to test the state of the API response. It extends Playwright’s built-in assertions with matchers for the Playwright’s APIResponse class. Hence, it suits our needs for testing an API.

Playwright and odottaa complement each other so well, so in this article, we will be using both.

Setting up an environment for testing

Before we start writing automated tests, let's spin up the testing server locally. For this article, I will be using a Node.js service powered by json-server that exposes a few endpoints:

Posts:

GET /posts — retrieves all available posts
GET /posts/id — retrieves specific post by id
POST /posts — creates new post
PUT /posts/id — updates specific post by id
PATCH /posts/id — modifies specific post by id
DELETE /posts/id — deletes specific post by id

Furthermore, there is the comments endpoint that supports the same methods as posts (e.g. GET /comments — retrieves all available comments). However, in this article, we will focus only on /posts.

You can find the source code on GitHub. Also, you can set up it locally.

Run the following commands to have it locally:

git clone https://github.com/elaichenkov/playwright-api-testing-example.git && cd playwright-api-testing-example

The first command copies an existing Git repository while the second change the directory.

Now install dependencies by running the command:

npm install

Start the server by the command:

npm start

Congratulations, you just have served json-server locally which can be accessed on the 1337 port.

Now all is ready for the development of automated tests.

Using Playwright and odottaa for API testing

Playwright can send requests to the server directly from Node.js without loading a page and running js code in it.

Playwright Test runner comes with the built-in request fixture.

First, we need to create an empty directory, move into it, and install all dependencies by running these commands in the terminal:

mkdir playwright-api-testing-with-odottaa && cd $_

The first command creates the directory while the second cd $_ change the directory to the retrieved the last argument of the previous command.

Now let’s initialize playwright project and install odottaa library:

npm init playwright

It will prompt you to press Enter key, select TypeScript and put test files inside tests directory and then press Enter key a few times again.

After Playwright has been installed and the project has been initialized, the next step is to install odottaa matchers:

npm i -D odottaa

We’re almost ready to start developing the automated tests for our service.

Last, open the playwright.config.ts file in your favorite IDE and replace its content with the following piece of code:

Set up playwright.config.ts

Now let me explain in detail what actually happens in the playwright.config.ts file, step by step:

Import dependencies
Extend expect with odottaa matchers
Declare config constant and assign an object to it
Set up webServer option to launch the development web server during the tests. Read more about all its properties here
Set up baseURL inside the use object. So that tests could use relative URLs

Writing your first Playwright API test with odottaa

Before creating our first test we can remove e2e test that was created automatically during the project initialization. You can do it manually in the folder or run the command in the terminal:

rm -rf tests/example.spec.ts

Now it is time for creating a new test file for API testing. In the tests directory, create posts.api.spec.ts file, which is where we will write our API tests. As a previous one, you can do it manually in the folder or run the command in the terminal:

touch tests/posts.api.spec.ts

Open the posts.api.spec.ts file to set up the test on the API GET request for the /posts endpoint. Add this code:

Get posts test #1

In this example, let’s learn some basics of Playwright Test runner. Some key things are used a lot in this article, and you will need to understand them:

Basic of Playwright:

test.describe — this optional method is used for grouping any number of test statements;
test — pass a function to this method, and Playwright Test runner would execute that function as a block of tests;
request — fixture that respects configuration options like baseURL or extraHTTPHeaders we specified and are ready to send some requests;
expect — this is the condition that the test needs to pass. It compares the received parameter to the matcher. It also gives you access to built-in matchers (e.g toBeOK) and odottaa matchers (e.g. toBeCreated, toBeForbidden, toContainJSON, and many more) that let you validate different things.

The test above demonstrates how to use Playwright Test runner and odottaa library to test /posts endpoint. The test suite does the following:

Send GET request to retrieve posts and store the response result in the constant;
Use toBeOK Playwright Test built-in matcher to assert that the response is between 200 and 299;
Use toContainJSON odottaa matcher to assert that the response body JSON contain default post object.

Let’s run it and verify that test is passed. Use the following command in the terminal:

npx playwright test

or you can use Playwright Test for VSCode plugin:

But, you may ask “What should I do if I don’t know the id of the post and I don’t want to send an additional request to get an ID?” If so, you can use expect.any(Number) instead of a literal value to verify that id in the post is a number.

Get posts test #1 but with expect.any(Number)

It looks like the same except the line #7:

diff id

NOTE: expect.any(constructor) matches anything that was created with the given constructor or if it’s a primitive that is of the passed type. You can use it inside toContainJSON ,toHaveJSON and toMatchJSON instead of a literal value.

Writing and adding more tests to our test suite

In the first test, we verified that an array of posts contain a specific object. Now, let’s verify that /posts returns an array of objects. We can do it with the toHaveJSON matcher.

posts.api.spec.ts

Let’s move onward to testing the post creation. But, before we start creating tests for post creation I need to say that our json-server accepts POST request only with authorization token or user credentials and Content-type: application/json headers. Let’s verify that we will get an error if try to create a post without an authorization token:

posts.api.spec.ts

Now, run the test:

npx playwright test

The output in your terminal should be like this:

After that, we can create a similar test but with an authorization token:

posts.api.spec.ts

That is awesome! In the test, we used authorization token to create a new post and then we verified that the response status code is 201 with toBeCreated method and the post response matches the object that we used for creation with toMatchJSON matcher.

Keep in mind that odottaa provides matchers for different status codes such as:

toBeCreated — verifies the status code is 201;
toBeUnauthorized — verifies the status code is 401;
toBeForbidden — verifies the status code is 403;
toBeNotFound — verifies the status code is 404.

Feel that we missed some needed method? Please, fill an issue or create a PR.

It seems that we have some duplicated code that we can optimize with playwright.config.ts . So, first of all, we can move our Content-type: application/json header property from test blocks to the extraHTTPHeaders property that is placed inside use. Moreover, we can move post variable at the top of the file because we use it in two tests:

config file with extraHTTPHeaders property

Even more, we could move our authorization token there too, but if we do that then we will not be able to test negative cases which we have for an unauthorized user.

NOTE: I have a hardcoded authorization token in the codebase. But, in the real world, you probably would have to retrieve this token or generate it. Moreover, you should not store it as a variable in the test block.

After we wrote tests for post creation we can write tests for the post deletion. But, before deleting we have to know id. Hence, we need to get all posts and get the id of the post:

posts.api.spec.ts

For this test, we need to verify that the status code is between 200 and 299 and that response is an empty object {}.

Furthermore, we can cover PUT and PATCH methods for this test suite, but I will leave it for you. It will be your homework to create a PR with the tests.

Instead of covering the remaining HTTP methods let’s focus more on test data because I can see there are some problems. Probably the hardest thing in the automated testing world is data management. We always should have fresh and actual test data to see green pipelines. If you didn’t get the point then just run again our test suite and you will see the following error:

Some of the tests failed. It happens because we have deleted and we have created some posts. But, our tests expect the exact data to be in the database. Let’s solve this problem by using globalSetup.

globalSetup — is used to set something up once before running all tests.

So, add the globalSetup property with the path to the file in the playwright.config.ts file like this:

config.ts

Now create globalSetup.ts file in the root directory and add this code:

global-setup.ts

As I already mentioned this function will be running once before tests execution. In the function, we retrieve all posts, remove them by id, and create a default one.

Now let’s run it just to make sure that our tests are stable and passed.

Run the command in the terminal:

npx playwright test

Our final test file should be like this:

final test suite

Setting up CI with GitHub Actions

Have you ever considered automating the API tests with GitHub Actions? It doesn’t matter if you haven’t gotten around to wrapping your head around GitHub Actions yet. I will show you how to do it. Since we decided to set up GitHub actions as CI during the initialization project we have playwright.yml in the .github/workflows directory. So, now just we need to update it with the following code:

playwright.yml

After that, we can create a repository on GitHub and push these changes. It will trigger the CI pipeline automatically, start the server, and run our tests on it. Then, you will see the result:

Moreover, you can add an HTML report and publish it on GitHub pages. Check out the published example here.

Conclusion

I hope my post here has given you a good introduction to API testing with Playwright and odottaa. I have also introduced you to the process of running API tests on CI.

If you still have any questions, you can leave them in the comments section below, and I will be really happy to answer every one and work through any issues with you.

Also, don’t forget to give it a star on GitHub.