Running Postman tests using CI has never been simpler

Do we even need to test our software?

Application Programming Interface (API) is something that web applications can't work without. They use it to integrate with each other and share data they would need to otherwise generate themselves. REST, GraphQL, and SOAP are the most popular ones nowadays.

It is obvious that we need to test our software. There were plenty of cases when applications were not tested seriously and we have seen spectacular disasters, like the famous Mars Climate Orbiter crash from 1999 or Mariner 1 explosion from 1962.

API testing tools

After all of those years of software development, we know more than ever. This is why we put so much effort into testing. API tests are part of the Web Applications development process and there are plenty of ways to do it. There are various frameworks that can be used to generate responses as well as check response types and data received. We can do it either using a programming language or GUI tools like Postman or Insomnia. The first one is especially popular among developers. I was told it is the first-choice tool the first time I heard about it. Keep in mind I had very little knowledge about API back in the day.

I feel like every backend developer has worked with Postman at least once in their career but only a few know that it is possible to create automated - parameterized tests using it. Little to none knows that it is possible to run them using CI tools.

Using Postman variables

Creating requests is very simple so I will just show you some examples of variables that can be created and used in our collection:

• global variables
• collection variables
• environment variables

Once we have them we can easily parametrize our requests using those variables with {{variable_name}}. For example, this is a parametrized POST request I created while playing with https://jsonplaceholder.typicode.com/ API.

Creating automated tests

Now, that we have our request, we can create various checks and tests to make sure everything works as it should. This is an example of me checking the status code and post title.

By simply sending that request we can check the server response. I have created 2 more tests to verify my variables working.

A Postman is a reliable tool that makes API testing easier. It empowers people without backend knowledge. It has an intuitive interface and once we are logged in we can save and share our requests and tests with a link as everything is stored in the Postman cloud.

Prepare to run Postman using CI

Those features are nice but there is more to it. It is possible to setup a CI environment and run regular API tests using the tool Postman provides - Newman. All we need to do is:

• install NodeJS
• install newman
• create some tests and export them as a file or link.

Once we have NodeJS installed we should install Newman:

npm install -g newman

Then we should export a collection via file or a link. In my case it is:

https://www.getpostman.com/collections/<collection_id>

We can do it the same way with variables. In order to execute tests we should run them in our terminal:

newman run <collection> -e <environment>

If our tests passed then output should look like:

→ GET Posts GET https://jsonplaceholder.typicode.com/posts?title_like=sunt [200 OK, 3.06kB, 156ms] → GET Comments GET https://jsonplaceholder.typicode.com/comments?postId=2&postId=5&_limit=2 [200 OK, 1.78kB, 51ms] ✓ Verify response status code ✓ Verify response time ✓ Verify status OK ✓ Verify post id ✓ Verify name length ✓ Verify response email ✓ Verify all response emails ✓ Verify content type header ✓ Verify content type header value → POST Post POST https://jsonplaceholder.typicode.com/posts [500 Internal Server Error, 1.63kB, 286ms] ✓ Status code is 201 ✓ Verify global post title ✓ Verify collection variable ✓ Verify environment variable → PUT User PUT https://jsonplaceholder.typicode.com/users/2 [200 OK, 1.33kB, 268ms] → Delete User DELETE https://jsonplaceholder.typicode.com/users/2 [200 OK, 993B, 291ms] ┌─────────────────────────┬────────────────────┬───────────────────┐ │ │ executed │ failed │ ├─────────────────────────┼────────────────────┼───────────────────┤ │ iterations │ 1 │ 0 │ ├─────────────────────────┼────────────────────┼───────────────────┤ │ requests │ 5 │ 0 │ ├─────────────────────────┼────────────────────┼───────────────────┤ │ test-scripts │ 7 │ 0 │ ├─────────────────────────┼────────────────────┼───────────────────┤ │ prerequest-scripts │ 6 │ 0 │ ├─────────────────────────┼────────────────────┼───────────────────┤ │ assertions │ 13 │ 0 │ ├─────────────────────────┴────────────────────┴───────────────────┤ │ total run duration: 1306ms │ ├──────────────────────────────────────────────────────────────────┤ │ total data received: 3.68kB (approx) │ ├──────────────────────────────────────────────────────────────────┤ │ average response time: 210ms [min: 51ms, max: 291ms, s.d.: 93ms] │ └──────────────────────────────────────────────────────────────────┘

Now we are set up. We can choose and configure CI that fits us best: Jenkins, CircleCI, Gitlab, or Github. We just need to install NodeJS, Newman and run tests from a collection that we exported using the link. We can save the output to the file using the Newman reporter:

newman run https://www.getpostman.com/collections/<collection_id> --reporters cli,json --reporter-json-export outputfile.json

Is it worth it?

Postman is a very powerful tool that people tend to forget about. I wanted to show you that you can use it for much more than just sending requests. This is only the tip of the iceberg. I highly recommend you play with it on your own and discover its full potential.

It is definitely worth using it when you are interested in testing API and you need something that works out of the box. Most backend developers would probably prefer to write tests using the programming language of their choice. However, if you are not experienced with backend development or just want to create fast and efficient API tests for your CI, then Newman is the way to go. If you have any question do not hesitate to contact me at kamil.senecki [at] deployed.pl

Till next time,

Kamil