Test automation for an API: Postman, Newman and Jenkins

What is test automation?

Test automation is the use of a software to control the execution of tests and comparison of the actual result and the expected result. To explain how to achieve a fully test automation of an API we need three programs: Postman, Newman and Jenkins.

Postman

The first program we are going to need to obtain a test automation is Postman, and it can be defined as follow:

“The main purpose of Postman is to help build APIs faster, allowing the creation of requests quickly and build workflows using collections”

Environment

Before we start to create tests, first we need to set an environment for the server we are going to execute the tests in. To do that we go to (top-right) Manage environments → Add. Then we can start to add pairs key-value with the variables we are going to use multiple times in the requests.

As an example, I chose the following API to show how this works: swapi.co/api, so we are going to save it in a variable called host:

This API doesn’t need the user to authenticate to use it, but if we use one that need it we can save in the environment variables parameters like id and password.

Collection

Postman lets you create collections to store the requests, so the first thing we need to do is create a Collection, giving it a name (required) and a description.

Then we can create our first request, that will look like this:

In this example we tried to get a list of starships, so in the screenshot we can observe:

  1. Request name.
  2. Method: It can be GET, POST, PUT, DELETE…
  3. API: The full path would be swapi.co/api/starships, but we store swapi.co/api in the variable host, then to use it we just need to type {{host}}.
  4. Environment.

The second part of a request is the Tests section, where we create asserts to verify that the information returned by the server is what we expected and also create new environment and global variables to store values. For example:

In this test we set an assert that checks that the status code of the request is OK (200) and a loop where we look for specific information to store in an environment variable. On the right side of the screen we have a series of snippets to help us build the tests.

The next step would be store the request, to do that we just click on “Save” and choose where to save it. We can choose to save it directly in the collection.

Or in a folder. Store the requests in folders is useful if we want to isolate the execution of some requests from the rest:

Executing Tests

To execute the requests we can do it by clicking on “Send” once we have save it (Important!). Then we can check the Body received in json format with all the information about the starships and the results of the tests.

In this case a test fails because we indicate that we expected more than 100 starships and if we observe the body there are less.

When we have all the requests with its tests, we’ll want to execute them. To do that we need to open the Collection Runner:

To execute the tests we have to follow this steps in order:

  1. Choose between execute the whole collection or one of its folders.
  2. Set the environment where we have stored the variables.
  3. Number of times we want to execute the tests.
  4. Start Test.

Now we can see the results of the test on the right side of the screen. At the top there is a summary of the whole process where we can observe the number of tests that passed, how many didn’t and how much time took the execution, and under it we can see the same summary for each request made.

Newman

Postman has a Command Line Version called Newman, it has many utilities but we only need one: It lets you Integrate Postman Collections with a Build System like Jenkins.

The installation of Newman is pretty simple:

npm install -g newman

But before we start to execute collections first we need to export them from Postman. We’ll have to export the environment:

And the Collection:

This actions will save a json of the environment and another of the collection in a folder, where we have to navigate using a terminal to execute Newman.

Now, we can execute to whole collection:

newman -c collection.json -e environment.json

Or just one folder:

newman -c collection.json -e environment.json -f Planets

And in the output we can see almost the same as we did in Postman’s output: The execution of each request and a summary with the tests that failed and those that didn’t:

Jenkins

The final step to complete the test automation for the API is to install newman in the same server we have Jenkins and install some plugins:

  • Log Parser Plugin: To parse the output.
    • Configuration:
      • Description: Error parsing
      • Parsing Rules File: File with the rules and with the following content:
error /✗/

Then we just need to create a new job with the following information:

Project name: Star Wars API (For example)

Build → Add build step → Execute Shell and we type the command lines to execute Newman as we did in local:

newman -c path/collection.json -e path/environment.json -f Starships

Post-build Actions → Add post-build action → Console output (build log) parsing

Then in Select Parsing Rules we select “Error parsing” (Description)

Post-build Actions → Add post-build action → Editable Email Notification

And we fill the following information:

  • Project Recipient List: Email addresses that should receive the email.
  • Advanced Settings → Add Trigger → Script – After Build
    • Trigger Script:
//only send an email if ✗ is found in build logs

build.logFile.text.readLines().any { it =~ /.*✗.*/ }

Save and Build Now…

The console output will show the same output that executing Newman in local does + some lines that appear when Jenkins checks stuff.

Test automation. Automatización de pruebas. Jenkins. Test output

I execute on purpose the folder Starships because I know a test will fail so it will send me an email. The email received will be:

Test automation. Automatización de pruebas. Jenkins. Email sent

In it we can observe that the Subject indicates that something went wrong and in the body that the build was successful and a link to the console output to check the results. Once we are in the console output we can navigate to “Parsed Console Output” to see underlined in red the tests that failed.

Achieve a fully test automation for any API is as easy as following the steps in this post, we just need to set up three programs and we don’t need to worry about anything else. Jenkins will send us an email if something went wrong… 🙂

 

Related articles

JMeter, an old friend in tip-top shape

Testing a mobile app using Sauce Labs, Jenkins and Appium

Experiences with ISTQB Foundation Level Course and test. Is it only worth for the certificate?

Continuous integration using Jenkins for iOS testing

 

5 thoughts on “Test automation for an API: Postman, Newman and Jenkins”

    • Thank you! The thing is, since its inception, postman has evolved to be a complete API lifecycle managing platform (you can find some examples about what it can do today in this article by the New Stack: https://thenewstack.io/postman-adds-api-consumption-api-lifecycle-platform; from mocking to API monitoring). It’s much more than the simple REST API client it used to be at its early stages and can improve your development activity from testing to API debugging, exploratory testing…

      Reply

Leave a Comment

¿Necesitas una estimación?

Calcula ahora