Run Postman API Tests in the Azure DevOps PipelinesBy Thilanka Weerasinghe
How’s your QA experience with API testing so far? Have you ever gone through a hazel of testing the same thing again and again when you wanted to do a release? Though all APIs work in the QA environment perfectly and does it fails in the UAT? As developers do you want to be bug free with your new integration? To solve all these problems we can introduce run your postman API tests in continues integration process. In this article I’m going to explain how you can configure your Azure DevOps pipelines to run your postman scripts.
Configuring pipeline will be divided to 3 main steps:
Create test artifacts
Run postman scripts
01. Create test artifacts
Test artifacts are the Postman scripts which needs to be execute. There are two different ways to create test artifacts for release pipeline
a. Through build pipeline
Most popular way is to use an existing build pipeline or new pipeline to create artifacts. Exported postman scripts and environment variables should be added to the source controller and in the build pipeline it should be copy and published as aftifacts.
1.Export postman scripts
Your postman collection which you wanted to run in the Azure DevOps pipeline should be exported.
2.Export environment variables
Environment variable which supports postman scripts should export as well.
3.Add it to source control
Exported postman and environment variable scripts should be added to the source control in order to access them in the pipeline.
4.Create build pipeline
If there is no exiting pipeline to create artifacts then create a new build pipeline to create the artifacts using following steps.
i. Select pipeline option of under pipelines menu
ii. Click on New Pipeline button
iii. Select ‘Use the Classic editor ‘ link
iv. Select the source
If the source is Azure Repos Git then project, repository and default branch should be selected
v. Click on Empty job
Empty job option needs to be selected to create the pipeline.
5.Copy postman scripts and environment variables
This task will copy relevant files to specific folder which you can be access in the release pipeline.
i. Contents: File paths to copy
ii. Target Folder: Target folder which files needs to be copied
Copied files will be published as artifacts folder called “drop” in order to access from the release pipeline.
i. Path to Publish: Path which will publish the artifacts
ii. Artifact Name: Name of the artifact
iii. Artifact Publish Location: Location which artifacts should store after publishing. It can be Azure pipelines, or copy it to a file share which can be accessible from the build agent.
b. Directly download from repository
If postman scripts are maintained in a different repository it can be downloaded directly to the release pipeline. In this scenario edit the release pipeline and select that artifacts from relevant repository.
02. Run scripts in release pipeline
API scripts needs to be run in the release pipeline after the deployment of new API. In order to run and get the results following steps needs to be followed.
1.Create new release pipeline or edit existing pipeline
This will be based on the place which you are supposed to run the API scripts. Either can use a new release pipeline to run scripts or use existing release pipeline to run the scripts after the deployment of API
Creation steps are as follow:
i. Select releases option under pipeline menu
ii. Click New button
iii. Select New release pipeline option
iv. Select template as Empty job
Artifacts need to link to the release pipeline. Artifacts can be coming from build pipeline or different Azure repository (As mentioned earlier).
Steps are as follow:
i. Click Add button of Artifacts
ii. If from build pipeline: Select relevant project and build pipeline and give a source alias
3.Create a stage by clicking Add button
4.Create task to Install newman
Task which needs to perform should configure inside the created stage. First task is to install newman.
Newman is required to run postman scripts. This task will install newman in agent machine before running postman scripts.
i. Add command line task
ii. Configure it to install newman
npm install -g newman
5.Create task to run postman scripts
To run postman scripts there should be a command line task to execute the command line scripts as below.
newman<<postman_script_name>>.json -e <<environment_variable_name>>.json --env-var "<<environment_var_to_override>> =<<new_value>>” --reporters cli,junit --reporter-junit-export <<report_name>>
newman run TestProject_API_SmokeTests.postman_collection.json -e TestProject_QA_Enviornment.postman_environment.json --env-var "url=http://api.qa.testproject.com" --reporters cli,junit --reporter-junit-export Results\\apiTestResults.xml
Working directly should be clearly mentioned according to the place which has postman scripts
03. Generate report
Tests results will be published as junit test results. For that it will use the test results file generated.
- Add publish test results task
Now it’s all set to go. Once you create the release you will be able to see the test results as below.