Atlassian Bamboo® + TestRail Integration Guide¶
Why you need Bumblebee
In many organizations, Atlassian Bamboo® is used to build software, run unit tests, and run various kinds of testing frameworks. Example: Selenium, JUnit, Pytest, TestNG, Visual Studio Test, etc. These test metrics are very important and should be reflected in TestRail to show the true health of any project. Agiletestware Pangolin TestRail Connector plugin for Atlassian Bamboo® allows users to integrate any testing framework with TestRail without making any code changes or writing custom tools.
Integrating of Atlassian Bamboo® and TestRail is very easy and requires user to install Pangolin TestRail Connector Add-on from Atlassian Bamboo® Marketplace and do a simple configuration
- Prerequisites¶
- Pangolin Server must be installed and accessible from Atlassian Bamboo® host
- Installation of add-on¶
- Download current version of add-on from https://agiletestware.com/pangolin#download
- Click on "Administration" -> "Add-ons"
- Click on "Upload add-on" -> "Browse" and select downloaded .jar file and click on "Upload" button
- Restart Atlassian Bamboo® server
- Add-on configuration¶
Add-on configuration consists of two parts:
- Global settings configuration¶
Pangolin global configurations are accessible on "Administration" - "Pangolin" - "Global Settings" page:
| Field Name | Description |
|---|---|
| Pangolin URL | URL for Pangolin server. Example: http://some-server:8888 |
| TestRail URL | URL for your TestRail instance. Example: https://testrail:8080 |
| TestRail User Name | Name of user in TestRail |
| TestRail Password | Password for TestRail user |
| Upload Timeout | The number of minutes to wait for the Pangolin server to process the request. 0 means wait indefinitely |
When all fields are set, please click on "Save" button - Pangolin will check availability of entered URLs and encrypt your password.
- "Pangolin: Upload test results into TestRail" build task configuration¶
To upload test results into TestRail, a new "Pangolin: Upload test results into TestRail" task has to be added:
| Field Name | Required | Description |
|---|---|---|
| TestRail User | No | If specified, overrides TestRail user name defined in Global Configuration |
| TestRail Password | No | If specified, overrides TestRail user password defined in Global Configuration |
| Project | Yes | The name of project in TestRail to which results should be exported |
| TestPath | Yes | Path to where Pangolin should test definitions, must contain suite name in the beginning (for single-suite project, suite name is always 'Master'), e.g. Master\Section1\Section2 |
| Format | Yes | Format of test results generated during the build. Available values: junit, nunit, testng, trx |
| Results File Pattern | Yes | Provide path to the report file(s) generated during the build. This is a comma separated list of test result directories. You can also use Ant style patterns such as **/surefire-reports/*.xml |
| Test Run | No | Name of test run in TestRail to which test results will be added |
| Test Plan | No | Name of test plan in TestRail to which test results will be added |
| Milestone Path | No | Path to a milestone in TestRail to which test run/plan will be added. E.g. Milestone1\Milestone2 |
| Case Fields | No | Values for case fields in TestRail can be specified in this field. The format is [TestRail field system name]=[value 1] and each field name\value pair should start with the new line. E.g.: custom_user_field1=foo |
| Result Fields | No | Values for result fields in TestRail can be specified in this field. The format is [TestRail field system name]=[value 1] and each field name\value pair should start with the new line. E.g.: custom_user_field1=foo |
| Fail build if upload unsuccessful | N/A | If checked, the build will be marked as failed if for any reason the plugin was not able to upload the results. This could be due to Pangolin server issues, TestRail server issues, network issues, etc. |
| Close Run | N/A | If checked, Pangolin will close the test run in TestRail and archive its tests and results |
| Disable Grouping | N/A | If checked, Pangolin will ignore structure in report files and upload all test cases into one Section, defined by the Test Path parameter. |
Make sure task is added to Final Tasks
In order to upload results to TestRail in case when test failed, just add "Pangolin: Upload test results into TestRail" task to Final Tasks section of Atlassian Bamboo® job.
- Execution¶
Once the job is configured, simply run the plan. Here is Bamboo plan execution output example:
And results in TestRail:
Test Cases:
Test Plan and Test Run:
Created Milestones hierarchy:
