JUnit

- Description

Pangolin provides a set of Java annotations which allows you to send your test results into TestRail from your favorite IDE or CI system.

Technically it consists of three parts:

• @Pangolin annotation
• JUnit test run listener: com.agiletestware.pangolin.annotations.junit.PangolinJUnitListener
• XML configuration file which contains TestRail connection and other details needed for functioning of pangolin annotations package

- Prerequisites

• Pangolin Server must be installed and be accessible via HTTP from user machine
• Maven project with JUnit tests

- Installation and configuration

• Modify your pom.xml file and add pangolin-annotations dependencies and Agiletestware repository information:

<dependencies>
    <dependency>
        <groupId>com.agiletestware</groupId>
        <artifactId>pangolin-annotations</artifactId>
        <version>2.5</version>
        <scope>test</scope>
    </dependency>   
</dependencies>
<repositories>
    <repository>
        <id>public.maven.agiletestware.com</id>
        <url>http://public.maven.agiletestware.com.s3-website-us-west-2.amazonaws.com/release</url>     
    </repository>       
</repositories>

• Add com.agiletestware.pangolin.annotations.junit.PangolinJUnitListener to maven-surefire-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.18.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <properties>
            <property>
                <name>listener</name>
                <value>com.agiletestware.pangolin.annotations.junit.PangolinJUnitListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

• Create pangolin_config.xml XML file in the project's root:

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>false</disabled>
    <!-- Whether a run in TestRail should be closed or not. If this is omitted, Pangolin will not close runs -->
    <close_run>true</close_run>
</pangolin>

Setting configuration file with Java System Properties

Sometimes it's not desired to store pangolin_config.xml file in project root directory, so it can be set with pangolin.config.xml Java System Property: "mvn -Dpangolin.config.xml=D:\temp\pangolin_config.xml clean test"

- Add Pangolin annotations to JUnit classes/methods

@Pangolin Java annotation can be added on class and method levels When class or method is marked with Pangolin annotation, its result will be exported to TestRail.

- Pangolin annotation attributes

Name	Description	Applicable	Required
sectionPath	Path to a section in TestRail project into which results should be exported. Must start with TestRail Suite name, e.g. Master\Section\SubSection	Class, method	Yes
runName	Name of a test run in TestRail to which results will be added. If run does not exist, it will be created automatically. If it is not set, a new run with time stamp will be created	Class, method	No
planName	Name of a test plan in TestRail to which results will be added. If plan does not exist, it will be created automatically	Class, method	No
milesonePath	A path to a milestone to which test results will be added. If milestone does not exist, it will be created. E.g.: Milestone1\Milestone2	Class, method	No
testName	Name of a test in TestRail. If it's not set, then method name is used	Method	No
customFields	An array of case fields name/value pairs to pass into TestRail	Class, method	No
disabled	Set it to com.agiletestware.pangolin.annotations.BooleanValue.TRUE to disable results export for a particular method or class	Class, method	No

Overriding annotations' values on method level

Pangolin annotation on method level overrides values set by Pangolin annotation on class level.

- Execute Maven test phase

To run JUnit tests and send results to TestRail, just run Maven test phase on your project: e.g.: mvn test

Support of WebDriver and Appium

Pangolin can be used easily together with WebDriver and Appium tests and do not require any additional configuration.

- Examples

- Pangolin annotation on class level

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Execution log:

Results in TestRail:

A new run with time stamp is created:

Test case result:

- Adding results to a particular Test Run in TestRail

To add results to a particular run in TestRail, just specify runName attribute:

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", runName = "Maven run")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Adding results to a particular Test Plan in TestRail

To add results to a particular run in TestRail, just specify planName attribute:

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", planName = "Maven plan")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Add test results to TestRail Milestone

To add tests results to a Milestone in TestRail, define a path to a milestone with milesonePath attribute of Pangolin annotation:

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", milesonePath = "Milestone1\\Milestone2")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Setting test name for a method

To set test name to a particular test method, add Pangolin annotation to a test method and set testName attribute:

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection")
public class SomePassSomeFailTest {

    @Pangolin(testName = "Some test name")
    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Passing values for custom case fields

If you need to pass value to some case fields in TestRail (e.g. for fields which are required in TestRail project), just add an array of com.agiletestware.pangolin.annotations.CustomField objects into customFields attribute.
CustomField object contains the following attributes:

• name - name of a field in TestRail, e.g. custom_test_field
• value - value to be set for a field

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.CustomField;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", customFields = { @CustomField(name = "custom_required_field", value = "Maven") })
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Passing valued for result fields

If you need to pass value to some fields of result in TestRail (e.g. for fields which are required in TestRail project), just add an array of com.agiletestware.pangolin.annotations.CustomField objects into customResultFields attribute.
CustomField object contains the following attributes:

• name - name of a field in TestRail, e.g. custom_test_field
• value - value to be set for a field

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.BooleanValue;
import com.agiletestware.pangolin.annotations.CustomField;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", customResultFields = { @CustomField(name = "custom_result_fields", value = "Maven"),
        @CustomField(name = "version", value = "12") })
public class SomePassSomeFailTest {

    @Pangolin(disabled = BooleanValue.FALSE)
    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results in TestRail:

- Adding arbitrary attachments to Test Step in TestRail (since 2.2)

In some cases it could be beneficial to have some test data like screenshots or logs accessible in TestRail, so Pangolin supports adding arbitrary attachments to test results.

Adding attachments in the code

To add an arbitrary attachment to your test results in TestRail you can use com.agiletestware.pangolin.annotations.CurrentTest class which provides several static overloaded addAttachment methods:

• addAttachment(File) - adds a given file into step results in TestRail
• addAttachments(List<File) - add a list of files into step results in TestRail
• addAttachment(String, InputStream) - add attachment from generic input stream

Example:

import static org.junit.Assert.assertEquals;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

import com.agiletestware.pangolin.annotations.CurrentTest;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\demo", runName = "Attachments")
public class WebDriverTest {

    private final WebDriver webDriver = new ChromeDriver();

    @Test
    public void testPageOpens() throws IOException {
        webDriver.navigate().to("http://agiletestware.com");
        final File screenshot = ((TakesScreenshot) webDriver).getScreenshotAs(OutputType.FILE);
        CurrentTest.addAttachment(screenshot);
    }

    @Test
    public void testTitleCorrect() throws IOException {     
        webDriver.navigate().to("http://agiletestware.com");
        try (InputStream stream = new ByteArrayInputStream(webDriver.getPageSource().getBytes())) {
            CurrentTest.addAttachment("page.html", stream);
        }
        assertEquals("something clearly wrong", webDriver.getTitle());
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

}

Results in TestRail:

- Mapping test method result into existing test in TestRail (since 2.5)

Pangolin Annotation package allows you to map results of your test into existing test in TestRail by simply marking your test method with Pangolin annotation with id attribute:

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

public class MapToExistingTest {

    @Pangolin(id = 14223)
    @Test
    public void mappedTest() {
        Assert.fail("fail");
    }

}

where id is an id of an existing test case in TestRail:

Note, that only 'C' character in the ID value should be omitted.

Results in TestRail:

- Disabling test results export

To disable exporting of test results into TestRail user can either:

• Set disabled element of pangolin_config.xml file to true

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>true</disabled>
</pangolin>

• Or set disabled attribute of Pangolin annotation to BooleanValue.TRUE for a test class or method:

package com.agiletestware.pangolin.dummytest;

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.BooleanValue;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\SubSection", disabled = BooleanValue.TRUE)
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {
        Assert.fail("test4 failed");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Pangolin will print the following message for tests which test results will not be exported:

Pangolin: Skipping com.agiletestware.pangolin.dummytest.SomePassSomeFailTest.test3 method because it's disabled