Selenium WebDriver + TestRail Integration Guide

Selenium WebDriver is one of the most widely used open source tool for automating browser interaction and is used by countless companies.

- Description

Pangolin's JUnit and TestNG solution allows you to easily integrate Selenium WebDriver tests with TestRail by using convenient set of Java annotations and Maven building system.

Technically it consists of three parts:

• @Pangolin annotation
• JUnit or TestNG test run listeners and publishers
• XML configuration file which contains TestRail connection and other details needed for functioning of pangolin annotations package

- Prerequisites

• Pangolin Server must be installed and be accessible via HTTP from user machine
• Maven project with JUnit/TestNG tests

- Installation and configuration

• Modify your pom.xml file and add pangolin-annotations dependencies and Agiletestware repository information:

<dependencies>
    <dependency>
        <groupId>com.agiletestware</groupId>
        <artifactId>pangolin-annotations</artifactId>
        <version>2.5</version>
        <scope>test</scope>
    </dependency>   
</dependencies>
<repositories>
    <repository>
        <id>public.maven.agiletestware.com</id>
        <url>http://public.maven.agiletestware.com.s3-website-us-west-2.amazonaws.com/release</url>     
    </repository>       
</repositories>

• Configure Maven surefire plugin for JUnit or TestNG frameworks:

JUnit:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.18.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <properties>
            <property>
                <name>listener</name>
                <value>com.agiletestware.pangolin.annotations.junit.PangolinJUnitListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

TestNG:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19</version>             
    <configuration>
        <properties>
            <property>
                <name>listener</name>
                <value>com.agiletestware.pangolin.annotations.testng.PangolinTestNGReporter,com.agiletestware.pangolin.annotations.testng.PangolinTestNGListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

• Create pangolin_config.xml XML file in the project's root:

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>false</disabled>
</pangolin>

Setting configuration file with Java System Properties

Sometimes it's not desired to store pangolin_config.xml file in project root directory, so it can be set with pangolin.config.xml Java System Property: "mvn -Dpangolin.config.xml=D:\temp\pangolin_config.xml clean test"

- Add Pangolin annotations to JUnit/TestNG classes/methods

@Pangolin Java annotation can be added on class and method levels When class or method is marked with Pangolin annotation, its result will be exported to TestRail.

- Pangolin annotation attributes

Name	Description	Applicable	Required
sectionPath	Path to a section in TestRail project into which results should be exported. Must start with TestRail Suite name, e.g. Master\Section\SubSection	Class, method	Yes
runName	Name of a test run in TestRail to which results will be added. If run does not exist, it will be created automatically. If it is not set, a new run with time stamp will be created	Class, method	No
planName	Name of a test plan in TestRail to which results will be added. If plan does not exist, it will be created automatically	Class, method	No
milesonePath	A path to a milestone to which test results will be added. If milestone does not exist, it will be created. E.g.: Milestone1\Milestone2	Class, method	No
testName	Name of a test in TestRail. If it's not set, then method name is used	Method	No
customFields	An array of case fields name/value pairs to pass into TestRail	Class, method	No
disabled	Set it to com.agiletestware.pangolin.annotations.BooleanValue.TRUE to disable results export for a particular method or class	Class, method	No

Overriding annotations' values on method level

Pangolin annotation on method level overrides values set by Pangolin annotation on class level.

- Execute Maven test phase

To run JUnit/TestNG tests and send results to TestRail, just run Maven test phase on your project: e.g.: mvn test

Support of WebDriver and Appium

Pangolin can be used easily together with WebDriver and Appium tests and do not require any additional configuration.

- Examples

Note

Unless stated, all the examples below are for JUnit. To run the same example on TestNG, simply replace JUnit annotations with corresponding TestNG ones.

- Pangolin annotation on class level

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Execution log:

Results in TestRail:

A new run with time stamp is created:

Test case result:

- Adding results to a particular Test Run in TestRail

To add results to a particular run in TestRail, just specify runName attribute:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", runName = "Maven run")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Adding results to a particular Test Plan in TestRail

To add results to a particular run in TestRail, just specify planName attribute:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", planName = "Maven plan")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Add test results to TestRail Milestone

To add tests results to a Milestone in TestRail, define a path to a milestone with milesonePath attribute of Pangolin annotation:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver",  milesonePath = "Milestone1\\Milestone2")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Setting test name for a method

To set test name to a particular test method, add Pangolin annotation to a test method and set testName attribute:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Pangolin(testName = "Find some pizzas")
    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Passing values for custom case fields

If you need to pass value to some case fields in TestRail (e.g. for fields which are required in TestRail project), just add an array of com.agiletestware.pangolin.annotations.CustomField objects into customFields attribute.
CustomField object contains the following attributes:

• name - name of a field in TestRail, e.g. custom_test_field
• value - value to be set for a field

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.CustomField;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", customFields = { @CustomField(name = "custom_required_field", value = "Maven") })
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Passing values for result fields (since 2.3)

If you need to pass value to some fields of result in TestRail (e.g. for fields which are required in TestRail project), just add an array of com.agiletestware.pangolin.annotations.CustomField objects into customResultFields attribute.
CustomField object contains the following attributes:

• name - name of a field in TestRail, e.g. custom_test_field
• value - value to be set for a field

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.CustomField;
import com.agiletestware.pangolin.annotations.Pangolin;
import com.agiletestware.pangolin.annotations.webdriver.PangolinRule;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", customResultFields = { @CustomField(name = "version", value = "12"),
        @CustomField(name = "custom_result_fields", value = "From annotations") })
public class WebDriverTest {

    @Rule
    public PangolinRule pangolinRule = PangolinRule.builder(() -> new ChromeDriver()).takeScreenshotOnFail(true).build();

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = pangolinRule.getWebDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Adding arbitrary attachments to Test Step in TestRail (since 2.2)

In some cases it could be beneficial to have some test data like screenshots or logs accessible in TestRail, so Pangolin supports adding arbitrary attachments to test results.

Adding attachments in the code

To add an arbitrary attachment to your test results in TestRail you can use com.agiletestware.pangolin.annotations.CurrentTest class which provides several static overloaded addAttachment methods:

• addAttachment(File) - adds a given file into step results in TestRail
• addAttachments(List<File) - add a list of files into step results in TestRail
• addAttachment(String, InputStream) - add attachment from generic input stream

Example:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.CurrentTest;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", runName = "Attachments")
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        final File screenshot = ((TakesScreenshot) webDriver).getScreenshotAs(OutputType.FILE);
        CurrentTest.addAttachment(screenshot);
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- Automatic screenshot capturing (since 2.2)

Selenium WebDriver tests failures are hard to debug if you only look at the stack trace. Pangolin can be configured to automatically capture screenshots for failed tests and send them to TestRail. This is super useful for quick debugging.

- JUnit

To capture screenshots automatically, add an instance of com.agiletestware.pangolin.annotations.webdriver.PangolinRule class as a JUnit rule to the test class as shown below:

@Rule
    public PangolinRule pangolinRule = PangolinRule.builder(() -> new ChromeDriver()).takeScreenshotOnFail(true).build();

PangolinRule automatically close associated WebDriver instance

Please note that PangolinRule automatically closes associated WebDriver instance by default. This behavior can be overridden by passing a different instance of com.agiletestware.pangolin.annotations.webdriver.WebDriverCloseBehavior to PangolinRule instance.

This can be done either with PangolinRule.setCloseBehavior(final WebDriverCloseBehavior closeBehavior) or with com.agiletestware.pangolin.annotations.webdriver.PangolinRule.Builder.closeBehavior methods.
e.g.:

public PangolinRule pangolinRule = PangolinRule.builder(() -> new ChromeDriver()).takeScreenshotOnFail(true).closeBehavior(PangolinRule.DO_NOTHING).build();

Full example:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.Pangolin;
import com.agiletestware.pangolin.annotations.webdriver.PangolinRule;

@Pangolin(sectionPath = "Master\\Section\\WebDriver")
public class WebDriverTest {

    @Rule
    public PangolinRule pangolinRule = PangolinRule.builder(() -> new ChromeDriver()).takeScreenshotOnFail(true).build();

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = pangolinRule.getWebDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Results in TestRail:

- TestNG

To capture screenshots automatically:

1. A new <screenshotOnFailure>true</screenshotOnFailure> element should be added to pangolin_config.xml file
2. Test class has to implement com.agiletestware.pangolin.annotations.webdriver.WebDriverTest interface and return valid WebDriver instance in getWebDriver() method.

e.g.:

pangolin_config.xml":

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>false</disabled>
    <screenshotOnFailure>true</screenshotOnFailure>
</pangolin>

Test class:

package com.agiletestware.pangolin.dummytest;

import static org.junit.Assert.assertTrue;
import static org.testng.Assert.assertEquals;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.BeforeClass;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;

import com.agiletestware.pangolin.annotations.Pangolin;
import com.agiletestware.pangolin.annotations.webdriver.WebDriverTest;

@Pangolin(sectionPath = "Master\\demo", runName = "Webdriver TestNG")
public class SimpleWebDriverTest implements WebDriverTest {
    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals(results.size(), 0, "Liar! There is no bad pizza!!!");
    }

    @Override
    public RemoteWebDriver getWebDriver() {
        return webDriver;
    }

    @AfterTest
    public void afterTest() {
        webDriver.quit();
    }

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }

}

Results in TestRail:

- Mapping test method result into existing test in TestRail (since 2.5)

Pangolin Annotation package allows you to map results of your test into existing test in TestRail by simply marking your test method with Pangolin annotation with id attribute:

import org.junit.Assert;
import org.junit.Test;

import com.agiletestware.pangolin.annotations.Pangolin;

public class MapToExistingTest {

    @Pangolin(id = 14223)
    @Test
    public void mappedTest() {
        Assert.fail("fail");
    }

}

where id is an id of an existing test case in TestRail:

Note, that only 'C' character in the ID value should be omitted.

Results in TestRail:

- Disabling test results export

To disable exporting of test results into TestRail user can either:

• Set disabled element of pangolin_config.xml file to true

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>true</disabled>
</pangolin>

• Or set disabled attribute of Pangolin annotation to BooleanValue.TRUE for a test class or method:

package com.agiletestware.pangolin.demo;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.agiletestware.pangolin.annotations.BooleanValue;
import com.agiletestware.pangolin.annotations.Pangolin;

@Pangolin(sectionPath = "Master\\Section\\WebDriver", disabled = BooleanValue.TRUE)
public class WebDriverTest {

    private static final String GOOGLE = "https://google.com";
    private final RemoteWebDriver webDriver = new ChromeDriver();

    @BeforeClass
    public static void setUp() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testFindSomePizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
    }

    @Test
    public void testThereAreNoBadPizzas() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("bad pizza");
        assertEquals("Liar! There is no bad pizza!!!", results.size(), 0);
    }

    @Test
    public void testFirstResultAboutPizza() throws IOException {
        webDriver.navigate().to(GOOGLE);
        final List<WebElement> results = getGoogleResults("pizza");
        assertTrue("Cannot find any pizzas", results.size() > 0);
        results.get(0).click();
        assertTrue(webDriver.getPageSource().contains("pizza"));
    }

    @After
    public void afterTest() {
        webDriver.quit();
    }

    private List<WebElement> getGoogleResults(final String query) {
        final WebElement element = webDriver.findElement(By.name("q"));
        element.sendKeys(query);
        element.submit();
        new WebDriverWait(webDriver, 1)
        .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.id("resultStats")));
        return webDriver.findElements(By.xpath("//*[@id='rso']//*[@class='r']/a[h3]"));
    }
}

Pangolin will print the following message for tests which test results will not be exported:

Pangolin: Skipping com.agiletestware.pangolin.demo.WebDriverTest.testThereAreNoBadPizzas method because it's disabled