Integrating with Sauce Labs

Pangolin provides a simple way to integrate TestRail with Sauce Labs and allows users to:

• Fetch test metadata from SauceLabs and publish it in TestRail
• Update actual status of a test in SauceLabs

- SauceLabs Connection Details

There are two ways to configure SauceLabs connection details

1. Pangolin Config File

Add the following element in the pangolin_config.xml file:

<remoteTestTool>
    <saucelabs>     
        <user>username</user>
        <apiKey>apikey123</apiKey>
    </saucelabs>
</remoteTestTool>

where user is the SauceLabs username and apiKey is SauceLabs user's API key.
By default, Pangolin will use https://saucelabs.com as a base URL for SauceLabs REST API calls. If needed, it can be changed by adding url element into saucelabs element:

<remoteTestTool>
    <saucelabs>
        <url>https://someotherurl</url>
        <user>username</user>
        <apiKey>apikey123</apiKey>
    </saucelabs>
</remoteTestTool>

Example:

<?xml version="1.0" encoding="UTF-8"?>
<pangolin>
    <!-- Pangolin Server URL -->
    <pangolin_url>http://pangolinurl:9090</pangolin_url>
    <!-- TestRail URL -->
    <testrail_url>https://testrailurl:8080</testrail_url>
    <!-- TestRail user name -->
    <testrail_user>username@company.domain</testrail_user>
    <!-- TestRail encrypted password: please use http://server_name:port/pangolin/password to encrypt your plain text password -->
    <testrail_encrypted_password>encryptedPassword</testrail_encrypted_password>
    <!-- TestRail project -->
    <testrail_project>project name</testrail_project>
    <!-- Whether the Pangolin is disabled or not. If this is omitted, Pangolin is enabled -->
    <disabled>false</disabled>
    <remoteTestTool>
        <saucelabs>
            <user>username</user>
            <apiKey>apikey123</apiKey>
        </saucelabs>
    </remoteTestTool>
</pangolin>

2. Java System Properties

There are three Java System Properties for setting SauceLabs connection details:

• pangolin.saucelabs_url - SauceLabs base URL. If this not set, Pangolin will use the value from XML configuration or https://saucelabs.com.
• pangolin.saucelabs_user - SauceLabs user name.
• pangolin.saucelabs_apiKey - SauceLabs API key.

Values passed as system properties override values in XML configuration.

- JUnit configuration

- Prerequisites

• Pangolin annotations library and com.agiletestware.pangolin.annotations.junit.PangolinJUnitListener must be added into your pom.xml. See details on JUnit documentation page

- Adding PangolinRule

In order to allow Pangolin to fetch SauceLabs test data, add an instance of com.agiletestware.pangolin.annotations.webdriver.PangolinRule to the test class:

package com.agiletestware.pangolin.dummytest;

import java.net.MalformedURLException;

import org.junit.After;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import com.agiletestware.pangolin.annotations.Pangolin;
import com.agiletestware.pangolin.annotations.webdriver.PangolinRule;

@Pangolin(sectionPath = "Master\\Section\\SubSection")
public class SauceLabsTest {
    private static final String USERNAME = "username";
    private static final String AUTOMATE_KEY = "apiKey123";
    private static final String URL = "http://" + USERNAME + ":" + AUTOMATE_KEY + "@ondemand.saucelabs.com:80/wd/hub";
    @Rule
    public PangolinRule rule = new PangolinRule(createRemoteDriver());
    private final RemoteWebDriver webDriver = rule.getWebDriver();

    private RemoteWebDriver createRemoteDriver() {
        try {
            final DesiredCapabilities caps = DesiredCapabilities.chrome();
            return new RemoteWebDriver(new java.net.URL(URL), caps);
        } catch (final MalformedURLException e) {
            throw new RuntimeException(e.getMessage(), e);
        }
    }

    @After
    public void tearDown() {
        webDriver.quit();
    }

    @Test
    public void passTest() {
        System.out.println("Test One");
        webDriver.get("http://agiletestware.com");
        System.out.println("Session id: " + webDriver.getSessionId());
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", webDriver.getTitle());
    }

    @Test
    public void failTest() {
        webDriver.get("http://agiletestware.com");
        Assert.assertEquals("Something wrong", webDriver.getTitle());
    }
}

Tests must use the same instance of WebDriver as PangolinRule

It is important to use the same instance of WebDriver which was passed to PangolinRule for all your tests.
WebDriver instance can be retrieved by calling PangolinRule.getWebDriver() method.

- Executing Maven test phase

To run tests and send results to TestRail, just run Maven test phase on your project: e.g.: mvn clean test

Results in TestRail:

Test Run in TestRail:

Test Results for the passed test:

In SauceLabs:

Test Results for the failed test:

In SauceLabs:

- TestNG Configuration

- Prerequisites

• Pangolin annotations library and com.agiletestware.pangolin.annotations.testng.PangolinTestNGReporter must be added into your pom.xml. See details on TestNG documentation page

- Configuring project pom.xml

To integrate with SauceLabs, add com.agiletestware.pangolin.annotations.testng.PangolinTestNGListener listener to the maven-surefire-plugin configuration in pom.xml file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19</version>
    <configuration>
        <properties>
            <property>
                <name>listener</name>
                <value>com.agiletestware.pangolin.annotations.testng.PangolinTestNGReporter,
                    com.agiletestware.pangolin.annotations.testng.PangolinTestNGListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

e.g.: this is how the whole pom.xml file might look like:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.agiletestware</groupId>
    <artifactId>pangolin-demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>Pangolin Demo</name>

    <properties>
        <java.version>1.8</java.version>
        <testng.version>6.14.3</testng.version>
        <pangolin-annotations.version>2.5</pangolin-annotations.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.7.0</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                    <debug>true</debug>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19</version>
                <configuration>
                    <properties>
                        <property>
                            <name>listener</name>
                            <value>com.agiletestware.pangolin.annotations.testng.PangolinTestNGReporter,
                                com.agiletestware.pangolin.annotations.testng.PangolinTestNGListener</value>
                        </property>
                    </properties>
                </configuration>
            </plugin>

        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.agiletestware</groupId>
            <artifactId>pangolin-annotations</artifactId>
            <version>${pangolin-annotations.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <repositories>
        <repository>
            <id>public.maven.agiletestware.com</id>
            <url>http://public.maven.agiletestware.com.s3-website-us-west-2.amazonaws.com/release</url>
        </repository>       
    </repositories>
</project>

- Implementing com.agiletestware.pangolin.annotations.webdriver.WebDriverTest interface

To enable automatic SauceLabs metadata publishing, test class must implement com.agiletestware.pangolin.annotations.webdriver.WebDriverTest interface and return the same WebDriver instance which is used for tests:

package com.agiletestware.pangolin.dummytest;

import java.net.MalformedURLException;

import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

import com.agiletestware.pangolin.annotations.Pangolin;
import com.agiletestware.pangolin.annotations.webdriver.WebDriverTest;

@Pangolin(sectionPath = "Master\\Section\\SubSection")
public class SauceLabsTest implements WebDriverTest {
    private static final String USERNAME = "username";
    private static final String AUTOMATE_KEY = "apiKey123";
    private static final String URL = "http://" + USERNAME + ":" + AUTOMATE_KEY + "@ondemand.saucelabs.com:80/wd/hub";
    private RemoteWebDriver webDriver;

    @BeforeTest
    public void setUp() throws MalformedURLException {
        final DesiredCapabilities caps = DesiredCapabilities.chrome();
        webDriver = new RemoteWebDriver(new java.net.URL(URL), caps);
    }

    @AfterTest
    public void tearDown() {
        webDriver.quit();
    }

    @Override
    public RemoteWebDriver getWebDriver() {
        return webDriver;
    }

    @Test
    public void passTest() {
        System.out.println("Test One");
        webDriver.get("http://agiletestware.com");
        System.out.println("Session id: " + webDriver.getSessionId());
        Assert.assertEquals(webDriver.getTitle(), "Agiletestware - Software for QA and Development Tools");
    }

    @Test
    public void failTest() {
        webDriver.get("http://agiletestware.com");
        Assert.assertEquals(webDriver.getTitle(), "Something wrong");
    }

}

- Executing Maven test phase

To run tests and send results to TestRail, just run Maven test phase on your project: e.g.: mvn clean test

Results in TestRail:

Test Run in TestRail:

Test Results for the passed test:

In SauceLabs:

Test Results for the failed test:

In SauceLabs:

Disabling status update in SauceLabs

If SauceLabs test status update is not desired, it could be switched off by adding <updateStatus>false</updateStatus> element to saucelabs element of configuration:

<remoteTestTool>
    <saucelabs>     
        <user>username</user>
        <apiKey>apikey123</apiKey>
        <updateStatus>false</updateStatus>
    </saucelabs>
</remoteTestTool>

The following log message will be added to log: Pangolin: Updating status in SauceLabs is disabled in configuration --> skipping SauceLabs status update