A new method to Integrate SoapUI test results in Yandex Allure Dashboard

Here is a quick but effective solution for embedding SoapUI into Yandex Allure. I know, SoapUI supports Maven, but the SmarBear guys are getting too restrictive in recent days. They are trying their best to enforce all users to go for Pro versions which is now all integrated into Ready!API package.

I tried to add Allure Junit listeners into SoapUI pom files, but wasn't proudly successful.  Then decided to make it work in my own way. This way works smoother and is not so sensitive to environment or version changes.

What we need:

• Having a local or remote database server, compatible with jdbc
• Add the connector jar to SoapUI ext folder. For Ready!API, I had to directly copy the mysql connector jar in /lib folder 
• In your Java/TestNG code, call SoapUI project test case using cmd and testrunner. You might find that assertion errors cause the process runner to wait till times out. To prevent this, you should pull a trick out of your sleeve: "cmd /B start /WAIT cmd /C" . This makes the process to immediately stop if it fails with stderr messages. 
• In addition, you need to assign an unique identifier to your testrunner process so you can extract the results using that identifier easily. Here is the function I wrote:

        @Step("Running SoapUI Process")
 public boolean processrun(String testRunnerPath, String suiteName, String testCaseName, String uuid, String project){
  boolean finished = false;
  try{
   String pString = "cmd /B start /WAIT cmd.exe /C \""+testRunnerPath+" -s"+suiteName+" -c"+testCaseName+" -r -Puuid="+uuid+" "+project+"\"";
    System.out.println(pString);
    
        String line;
        Process p = Runtime.getRuntime().exec(pString);
        BufferedReader input =  new BufferedReader(new InputStreamReader(p.getInputStream()));
        while ((line = input.readLine()) != null) {
         System.out.println(line);
         if (line.contains("TestCaseRunner Summary")){
          finished = true;
           Thread.sleep(3000);
           p.destroy();
           p.wait(1l);
           break;
         }
        }
        input.close();
        if (finished) return true; else  return false;
        
  }catch (Exception e){
   if (finished) return true;
   else {
   System.out.println("--------------->Error Occured!");
   saveTextLog("Unexpected-Error", e.getMessage());
   System.out.println(e.getMessage());
   Assert.fail("Unexpected Test Execution Error");
   return false;
   }
  }
 }

• Please note that above function looks for "TestCaseRunner Summary" string, which is always generated when you use "-r" flag. 
• The uuid is a unique identifier string. You can simply generate it like "uuid = java.util.UUID.randomUUID()"
• Write some groovy code in TearDown pane and record the first assertion failure in your table including the uuid. Here is how to get the step name, status, assertion errors:

/*
 * Recording the results from each test case into MySQl
 * Author: ppirooznia
 */
import com.eviware.soapui.impl.wsdl.teststeps.*
boolean next = true
import groovy.sql.*;
import java.util.zip.GZIPOutputStream
def driver = 'com.mysql.jdbc.Driver'
def con = Sql.newInstance("jdbc:mysql://127.0.0.1:3306/test?autoReconnect=true&useSSL=false","username","passsword",driver);
def uuid = context.expand( '${#Project#uuid}' )
log.info( testRunner.testCase.testSuite.name)// + testCase.testSuite.project.name )
assertion = ""
message = ""
request = ""
response = ""
boolean comma = false
boolean getinfo = false
lquery = ""
testsuitename = testRunner.testCase.testSuite.name
testcasename = testRunner.testCase.name
//testRunner.testCase.testSteps.each{ name,props ->

testRunner.testCase.testSuite.project.testSuites[testsuitename].getTestCaseByName(testcasename).testSteps.each{ name,props ->
step = testRunner.testCase.testSuite.project.testSuites[testsuitename].getTestCaseByName(testcasename).getTestStepByName("$name")
if (step instanceof WsdlTestRequestStep || step instanceof RestTestRequestStep || step instanceof JdbcRequestTestStep || step instanceof HttpTestRequestStep)
{
     props.getAssertionList().each{
        log.info "$it.label - $it.status - $it.errors - $uuid"
  tname = "$name"
        if ("$it.status"=="FAILED") {
         getinfo = true
         if (comma){
          assertion += ", $it.label"
          message += ", $it.errors"
         } else {
          assertion += "$it.label" 
          message += "$it.errors"
          comma = true
         }
        }
        }

        if (next && getinfo) {
          rawResponse = context.expand( '${'+"$name"+'#RawResponse}' )
           zResponse = zip(rawResponse)
   request = context.expand( '${'+"$name"+'#RawRequest}' )
   zRequest = zip(request)
         assertion = assertion.replaceAll( /([^a-zA-Z0-9 _,:])/, '-' )
         message = message.replaceAll( /([^a-zA-Z0-9 _,:\[\]])/, '-' )
         if (assertion.length() > 128) {assertion = assertion.substr(0,128)}
         if (message.length() > 1024) {message = message.substr(0,1024)}
         lquery = "INSERT INTO test.soapuiresults (uuid,assertion,tstatus,message,step,request,response,tsuite,tcase) VALUES ('"+uuid+"','"+assertion+"','F','"+message+"','"+tname+"','"+zRequest+"','"+zResponse+"','"+testsuitename+"','"+testcasename+"');"
         try {
          if (next) {
           con.execute(lquery) 
           next = false
          }
         } catch (Exception e) {
          log.error e.message
         }
         comma = false
         getinfo = false
        }
}
}

 if (next) {
  lquery = 'INSERT INTO test.soapuiresults (uuid,assertion,tstatus,message,step,tsuite,tcase) VALUES ("'+uuid+'","No Errors","P","All Passed","All","'+testsuitename+'","'+testcasename+'");'
  try {
  con.execute(lquery) 
  } catch (Exception e) {
          log.error e.message
         }
 }
  con.close()  
def zip(String s){
def targetStream = new ByteArrayOutputStream()
def zipStream = new GZIPOutputStream(targetStream)
zipStream.write(s.getBytes())
zipStream.close()
def zipped = targetStream.toByteArray()
targetStream.close()
return zipped.encodeBase64()

• Please note that in order to save HTML/XML requests and responses, I preferred to zip them and put them into database as a blob field. Later, I will use the unzipped text as a text attachment in Allure dashboard
• If you have the Ready!API Pro version, you can add this code only once to the event handler (right-click on project node, select events, and create a "TestRunListener.afterRun" event)
• now you need to retrieve data from database (using uuid selector) in your Allure Maven TestNG suite, and simply pass or fail them with regular Assert() method.

A combination of SoapUI, Selenium WebDriver, PhantomJS and Groovy!

      Our company has a monitoring system for most of sensitive web services and websites, including local and intranet sites. Most of this monitoring is being done in few Linux centos boxes that execute SoapUI tesrunner.sh tool using NRPE-Nagios and return the final results into a dashboard.

Writing SoapUI projects and scripts for web services or even web GUIs with 1-2 pages are not hard, even in our case, that most of the page elements require extensive authentication process and passing security tokens in various ways. However, the job gets painful when a full user scenario should be created as a monitoring script. 

Lets give you an example: If the usecase is to open a login page, input the credentials and ensure all elements in the main page exist and are valid, it is possible to be done using SoapUI http methods by following network activity sequences. This activities can be sniffed using any browser "developer" tool, or SoapUI recorder itself.

Now, if the use case is more complicated, like filling up  a form after login, submit it, check the next page, submit the second form, wait for an email to come and reply it for confirmation, then ensure the subscription is finished successfully, that would be no easy way to do it using SoapUI steps and simple groovy coding.

The solution here is using a WebDriver in a Headless browser such as PhantomJS.

I managed to make it work after 2 days, and here are the lessons I learnt:

1- After unzipping the phantomjs tar file, make sure the /bin/phantomjs binary file is executable for all groups.

2-Do a "ldd phantomjs" and ensure all libraries are installed. Most probably you'll need to update your GLIBC and GLIBCXX (via libstdc++).

3- Make sure your phantomjs works. to do that create a file like loadepage.js :

var page = require('webpage').create();

page.open('http://www.google.com', function(status) {

  console.log("Status: " + status);

  if(status === "success") {

    page.render('example.png');

  }

  phantom.exit();

});

4- Then execute it like this "./phantomjs loadpage.js" . If you received a failure and you are sure that there is no network restriction to your destination, try this "./phantomjs --ignore-ssl-errors=true loadpage.js"

5- If your machine has jre 1.6 , then you have to use older Selenium jars (I recommend selenium-java-2.45.0 because it has most recent phantomservice jar)

6- Here is how I opened my groovy script, every piece of this code is vital and has its good reason:

import org.openqa.selenium.*

import org.openqa.selenium.firefox.FirefoxDriver

import org.openqa.selenium.phantomjs.PhantomJSDriver;

import org.openqa.selenium.phantomjs.PhantomJSDriverService;

import org.openqa.selenium.interactions.Actions

import org.openqa.selenium.remote.DesiredCapabilities

import org.openqa.selenium.support.ui.Select

import java.io.*

import java.util.concurrent.TimeUnit

import org.openqa.selenium.support.ui.ExpectedCondition

import org.apache.commons.io.FileUtils

import org.openqa.selenium.support.ui.WebDriverWait;

import org.openqa.selenium.support.ui.ExpectedConditions;

import java.util.Random

import java.util.logging.Level;

import java.util.logging.Logger;

def baseurl = context.expand( '${#TestCase#baseurl}' )

def username = context.expand( '${#TestCase#username}' )

def password = context.expand( '${#TestCase#password}' )

def outfolder = context.expand( '${#TestCase#outfolder}' )

def phantomfile = context.expand( '${#TestCase#phantomdriver}' )

dCaps = new DesiredCapabilities();

 dCaps.setJavascriptEnabled(true);

 dCaps.setCapability("elementScrollBehavior", true);

 dCaps.setCapability("phantomjs.page.settings.userAgent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36");

 ArrayList<String> cliArgsCap = new ArrayList<String>();

 cliArgsCap.add("--webdriver-loglevel=NONE");

 cliArgsCap.add("--ignore-ssl-errors=true");

 cliArgsCap.add("--web-security=false");

 cliArgsCap.add("--ssl-protocol=any");

 cliArgsCap.add("--webdriver-logfile=none");

 dCaps.setCapability(PhantomJSDriverService.PHANTOMJS_GHOSTDRIVER_CLI_ARGS,"--ignore-ssl-errors=yes");

 dCaps.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, cliArgsCap);

 dCaps.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY,phantomfile);

def WebDriver driver = new PhantomJSDriver(dCaps)

Logger.getLogger(PhantomJSDriverService.class.getName()).setLevel(Level.OFF);

Logger.getLogger(org.openqa.selenium.phantomjs.PhantomJSDriverService.class.getName()).setLevel(Level.OFF);

Dimension d = new Dimension(1280,1024);

driver.manage().window().setSize(d);

driver.manage().window().maximize();

driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);

7- Now go ahead and write your code as you usually do. Java and groovy are very similar when it comes to WebDriver. Just mae sure you use a bunch of try{..}catch{..}

8- Don't forget to log,assert, fail, and quit properly when you got to the catch{} part. The proper way would be in following order:

1. logs
2. quit the driver
3. fail the test case
4. assert

     log.error (failure_text_message)

    driver.quit()

    testRunner.fail(failure_text_message);
      assert "XX Page Elements Not found, please chaeck the logs"==failure_text_message

   

    return