Awesum Stuff

How to set Jenkins Job status based on Test Case Pass % ?

We all love to run our tests from Jenkins ! Its quite easy to run your test script from the Jenkins command line execution. What is “not so easy” is setting the status of the Job depending on what percentage of your tests have passed.

For example: There can be a type of Automation test which needs to be 100% passed in order for the Jenkins job to pass. In other cases, that threshold may be 80%.What we intend to do is to figure out a testing framework agnostic way of doing this.

Following is my shot at solving this problem:-

  1. Add a Jenkins Hidden Parameter to your Job through the Hidden Parameter Plugin:-

  1.  Run your tests from the command line. After the execution finishes, just make sure you print the Test Run specific information on the console(you might need to write some code here if your testing framework does not provide this on the console), something like – 98% Tests Passed. For the sake of simplicity, I will just print the message directly:-

 

Note the “exit /b 0” command, this will make sure your batch command does not decide the fate of your Jenkins job. It simply means “whatever happens in the command, always return a status code of 0.

  1. Using the PostBuild Groovy Plugin, analyse the results from the console and set the job status. The Groovy code for this is:-

import hudson.model.*
def matcher = manager.getLogMatcher("(.*)% Tests Passed\$")

if (matcher?.matches()) {
 testinfo=matcher.group(1) + "% Tests Passed"
 
 manager.build.addAction(
 new ParametersAction([
 new StringParameterValue("TEST_INFO", testinfo)
 ]))
 
 manager.addShortText(testinfo) 

 def intValue = matcher.group(1) as int
 if(intValue>95){
 manager.listener.logger.println("More than 95% test method passed.Build is marked as pass")
 manager.buildSuccess()
 manager.addBadge("green.gif",testinfo)
 }
 else if(intValue>90 && intValue<=95){
 manager.listener.logger.println("Between 90 to 95% test method passed.Build is marked as unstable")
 manager.buildUnstable()
 manager.addBadge("yellow.gif",testinfo)
 }
 else{
 manager.listener.logger.println("Less than 90% test method passed.Build is marked as fail")
 manager.buildFailure()
 manager.addBadge("red.gif",testinfo)
 }
}
else{ 
 manager.listener.logger.println("Overall Build is marked as fail")
 manager.buildFailure()
 manager.addBadge("red.gif",testinfo)
}

A better way of doing this can be to put the above code in a .groovy file and mention it in the job like this:-

 

  1. Going one step ahead, we can also pass the variable TEST_INFO’s value to the Email Subject just to make our subject more informative:-

 

Happy Automating!

Harshit Kohli

Advertisements

Categories: Awesum Stuff

Tagged as: , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s