How to set Jenkins Job status based on Test Case Pass % ?

We all love to run our tests from Jenkins ! Its quite easy to run your test script from the Jenkins command line execution. What is “not so easy” is setting the status of the Job depending on what percentage of your tests have passed.

For example: There can be a type of Automation test which needs to be 100% passed in order for the Jenkins job to pass. In other cases, that threshold may be 80%.What we intend to do is to figure out a testing framework agnostic way of doing this.

Following is my shot at solving this problem:-

  1. Add a Jenkins Hidden Parameter to your Job through the Hidden Parameter Plugin:-

  1.  Run your tests from the command line. After the execution finishes, just make sure you print the Test Run specific information on the console(you might need to write some code here if your testing framework does not provide this on the console), something like – 98% Tests Passed. For the sake of simplicity, I will just print the message directly:-


Note the “exit /b 0” command, this will make sure your batch command does not decide the fate of your Jenkins job. It simply means “whatever happens in the command, always return a status code of 0.

  1. Using the PostBuild Groovy Plugin, analyse the results from the console and set the job status. The Groovy code for this is:-

import hudson.model.*
def matcher = manager.getLogMatcher("(.*)% Tests Passed\$")

if (matcher?.matches()) { + "% Tests Passed"
 new ParametersAction([
 new StringParameterValue("TEST_INFO", testinfo)

 def intValue = as int
 manager.listener.logger.println("More than 95% test method passed.Build is marked as pass")
 else if(intValue>90 && intValue<=95){
 manager.listener.logger.println("Between 90 to 95% test method passed.Build is marked as unstable")
 manager.listener.logger.println("Less than 90% test method passed.Build is marked as fail")
 manager.listener.logger.println("Overall Build is marked as fail")

A better way of doing this can be to put the above code in a .groovy file and mention it in the job like this:-


  1. Going one step ahead, we can also pass the variable TEST_INFO’s value to the Email Subject just to make our subject more informative:-


Happy Automating!

Harshit Kohli


5 thoughts on “How to set Jenkins Job status based on Test Case Pass % ?

  1. Hi Harshit,
    I tried following all the steps mentioned but I am getting below error for groovy file. Please help. Thanks.

    Process leaked file descriptors. See for more information
    Build step ‘Execute shell’ marked build as failure
    Overall Build is marked as fail
    ERROR: Failed to evaluate groovy script.
    groovy.lang.MissingPropertyException: No such property: testinfo for class: DataPercent
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(
    at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(
    at groovy.lang.GroovyShell.evaluate(
    at groovy.lang.GroovyShell.evaluate(
    at groovy.lang.Script.evaluate(
    at groovy.lang.Script$evaluate.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(
    at groovy.lang.GroovyShell.evaluate(
    at groovy.lang.GroovyShell.evaluate(
    at groovy.lang.GroovyShell.evaluate(
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SecureGroovyScript.evaluate(
    at org.jvnet.hudson.plugins.groovypostbuild.GroovyPostbuildRecorder.perform(
    at hudson.tasks.BuildStepMonitor$1.perform(
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(
    at hudson.model.Build$BuildExecution.post2(
    at hudson.model.AbstractBuild$
    at hudson.model.Run.execute(
    at hudson.model.ResourceController.execute(


  2. Hi Rohit,

    Please make sure that the names of the variables are exactly the same as in the code snipped share above. Also, you can print inside the groovyscript by using the following line:-

    manager.listener.logger.println(“Value of testinfo = ” + testinfo )

  3. Hi Harshit,

    Appreciate your approach. Can you please elaborate on Step 2. Currently I’ve build my jobs by Invoking ANT and I do not use command prompt to kick start the jobs. So my question here is how do you get the percentage dynamically (Ex: 98% in step 2 which you have hard coded)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s