xfail in Testng

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

xfail in Testng

Umang
I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: xfail in Testng

Todd Bradley-2
I've never heard of anything like that in TestNG. You could fake it by writing your own test runner or report generator.


Todd.

On Thu, Mar 29, 2018 at 5:37 AM, Umang <[hidden email]> wrote:
I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: xfail in Testng

Umang
Hi Todd,
Thanks for your input. It would be great if you could show some examples for the test runner.

On Thursday, March 29, 2018 at 7:29:10 PM UTC+5:30, Todd Bradley wrote:
I've never heard of anything like that in TestNG. You could fake it by writing your own test runner or report generator.


Todd.

On Thu, Mar 29, 2018 at 5:37 AM, Umang <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="eViBfiK_AgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">yoyote...@...> wrote:
I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="eViBfiK_AgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="eViBfiK_AgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: xfail in Testng

Krishnan Mahadevan

Umang,

 

Here’s how you can get this done in your project. Please make sure you are making use of TestNG 6.14.3

 

Sample test class:

 

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(XFailTestPruner.class)
public class SampleTestClassForXFail {

   
@Test
   
public void happyFlow() {
        System.
err.println("Happy flow");
    }

   
@BrokenTest
    @Test
   
public void brokenTest() {
       
throw new RuntimeException("Simulating a broken test");
    }
}

 

Here’s how the listener would look like:

 

import org.testng.ITestContext;
import org.testng.ITestNGMethod;
import org.testng.TestListenerAdapter;

public class XFailTestPruner extends TestListenerAdapter {
   
@Override
   
public void onFinish(ITestContext testContext) {
       
//We are looking for our annotation only on the methods.
        //This can be enhanced to look for the annotation at the class level as well.
       
testContext.getFailedTests().getAllResults().removeIf(each -> isBrokenTest(each.getMethod()));
    }

   
private static boolean isBrokenTest(ITestNGMethod tngMethod) {
       
return tngMethod.getConstructorOrMethod().getMethod().getAnnotation(BrokenTest.class) != null;
    }
}

 

Here’s how the execution output look like:

 

java.lang.RuntimeException: Simulating a broken test

    at com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail.brokenTest(SampleTestClassForXFail.java:17)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:498)

    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)

    at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)

    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)

    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)

    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)

    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)

    at org.testng.TestRunner.privateRun(TestRunner.java:648)

    at org.testng.TestRunner.run(TestRunner.java:505)

    at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)

    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)

    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)

    at org.testng.SuiteRunner.run(SuiteRunner.java:364)

    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)

    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)

    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)

    at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)

    at org.testng.TestNG.runSuites(TestNG.java:1049)

    at org.testng.TestNG.run(TestNG.java:1017)

    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)

    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

 

Happy flow

===============================================

Default Suite

Total tests run: 1, Failures: 0, Skips: 0

===============================================

 

Process finished with exit code 0

 

 

Here’s screenshots from TestNG default reports.

 

 

 

<?xml version="1.0" encoding="UTF-8"?>

<testng-results skipped="0" failed="0" ignored="0" total="1" passed="1">

  <reporter-output>

  </reporter-output>

  <suite name="Default Suite" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

    <groups>

    </groups>

    <test name="testbed" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

      <class name="com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail">

        <test-method status="PASS" signature="happyFlow()[pri:0, instance:com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail@12cdcf4]" name="happyFlow" duration-ms="1" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

          <reporter-output>

          </reporter-output>

        </test-method> <!-- happyFlow -->

      </class> <!-- com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail -->

    </test> <!-- testbed -->

  </suite> <!-- Default Suite -->

</testng-results>

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

Reply-To: <[hidden email]>
Date: Friday, March 30, 2018 at 12:49 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] xfail in Testng

 

Hi Todd,
Thanks for your input. It would be great if you could show some examples for the test runner.

On Thursday, March 29, 2018 at 7:29:10 PM UTC+5:30, Todd Bradley wrote:

I've never heard of anything like that in TestNG. You could fake it by writing your own test runner or report generator.

 

 

Todd.

 

On Thu, Mar 29, 2018 at 5:37 AM, Umang <<a href="javascript:" target="_blank">yoyote...@...> wrote:

I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
<a href="javascript:" target="_blank">testng-users...@....
To post to this group, send email to
<a href="javascript:" target="_blank">testng...@....
Visit this group at
https://groups.google.com/group/testng-users.
For more options, visit
https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[hidden email].
To post to this group, send email to
[hidden email].
Visit this group at
https://groups.google.com/group/testng-users.
For more options, visit
https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: xfail in Testng

Umang
Hi Krishnan,
Thanks for your reply. Is there anyway that I can show the result into a new label "broken" rather than a pass state,like pass:0,skip:0,broken:1

On Friday, March 30, 2018 at 3:32:45 PM UTC+5:30, Krishnan Mahadevan wrote:

Umang,

 

Here’s how you can get this done in your project. Please make sure you are making use of TestNG 6.14.3

 

Sample test class:

 

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(XFailTestPruner.class)
public class SampleTestClassForXFail {

   
@Test
   
public void happyFlow() {
        System.
err.println("Happy flow");
    }

   
@BrokenTest
    @Test
   
public void brokenTest() {
       
throw new RuntimeException("Simulating a broken test");
    }
}

 

Here’s how the listener would look like:

 

import org.testng.ITestContext;
import org.testng.ITestNGMethod;
import org.testng.TestListenerAdapter;

public class XFailTestPruner extends TestListenerAdapter {
   
@Override
   
public void onFinish(ITestContext testContext) {
       
//We are looking for our annotation only on the methods.
        //This can be enhanced to look for the annotation at the class level as well.
       
testContext.getFailedTests().getAllResults().removeIf(each -> isBrokenTest(each.getMethod()));
    }

   
private static boolean isBrokenTest(ITestNGMethod tngMethod) {
       
return tngMethod.getConstructorOrMethod().getMethod().getAnnotation(BrokenTest.class) != null;
    }
}

 

Here’s how the execution output look like:

 

java.lang.RuntimeException: Simulating a broken test

    at com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail.brokenTest(SampleTestClassForXFail.java:17)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:498)

    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)

    at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)

    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)

    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)

    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)

    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)

    at org.testng.TestRunner.privateRun(TestRunner.java:648)

    at org.testng.TestRunner.run(TestRunner.java:505)

    at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)

    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)

    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)

    at org.testng.SuiteRunner.run(SuiteRunner.java:364)

    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)

    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)

    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)

    at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)

    at org.testng.TestNG.runSuites(TestNG.java:1049)

    at org.testng.TestNG.run(TestNG.java:1017)

    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)

    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

 

Happy flow

===============================================

Default Suite

Total tests run: 1, Failures: 0, Skips: 0

===============================================

 

Process finished with exit code 0

 

 

Here’s screenshots from TestNG default reports.

 

 

 

<?xml version="1.0" encoding="UTF-8"?>

<testng-results skipped="0" failed="0" ignored="0" total="1" passed="1">

  <reporter-output>

  </reporter-output>

  <suite name="Default Suite" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

    <groups>

    </groups>

    <test name="testbed" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

      <class name="com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail">

        <test-method status="PASS" signature="happyFlow()[pri:0, instance:com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail@12cdcf4]" name="happyFlow" duration-ms="1" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

          <reporter-output>

          </reporter-output>

        </test-method> <!-- happyFlow -->

      </class> <!-- com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail -->

    </test> <!-- testbed -->

  </suite> <!-- Default Suite -->

</testng-results>

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="EVNSfdAAAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Friday, March 30, 2018 at 12:49 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="EVNSfdAAAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: Re: [testng-users] xfail in Testng

 

Hi Todd,
Thanks for your input. It would be great if you could show some examples for the test runner.

On Thursday, March 29, 2018 at 7:29:10 PM UTC+5:30, Todd Bradley wrote:

I've never heard of anything like that in TestNG. You could fake it by writing your own test runner or report generator.

 

 

Todd.

 

On Thu, Mar 29, 2018 at 5:37 AM, Umang <[hidden email]> wrote:

I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
testng-users...@googlegroups.com.
To post to this group, send email to
[hidden email].
Visit this group at
<a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit
<a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="EVNSfdAAAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users+unsubscribe@googlegroups.com.
To post to this group, send email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="EVNSfdAAAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users@....
Visit this group at
<a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit
<a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: xfail in Testng

Krishnan Mahadevan
Nope. Not straight out of the box. 

But you should be able to do that by having your custom reporter (default reports won't have it) look for the custom annotation and then classify and show it accordingly. 

On Fri, Mar 30, 2018, 17:21 Umang <[hidden email]> wrote:
Hi Krishnan,
Thanks for your reply. Is there anyway that I can show the result into a new label "broken" rather than a pass state,like pass:0,skip:0,broken:1


On Friday, March 30, 2018 at 3:32:45 PM UTC+5:30, Krishnan Mahadevan wrote:

Umang,

 

Here’s how you can get this done in your project. Please make sure you are making use of TestNG 6.14.3

 

Sample test class:

 

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(XFailTestPruner.class)
public class SampleTestClassForXFail {

   
@Test
   
public void happyFlow() {
        System.
err.println("Happy flow");
    }

   
@BrokenTest
    @Test
   
public void brokenTest() {
       
throw new RuntimeException("Simulating a broken test");
    }
}

 

Here’s how the listener would look like:

 

import org.testng.ITestContext;
import org.testng.ITestNGMethod;
import org.testng.TestListenerAdapter;

public class XFailTestPruner extends TestListenerAdapter {
   
@Override
   
public void onFinish(ITestContext testContext) {
       
//We are looking for our annotation only on the methods.
        //This can be enhanced to look for the annotation at the class level as well.
       
testContext.getFailedTests().getAllResults().removeIf(each -> isBrokenTest(each.getMethod()));
    }

   
private static boolean isBrokenTest(ITestNGMethod tngMethod) {
       
return tngMethod.getConstructorOrMethod().getMethod().getAnnotation(BrokenTest.class) != null;
    }
}

 

Here’s how the execution output look like:

 

java.lang.RuntimeException: Simulating a broken test

    at com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail.brokenTest(SampleTestClassForXFail.java:17)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:498)

    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)

    at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)

    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)

    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)

    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)

    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)

    at org.testng.TestRunner.privateRun(TestRunner.java:648)

    at org.testng.TestRunner.run(TestRunner.java:505)

    at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)

    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)

    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)

    at org.testng.SuiteRunner.run(SuiteRunner.java:364)

    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)

    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)

    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)

    at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)

    at org.testng.TestNG.runSuites(TestNG.java:1049)

    at org.testng.TestNG.run(TestNG.java:1017)

    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)

    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

 

Happy flow

===============================================

Default Suite

Total tests run: 1, Failures: 0, Skips: 0

===============================================

 

Process finished with exit code 0

 

 

Here’s screenshots from TestNG default reports.

 

 

 

<?xml version="1.0" encoding="UTF-8"?>

<testng-results skipped="0" failed="0" ignored="0" total="1" passed="1">

  <reporter-output>

  </reporter-output>

  <suite name="Default Suite" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

    <groups>

    </groups>

    <test name="testbed" duration-ms="64" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

      <class name="com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail">

        <test-method status="PASS" signature="happyFlow()[pri:0, instance:com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail@12cdcf4]" name="happyFlow" duration-ms="1" started-at="2018-03-30T09:43:11Z" finished-at="2018-03-30T09:43:11Z">

          <reporter-output>

          </reporter-output>

        </test-method> <!-- happyFlow -->

      </class> <!-- com.rationaleemotions.googleforums.xfail.SampleTestClassForXFail -->

    </test> <!-- testbed -->

  </suite> <!-- Default Suite -->

</testng-results>

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

Reply-To: <[hidden email]>


Date: Friday, March 30, 2018 at 12:49 PM

To: testng-users <[hidden email]>


Subject: Re: [testng-users] xfail in Testng

 

Hi Todd,
Thanks for your input. It would be great if you could show some examples for the test runner.

On Thursday, March 29, 2018 at 7:29:10 PM UTC+5:30, Todd Bradley wrote:

I've never heard of anything like that in TestNG. You could fake it by writing your own test runner or report generator.

 

 

Todd.

 

On Thu, Mar 29, 2018 at 5:37 AM, Umang <[hidden email]> wrote:

I would like to know, if there is some behaviour equivalent to Xfail similar to that in Pytest. This will be useful when we find a bug in our software and we write a test to document how the software should behave. This test will fail until we fix the bug. To avoid having a failing test we mark the test as xfail. Once the bug is fixed, we remove the xfail marker. I'm don't want the test to be skipped, rather, I want the test to be run but not reported as a failure(or success as the case might be) in the report.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[hidden email].
To post to this group, send email to
[hidden email].
Visit this group at
https://groups.google.com/group/testng-users.
For more options, visit
https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[hidden email].
To post to this group, send email to
[hidden email].
Visit this group at
https://groups.google.com/group/testng-users.
For more options, visit
https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.