False positive in TestNG (with TestListenerAdapter)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

False positive in TestNG (with TestListenerAdapter)

Parthy Chandrasekaran

<a class="header" href="eclipse-javadoc:%E2%98%82=dsaf/%5C/Users%5C/parthyc%5C/.m2%5C/repository%5C/org%5C/testng%5C/testng%5C/6.11%5C/testng-6.11.jar%3Corg">org.<a class="header" href="eclipse-javadoc:%E2%98%82=dsaf/%5C/Users%5C/parthyc%5C/.m2%5C/repository%5C/org%5C/testng%5C/testng%5C/6.11%5C/testng-6.11.jar%3Corg.testng">testng.TestListenerAdapter reports no config failures if there is a @BeforeSuite method in the test suite and if a @BeforeClass in the same suite fails.

public void test() {

       class BeforeSuiteTestClass {


           public void beforeSuite() {



        abstract class AbstractTestClass {

           public void fail() {

               throw new RuntimeException("kaboom");



        class TestClass extends AbstractTestClass {


            public void setup() {




           public void test() {



        XmlClass c = new XmlClass(BeforeSuiteTestClass.class);

       XmlClass c2 = new XmlClass(TestClass.class);

        XmlSuite suite = new XmlSuite();

       XmlTest test = new XmlTest();




        XmlTest test2 = new XmlTest();






        TestListenerAdapter testListenerAdapter = new TestListenerAdapter();

       TestNG testNg = new TestNG();





        long numTestSkips = testListenerAdapter.getSkippedTests().size();

       long numConfigSkips = testListenerAdapter.getConfigurationSkips().size();

       long numTestFailures = testListenerAdapter.getFailedTests().size();

       long numConfigFailures = testListenerAdapter.getConfigurationFailures().size();


       System.out.println("numTestSkips " + numTestSkips);

       System.out.println("numConfigSkips " + numConfigSkips);

       System.out.println("numTestFailures " + numTestFailures);

       System.out.println("numConfigFailures " + numConfigFailures);




Default Suite

Total tests run: 1, Failures: 0, Skips: 1

Configuration Failures: 1, Skips: 0


numTestSkips 1

numConfigSkips 0

numTestFailures 0

numConfigFailures 0

Notice that TestListenerAdapter says 0 config failures when there are.

TestNG version: 6.11

Is this a known bug? Is this already fixed?



You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.