False positive in TestNG (with TestListenerAdapter)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

False positive in TestNG (with TestListenerAdapter)

Parthy Chandrasekaran
Hi,

<a class="header" href="eclipse-javadoc:%E2%98%82=dsaf/%5C/Users%5C/parthyc%5C/.m2%5C/repository%5C/org%5C/testng%5C/testng%5C/6.11%5C/testng-6.11.jar%3Corg">org.<a class="header" href="eclipse-javadoc:%E2%98%82=dsaf/%5C/Users%5C/parthyc%5C/.m2%5C/repository%5C/org%5C/testng%5C/testng%5C/6.11%5C/testng-6.11.jar%3Corg.testng">testng.TestListenerAdapter reports no config failures if there is a @BeforeSuite method in the test suite and if a @BeforeClass in the same suite fails.

public void test() {

       class BeforeSuiteTestClass {

           @BeforeSuite

           public void beforeSuite() {

           }

       };

        abstract class AbstractTestClass {

           public void fail() {

               throw new RuntimeException("kaboom");

           }

       };


        class TestClass extends AbstractTestClass {


            @BeforeClass

            public void setup() {

               super.fail();

           }


            @Test

           public void test() {

           }


        }




        XmlClass c = new XmlClass(BeforeSuiteTestClass.class);

       XmlClass c2 = new XmlClass(TestClass.class);


        XmlSuite suite = new XmlSuite();

       XmlTest test = new XmlTest();

       test.setName("BeforeSuiteTestClass");

       test.setClasses(Collections.singletonList(c));

       test.setSuite(suite);


        XmlTest test2 = new XmlTest();

       test2.setName("TestClass");

       test2.setClasses(Collections.singletonList(c2));

       test2.setSuite(suite);



        suite.addTest(test);

       suite.addTest(test2);


        TestListenerAdapter testListenerAdapter = new TestListenerAdapter();

       TestNG testNg = new TestNG();

       testNg.setPreserveOrder(true);

       testNg.addListener((ITestNGListener)testListenerAdapter);


       testNg.setXmlSuites(Collections.singletonList(suite));

       testNg.run();


        long numTestSkips = testListenerAdapter.getSkippedTests().size();

       long numConfigSkips = testListenerAdapter.getConfigurationSkips().size();

       long numTestFailures = testListenerAdapter.getFailedTests().size();

       long numConfigFailures = testListenerAdapter.getConfigurationFailures().size();

       

       System.out.println("numTestSkips " + numTestSkips);

       System.out.println("numConfigSkips " + numConfigSkips);

       System.out.println("numTestFailures " + numTestFailures);

       System.out.println("numConfigFailures " + numConfigFailures);

   }


Output:

===============================================

Default Suite

Total tests run: 1, Failures: 0, Skips: 1

Configuration Failures: 1, Skips: 0

===============================================


numTestSkips 1

numConfigSkips 0

numTestFailures 0

numConfigFailures 0



Notice that TestListenerAdapter says 0 config failures when there are.


TestNG version: 6.11


Is this a known bug? Is this already fixed?


thanks

parthy

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.