[testng-dev] Connaot run tests when group-by-instances and preserve-order used together

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[testng-dev] Connaot run tests when group-by-instances and preserve-order used together

Pawel Kowalski
Hi Cedric,

Since new  Gradle 1-milestone-7 has been released, which now supports TestNG >6.2, I decided to upgrade test projects to use TestNG 6.3.1, but I was surprised that none of my suites could be execuded. I always got exception:
 
org.testng.TestNGException:
No free nodes found in:[DynamicGraph
...

    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
...

I have spent some time figuring out what is going on and I've discovered that is caused by using group-by-instance and preserve-order set to true (preserve-order default is true,) at the same time. When the mentioned options are set to true  and there are at least two classes with tests to run , than it is very likely that TestNG will build dependency graph with cycles even if tests don't have any explicit dependency. The built graph will have dependency on classes to satisfy order and then dependency on instances to guarantee grouping. The more classes is the probability is higher. In my real projects it was impossible to run anything anytime.  I was able to reproduced the problem with only two classes (it is not so easy ) but with three classes the probability is almost 100%.

Below is my "almost always reproducible" example:

//  1st class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

//  2nd class
package test;
import org.testng.annotations.Test;

public class Y {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}


//  3rd class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

Run with  following suite:

<suite name="Suite1" verbose="1" group-by-instances="true">
  <test name="Test" preserve-order="true">
    <packages>
       <package name="test.*" />
    </packages>
  </test>
</suite>

An almost every time get:


org.testng.TestNGException:
No free nodes found in:[DynamicGraph
  Ready:[X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4], X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4], Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce], Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce], Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9], Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]]
  Running:[]
  Finished:[]
  Edges:
     X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
     X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
]
    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1142)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.run(TestNG.java:979)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:202)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)

Regards,
Paweł Kowalski

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|

[testng-dev] Re: Connaot run tests when group-by-instances and preserve-order used together

Pawel Kowalski
Small update
In my  example 3rd class should be  of course Z (or any different than X and Y) not X. 

W dniu 11 stycznia 2012 18:22 użytkownik Pawel Kowalski <[hidden email]> napisał:
Hi Cedric,

Since new  Gradle 1-milestone-7 has been released, which now supports TestNG >6.2, I decided to upgrade test projects to use TestNG 6.3.1, but I was surprised that none of my suites could be execuded. I always got exception:
 
org.testng.TestNGException:
No free nodes found in:[DynamicGraph
...

    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
...

I have spent some time figuring out what is going on and I've discovered that is caused by using group-by-instance and preserve-order set to true (preserve-order default is true,) at the same time. When the mentioned options are set to true  and there are at least two classes with tests to run , than it is very likely that TestNG will build dependency graph with cycles even if tests don't have any explicit dependency. The built graph will have dependency on classes to satisfy order and then dependency on instances to guarantee grouping. The more classes is the probability is higher. In my real projects it was impossible to run anything anytime.  I was able to reproduced the problem with only two classes (it is not so easy ) but with three classes the probability is almost 100%.

Below is my "almost always reproducible" example:

//  1st class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

//  2nd class
package test;
import org.testng.annotations.Test;

public class Y {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}


//  3rd class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

Run with  following suite:

<suite name="Suite1" verbose="1" group-by-instances="true">
  <test name="Test" preserve-order="true">
    <packages>
       <package name="test.*" />
    </packages>
  </test>
</suite>

An almost every time get:


org.testng.TestNGException:
No free nodes found in:[DynamicGraph
  Ready:[X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4], X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4], Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce], Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce], Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9], Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]]
  Running:[]
  Finished:[]
  Edges:
     X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
     X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
]
    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1142)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.run(TestNG.java:979)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:202)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)

Regards,
Paweł Kowalski



--
Pozdrawiam,
Paweł Kowalski

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.