Wrong number of test executions when a TestNG suite xml calls other suite xmls
Locked
Wrong number of test executions when a TestNG suite xml calls other suite xmls
 
|
Hello TestNG guys,
We have hit a problem with latest TestNG Eclipse plugin when calling a testNG suite xml from another suite xml. Here is how to reproduce the problem: 1. Create a simple java project with 1 class <PROJECT ROOT>/src/com/ Test1Class.java which has just 1 test method. 2. Create 2 suite xml files placed in <PROJECT ROOT>/suites: 2.1. the first xml is called suite1.xml and has the following content: <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite name="Suite1" verbose="1"> <test name="Suite1 tests"> <classes> <class name="com.Test1Class" /> </classes> </test> </suite> 2.2. the second xml is called complex_suite.xml and has the following content: <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite name="Complex suite" verbose="1"> <suite-files> <suite-file path="suites/suite1.xml" /> </suite-files> </suite> Now run the complex_suite.xml and you will get the following output in the console: [TestNG] Running: D:\Eclipse_workspace\TestTestNG\suites\suite1.xml running testMethod1() =============================================== Suite1 Total tests run: 1, Failures: 0, Skips: 0 =============================================== [TestNG] Running: D:\Eclipse_workspace\TestTestNG\suites\suite1.xml running testMethod1() =============================================== Suite1 Total tests run: 2, Failures: 0, Skips: 0 =============================================== [TestNG] Running: D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml =============================================== Complex suite Total tests run: 4, Failures: 0, Skips: 0 =============================================== As you can see the test method is called twice, while expected only one execution. It is interestig to note that changing the complex_suite.xml to: <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite name="Complex suite" verbose="1"> <suite-files> <suite-file path="suite1.xml" /> </suite-files> </suite> produces the following output in the console: java.io.FileNotFoundException: D:\Eclipse_workspace\TestTestNG \suite1.xml (The system cannot find the file specified) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:106) at java.io.FileInputStream.<init>(FileInputStream.java:66) at org.testng.xml.Parser.parse(Parser.java:165) at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java: 271) at org.testng.TestNG.run(TestNG.java:980) at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109) at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java: 202) at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173) [TestNG] Running: D:\Eclipse_workspace\TestTestNG\suites\suite1.xml running testMethod1() =============================================== Suite1 Total tests run: 1, Failures: 0, Skips: 0 =============================================== [TestNG] Running: D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml =============================================== Complex suite Total tests run: 1, Failures: 0, Skips: 0 =============================================== So it complains it cannot find the internal suite1.xml, but indeed runs the test method just once as expected. I've tried calling many internal suites from the complex one and the issue is the same: all tests are run once and then are run once again in the same order. thank you for your help :) Nikolay -- You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
|
|
I've seen this issue too, and it's not limited to the TestNG Eclipse
plugin. I'm using the maven surefire plugin to fire of my parent suite, and when I build the project I get the same issue. I'm guessing the problem is in TestNG itself, not any related plugin. On Feb 28, 3:27 am, ngogovski <[hidden email]> wrote: > Hello TestNG guys, > > We have hit a problem with latest TestNG Eclipse plugin when calling a > testNG suite xml from another suite xml. > > Here is how to reproduce the problem: > 1. Create a simple java project with 1 class <PROJECT ROOT>/src/com/ > Test1Class.java which has just 1 test method. > 2. Create 2 suite xml files placed in <PROJECT ROOT>/suites: > 2.1. the first xml is called suite1.xml and has the following content: > > <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> > <suite name="Suite1" verbose="1"> > <test name="Suite1 tests"> > <classes> > <class name="com.Test1Class" /> > </classes> > </test> > </suite> > > 2.2. the second xml is called complex_suite.xml and has the following > content: > > <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> > <suite name="Complex suite" verbose="1"> > <suite-files> > <suite-file path="suites/suite1.xml" /> > </suite-files> > </suite> > > Now run the complex_suite.xml and you will get the following output in > the console: > > [TestNG] Running: > D:\Eclipse_workspace\TestTestNG\suites\suite1.xml > > running testMethod1() > > =============================================== > Suite1 > Total tests run: 1, Failures: 0, Skips: 0 > =============================================== > > [TestNG] Running: > D:\Eclipse_workspace\TestTestNG\suites\suite1.xml > > running testMethod1() > > =============================================== > Suite1 > Total tests run: 2, Failures: 0, Skips: 0 > =============================================== > > [TestNG] Running: > D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml > > =============================================== > Complex suite > Total tests run: 4, Failures: 0, Skips: 0 > =============================================== > > As you can see the test method is called twice, while expected only > one execution. > > It is interestig to note that changing the complex_suite.xml to: > > <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> > <suite name="Complex suite" verbose="1"> > <suite-files> > <suite-file path="suite1.xml" /> > </suite-files> > </suite> > > produces the following output in the console: > > java.io.FileNotFoundException: D:\Eclipse_workspace\TestTestNG > \suite1.xml (The system cannot find the file specified) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:106) > at java.io.FileInputStream.<init>(FileInputStream.java:66) > at org.testng.xml.Parser.parse(Parser.java:165) > at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java: > 271) > at org.testng.TestNG.run(TestNG.java:980) > at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109) > at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java: > 202) > at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173) > [TestNG] Running: > D:\Eclipse_workspace\TestTestNG\suites\suite1.xml > > running testMethod1() > > =============================================== > Suite1 > Total tests run: 1, Failures: 0, Skips: 0 > =============================================== > > [TestNG] Running: > D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml > > =============================================== > Complex suite > Total tests run: 1, Failures: 0, Skips: 0 > =============================================== > > So it complains it cannot find the internal suite1.xml, but indeed > runs the test method just once as expected. > > I've tried calling many internal suites from the complex one and the > issue is the same: all tests are run once and then are run once again > in the same order. > > thank you for your help :) > Nikolay -- You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
Re: Re: Wrong number of test executions when a TestNG suite xml calls other suite xmls
|
|
Yes, it looks like a TestNG issue, I'll look into it.
-- Cédric On Thu, Mar 8, 2012 at 4:37 PM, ssimmons <[hidden email]> wrote: I've seen this issue too, and it's not limited to the TestNG Eclipse You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
|
Hi,
Any progress about that? It's really disappointing not beeing able to run nested suites properly any more (i don't remember having that problem in the past). Using testng-6.5.1 greets, Arno
|
Re: Re: Wrong number of test executions when a TestNG suite xml calls other suite xmls
|
|
Sorry, no, I haven't had the time to look into it.
-- Cédric On Fri, Jun 8, 2012 at 9:23 AM, Jostar <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
|
|
Is there a previous version that can be reverted to where this is not an issue?
On Friday, June 8, 2012 10:14:14 AM UTC-7, Cédric Beust ♔ wrote: Sorry, no, I haven't had the time to look into it.-- You received this message because you are subscribed to the Google Groups "testng-users" group. To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/t-c1beQ87XsJ. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
Re: Re: Wrong number of test executions when a TestNG suite xml calls other suite xmls
|
|
All the previous versions have their own update site at http://beust.com/eclipse-old
-- Cédric On Fri, Jun 15, 2012 at 11:35 AM, justme <[hidden email]> wrote: Is there a previous version that can be reverted to where this is not an issue? You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
|
|
In reply to this post by Cédric Beust ♔-2
Can you maybe describe in a few lines how the "collecting" of classes/methods to execute is working, so i'll have a look at it myself. May i could help to "faire avancer le schmilblick" ;-)
That issue is really bothering me!!! Greets & Thx in advance, Arno <quote author="Cedric Beust"> Sorry, no, I haven't had the time to look into it. -- Cédric On Fri, Jun 8, 2012 at 9:23 AM, Jostar <arno.jost@hispeed.ch> wrote: > > Hi, > > Any progress about that? > It's really disappointing not beeing able to run nested suites properly any > more (i don't remember having that problem in the past). > Using testng-6.5.1 > > greets, > Arno > > > |
Re: Re: Wrong number of test executions when a TestNG suite xml calls other suite xmls
|
|
Hi,
I actually fixed the duplicate bug in the Eclipse plug-in over the weekend, can you please update your plug-in and confirm it's working for you? Thanks. -- Cédric On Mon, Jun 18, 2012 at 8:12 AM, Jostar <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "testng-users" group. To post to this group, send email to [hidden email]. To unsubscribe from this group, send email to [hidden email]. For more options, visit this group at http://groups.google.com/group/testng-users?hl=en. |
|
|
I did and yes it does.
Thx a lot!
|
«
Return to testng-users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |