Problems combining threadPoolSize, invocationCount and dataProvider

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems combining threadPoolSize, invocationCount and dataProvider

tarun3kumar
Hi,

While trying to change a bunch of testcases into a rudimentary load test, I ran into the problem that the test runner seems to miss results of some of the testcases (it only sees the last 'invocationCount' results). I was wondering if this is due to some misunderstanding or testNG abuse of my side or that I triggered a bug/feature.

I attached a small example demonstrating the problem. (the code is an adaption of the code found in this thread http://forums.opensymphony.com/thread.jspa?messageID=93663)

The attached testcase gives this output in the latest eclipse plugin (version 5.3.2.0):

pool-1-thread-3 Start: []
pool-1-thread-1 Start: [foo]
pool-1-thread-2 Start: [bar]
pool-1-thread-3 Done : []
pool-1-thread-1 Done : [foo]
pool-1-thread-2 Done : [bar]
pool-1-thread-1 Start: [baz]
pool-1-thread-3 Start: [foo, baz]
pool-1-thread-1 Done : [baz]
pool-1-thread-3 Done : [foo, baz]
pool-1-thread-2 Start: [foo, bar]
pool-1-thread-2 Done : [foo, bar]
pool-1-thread-1 Start: [baz, bar]
pool-1-thread-1 Done : [baz, bar]
pool-1-thread-3 Start: [foo, baz, bar]
pool-1-thread-3 Done : [foo, baz, bar]
PASSED: actionWithFailuresOn([baz, bar])
PASSED: actionWithFailuresOn([foo, bar])
PASSED: actionWithFailuresOn([foo, baz, bar])

===============================================
    TestNG
    Tests run: 3, Failures: 0, Skips: 0
===============================================

If one changes invocationCount then the number of passed tests follows suit.

Running the testcase on a vanilla testng 5.3 jar on the same windows XP box with java version "1.5.0_09" gives the same results. Yet on Solaris 5.8 with java version "1.5.0_06" it yields:

java -cp $HOME/lib/java/testng-5.3-jdk15.jar:. org.testng.TestNG -testclass TestNG
[Parser] Running:
  Command line suite

pool-1-thread-2 Start: []
pool-1-thread-2 Done : []
pool-1-thread-2 Start: [bar]
pool-1-thread-2 Done : [bar]
pool-1-thread-2 Start: [baz]
pool-1-thread-2 Done : [baz]
pool-1-thread-2 Start: [foo, bar]
pool-1-thread-2 Done : [foo, bar]
pool-1-thread-2 Start: [foo, baz]
pool-1-thread-2 Done : [foo, baz]
pool-1-thread-2 Start: [baz, bar]
pool-1-thread-2 Done : [baz, bar]
pool-1-thread-2 Start: [foo, baz, bar]
pool-1-thread-2 Done : [foo, baz, bar]
pool-1-thread-3 Start: [foo]
pool-1-thread-3 Done : [foo]

===============================================
Command line suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

E.g. only 2 testcases of the total are seen (enabling the sleep in the testcase makes the third 'visible' as well)

Any ideas?

Cheers,

Ric
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53196&messageID=106980#106980


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

TestNG.java (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problems combining threadPoolSize, invocationCount and dataProvider

Alexandru Popescu ☀

I am pretty sure this is related to a problem with the reporters in
fact. Can you please check it against the TestNG 5.4 distro?

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator


On 12/12/06, Ric Klaren <[hidden email]> wrote:

> Hi,
>
> While trying to change a bunch of testcases into a rudimentary load test, I ran into the problem that the test runner seems to miss results of some of the testcases (it only sees the last 'invocationCount' results). I was wondering if this is due to some misunderstanding or testNG abuse of my side or that I triggered a bug/feature.
>
> I attached a small example demonstrating the problem. (the code is an adaption of the code found in this thread http://forums.opensymphony.com/thread.jspa?messageID=93663)
>
> The attached testcase gives this output in the latest eclipse plugin (version 5.3.2.0):
>
> pool-1-thread-3 Start: []
> pool-1-thread-1 Start: [foo]
> pool-1-thread-2 Start: [bar]
> pool-1-thread-3 Done : []
> pool-1-thread-1 Done : [foo]
> pool-1-thread-2 Done : [bar]
> pool-1-thread-1 Start: [baz]
> pool-1-thread-3 Start: [foo, baz]
> pool-1-thread-1 Done : [baz]
> pool-1-thread-3 Done : [foo, baz]
> pool-1-thread-2 Start: [foo, bar]
> pool-1-thread-2 Done : [foo, bar]
> pool-1-thread-1 Start: [baz, bar]
> pool-1-thread-1 Done : [baz, bar]
> pool-1-thread-3 Start: [foo, baz, bar]
> pool-1-thread-3 Done : [foo, baz, bar]
> PASSED: actionWithFailuresOn([baz, bar])
> PASSED: actionWithFailuresOn([foo, bar])
> PASSED: actionWithFailuresOn([foo, baz, bar])
>
> ===============================================
>     TestNG
>     Tests run: 3, Failures: 0, Skips: 0
> ===============================================
>
> If one changes invocationCount then the number of passed tests follows suit.
>
> Running the testcase on a vanilla testng 5.3 jar on the same windows XP box with java version "1.5.0_09" gives the same results. Yet on Solaris 5.8 with java version "1.5.0_06" it yields:
>
> java -cp $HOME/lib/java/testng-5.3-jdk15.jar:. org.testng.TestNG -testclass TestNG
> [Parser] Running:
>   Command line suite
>
> pool-1-thread-2 Start: []
> pool-1-thread-2 Done : []
> pool-1-thread-2 Start: [bar]
> pool-1-thread-2 Done : [bar]
> pool-1-thread-2 Start: [baz]
> pool-1-thread-2 Done : [baz]
> pool-1-thread-2 Start: [foo, bar]
> pool-1-thread-2 Done : [foo, bar]
> pool-1-thread-2 Start: [foo, baz]
> pool-1-thread-2 Done : [foo, baz]
> pool-1-thread-2 Start: [baz, bar]
> pool-1-thread-2 Done : [baz, bar]
> pool-1-thread-2 Start: [foo, baz, bar]
> pool-1-thread-2 Done : [foo, baz, bar]
> pool-1-thread-3 Start: [foo]
> pool-1-thread-3 Done : [foo]
>
> ===============================================
> Command line suite
> Total tests run: 2, Failures: 0, Skips: 0
> ===============================================
>
> E.g. only 2 testcases of the total are seen (enabling the sleep in the testcase makes the third 'visible' as well)
>
> Any ideas?
>
> Cheers,
>
> Ric
> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=53196&messageID=106980#106980
>
>
> >
>
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Problems combining threadPoolSize, invocationCount and dataProvider

Ric Klaren

Hi,

On 12/12/06, Alexandru Popescu <[hidden email]> wrote:
> I am pretty sure this is related to a problem with the reporters in
> fact. Can you please check it against the TestNG 5.4 distro?

It's fixed in 5.4 :)

I had missed that there was a new download. Sorry for the bandwidth.

Cheers,

Ric

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---