Threading test failure in latest code from SVN trunk

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Threading test failure in latest code from SVN trunk

Daniel Dyer

The failure below occurs in ThreadPoolSizeTest.verify.  Is this a
result of a fauly assumption in the test case?  Although the thread
pool size is set to 3, this test case is non-deterministic and there is
no guarantee that the executor will use all three threads to execute
the 10 invocations.  In fact, if my understanding of the
java.util.concurrent API is correct, the threads are created lazily, so
the executor may not even create 3 threads.  Now in most cases we might
expect that it would create 3 threads, and if we modified the
ThreadUtils class to call prestartAllCoreThreads(), we could ensure
that it does.  Regardless, it would not guarantee that they all get
used, and it seems that on my dual core Mac using Apple's 1.5 VM, only
2 threads get used.

Dan.

java.lang.AssertionError: Should have run on 3 threads but ran on 2
expected:<3> but was:<2>
at org.testng.Assert.fail(Assert.java:84)
at org.testng.Assert.failNotEquals(Assert.java:438)
at org.testng.Assert.assertEquals(Assert.java:108)
at org.testng.Assert.assertEquals(Assert.java:323)
at test.thread.ThreadPoolSizeTest.verify(ThreadPoolSizeTest.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:639)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:454)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:683)
at
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:115)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:101)
at org.testng.TestRunner.runWorkers(TestRunner.java:665)
at org.testng.TestRunner.privateRun(TestRunner.java:636)
at org.testng.TestRunner.run(TestRunner.java:513)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:279)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:264)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:244)
at org.testng.SuiteRunner.run(SuiteRunner.java:168)
at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:969)
at org.testng.TestNG.runSuitesLocally(TestNG.java:933)
at org.testng.TestNG.run(TestNG.java:701)
at org.testng.TestNG.privateMain(TestNG.java:1001)
at org.testng.TestNG.main(TestNG.java:979)


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Threading test failure in latest code from SVN trunk

Alexandru Popescu ☀

On 12/15/06, Daniel Dyer <[hidden email]> wrote:

>
> The failure below occurs in ThreadPoolSizeTest.verify.  Is this a
> result of a fauly assumption in the test case?  Although the thread
> pool size is set to 3, this test case is non-deterministic and there is
> no guarantee that the executor will use all three threads to execute
> the 10 invocations.  In fact, if my understanding of the
> java.util.concurrent API is correct, the threads are created lazily, so
> the executor may not even create 3 threads.  Now in most cases we might
> expect that it would create 3 threads, and if we modified the
> ThreadUtils class to call prestartAllCoreThreads(), we could ensure
> that it does.  Regardless, it would not guarantee that they all get
> used, and it seems that on my dual core Mac using Apple's 1.5 VM, only
> 2 threads get used.
>

If you check the  ThreadUtil.execute method then you will see that I
am synchronizing the start of the first threadPoolSize invocations (to
emulate load testing). So, with the numbers set on the
ThreadPoolSizeTest.f1() I would expect to really have 3 threads.

Can you please change ThreadPoolSizeTest and use the
ThreadUtil.currentThreadInfo() in all its methods to print the
involved threads and send me the log? Unfortunately, I don't have a
dual-core machine and on mine the wrong behavior never happened.

TIA,

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> Dan.
>
> java.lang.AssertionError: Should have run on 3 threads but ran on 2
> expected:<3> but was:<2>
> at org.testng.Assert.fail(Assert.java:84)
> at org.testng.Assert.failNotEquals(Assert.java:438)
> at org.testng.Assert.assertEquals(Assert.java:108)
> at org.testng.Assert.assertEquals(Assert.java:323)
> at test.thread.ThreadPoolSizeTest.verify(ThreadPoolSizeTest.java:33)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:639)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:454)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:683)
> at
> org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:115)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:101)
> at org.testng.TestRunner.runWorkers(TestRunner.java:665)
> at org.testng.TestRunner.privateRun(TestRunner.java:636)
> at org.testng.TestRunner.run(TestRunner.java:513)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:279)
> at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:264)
> at org.testng.SuiteRunner.privateRun(SuiteRunner.java:244)
> at org.testng.SuiteRunner.run(SuiteRunner.java:168)
> at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:969)
> at org.testng.TestNG.runSuitesLocally(TestNG.java:933)
> at org.testng.TestNG.run(TestNG.java:701)
> at org.testng.TestNG.privateMain(TestNG.java:1001)
> at org.testng.TestNG.main(TestNG.java:979)
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Threading test failure in latest code from SVN trunk

Daniel Dyer

Sorry, I only looked at the line that set up the thread pool and
completely ignored the rest.  I guess it is me that is making faulty
assumptions :)

Unfortunately, like all good concurrency problems, it's not easy to
reproduce.  I originally saw the problem building TestNG with TeamCity.
 I rebuilt with TeamCity and got the same problem so I posted here.
Having since tried to build TestNG both from IDEA and the command-line,
I can't get the problem to happen again.

Having looked at the code again (in more detail this time), and having
looked at the ThreadPoolExecutor source code to confirm exactly how it
works, I can't see anything obviously wrong in ThreadUtil.

I can only think of two things:

1).  Submitting any task after the first 2 results in a
RejectedExecutionException that is swallowed by the empty catch block,
and the latch is later released oblivious to the failures above.  I
don't believe this is happening, since it doesn't make much sense, but
I would add a printStackTrace to the catch block as a sanity check.

2).  The threads are being interrupted while waiting for the latch to
be released and I didn't see the message because the log level wasn't
set high enough.


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Threading test failure in latest code from SVN trunk

Alexandru Popescu ☀

On 12/16/06, Daniel Dyer <[hidden email]> wrote:

>
> Sorry, I only looked at the line that set up the thread pool and
> completely ignored the rest.  I guess it is me that is making faulty
> assumptions :)
>
> Unfortunately, like all good concurrency problems, it's not easy to
> reproduce.  I originally saw the problem building TestNG with TeamCity.
>  I rebuilt with TeamCity and got the same problem so I posted here.
> Having since tried to build TestNG both from IDEA and the command-line,
> I can't get the problem to happen again.
>
> Having looked at the code again (in more detail this time), and having
> looked at the ThreadPoolExecutor source code to confirm exactly how it
> works, I can't see anything obviously wrong in ThreadUtil.
>
> I can only think of two things:
>
> 1).  Submitting any task after the first 2 results in a
> RejectedExecutionException that is swallowed by the empty catch block,
> and the latch is later released oblivious to the failures above.  I
> don't believe this is happening, since it doesn't make much sense, but
> I would add a printStackTrace to the catch block as a sanity check.
>

This is not possible as the ExecutorService is only under ThreadUtil
control and as you see the shutdown call in far down the line.

> 2).  The threads are being interrupted while waiting for the latch to
> be released and I didn't see the message because the log level wasn't
> set high enough.
>

There is no timeout involved or any other mechanism to interrupt those
threads, so I would say that this is also not happening.

The problem may be the m_threadIds map which is not synchronized so
recording those thread ids is not quite correct.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator


>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Threading test failure in latest code from SVN trunk

Daniel Dyer

Yes, my guesses were very speculative.  Your idea about the HashMap
sounds much more likely to be the cause of the problem.


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---