[testng-dev] How to run tests in parallel with constant thread count

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[testng-dev] How to run tests in parallel with constant thread count

shankarkc78@gmail.com
Hi,
Our product is tweaked to support say 60 concurrent users. 
Say I have 2 maven projects with 5 test suites. each having 10 tests. For convenience lets say all tests except one each takes 1min to run. Long running test takes 10 min to complete.
Lets assume i run these 2 projects in parallel in 2 jenkins job. Each job runs 3 test suites in parallel with 10 tests in parallel( using parallel=methods and thread-count =5)
Now when i start the tests during first minute we have 2 * 3 * 10. After first min all tests in a suite completed execution except the long running test. It means only 6 tests are running. Rest of the threads (2 * 3 * 9) are idle and will not run remaining suites as current thread didnt finish because of long running test. It cannot pick remaining 2 suites though they are idle. How do we solve this and put a constant load on our server. Now 2nd batch of test suite execution happens only after 10 minutes. All the threads were idle expect one for 9 minutes. This is a common issue in product companies i think. But I am not sure how to tackle it properly without wasting lot of time. How you all handle this at your organization?
Thanks
Shankar KC 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] How to run tests in parallel with constant thread count

shankarkc78@gmail.com

Hi Cedric
Do you have any recommendations to run this and achieve constant load on server?
Thank you
Shankar

On Jul 5, 2016 1:53 PM, "shankar KC" <[hidden email]> wrote:
Hi,
Our product is tweaked to support say 60 concurrent users. 
Say I have 2 maven projects with 5 test suites. each having 10 tests. For convenience lets say all tests except one each takes 1min to run. Long running test takes 10 min to complete.
Lets assume i run these 2 projects in parallel in 2 jenkins job. Each job runs 3 test suites in parallel with 10 tests in parallel( using parallel=methods and thread-count =5)
Now when i start the tests during first minute we have 2 * 3 * 10. After first min all tests in a suite completed execution except the long running test. It means only 6 tests are running. Rest of the threads (2 * 3 * 9) are idle and will not run remaining suites as current thread didnt finish because of long running test. It cannot pick remaining 2 suites though they are idle. How do we solve this and put a constant load on our server. Now 2nd batch of test suite execution happens only after 10 minutes. All the threads were idle expect one for 9 minutes. This is a common issue in product companies i think. But I am not sure how to tackle it properly without wasting lot of time. How you all handle this at your organization?
Thanks
Shankar KC 

--
You received this message because you are subscribed to a topic in the Google Groups "testng-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/testng-dev/7NMM0UfMNds/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] How to run tests in parallel with constant thread count

shankarkc78@gmail.com
Hi All this is the response i got from testng users group.

=======================
Yes, and we parallelize tests as much as we reasonably can. But ultimately we have the same issue you do, and just accept that we can't perfectly balance all the test execution threads without a lot more work. And for us, at least, that extra work isn't worth it because all it would gain is a small improvement in test run time. So we do other things to improve test run time - running tests of slow changing features less often, running slow tests less often, etc.
=======================
Is there any better idea to handle this? After first minute most of the threads are idle thought there are  tests to be executed. We have thousands of tests to run and this behavior slowing down our execution. Please let me know your thoughts on problems like this.
Thanks
Shankar KC

On Tuesday, July 5, 2016 at 5:32:37 PM UTC+5:30, shankar KC wrote:

Hi Cedric
Do you have any recommendations to run this and achieve constant load on server?
Thank you
Shankar

On Jul 5, 2016 1:53 PM, "shankar KC" <[hidden email]> wrote:
Hi,
Our product is tweaked to support say 60 concurrent users. 
Say I have 2 maven projects with 5 test suites. each having 10 tests. For convenience lets say all tests except one each takes 1min to run. Long running test takes 10 min to complete.
Lets assume i run these 2 projects in parallel in 2 jenkins job. Each job runs 3 test suites in parallel with 10 tests in parallel( using parallel=methods and thread-count =5)
Now when i start the tests during first minute we have 2 * 3 * 10. After first min all tests in a suite completed execution except the long running test. It means only 6 tests are running. Rest of the threads (2 * 3 * 9) are idle and will not run remaining suites as current thread didnt finish because of long running test. It cannot pick remaining 2 suites though they are idle. How do we solve this and put a constant load on our server. Now 2nd batch of test suite execution happens only after 10 minutes. All the threads were idle expect one for 9 minutes. This is a common issue in product companies i think. But I am not sure how to tackle it properly without wasting lot of time. How you all handle this at your organization?
Thanks
Shankar KC 

--
You received this message because you are subscribed to a topic in the Google Groups "testng-dev" group.
To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/testng-dev/7NMM0UfMNds/unsubscribe" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/topic/testng-dev/7NMM0UfMNds/unsubscribe&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/topic/testng-dev/7NMM0UfMNds/unsubscribe&#39;;return true;">https://groups.google.com/d/topic/testng-dev/7NMM0UfMNds/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at <a href="https://groups.google.com/group/testng-dev" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-dev&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-dev&#39;;return true;">https://groups.google.com/group/testng-dev.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/d/optout.