command line options aren't the same as suite.xml options?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

command line options aren't the same as suite.xml options?

Todd Bradley-2
Hi, I just ran into something so strange I can barely believe it. We have a nice healthy suite.xml file that defines our test suite. And for years we specified these settings in the file:

<suite name="Bronze Tests" parallel="tests" data-provider-thread-count="2" thread-count="5" preserve-order="false">
Recently, I changed things to move these parameters to being specified on the command line. I removed them from this <suite> element, and instead started doing this:

-suitethreadpoolsize 3 -parallel tests -dataproviderthreadcount 2 -threadcount 5

We only specify 2 suites on our Java command line, so I think suitethreadpoolsize 3 is irrelevant, though I'm not sure.

The important part is that in the old way, where the parameters are specified in XML, the test suites take 44 minutes. With the new way, we killed them after 2 hours 10 minutes, and they were still running.

So, is it true that these should be equivalent ways of doing things? And if so, has anyone else seen issues like this, or have any clues as to what might be going on?


Thanks,
Todd.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: command line options aren't the same as suite.xml options?

Krishnan Mahadevan
Todd,

AFAIK, you cannot run parallel suites out of the box through the XML mechanism.

The argument "suitethreadpoolsize" actually is telling TestNG to start running both your suites in parallel (you mentioned size of 3, so at max 3 suites can run in parallel).

The equivalent way of doing this via your suite xml file :

You can try running parallel suites via surefire plugin as well, by referring to this blog post of mine which shows how to do this.

The fact that your tests are getting stuck when you run things in parallel hints that perhaps there's some issues with your tests [ something that you are doing at the suite level setup perhaps is not threadsafe ]


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Fri, Mar 10, 2017 at 2:48 AM, Todd Bradley <[hidden email]> wrote:
Hi, I just ran into something so strange I can barely believe it. We have a nice healthy suite.xml file that defines our test suite. And for years we specified these settings in the file:

<suite name="Bronze Tests" parallel="tests" data-provider-thread-count="2" thread-count="5" preserve-order="false">
Recently, I changed things to move these parameters to being specified on the command line. I removed them from this <suite> element, and instead started doing this:

-suitethreadpoolsize 3 -parallel tests -dataproviderthreadcount 2 -threadcount 5

We only specify 2 suites on our Java command line, so I think suitethreadpoolsize 3 is irrelevant, though I'm not sure.

The important part is that in the old way, where the parameters are specified in XML, the test suites take 44 minutes. With the new way, we killed them after 2 hours 10 minutes, and they were still running.

So, is it true that these should be equivalent ways of doing things? And if so, has anyone else seen issues like this, or have any clues as to what might be going on?


Thanks,
Todd.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: command line options aren't the same as suite.xml options?

Todd Bradley-2
I think the suite thread pool is probably the least of my worries. The main thing is that for data-provider-thread-count and thread-count, the behavior I see when I set those in my XML and when I set those on the command line is different. And I assumed the behavior should be exactly the same. Should it?



On Thu, Mar 9, 2017 at 8:23 PM, ⇜Krishnan Mahadevan⇝ <[hidden email]> wrote:
Todd,

AFAIK, you cannot run parallel suites out of the box through the XML mechanism.

The argument "suitethreadpoolsize" actually is telling TestNG to start running both your suites in parallel (you mentioned size of 3, so at max 3 suites can run in parallel).

The equivalent way of doing this via your suite xml file :

You can try running parallel suites via surefire plugin as well, by referring to this blog post of mine which shows how to do this.

The fact that your tests are getting stuck when you run things in parallel hints that perhaps there's some issues with your tests [ something that you are doing at the suite level setup perhaps is not threadsafe ]


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Fri, Mar 10, 2017 at 2:48 AM, Todd Bradley <[hidden email]> wrote:
Hi, I just ran into something so strange I can barely believe it. We have a nice healthy suite.xml file that defines our test suite. And for years we specified these settings in the file:

<suite name="Bronze Tests" parallel="tests" data-provider-thread-count="2" thread-count="5" preserve-order="false">
Recently, I changed things to move these parameters to being specified on the command line. I removed them from this <suite> element, and instead started doing this:

-suitethreadpoolsize 3 -parallel tests -dataproviderthreadcount 2 -threadcount 5

We only specify 2 suites on our Java command line, so I think suitethreadpoolsize 3 is irrelevant, though I'm not sure.

The important part is that in the old way, where the parameters are specified in XML, the test suites take 44 minutes. With the new way, we killed them after 2 hours 10 minutes, and they were still running.

So, is it true that these should be equivalent ways of doing things? And if so, has anyone else seen issues like this, or have any clues as to what might be going on?


Thanks,
Todd.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: command line options aren't the same as suite.xml options?

Krishnan Mahadevan
Todd,
What happens when you remove "-suitethreadpoolsize 3" and just run your tests with "-parallel tests -dataproviderthreadcount 2 -threadcount 5" ?
Do you still see the same behaviour ?

The command line arguments and the XML parameters are just different ways of configuring TestNG. Their behaviour should exactly be the same irrespective of how you are executing TestNG tests.

 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Fri, Mar 10, 2017 at 9:43 AM, Todd Bradley <[hidden email]> wrote:
I think the suite thread pool is probably the least of my worries. The main thing is that for data-provider-thread-count and thread-count, the behavior I see when I set those in my XML and when I set those on the command line is different. And I assumed the behavior should be exactly the same. Should it?



On Thu, Mar 9, 2017 at 8:23 PM, ⇜Krishnan Mahadevan⇝ <[hidden email]> wrote:
Todd,

AFAIK, you cannot run parallel suites out of the box through the XML mechanism.

The argument "suitethreadpoolsize" actually is telling TestNG to start running both your suites in parallel (you mentioned size of 3, so at max 3 suites can run in parallel).

The equivalent way of doing this via your suite xml file :

You can try running parallel suites via surefire plugin as well, by referring to this blog post of mine which shows how to do this.

The fact that your tests are getting stuck when you run things in parallel hints that perhaps there's some issues with your tests [ something that you are doing at the suite level setup perhaps is not threadsafe ]


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Fri, Mar 10, 2017 at 2:48 AM, Todd Bradley <[hidden email]> wrote:
Hi, I just ran into something so strange I can barely believe it. We have a nice healthy suite.xml file that defines our test suite. And for years we specified these settings in the file:

<suite name="Bronze Tests" parallel="tests" data-provider-thread-count="2" thread-count="5" preserve-order="false">
Recently, I changed things to move these parameters to being specified on the command line. I removed them from this <suite> element, and instead started doing this:

-suitethreadpoolsize 3 -parallel tests -dataproviderthreadcount 2 -threadcount 5

We only specify 2 suites on our Java command line, so I think suitethreadpoolsize 3 is irrelevant, though I'm not sure.

The important part is that in the old way, where the parameters are specified in XML, the test suites take 44 minutes. With the new way, we killed them after 2 hours 10 minutes, and they were still running.

So, is it true that these should be equivalent ways of doing things? And if so, has anyone else seen issues like this, or have any clues as to what might be going on?


Thanks,
Todd.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Loading...