[testng-dev] New Feature: Can configure parallel tests to run exclusive to one and another

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[testng-dev] New Feature: Can configure parallel tests to run exclusive to one and another

lindsey sawatzky
In general, I can run my tests all in parallel with no issues.  However, there are some circumstances when I would like to run a set of tests such that all can run in parallel, except for a subset which can run in parallel with the rest of the set but not with themselves.

An example of this would work as follows.  Run set S in parallel, but do not allow any test from subset T to run at the same time
S := {Test1, Test2, Test3, Test4, Test5}
T := {Test1, Test2}

A run execution for this configuration, when thread-count is set to 3, may look like this:
Test1 start
Test3 start
Test4 start
Test3 end
Test5 start
Test1 end
Test2 start
Test4 end
Test5 end
Test2 end


The key is that this still utilizes all threads efficiently, while at the same time allowing specific tests to be mutually exclusive.

You'll note that using the current dependency mechanism does not exactly fulfil this use case.  For example, if I had set Test2 to be dependant on Test1 then if I wanted to run Test2 alone, first Test1 would have to run.

To re-iterate, dependency isn't the use case - exclusion is the use case.  Another similar terminology would be being able to configure locks.  Test1 and Test2 each require the lock L, and if one has acquired L the other must wait.

I would be happy to work on this feature, if need be.

~Lindsey

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/-r4hjxQ7bfAJ.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] New Feature: Can configure parallel tests to run exclusive to one and another

Cédric Beust ♔-2
Hi Lindsey,

Thanks for your offer to help!

It seems to me TestNG already offers something similar with the @Test(sequential) attribute. When you put this annotation on top of a class, all the methods in that class are guaranteed to run in the same thread, even if the overall strategy for the suite is parallel.

The only constraint is that these tests need to be in the same class.

-- 
Cédric




On Wed, Sep 5, 2012 at 11:14 AM, lindsey sawatzky <[hidden email]> wrote:
In general, I can run my tests all in parallel with no issues.  However, there are some circumstances when I would like to run a set of tests such that all can run in parallel, except for a subset which can run in parallel with the rest of the set but not with themselves.

An example of this would work as follows.  Run set S in parallel, but do not allow any test from subset T to run at the same time
S := {Test1, Test2, Test3, Test4, Test5}
T := {Test1, Test2}

A run execution for this configuration, when thread-count is set to 3, may look like this:
Test1 start
Test3 start
Test4 start
Test3 end
Test5 start
Test1 end
Test2 start
Test4 end
Test5 end
Test2 end


The key is that this still utilizes all threads efficiently, while at the same time allowing specific tests to be mutually exclusive.

You'll note that using the current dependency mechanism does not exactly fulfil this use case.  For example, if I had set Test2 to be dependant on Test1 then if I wanted to run Test2 alone, first Test1 would have to run.

To re-iterate, dependency isn't the use case - exclusion is the use case.  Another similar terminology would be being able to configure locks.  Test1 and Test2 each require the lock L, and if one has acquired L the other must wait.

I would be happy to work on this feature, if need be.

~Lindsey

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/-r4hjxQ7bfAJ.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] New Feature: Can configure parallel tests to run exclusive to one and another

lindsey sawatzky
Oh really, cool I didn't realize that existed!

Could some thought still be put into my request?    Maybe this feature can simply be expanded upon to not be constrained within one test class?

Although we can refactor these tests into one class, I can see that growing a bit too large without too much effort :(

~Lindsey

On Wednesday, September 5, 2012 1:41:52 PM UTC-6, Cédric Beust ♔ wrote:
Hi Lindsey,

Thanks for your offer to help!

It seems to me TestNG already offers something similar with the @Test(sequential) attribute. When you put this annotation on top of a class, all the methods in that class are guaranteed to run in the same thread, even if the overall strategy for the suite is parallel.

The only constraint is that these tests need to be in the same class.

-- 
Cédric




On Wed, Sep 5, 2012 at 11:14 AM, lindsey sawatzky <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="gzNSjNB1zAEJ">dion...@...> wrote:
In general, I can run my tests all in parallel with no issues.  However, there are some circumstances when I would like to run a set of tests such that all can run in parallel, except for a subset which can run in parallel with the rest of the set but not with themselves.

An example of this would work as follows.  Run set S in parallel, but do not allow any test from subset T to run at the same time
S := {Test1, Test2, Test3, Test4, Test5}
T := {Test1, Test2}

A run execution for this configuration, when thread-count is set to 3, may look like this:
Test1 start
Test3 start
Test4 start
Test3 end
Test5 start
Test1 end
Test2 start
Test4 end
Test5 end
Test2 end


The key is that this still utilizes all threads efficiently, while at the same time allowing specific tests to be mutually exclusive.

You'll note that using the current dependency mechanism does not exactly fulfil this use case.  For example, if I had set Test2 to be dependant on Test1 then if I wanted to run Test2 alone, first Test1 would have to run.

To re-iterate, dependency isn't the use case - exclusion is the use case.  Another similar terminology would be being able to configure locks.  Test1 and Test2 each require the lock L, and if one has acquired L the other must wait.

I would be happy to work on this feature, if need be.

~Lindsey

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/-r4hjxQ7bfAJ.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="gzNSjNB1zAEJ">testn...@....
To unsubscribe from this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="gzNSjNB1zAEJ">testng-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/hk8oUyP3Q8EJ.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.