Different configurations for different test class instances?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Different configurations for different test class instances?

Phillip Ross-2

I hope someone here can help me find a clean solution to a kind of tricky TestNG problem.

I'm instantiating (programmatically) TestNG instances on a server in response to requests from multiple clients (via a web browser).  The test classes also all reside on the server.  The test class instances each require different configuration information (such as the URL to test against, where to store resulting data, and test run identification information) that is provided by each user from the user's web browser.  So I need to provide a different set of configurations for each TestNG instance for use by the test class instance created and run by each individual TestNG instance.  Is that at all clear?

Is there a way I can get a reference to my test class (perhaps from a listener) after it is created by the TestNG instance?  This would allow me to pass the configuration information to each instance of my test class.  

Or am I completely off base and there is a recommended approach to supplying different test case configuration information to each test case instance.  Maybe I should use parameters?

Thanks in advance for any help.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=52338&messageID=105627#105627

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Hani Suleiman

Yeah why not just pass it all in as parameters to each tng instance  
you create?

By the way, is it possible to have a look at your code? It'd make for  
a pretty cool sample app for people to play with!

On Dec 6, 2006, at 6:33 PM, Ataraxian wrote:

>
> I hope someone here can help me find a clean solution to a kind of  
> tricky TestNG problem.
>
> I'm instantiating (programmatically) TestNG instances on a server  
> in response to requests from multiple clients (via a web browser).  
> The test classes also all reside on the server.  The test class  
> instances each require different configuration information (such as  
> the URL to test against, where to store resulting data, and test  
> run identification information) that is provided by each user from  
> the user's web browser.  So I need to provide a different set of  
> configurations for each TestNG instance for use by the test class  
> instance created and run by each individual TestNG instance.  Is  
> that at all clear?
>
> Is there a way I can get a reference to my test class (perhaps from  
> a listener) after it is created by the TestNG instance?  This would  
> allow me to pass the configuration information to each instance of  
> my test class.
>
> Or am I completely off base and there is a recommended approach to  
> supplying different test case configuration information to each  
> test case instance.  Maybe I should use parameters?
>
> Thanks in advance for any help.
> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?
> threadID=52338&messageID=105627#105627
>
> >


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Alexandru Popescu ☀
In reply to this post by Phillip Ross-2

On 12/7/06, Ataraxian <[hidden email]> wrote:

>
> I hope someone here can help me find a clean solution to a kind of tricky TestNG problem.
>
> I'm instantiating (programmatically) TestNG instances on a server in response to requests from multiple clients (via a web browser).  The test classes also all reside on the server.  The test class instances each require different configuration information (such as the URL to test against, where to store resulting data, and test run identification information) that is provided by each user from the user's web browser.  So I need to provide a different set of configurations for each TestNG instance for use by the test class instance created and run by each individual TestNG instance.  Is that at all clear?
>
> Is there a way I can get a reference to my test class (perhaps from a listener) after it is created by the TestNG instance?  This would allow me to pass the configuration information to each instance of my test class.
>
> Or am I completely off base and there is a recommended approach to supplying different test case configuration information to each test case instance.  Maybe I should use parameters?
>
> Thanks in advance for any help.

Not sure I have fully understood your scenario, but I am wondering if
@DataProvider may be of any help.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=52338&messageID=105627#105627
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Cédric Beust ♔
In reply to this post by Phillip Ross-2
Hi Ataraxian,

Note that I'm CC'ing testng-users, and I suggest we continue the discussion there (your question is general enough).

On 12/6/06, Ataraxian <[hidden email]> wrote:

I hope someone here can help me find a clean solution to a kind of tricky TestNG problem.

I'm instantiating (programmatically) TestNG instances on a server in response to requests from multiple clients (via a web browser).  The test classes also all reside on the server.  The test class instances each require different configuration information (such as the URL to test against, where to store resulting data, and test run identification information) that is provided by each user from the user's web browser.  So I need to provide a different set of configurations for each TestNG instance for use by the test class instance created and run by each individual TestNG instance.  Is that at all clear?

Is there a way I can get a reference to my test class (perhaps from a listener) after it is created by the TestNG instance?  This would allow me to pass the configuration information to each instance of my test class.

Like Hani and Alexandru, I'm not 100% sure what you are trying to do (would love to see some code), so my answer might be a bit off base, but let me point out that if you declare your Data Provider as taking a java.lang.reflect.Method as its first parameter, TestNG will use this parameter to pass the test method about to be called.  This way, you Data Provider can return different values depending on which test method is about to be invoked.

Let us know if any of these suggestions help, and in the negative, please post some of your code...

Thanks!

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Phillip Ross-2

Thanks for all the advice. I'm now much closer than I was yesterday but I would still like some advice before I plunge ahead.  I'd also like to apologize in advance for the long following dissertation.

The short version:  I have multiple instances of TestNG running on a server and I'd like each TestNG instance to supply a different set of parameters to its test class instances.

But here's the long version:

First let me try to explain my problem better.  

I'm using TestNG to drive automated regression tests for validating the web site (across an HTTP connection) of a very large financial institution.  Because these are regression tests, not unit tests, there are many configuration parameters (currently 22) which address such things as the Sign On URL, test run identification, security levels, and verbosity of the test run (show HTML pages? Show http headers?).   Up until recently, we have run these tests using the TestNG eclipse plug-in on each of our individual machines thus avoiding this problem because we each had our own local copy of the configuration information.  I am now providing a web front end to allow anyone in the department to select and run test cases from their web browser.  The web user selects a set of predefined configuration parameters, selects the test cases they would like to run, and then launches the run on the server.  The information on the run is both stored to a database and reflected back to the user's screen.   Up until now, I have just used a static configuration singleton so that I can set the configuration (out of a file) once for an entire test run but now I have multiple users with multiple configurations all running on the same machine, the server.  Other than the configuration problem, all works very well.  I just create a new TestNG instance for each user, store it in a hashmap keyed on their username (basically also a session id), and do any requested operations on their own private copy of TestNG.  But how do I put a private copy of the configuration object into each user's private copy of TestNG?  I have no handle to the test class instances, just the TestNG instances.  

I might not understand the data provider but it seems geared towards providing data that differs from one test case definition to another rather than from one test instance to another.  Is that a correct assessment?  

I've considered sending these configuration parameters in as TestNG parameters but I'm less than happy with that solution because I have so many configuration parameters and I want to keep the individual test cases as free of repetitive code as possible.  I don't want to send each of the 22 parameters individually and sending a long string of key/value pairs seems unclean to me.  

The approach I'm currently considering is this.  Create a static class (perhaps a data provider?) that allows me to register configuration parameters under a unique key.  Pass that key as a parameter to each BeforeClass method.  The BeforeClass method retrieves the information, based on the key, from the static registration class, instantiates a private copy of the Configuration class, and installs it into the current class.  Finally I remove registered object once the test has completed.  I know that sounds awfully complicated but I'm just not seeing a cleaner solution.

Also, I hope to share the code as a sample app soon but you might (or might not) be disappointed to learn that the TestNG driver is implemented as a Google Web Toolkit RPC service.  Also, I'm afraid that I have to deal with our legal department here before I can release anything into the wild but I have hopes that I can manage to push it out there towards the beginning of next year.

Thanks again for any advice and direction.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=52338&messageID=105820#105820

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Hani Suleiman


On Dec 7, 2006, at 1:46 PM, Ataraxian wrote:

> First let me try to explain my problem better.
>
<snip>
>
> I might not understand the data provider but it seems geared  
> towards providing data that differs from one test case definition  
> to another rather than from one test instance to another.  Is that  
> a correct assessment?
>
Yes, data provider is the wrong solution.

> I've considered sending these configuration parameters in as TestNG  
> parameters but I'm less than happy with that solution because I  
> have so many configuration parameters and I want to keep the  
> individual test cases as free of repetitive code as possible.  I  
> don't want to send each of the 22 parameters individually and  
> sending a long string of key/value pairs seems unclean to me.
>
Hm, I still think this is the right approach. What you need though is  
a separate layer in front of testng that does the parameter  
validation. For example, have defaults for each of them, specify that  
some are required, etc. So you'd perform the parameter validation up  
front.

In your classes, you don't need to change every test to specify its  
parameters. You can have one Settings class for example, with a test  
that takes in all the parameters you need and stores them.

So for a test that needs access to its configuration, it'd call  
settings.getURL() etc.

The tricky part now is figuring out how a test can access this  
settings class without making it static (since you'd want to support  
more than one user at a time!)

Since it's a webapp, you can go with a ThreadLocal here. So any  
client can call Settings.getInstance() to get its own thread bound  
settings.

> Also, I hope to share the code as a sample app soon but you might  
> (or might not) be disappointed to learn that the TestNG driver is  
> implemented as a Google Web Toolkit RPC service.

No problem at all.

> Also, I'm afraid that I have to deal with our legal department here  
> before I can release anything into the wild but I have hopes that I  
> can manage to push it out there towards the beginning of next year.

This would be hugely appreciated, otherwise I'd have to write one  
from scratch anyway, and feels a shame to duplicate efforts!



--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Different configurations for different test class instances?

Phillip Ross-2

Thanks, Hani!  That's exactly the suggestion I was hoping for.  Busy day so no time to implement but I'm pretty sure that will work very nicely.  ThreadLocal seems to be the piece of the jigsaw that makes the picture suddenly clear.  

Also...I will make clean up and release of the code a top priority.  With luck, I can put something out for review in the first couple of weeks of January.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=52338&messageID=106107#106107

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---