Re: TestNG questions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: TestNG questions

Cédric Beust ♔
Hi Sandeep

On 12/27/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
 
Hello Cedric --
 
Thanks for your quick response.
 
In my opinion a static method implementation such as TestNG.getDefault().getTestContext() or something
similar would be much useful. The test context (or similar object) would then return
test name, test method name, data provider name, group being executed, handle to class instance and all such info.
This would be easily acessible from any test method as well as data provider implementation too.

I try to stay away from static methods whenever I can, and I don't think we need one here.

What I have in mind right now is a similar trick to the Method one:  if your DataProvider declares a parameter of a certain type (probably ITestContext), then TestNG will inject the correct value so you can determine what groups have been included in this run.

I'll have to read the second part of your email more closely to give you an informed response...

--
Cedric


############
 
I have another question. (So far I could not find the solution to this so I am guessing that its not
possible in TestNG either).
 
I want to run certain test methods in pre-defined order to create a test scenario. These methods could be
part of multiple classes. Each individual method is a test case in itself. And group of these methods in certain
order forms what we call a "scenario".
 
For example, consider following 2 classes and their test methods.
 
UI Class
  DefineDeviceObject - Defines a device object in a software
  DefineDeviceConfiguration - Defines the configuration template for the device
  DeleteDeviceObject - Deletes device objects from software 
  ChangeDeviceConfiguration - change device config
 
Device Class
  ReadConfig
  ActivatePhysicalDevice
  AssociatePhysicalDevice
 
Note that each one of this test methods (i.e. test cases) could be tested independently.
 
 
Now, for end-to-end workflows, consider following scenarios based on above test methods.
Here the order of test method execution is very important.
 
Scenario 1 ....
UI.DefineDeviceObject
UI.DefineDeviceConfiguration
Device.ActivatePhysicalDevice
Device.AssociatePhysicalDevice
UI.ChangeDeviceConfiguration
Device.ReadConfig
 
Scenario 2 ... (only if Scenario 1 is successful)
UI.ChangeDeviceConfiguration
Device.ReadConfig
UI.ChangeDeviceConfiguration
Device.ReadConfig
UI.DeleteDeviceObject
 
Note that each test scenario becomes end-to-end test case.
 
So far I haven't found any way to implement the scenarios in TestNG. If I use groups and DependsOnGroups attributes -
because of the complexity of the Tests and the dependency might change from scenario to scenario. Is this do-able in
Suite XML file ?
 
Thanks,
Sandeep
 


From: [hidden email] [mailto:[hidden email]] On Behalf Of C�dric Beust ?
Sent: Tuesday, December 26, 2006 11:15 PM
To: Sandeep Girgaonkar (sgirgaon); [hidden email]
Subject: Re: TestNG questions

Hi Sandeep,

It's not possible currently, but your idea makes a lot of sense to me.

Let me think about it and I'll get back to you shortly.

Thanks!

--
Cedric


On 12/26/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
Hello Cedric --
 
Since I could not access the Web Forum, I took liberty to send this e-mail
directly to you.
 
Quick question on TestNG data provider...
 
I have a test method say "testOne" and its data provider is "dp".
"dp" returns collection of 1000 data objects which means testOne
willl be called 1000 times.
 
I have 2 groups - "smoke" and "complete". Now for "smoke" tests I want
to run testOne for say only 25 data objects and for "complete" tests I want
to run it for all 1000 data objects.
 
So the question is - Is there any way to know for what group the given test method is
being run for ? I need this information to return appropriate number of objects. Is this information
available at run-time in test method as well as data provider ?
 
Is/are there any alternate ways to achieve this ?
 
Thanks in advance,
Sandeep



--
C dric



--
C�dric

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG questions

Alexandru Popescu ☀
On 12/28/06, C dric Beust   <[hidden email]> wrote:

> Hi Sandeep
>
> On 12/27/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
> >
> >
> >
> > Hello Cedric --
> >
> > Thanks for your quick response.
> >
> > In my opinion a static method implementation such as
> TestNG.getDefault().getTestContext() or something
> > similar would be much useful. The test context (or similar object) would
> then return
> > test name, test method name, data provider name, group being executed,
> handle to class instance and all such info.
> > This would be easily acessible from any test method as well as data
> provider implementation too.
>
> I try to stay away from static methods whenever I can, and I don't think we
> need one here.
>
> What I have in mind right now is a similar trick to the Method one:  if your
> DataProvider declares a parameter of a certain type (probably ITestContext),
> then TestNG will inject the correct value so you can determine what groups
> have been included in this run.
>

I really think Cedric proposal is better then a static access based API.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> I'll have to read the second part of your email more closely to give you an
> informed response...
>
> --
> Cedric
>
>
> >
> >
> > ############
> >
> > I have another question. (So far I could not find the solution to this so
> I am guessing that its not
> > possible in TestNG either).
> >
> > I want to run certain test methods in pre-defined order to create a test
> scenario. These methods could be
> > part of multiple classes. Each individual method is a test case in itself.
> And group of these methods in certain
> > order forms what we call a "scenario".
> >
> > For example, consider following 2 classes and their test methods.
> >
> > UI Class
> >   DefineDeviceObject - Defines a device object in a software
> >   DefineDeviceConfiguration - Defines the configuration template for the
> device
> >   DeleteDeviceObject - Deletes device objects from software
> >
> >   ChangeDeviceConfiguration - change device config
> >
> > Device Class
> >
> >   ReadConfig  ActivatePhysicalDevice
> >   AssociatePhysicalDevice
> >
> > Note that each one of this test methods (i.e. test cases) could be tested
> independently.
> >
> >
> > Now, for end-to-end workflows, consider following scenarios based on above
> test methods.
> > Here the order of test method execution is very important.
> >
> > Scenario 1 ....
> > UI.DefineDeviceObject
> > UI.DefineDeviceConfiguration
> > Device.ActivatePhysicalDevice
> > Device.AssociatePhysicalDevice
> > UI.ChangeDeviceConfiguration
> > Device.ReadConfig
> >
> > Scenario 2 ... (only if Scenario 1 is successful)
> >
> > UI.ChangeDeviceConfiguration
> > Device.ReadConfig
> >
> > UI.ChangeDeviceConfiguration
> > Device.ReadConfig
> > UI.DeleteDeviceObject
> >
> > Note that each test scenario becomes end-to-end test case.
> >
> > So far I haven't found any way to implement the scenarios in TestNG. If I
> use groups and DependsOnGroups attributes -
> > because of the complexity of the Tests and the dependency might change
> from scenario to scenario. Is this do-able in
> > Suite XML file ?
> >
> > Thanks,
> > Sandeep
> >
> >
> >
> > ________________________________
>  From: [hidden email] [mailto: [hidden email]] On Behalf Of C dric Beust
> ?
> > Sent: Tuesday, December 26, 2006 11:15 PM
> > To: Sandeep Girgaonkar (sgirgaon); [hidden email]
> > Subject: Re: TestNG questions
> >
> >
> >
> > Hi Sandeep,
> >
> > It's not possible currently, but your idea makes a lot of sense to me.
> >
> > Let me think about it and I'll get back to you shortly.
> >
> > Thanks!
> >
> > --
> > Cedric
> >
> >
> >
> > On 12/26/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
> > >
> > >
> > > Hello Cedric --
> > >
> > > Since I could not access the Web Forum, I took liberty to send this
> e-mail
> > > directly to you.
> > >
> > > Quick question on TestNG data provider...
> > >
> > > I have a test method say "testOne" and its data provider is "dp".
> > > "dp" returns collection of 1000 data objects which means testOne
> > > willl be called 1000 times.
> > >
> > > I have 2 groups - "smoke" and "complete". Now for "smoke" tests I want
> > > to run testOne for say only 25 data objects and for "complete" tests I
> want
> > > to run it for all 1000 data objects.
> > >
> > > So the question is - Is there any way to know for what group the given
> test method is
> > > being run for ? I need this information to return appropriate number of
> objects. Is this information
> > > available at run-time in test method as well as data provider ?
> > >
> > > Is/are there any alternate ways to achieve this ?
> > >
> > > Thanks in advance,
> > > Sandeep
> >
> >
> >
> > --
> > C dric
> >
>
>
>
> --
> C dric
>
>  >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG questions

Cédric Beust ♔
In reply to this post by Cédric Beust ♔
Hi Sandeep,

I have implemented the fix.  You can now declare your DataProvider as follows:

import org.testng.ITestContext;

  @DataProvider(name = "testContext")
  public Object[][] createContext(ITestContext ctx) {
    String[] groups = ctx.getIncludedGroups();

and then decide what to return based on the groups included in the current run (you can also query other useful info on the ITestContext object).

Please try it out and let us know if it fixes your problem:

http://testng.org/testng-5.5beta-jdk15.jar

--
Cedric


On 12/27/06, C dric Beust <[hidden email]> wrote:
Hi Sandeep

On 12/27/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
 
Hello Cedric --
 
Thanks for your quick response.
 
In my opinion a static method implementation such as TestNG.getDefault().getTestContext() or something
similar would be much useful. The test context (or similar object) would then return
test name, test method name, data provider name, group being executed, handle to class instance and all such info.
This would be easily acessible from any test method as well as data provider implementation too.

I try to stay away from static methods whenever I can, and I don't think we need one here.

What I have in mind right now is a similar trick to the Method one:  if your DataProvider declares a parameter of a certain type (probably ITestContext), then TestNG will inject the correct value so you can determine what groups have been included in this run.

I'll have to read the second part of your email more closely to give you an informed response...

--
Cedric


############
 
I have another question. (So far I could not find the solution to this so I am guessing that its not
possible in TestNG either).
 
I want to run certain test methods in pre-defined order to create a test scenario. These methods could be
part of multiple classes. Each individual method is a test case in itself. And group of these methods in certain
order forms what we call a "scenario".
 
For example, consider following 2 classes and their test methods.
 
UI Class
  DefineDeviceObject - Defines a device object in a software
  DefineDeviceConfiguration - Defines the configuration template for the device
  DeleteDeviceObject - Deletes device objects from software 
  ChangeDeviceConfiguration - change device config
 
Device Class
  ReadConfig
  ActivatePhysicalDevice
  AssociatePhysicalDevice
 
Note that each one of this test methods (i.e. test cases) could be tested independently.
 
 
Now, for end-to-end workflows, consider following scenarios based on above test methods.
Here the order of test method execution is very important.
 
Scenario 1 ....
UI.DefineDeviceObject
UI.DefineDeviceConfiguration
Device.ActivatePhysicalDevice
Device.AssociatePhysicalDevice
UI.ChangeDeviceConfiguration
Device.ReadConfig
 
Scenario 2 ... (only if Scenario 1 is successful)
UI.ChangeDeviceConfiguration
Device.ReadConfig
UI.ChangeDeviceConfiguration
Device.ReadConfig
UI.DeleteDeviceObject
 
Note that each test scenario becomes end-to-end test case.
 
So far I haven't found any way to implement the scenarios in TestNG. If I use groups and DependsOnGroups attributes -
because of the complexity of the Tests and the dependency might change from scenario to scenario. Is this do-able in
Suite XML file ?
 
Thanks,
Sandeep
 


From: [hidden email] [mailto:[hidden email]] On Behalf Of C dric Beust ?
Sent: Tuesday, December 26, 2006 11:15 PM
To: Sandeep Girgaonkar (sgirgaon); [hidden email]
Subject: Re: TestNG questions

Hi Sandeep,

It's not possible currently, but your idea makes a lot of sense to me.

Let me think about it and I'll get back to you shortly.

Thanks!

--
Cedric


On 12/26/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
Hello Cedric --
 
Since I could not access the Web Forum, I took liberty to send this e-mail
directly to you.
 
Quick question on TestNG data provider...
 
I have a test method say "testOne" and its data provider is "dp".
"dp" returns collection of 1000 data objects which means testOne
willl be called 1000 times.
 
I have 2 groups - "smoke" and "complete". Now for "smoke" tests I want
to run testOne for say only 25 data objects and for "complete" tests I want
to run it for all 1000 data objects.
 
So the question is - Is there any way to know for what group the given test method is
being run for ? I need this information to return appropriate number of objects. Is this information
available at run-time in test method as well as data provider ?
 
Is/are there any alternate ways to achieve this ?
 
Thanks in advance,
Sandeep



--
C dric



--
C dric





--
C dric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG questions

Cédric Beust ♔
In reply to this post by Cédric Beust ♔


On 12/28/06, Sandeep Girgaonkar (sgirgaon) <[hidden email]> wrote:
Hello Cedric --
 
Here are my comments ...
 
1. I am looking for following information from the ITestContext which I could not get.
    - Current method name (with class name and package name) for which data provider was called
    - any other relevent details about this method such as defined groups, dependsOn groups, invocationCount etc.

You can declare a Method as parameter in your DataProvider, and you'll receive the test method that's about to be run:

@DataProvider(...)
public void f(Method m, ITestContext tc) { ... }

The ITestContext contains the information above, as far as I can tell (I thought you only needed the included groups anyway?).


    Without this information I cannot construct different data sets for different methods/groups.
 
2. I believe getClass() and toString() methods are Java Object default implementaion. Just want to confirm.   
 
3. My observation is that ITestContext is partially filled - which makes sense
 
4. ITestContext gives access to the methods which are probably used only by the underlying framework.
   For example,
   ITestNGMethod.setInvocationCount()
   ITestNGMethod.setMissingGroup()
   ITestNGMethod.setDate()
 
   Calling these methods have some adverse effects on the test execution. Since altering test execution
   behavior is not the intent, in my opinion, exposing such methods should be avoided as far as possible.
   Is there any other Interface/class that gives access to only relevant and read only information ?
 

I should probably restrict the interface I expose in this method so that developer are not tempted to tamper with it...


--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---