Dependency organization when dealing with persistent external objects

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Dependency organization when dealing with persistent external objects

tarun3kumar

Hi all,

I've just started to make myself familiar with TestNG, and I've run into a problem in my test organization.

Say I'm testing an application that stores some information in a persistent external object, such as a file or a database entry. Suppose there are test methods create(), modify() and destroy() to deal with the object, and they only work if run in this order - you can't modify the object before it has been created, and so on. I've tried to express this with dependencies:

  create(): none
  modify(): dependsOnMethds = "create"
  destroy(): dependsOnMethds = "modify"

This works fine if all three tests pass. But now imagine that modify() fails (either because of an error in the application or, as is often the case for me while I'm developing the tests, in the test method itself). After I've fixed the error, I'd like to continue my test; because the object created in the first test run still exists and create() fails in this situation, I only want to run modify() and destroy() on the existing object. But due to the dependencies, the testng-failed.xml also contains create() again, and since that fails now, the other tests are skipped.

Any suggestions how to deal with this? Currently I'm telling TestNG "if modify() is run, then run create() first", but what I'd actually need is something different, along the lines of "if create() and modify() are [i]both[/i] run, [i]then[/i] start with create()", and I haven't found a way to do this yet. But as I said, I'm quite new to the tool, maybe I'm just overlooking something basic? Or am I perhaps using a wrong approach anyway? Should I organize my tests in a different way?
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107843#107843


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Alexandru Popescu ☀

On 12/15/06, Holger Wahlen <[hidden email]> wrote:

>
> Hi all,
>
> I've just started to make myself familiar with TestNG, and I've run into a problem in my test organization.
>
> Say I'm testing an application that stores some information in a persistent external object, such as a file or a database entry. Suppose there are test methods create(), modify() and destroy() to deal with the object, and they only work if run in this order - you can't modify the object before it has been created, and so on. I've tried to express this with dependencies:
>
>   create(): none
>   modify(): dependsOnMethds = "create"
>   destroy(): dependsOnMethds = "modify"
>
> This works fine if all three tests pass. But now imagine that modify() fails (either because of an error in the application or, as is often the case for me while I'm developing the tests, in the test method itself). After I've fixed the error, I'd like to continue my test; because the object created in the first test run still exists and create() fails in this situation, I only want to run modify() and destroy() on the existing object. But due to the dependencies, the testng-failed.xml also contains create() again, and since that fails now, the other tests are skipped.
>
> Any suggestions how to deal with this? Currently I'm telling TestNG "if modify() is run, then run create() first", but what I'd actually need is something different, along the lines of "if create() and modify() are [i]both[/i] run, [i]then[/i] start with create()", and I haven't found a way to do this yet. But as I said, I'm quite new to the tool, maybe I'm just overlooking something basic? Or am I perhaps using a wrong approach anyway? Should I organize my tests in a different way?

The problem I am seeing is that after a failure you leave some
temporary data in an unknown, or unwanted state; moreover you would
like to that your test to pick up from where it left last time. So,
probably I would look into:

1/ trying to clean up after the test (whatever the test status is).
Adding a@BeforeClass or @AfterClass method that cleans the test data
will assure that each time you run the test it will start all over.

An important aspect of testing is that the results are reproducible:
you definitely want to make sure that your tests are doing the same
thing each time and are yielding the same results.

2/ I am not sure about the condition: if "both run then run create",
and I think you are somehow refering to something similar to my above
point.

3/ other than these, I think your organization of code is quite what I would do.

HTH,

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator


> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107843#107843
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

tarun3kumar

Alexandru Popescu wrote:
> 1/ trying to clean up after the test (whatever the test status is).
> Adding a@BeforeClass or @AfterClass method that cleans the test data
> will assure that each time you run the test it will start all over.

Hm, okay, destroy() is basically a cleanup method, so I could annotate that as @AfterClass. But how would that look in detail, particularly with regard to "alwaysRun"? The cleanup should be run if modify() fails, but shouldn't if create() fails because then there's no object to clean up anyway. How do I express that?

Moreover, though, in my particular case I'm not really happy with the approach to run the complete test again simply because create() can be time-consuming. If I were to perform the tests manually, I'd be rather content with the fact that the object has been created already and I could just start again with modify(), and I'd like to see the test tool support that somehow as well.

> An important aspect of testing is that the results are reproducible:
> you definitely want to make sure that your tests are doing the same
> thing each time and are yielding the same results.

Okay, I want each [i]method[/i] to do the same thing each time it's called. But I don't want my entire [i]test[/i] to consist of the same methods every time, do I? After all, TestNG already creates the testng-failed.xml so that I can rerun only the failed methods, my problem is just that the dependencies there cause other methods to be run as well even if I don't want that in this situation.

> 2/ I am not sure about the condition: if "both run then run create",
> and I think you are somehow refering to something similar to my above
> point.

I'm looking for a possibility to say:

(a) if the test (testng.xml) contains both create() and modify(), run create() first,
(b) but if it contains only modify(), run only that, i.e., there's no need to include create() in that case.

My current "dependsOnMethods" covers (a), but also means that I can't build a test that only runs modify(), without create(), so I can't accomplish (b).

(Of course, this would only be really helpful if the testng-failed.xml was constructed accordingly then and didn't contain create() explicitly.)
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107879#107879


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Alexandru Popescu ☀

On 12/15/06, Holger Wahlen <[hidden email]> wrote:
>
> Alexandru Popescu wrote:
> > 1/ trying to clean up after the test (whatever the test status is).
> > Adding a@BeforeClass or @AfterClass method that cleans the test data
> > will assure that each time you run the test it will start all over.
>
> Hm, okay, destroy() is basically a cleanup method, so I could annotate that as @AfterClass. But how would that look in detail, particularly with regard to "alwaysRun"? The cleanup should be run if modify() fails, but shouldn't if create() fails because then there's no object to clean up anyway. How do I express that?
>

Ahhh... I haven't seen any code in this thread, so detailing how it
should look it pretty difficult. I would probably set alwaysRun, and
use some booleans to determine what was run and what was not and drive
the code in @AfterClass method according to these.

> Moreover, though, in my particular case I'm not really happy with the approach to run the complete test again simply because create() can be time-consuming. If I were to perform the tests manually, I'd be rather content with the fact that the object has been created already and I could just start again with modify(), and I'd like to see the test tool support that somehow as well.
>

I am not sure what you are saying here. If your object creation is too
complex/too expensive then just do it once in a @BeforeTest or
something. And test is completely separately.

> > An important aspect of testing is that the results are reproducible:
> > you definitely want to make sure that your tests are doing the same
> > thing each time and are yielding the same results.
>
> Okay, I want each [i]method[/i] to do the same thing each time it's called. But I don't want my entire [i]test[/i] to consist of the same methods every time, do I?

Yes, you do... at least this is the intention of testing: to have
their same behavior repeated all over again so that you can identify
if your latest changes have broken anything or not (imagine you run
your test: create succeeds and modify fails. You go change the code
for modify and touch the code for create. Re-run: this time the test
starts from modify() this time, but you have broken the create(), so
you are entering a vicious circle).

After all, TestNG already creates the testng-failed.xml so that I can
rerun only the failed methods, my problem is just that the
dependencies there cause other methods to be run as well even if I
don't want that in this situation.

>
> > 2/ I am not sure about the condition: if "both run then run create",
> > and I think you are somehow refering to something similar to my above
> > point.
>
> I'm looking for a possibility to say:
>
> (a) if the test (testng.xml) contains both create() and modify(), run create() first,
> (b) but if it contains only modify(), run only that, i.e., there's no need to include create() in that case.
>
> My current "dependsOnMethods" covers (a), but also means that I can't build a test that only runs modify(), without create(), so I can't accomplish (b).
>
> (Of course, this would only be really helpful if the testng-failed.xml was constructed accordingly then and didn't contain create() explicitly.)

Oke. Then what I think you need are MethodSelector and the scripting
support TestNG is offering. Please check the documentation for these
and post back your advance.

HTH,

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107879#107879
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔
In reply to this post by tarun3kumar
Hi Holger,

I think Alexandru already gave you a few good pointers, which I'll reinforce:

On 12/15/06, Holger Wahlen <[hidden email]> wrote:


Moreover, though, in my particular case I'm not really happy with the approach to run the complete test again simply because create() can be time-consuming. If I were to perform the tests manually, I'd be rather content with the fact that the object has been created already and I could just start again with modify(), and I'd like to see the test tool support that somehow as well.

It's not very easy to express this with TestNG, but nothing stops you from storing some information in your tests (remember:  TestNG doesn't reinitialize your test class between methods, so you can safely store information in fields).  For example, you could maintain a map of <Object ids, boolean> that keeps track of objects that have been successfully created, which create() would consult first before doing its own creation.

Would this help?

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

tarun3kumar

Cedric Beust wrote:
> It's not very easy to express this with TestNG, but nothing stops you from
> storing some information in your tests (remember: TestNG doesn't
> reinitialize your test class between methods, so you can safely store
> information in fields).

I can do that while I'm in a single test run, yes; but the class [i]is[/i] reinitialized between test runs, of course, so the fields from the initial run wouldn't be available any more in the rerun for the failed test. I guess I could write the data to a file or something, but that doesn't seem a very elegant solution either.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107962#107962


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔


On 12/15/06, Holger Wahlen <[hidden email]> wrote:

Cedric Beust wrote:
> It's not very easy to express this with TestNG, but nothing stops you from
> storing some information in your tests (remember: TestNG doesn't
> reinitialize your test class between methods, so you can safely store
> information in fields).

I can do that while I'm in a single test run, yes; but the class [i]is[/i] reinitialized between test runs, of course, so the fields from the initial run wouldn't be available any more in the rerun for the failed test. I guess I could write the data to a file or something, but that doesn't seem a very elegant solution either.

True, but you need to realize that you are asking TestNG to keep track of a state that even yourself find hard to maintain in your own code :-)

Is it too expensive to do a quick select on the database to verify if the object has already been created?

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

isterin
In reply to this post by tarun3kumar

I think you're taking the wrong approach, at least from my
understanding.  There is no persistent state of the test invocations,
so you can't say, if something fails, continue after I fix it.  You
have to rerun your test suite and/or class.

The best way to fix your dilemma is to provide a destroy method
annotated with @AfterTestClass and that way if anything fails you can
do the cleanup there and bring your database to a consistent state
needed to rerun the tests.  In your case, it would be checking whether
the object exists and if so, deleting it.

Ilya

On 12/15/06, Holger Wahlen <[hidden email]> wrote:

>
> Hi all,
>
> I've just started to make myself familiar with TestNG, and I've run into a problem in my test organization.
>
> Say I'm testing an application that stores some information in a persistent external object, such as a file or a database entry. Suppose there are test methods create(), modify() and destroy() to deal with the object, and they only work if run in this order - you can't modify the object before it has been created, and so on. I've tried to express this with dependencies:
>
>   create(): none
>   modify(): dependsOnMethds = "create"
>   destroy(): dependsOnMethds = "modify"
>
> This works fine if all three tests pass. But now imagine that modify() fails (either because of an error in the application or, as is often the case for me while I'm developing the tests, in the test method itself). After I've fixed the error, I'd like to continue my test; because the object created in the first test run still exists and create() fails in this situation, I only want to run modify() and destroy() on the existing object. But due to the dependencies, the testng-failed.xml also contains create() again, and since that fails now, the other tests are skipped.
>
> Any suggestions how to deal with this? Currently I'm telling TestNG "if modify() is run, then run create() first", but what I'd actually need is something different, along the lines of "if create() and modify() are [i]both[/i] run, [i]then[/i] start with create()", and I haven't found a way to do this yet. But as I said, I'm quite new to the tool, maybe I'm just overlooking something basic? Or am I perhaps using a wrong approach anyway? Should I organize my tests in a different way?
> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=107843#107843
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Thomas.M.Stahl@googlemail.com
In reply to this post by tarun3kumar

I had similar problems testing my site (a chess server) with TestNG and
Selenium. Before I can test, say, playing a game, I need two players.
So both must register and login, create and navigate to the relevant
game. This is quite time consuming and too complex to be handled with
the TestNG dependencies.

I distinguish between *incremental* and *complete* tests (switching
with a TestNG parameter).

For complete tests, TestNG and Selenium do just the above. Registration
and login are considered as setup methods here (not as tests). At the
end of this setup I store the complete system state into a file
(exporting the DB tables and cookies). Then I run the game tests.
For incremental tests, the setup method does *not* walk through the
registration and login process, but simply loads the relevant DB tables
and cookies. This is *much* faster than the walk-through.

Every test class has its own BeforeClass method that uses the above
mechanism (which is made available by a common base class). This way,
every test class starts with a *clean* state, all my test classes are
independent from another.

Applied to your problem, once create has been successful, the
corresponding state is stored (into files). A repeated, incremental
test would just restore that state and then test modify and/or delete.
As soon as you change your create process, you run a complete test,
thus creating an updated setup state for future incremental tests.

My experience is that the TestNG dependencies are nice, but can easily
lead to very convoluted test systems. I try to use even my functional
tests with as much *unit* test philosophy as possible (keeping them
independent and reproducable). I use dependsOnMethod only sparingly.
Very often I resolve these dependencies with independent test classes,
as soon as I need dependsOn too often (say 3-5 in a test class). My
advice is: do not use TestNG to handle dependencies, but try to avoid
them alltogether - even if this means increased effort at first. In the
long run, you will save a lot of time.

Thomas - www.playchess.de


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔
All great advice, Thomas, thanks a lot!

Like you say, dependencies are useful for a very specific niche of problems, but by and large, the majority of your test code base should typically not be using them.

--
Cedric


On 12/16/06, [hidden email] <[hidden email]> wrote:

I had similar problems testing my site (a chess server) with TestNG and
Selenium. Before I can test, say, playing a game, I need two players.
So both must register and login, create and navigate to the relevant
game. This is quite time consuming and too complex to be handled with
the TestNG dependencies.

I distinguish between *incremental* and *complete* tests (switching
with a TestNG parameter).

For complete tests, TestNG and Selenium do just the above. Registration
and login are considered as setup methods here (not as tests). At the
end of this setup I store the complete system state into a file
(exporting the DB tables and cookies). Then I run the game tests.
For incremental tests, the setup method does *not* walk through the
registration and login process, but simply loads the relevant DB tables
and cookies. This is *much* faster than the walk-through.

Every test class has its own BeforeClass method that uses the above
mechanism (which is made available by a common base class). This way,
every test class starts with a *clean* state, all my test classes are
independent from another.

Applied to your problem, once create has been successful, the
corresponding state is stored (into files). A repeated, incremental
test would just restore that state and then test modify and/or delete.
As soon as you change your create process, you run a complete test,
thus creating an updated setup state for future incremental tests.

My experience is that the TestNG dependencies are nice, but can easily
lead to very convoluted test systems. I try to use even my functional
tests with as much *unit* test philosophy as possible (keeping them
independent and reproducable). I use dependsOnMethod only sparingly.
Very often I resolve these dependencies with independent test classes,
as soon as I need dependsOn too often (say 3-5 in a test class). My
advice is: do not use TestNG to handle dependencies, but try to avoid
them alltogether - even if this means increased effort at first. In the
long run, you will save a lot of time.

Thomas - www.playchess.de







--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Hani Suleiman
In reply to this post by Thomas.M.Stahl@googlemail.com

On Dec 16, 2006, at 1:30 PM, [hidden email] wrote:

>
> My experience is that the TestNG dependencies are nice, but can easily
> lead to very convoluted test systems. I try to use even my functional
> tests with as much *unit* test philosophy as possible (keeping them
> independent and reproducable). I use dependsOnMethod only sparingly.
> Very often I resolve these dependencies with independent test classes,
> as soon as I need dependsOn too often (say 3-5 in a test class). My
> advice is: do not use TestNG to handle dependencies, but try to avoid
> them alltogether - even if this means increased effort at first. In  
> the
> long run, you will save a lot of time.

Interesting, I actually likewise have another approach to dealing  
with too many explicit dependencies (I have an aversion to too many  
dependsOnGroups), but my approach is instead to use implicit  
dependencies, and ensure my tests are all grouped correctly.

An example might make this clearer. Let's say I have 3 layers of  
tests, with each layer depending on one below it. So instead of  
declaring the dependencies in every test, I just put each test in the  
right group: layer1, layer2, or layer3. Each layer has its setup  
method, and it's in there that I specify the dependency.

Basically I tend to avoid putting dependencies in test methods,  
preferring instead to put them in setup methods.

I do agree though with the saved state approach, I often will use  
dumps from a 'real' run and check in a bunch of them as part of the  
test suite. It's actually very useful for regression testing, not  
just unit tests.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Thomas.M.Stahl@googlemail.com
In reply to this post by Cédric Beust ♔

Cédric Beust ♔  wrote:
> Like you say, dependencies are useful for a very specific niche of problems,
> but by and large, the majority of your test code base should typically not
> be using them.

This is a surpising answer from *you*. I had expected that you would
argue *for* these dependsOn methods - after all you "invented" them
(didn't you?).

BTW, even if you think that"unit test philosophy" such a good idea,
what do you consider to be the most significant advantages of TestNG
compared to JUnit - for functional (regression) testing? What arguments
should I use to convince my collegues? ;-)

Thomas

PS: don't misunderstand me - TestNG is a great system!


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔


On 12/16/06, [hidden email] <[hidden email]> wrote:

Cédric Beust ♔  wrote:
> Like you say, dependencies are useful for a very specific niche of problems,
> but by and large, the majority of your test code base should typically not
> be using them.

This is a surpising answer from *you*. I had expected that you would
argue *for* these dependsOn methods - after all you "invented" them
(didn't you?).

I'm just recommending to use this feature with moderation and common sense.  I think dependencies are the kind of feature that's not commonly needed, but when you do, you really need it to be supported natively by your testing framework.

BTW, even if you think that"unit test philosophy" such a good idea,
what do you consider to be the most significant advantages of TestNG
compared to JUnit - for functional (regression) testing? What arguments
should I use to convince my collegues? ;-)

Groups, data providers, the fine granularity for control (before suite/test/class/method), test time outs, concurrency...



PS: don't misunderstand me - TestNG is a great system!


No need to justify yourself, criticism is good!

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Thomas.M.Stahl@googlemail.com
In reply to this post by Hani Suleiman

Thanks for this hint. Sounds reasonable. I will reconsider my strategy.
Maybe, a combination of your "setup dependencies approach" with my
"incremental save state approach" is even better. In fact, I had
already experimented with such a layered test strategy - after all,
every Star Trek fan knows what a "level 2 diagnosis" means... ;-)

Thomas - www.playchess.de


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

tarun3kumar
In reply to this post by Cédric Beust ♔

Cedric Beust wrote:

> > I can do that while I'm in a single test run, yes; but the class [i]is[/i]
> > reinitialized between test runs, of course, so the fields from the initial
> > run wouldn't be available any more in the rerun for the failed test. I guess
> > I could write the data to a file or something, but that doesn't seem a very
> > elegant solution either.
> True, but you need to realize that you are asking TestNG to keep track of a
> state that even yourself find hard to maintain in your own code :-)
>
> Is it too expensive to do a quick select on the database to verify if the
> object has already been created?

My test methods don't really know anything about the database. I'm dealing with a GUI application, and from within my test I can only check for the object's existence by performing a search in the GUI. Possible, yes, but not very fast and not all that simple either.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=109193#109193


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

tarun3kumar
In reply to this post by Alexandru Popescu ☀

One more thing that didn't cross my mind earlier:

Alexandru Popescu wrote:
> 1/ trying to clean up after the test (whatever the test status is).
> Adding a@BeforeClass or @AfterClass method that cleans the test data
> will assure that each time you run the test it will start all over.

As I said, I could use my destroy() method for that. But that's a test itself and can fail, so even that approach doesn't necessarily prevent the system from being left in some unwanted state, where the object still exists after the test and causes problems for later runs.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=109195#109195


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Alexandru Popescu ☀

On 12/20/06, Holger Wahlen <[hidden email]> wrote:
>
> One more thing that didn't cross my mind earlier:
>
> Alexandru Popescu wrote:
> > 1/ trying to clean up after the test (whatever the test status is).
> > Adding a@BeforeClass or @AfterClass method that cleans the test data
> > will assure that each time you run the test it will start all over.
>
> As I said, I could use my destroy() method for that. But that's a test itself and can fail, so even that approach doesn't necessarily prevent the system from being left in some unwanted state, where the object still exists after the test and causes problems for later runs.

This is a scenario that may happen, and there is not much you can do
about it in some corner cases. TestNG will make sure that if a
configuration method is failing the test methods will be reported as
skipped and not as failures, so that you will know that something went
wrong.

When this type of cases may occur with a high probability I tend to
include a @Before that is checking the status before running any test,
so that I know that my tests are run against the environment I am
targeting and not against an env in an unknown status.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> ---------------------------------------------------------------------
> Posted via Jive Forums
> http://forums.opensymphony.com/thread.jspa?threadID=53691&messageID=109195#109195
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔
In reply to this post by tarun3kumar


On 12/20/06, Holger Wahlen <[hidden email]> wrote:

One more thing that didn't cross my mind earlier:

Alexandru Popescu wrote:
> 1/ trying to clean up after the test (whatever the test status is).
> Adding a@BeforeClass or @AfterClass method that cleans the test data
> will assure that each time you run the test it will start all over.

As I said, I could use my destroy() method for that. But that's a test itself and can fail, so even that approach doesn't necessarily prevent the system from being left in some unwanted state, where the object still exists after the test and causes problems for later runs.

What do you mean by "it's a test and it can fail"?

Surely, you can make sure that destroy() will always be called, regardless of what happens, right?  (just put it as the very first statement of the @AfterClass method).

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Alexandru Popescu ☀
On 12/20/06, C�dric Beust   <[hidden email]> wrote:

>
>
> On 12/20/06, Holger Wahlen <[hidden email]> wrote:
> >
> > One more thing that didn't cross my mind earlier:
> >
> > Alexandru Popescu wrote:
> > > 1/ trying to clean up after the test (whatever the test status is).
> > > Adding a@BeforeClass or @AfterClass method that cleans the test data
> > > will assure that each time you run the test it will start all over.
> >
> > As I said, I could use my destroy() method for that. But that's a test
> itself and can fail, so even that approach doesn't necessarily prevent the
> system from being left in some unwanted state, where the object still exists
> after the test and causes problems for later runs.
>
> What do you mean by "it's a test and it can fail"?
>
> Surely, you can make sure that destroy() will always be called, regardless
> of what happens, right?  (just put it as the very first statement of the
> @AfterClass method).
>

My understanding of OP's point is that whatever you call the destroy
method it still can failed, and the rest of the tests will run on a
bad environment. That's the reason I have suggested the usage of a
@Before environment check.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> --
> C�dric
>
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Dependency organization when dealing with persistent external objects

Cédric Beust ♔


On 12/20/06, Alexandru Popescu <[hidden email]> wrote:

>
> What do you mean by "it's a test and it can fail"?
>
> Surely, you can make sure that destroy() will always be called, regardless
> of what happens, right?  (just put it as the very first statement of the
> @AfterClass method).
>

My understanding of OP's point is that whatever you call the destroy
method it still can failed, and the rest of the tests will run on a
bad environment. That's the reason I have suggested the usage of a
@Before environment check.

In this case, we're back to what we were saying earlier:  this is really a problem of the system being apparently very hard to put into a consistent state. 

This is a serious problem that should be solved first and foremost, because I don't believe it will be possible to set up any reasonable testing infrastructure until this is addressed.

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

12