[testng-dev] Implementing multiple listener interfaces

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[testng-dev] Implementing multiple listener interfaces

Dimitar Dimitrov
I have a class that implements both IInvokedMethodListener and IReporter - the use case is that I record some information while the test runs and I output it at the end.

What I found is that my listener gets called multiple times per test method invocation. The reason I have found so far is that the TestRunner#initListeners method calls SuiteRunner#addListener multiple times if it happens to implement two or more of ISuiteListener, IInvokedMethodListener, IReporter. This in turn causes the listener to be added as many times to each collection. 

A fix I would suggest is to either change SuiteRunner#addListener to check that the listener has not been already added to the collection.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Cédric Beust ♔-2
But that would break the behavior. These interfaces implement different methods, if you're adding a listener implementing two interfaces, surely you expect all the methods that you implement to be called, right?

For example, with your logic, you would add a reporter listener but its methods would never be invoked...

-- 
Cédric


-- 
Cédric



On Tue, May 7, 2013 at 12:40 AM, ddimitrov <[hidden email]> wrote:
I have a class that implements both IInvokedMethodListener and IReporter - the use case is that I record some information while the test runs and I output it at the end.

What I found is that my listener gets called multiple times per test method invocation. The reason I have found so far is that the TestRunner#initListeners method calls SuiteRunner#addListener multiple times if it happens to implement two or more of ISuiteListener, IInvokedMethodListener, IReporter. This in turn causes the listener to be added as many times to each collection. 

A fix I would suggest is to either change SuiteRunner#addListener to check that the listener has not been already added to the collection.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Dimitar Dimitrov
Cedric, I think I am not conveying what I mean - I guess it would be better explained by example:

public class TestCounter implements IReporter, IInvokedMethodListener  {
    int count;
    public void beforeInvocation(IInvokedMethod m, ITestResult r) { 
       count++; 
    }
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) { }
    public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) { 
        System.out.println(count);
    }
}

@Listeners(TestCounter)
public class TestWithOneMethod {
    @Test public void test() { assert System.nanoTime() > 0; }
}

As there is only one test method, I would expect that the counter would print "1" - with the current implementation it would print "2". If that is on purpose, it is worth mentioning it in the docs, as I and my colleagues find it counterintuitive. If not, I would be glad to work with you on fixing it.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Cédric Beust ♔-2
Hi Dimitar,

This behavior is necessary and intentional. You are suggesting that if I'm trying to add a listener, any listener, I should double check if that instance hasn't been added already, and if it has, not add it.

Such implementation would make it impossible to define a listener that implements two interfaces since only one of the listeners would be added (either IReporter or IInvokedMethodListener).

-- 
Cédric


-- 
Cédric



On Sun, May 19, 2013 at 6:08 PM, ddimitrov <[hidden email]> wrote:
Cedric, I think I am not conveying what I mean - I guess it would be better explained by example:

public class TestCounter implements IReporter, IInvokedMethodListener  {
    int count;
    public void beforeInvocation(IInvokedMethod m, ITestResult r) { 
       count++; 
    }
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) { }
    public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) { 
        System.out.println(count);
    }
}

@Listeners(TestCounter)
public class TestWithOneMethod {
    @Test public void test() { assert System.nanoTime() > 0; }
}

As there is only one test method, I would expect that the counter would print "1" - with the current implementation it would print "2". If that is on purpose, it is worth mentioning it in the docs, as I and my colleagues find it counterintuitive. If not, I would be glad to work with you on fixing it.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Dimitar Dimitrov
Hi Cedric, I can see why the current implementation behaves the way it does, but that doesn't help me rationalize why does TestNG call the 'beforeInvoke' different number of times depending on whether we implement the counting ant the printing part in the same class or in different classes.

I am suggesting that the behavior should be changed, so that the "beforeInvoke" method of each listener would be called once and only once before a test method is invoked, and the "generateReport" method would be invoked once and only once when all tests have run and all other listeners invoked. 

This behavior should be consistent, regardless of which parts of the test lifecycle a listener listens to (i.e. a listener class should be able to implement all interfaces and not receive duplicate notifications).


--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Cédric Beust ♔-2
Ok I'll take a closer look.


-- 
Cédric



On Mon, May 20, 2013 at 12:26 AM, ddimitrov <[hidden email]> wrote:
Hi Cedric, I can see why the current implementation behaves the way it does, but that doesn't help me rationalize why does TestNG call the 'beforeInvoke' different number of times depending on whether we implement the counting ant the printing part in the same class or in different classes.

I am suggesting that the behavior should be changed, so that the "beforeInvoke" method of each listener would be called once and only once before a test method is invoked, and the "generateReport" method would be invoked once and only once when all tests have run and all other listeners invoked. 

This behavior should be consistent, regardless of which parts of the test lifecycle a listener listens to (i.e. a listener class should be able to implement all interfaces and not receive duplicate notifications).


--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Cédric Beust ♔-2
In reply to this post by Dimitar Dimitrov
Hi Dimitar,

You are right, there was a bug in how I handle these listeners. I just uploaded a fix, can you try it? It's at http://testng.org/beta

Thanks for your patience!

-- 
Cédric


-- 
Cédric



On Mon, May 20, 2013 at 12:26 AM, ddimitrov <[hidden email]> wrote:
Hi Cedric, I can see why the current implementation behaves the way it does, but that doesn't help me rationalize why does TestNG call the 'beforeInvoke' different number of times depending on whether we implement the counting ant the printing part in the same class or in different classes.

I am suggesting that the behavior should be changed, so that the "beforeInvoke" method of each listener would be called once and only once before a test method is invoked, and the "generateReport" method would be invoked once and only once when all tests have run and all other listeners invoked. 

This behavior should be consistent, regardless of which parts of the test lifecycle a listener listens to (i.e. a listener class should be able to implement all interfaces and not receive duplicate notifications).


--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Dimitar Dimitrov
Hi Cedric, I can confirm that the Beta build foxes my issue.

While testing, I found another peculiarity (I am mentioning it just in case it matters) - the double invocation was happening when multi-interface listener was registed using @Listeners annotation, but not when I specified it in the "Listeners" tab of the IntelliJ test runner...

Thank you!
Dimitar


On Sunday, May 26, 2013 2:18:02 AM UTC+9, Cédric Beust ♔ wrote:
Hi Dimitar,

You are right, there was a bug in how I handle these listeners. I just uploaded a fix, can you try it? It's at http://testng.org/beta

Thanks for your patience!

-- 
Cédric


-- 
Cédric



On Mon, May 20, 2013 at 12:26 AM, ddimitrov <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="F76eW-zLg4sJ">dimitar....@...> wrote:
Hi Cedric, I can see why the current implementation behaves the way it does, but that doesn't help me rationalize why does TestNG call the 'beforeInvoke' different number of times depending on whether we implement the counting ant the printing part in the same class or in different classes.

I am suggesting that the behavior should be changed, so that the "beforeInvoke" method of each listener would be called once and only once before a test method is invoked, and the "generateReport" method would be invoked once and only once when all tests have run and all other listeners invoked. 

This behavior should be consistent, regardless of which parts of the test lifecycle a listener listens to (i.e. a listener class should be able to implement all interfaces and not receive duplicate notifications).


--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="F76eW-zLg4sJ">testng-dev+...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="F76eW-zLg4sJ">testn...@....
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Dimitar Dimitrov
Is there any plan to release the patch I tested? 
I don't see it mentioned in the Github release notes for the next release.

Dimitar

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

Cédric Beust ♔-2
Since it was in the beta build I sent you, the patch is in, even if I forgot to add it to the release notes :)


-- 
Cédric



On Wed, Aug 14, 2013 at 3:40 PM, ddimitrov <[hidden email]> wrote:
Is there any plan to release the patch I tested? 
I don't see it mentioned in the Github release notes for the next release.

Dimitar

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: [testng-dev] Implementing multiple listener interfaces

jitendra kandula
Hi Cedric,

Is this part of latest release (6.8.5)? Would it solve the below problem?

I am currently using 6.8.5 and I have registered an ISuiteListener using annotation. In suite.xml have two set of test groups. In this scenario Suite listener is getting registered with two different instances per TestRunner and finally getting triggered twice. 

Thanks for you time.
Jitendra

On Wednesday, August 14, 2013 9:11:07 PM UTC-7, Cédric Beust ♔ wrote:
Since it was in the beta build I sent you, the patch is in, even if I forgot to add it to the release notes :)


-- 
Cédric



On Wed, Aug 14, 2013 at 3:40 PM, ddimitrov <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="uWncPda4g9wJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">dimitar....@...> wrote:
Is there any plan to release the patch I tested? 
I don't see it mentioned in the Github release notes for the next release.

Dimitar

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="uWncPda4g9wJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">testng-dev+...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="uWncPda4g9wJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">testn...@....
Visit this group at <a href="http://groups.google.com/group/testng-dev" target="_blank" onmousedown="this.href='http://groups.google.com/group/testng-dev';return true;" onclick="this.href='http://groups.google.com/group/testng-dev';return true;">http://groups.google.com/group/testng-dev.

For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/groups/opt_out.