Remove retried test methods from report

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Remove retried test methods from report

stefanschuri
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/QO_cSVjR4x8J.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Giannis Papadakis
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

stefanschuri
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Shu
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Shu
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Shu
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Shu
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

kamalakar vadla
Hi

Could please let me how to override the default test report??

Regards
KV



On Wednesday, 13 March 2013 00:21:52 UTC+5:30, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Indrani
Can you let us know how to override the test method to remove duplicate test after retry

On Friday, April 12, 2013 5:38:15 AM UTC-7, kamalakar chary wrote:
Hi

Could please let me how to override the default test report??

Regards
KV



On Wednesday, 13 March 2013 00:21:52 UTC+5:30, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Govindaram PS
I am able successful remove all the failed results from TestNG which are passed in second attempt.
I am running the test through maven. The problem here is maven fail safe plugin is making the build fail.
Maven fail safe plugin holds its own test results. 

How to update maven fail safe's results programatically?




On Saturday, October 19, 2013 3:07:25 AM UTC+5:30, Indrani biswas wrote:
Can you let us know how to override the test method to remove duplicate test after retry

On Friday, April 12, 2013 5:38:15 AM UTC-7, kamalakar chary wrote:
Hi

Could please let me how to override the default test report??

Regards
KV



On Wednesday, 13 March 2013 00:21:52 UTC+5:30, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Owen Fletcher
In reply to this post by stefanschuri
"The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?
"  I would also be interested in a resolution to this problem.

On Thursday, 31 January 2013 10:07:00 UTC, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Indrani
In reply to this post by Govindaram PS
Hi Govindaram,

Could you please share the code on how you have achieved to remove duplicate test results from reports. Will be really helpful.

Thanks,
--Indrani

On Friday, December 6, 2013 12:10:11 AM UTC-8, Govindaram PS wrote:
I am able successful remove all the failed results from TestNG which are passed in second attempt.
I am running the test through maven. The problem here is maven fail safe plugin is making the build fail.
Maven fail safe plugin holds its own test results. 

How to update maven fail safe's results programatically?




On Saturday, October 19, 2013 3:07:25 AM UTC+5:30, Indrani biswas wrote:
Can you let us know how to override the test method to remove duplicate test after retry

On Friday, April 12, 2013 5:38:15 AM UTC-7, kamalakar chary wrote:
Hi

Could please let me how to override the default test report??

Regards
KV



On Wednesday, 13 March 2013 00:21:52 UTC+5:30, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Indrani
In reply to this post by Owen Fletcher
Hi fletch,

Could you please share the code on how you have achieved to remove duplicate test results from reports. Will be really helpful.

Thanks,
--Indrani

On Thursday, February 20, 2014 7:22:58 AM UTC-8, fletch wrote:
"The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?
"  I would also be interested in a resolution to this problem.

On Thursday, 31 January 2013 10:07:00 UTC, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Martino Turturiello-2
Hi all,

can someone share its working solution?



On Monday, March 24, 2014 at 5:29:36 PM UTC-7, Indrani biswas wrote:
Hi fletch,

Could you please share the code on how you have achieved to remove duplicate test results from reports. Will be really helpful.

Thanks,
--Indrani

On Thursday, February 20, 2014 7:22:58 AM UTC-8, fletch wrote:
"The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?
"  I would also be interested in a resolution to this problem.

On Thursday, 31 January 2013 10:07:00 UTC, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ';return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

Automation Enthusiast
In reply to this post by Shu
Shu,

I know this is now a #+ years old thread. Can you post your solution here since looks like a bunch of us would like to do something similar but unable to find the exact solution.

My TestNG results are showing accurately and are excluding any retried tests that PASS, but the build is still being marked unsuccessful and in Jenkins, the default test results are also being displayed with all the retries and failures being shown. Is there a way to turn off the default test results if one is using the TestNG results plugin ?

Thanks!

On Tuesday, March 12, 2013 at 11:51:52 AM UTC-7, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\46sa\75D\46sntz\0751\46usg\75AFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ&#39;;return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Remove retried test methods from report

aswin hari rao
Hi,

Any solution to this, I am able to remove duplicate failed test cases from the report,testng report looks fine after the retries, but the maven build is always marked failed even if retry is successful.

Can any one help with solution

On Friday, April 1, 2016 at 8:34:00 AM UTC-5, Automation Enthusiast wrote:
Shu,

I know this is now a #+ years old thread. Can you post your solution here since looks like a bunch of us would like to do something similar but unable to find the exact solution.

My TestNG results are showing accurately and are excluding any retried tests that PASS, but the build is still being marked unsuccessful and in Jenkins, the default test results are also being displayed with all the retries and failures being shown. Is there a way to turn off the default test results if one is using the TestNG results plugin ?

Thanks!

On Tuesday, March 12, 2013 at 11:51:52 AM UTC-7, Shu wrote:
Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:
Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

On Thursday, January 31, 2013 2:07:00 AM UTC-8, [hidden email] wrote:
Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan




On Friday, January 25, 2013 9:34:55 PM UTC+1, John Papadakis wrote:
Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {
for(int i=0;i<context.getAllTestMethods().length;i++){
if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){
if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){
context.getFailedTests().removeResult(context.getAllTestMethods()[i]);
}
}
}
}

On Thursday, January 17, 2013 12:25:41 PM UTC+2, [hidden email] wrote:
Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
<a href="http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F7803691%2Fhow-to-optimize-testng-and-seleniums-tests\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG2fnmP_bAAT_z00ml-wnDFW-jyyQ&#39;;return true;">http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Loading...