Removing failed tests from within an IRetryAnalyzer

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Removing failed tests from within an IRetryAnalyzer

jpicton
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing failed tests from within an IRetryAnalyzer

Sabaat Ahmad
I am using TestNG 6.8.7 which has a method
result.getTestContext().getFailedTests().removeResult( result );

This successfully remove the previous result if I invoke the retry() method once. But if the test fails a second time, the result.getTestContext().getFailedTests().size() is somehow 2 (not 1). In addition to this I always get a failed + pass result in the report. If I retry only once, I get only a pass result.

This seems odd. Has anybody else faced this issue?

On Thursday, October 17, 2013 5:46:58 AM UTC+5:30, [hidden email] wrote:
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing failed tests from within an IRetryAnalyzer

Sabaat Ahmad
the reason I don't want to remove duplicates in onFinish() method of the listener is that if the test have dependencies, then the dependent methods always get skipped as for them the test had failed. Has someone found a perfect solution to removing duplicates?

On Sunday, January 4, 2015 5:18:03 AM UTC+5:30, Sabaat Ahmad wrote:
I am using TestNG 6.8.7 which has a method
result.getTestContext().getFailedTests().removeResult( result );

This successfully remove the previous result if I invoke the retry() method once. But if the test fails a second time, the result.getTestContext().getFailedTests().size() is somehow 2 (not 1). In addition to this I always get a failed + pass result in the report. If I retry only once, I get only a pass result.

This seems odd. Has anybody else faced this issue?

On Thursday, October 17, 2013 5:46:58 AM UTC+5:30, [hidden email] wrote:
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing failed tests from within an IRetryAnalyzer

Devendra Raju
In reply to this post by jpicton

Hi Ahmad,

Please try the below code:

public void onFinish(ITestContext context) {
    Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
    for (ITestResult temp : failedTests) {
        ITestNGMethod method = temp.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            failedTests.remove(temp);
        } else {
            if (context.getPassedTests().getResults(method).size() > 0) {
                failedTests.remove(temp);
            }
        }
    }

}

For more details try to check in TestNG Selenium Tutorials


On Thursday, October 17, 2013 5:46:58 AM UTC+5:30, [hidden email] wrote:
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing failed tests from within an IRetryAnalyzer

Devendra Raju
In reply to this post by jpicton
Hi Ahmad,

Please try the below code:

public void onFinish(ITestContext context) {
    Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
    for (ITestResult temp : failedTests) {
        ITestNGMethod method = temp.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            failedTests.remove(temp);
        } else {
            if (context.getPassedTests().getResults(method).size() > 0) {
                failedTests.remove(temp);
            }
        }
    }

}

For more details try to check in TestNG Selenium Tutorials



On Thursday, October 17, 2013 5:46:58 AM UTC+5:30, [hidden email] wrote:
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing failed tests from within an IRetryAnalyzer

Manas Gupta
In reply to this post by Sabaat Ahmad
Hi Sabaat,

I'm facing the same problem but not able to resolve it yet even after trying that solution to remove failedTest in onFinishMethod(). Do you have any solution for this. Thanks in advance.

On Sunday, January 4, 2015 at 5:18:03 AM UTC+5:30, Sabaat Ahmad wrote:
I am using TestNG 6.8.7 which has a method
result.getTestContext().getFailedTests().removeResult( result );

This successfully remove the previous result if I invoke the retry() method once. But if the test fails a second time, the result.getTestContext().getFailedTests().size() is somehow 2 (not 1). In addition to this I always get a failed + pass result in the report. If I retry only once, I get only a pass result.

This seems odd. Has anybody else faced this issue?

On Thursday, October 17, 2013 5:46:58 AM UTC+5:30, [hidden email] wrote:
Hi. What I am trying to do is to create a RetryAnalyser class that will retry failed test methods two more times. However, if one of the attempts is successful, I want the test suite to pass. The only way I can think of to achieve this is to remove the failed test attempts from the TestContext.

My RetryAnalyser is below:

import org.testng.IRetryAnalyzer;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;

public class RetryAnalyser
implements IRetryAnalyzer
{
    private static final int MAX_COUNT = 2;
    private int myCount = 0;

    @Override
    public boolean retry( final ITestResult result )
    {
        if ( !result.isSuccess() )
        {
            if ( myCount < MAX_COUNT )
            {
                ++myCount;

                result.getTestContext().getFailedTests().removeResult( result.getMethod() );

                final String message =
                    Thread.currentThread().getName() + ": Error in " +
                        result.getName() + " with status " + result.getStatus() +
                        " Retrying " + myCount + " times";

                System.out.println( message );
                Reporter.log( message );

                return true;
            }
        }

        return false;
    }
}


The problem I seem to be having is that after I remove the method from the failed tests on the last attempt it is being reinserted. i.e. If the test method fails 3 times, I expect the suite to only report one failure (the last attempt) but instead it is reporting 2 failures? Can somebody please explain this, or show me a better way to achieve what I'm trying to do?

Cheers,
Jake.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Loading...