Soft asserts

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Soft asserts

Yury Kryazhev
Hi,

I'm using soft asserts in my tests as described in the article
http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/

And since testng 5.14 me and some other guys are facing with
duplicating problem:
PK
>If my test method A failed with soft assertion, the report shows two instances of method A – one >failed and one succeeded. Both having the same exception list.
>However, if method A succeeded, there is just one instance of A in the report – the succeeded >one.
>Also, I am sure the test method is getting executed only once – it is only the report that has the >duplicate exception list.

Any ideas how to resolve this issue?

Thx in advanced,
Yury

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Cédric Beust ♔-2
Hi Yury,

Can you show a small code example that produces this problem so I can try it?

Thanks.

-- 
Cédric




On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]> wrote:
Hi,

I'm using soft asserts in my tests as described in the article
http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/

And since testng 5.14 me and some other guys are facing with
duplicating problem:
PK
>If my test method A failed with soft assertion, the report shows two instances of method A – one >failed and one succeeded. Both having the same exception list.
>However, if method A succeeded, there is just one instance of A in the report – the succeeded >one.
>Also, I am sure the test method is getting executed only once – it is only the report that has the >duplicate exception list.

Any ideas how to resolve this issue?

Thx in advanced,
Yury

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Yury Kryazhev
Here you are )

public class InvokedMethodListener implements IInvokedMethodListener {

    @Override
    public void beforeInvocation(IInvokedMethod iInvokedMethod,
ITestResult iTestResult) {
    }ndedSeleneseTestBaseCommon.setPassed();
    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult
result) {
        Reporter.setCurrentTestResult(result);
        if (method.isTestMethod()) {
            if (!ExtendedSeleneseTestBaseCommon.getPassed()) {
                result.setStatus(ITestResult.FAILURE);
            }
        }
    }
}


public class ExtendedSeleneseTestBaseCommon extends SeleneseTestBase {

    private static boolean passed = true;

    public static void setFailed() {
        passed = false;
    }

    public static boolean getPassed() {
        return passed;
    }
}


public class UIA26_Vanity_Number extends ExtendedSeleneseTestBase {

    @Test
    public void test_1() throws Exception {
                ExtendedSeleneseTestBaseCommon.setFailed();
    }
}


On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:

> Hi Yury,
>
> Can you show a small code example that produces this problem so I can try
> it?
>
> Thanks.
>
> --
> Cédric
>
> On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]>wrote:
>
>
>
>
>
>
>
> > Hi,
>
> > I'm using soft asserts in my tests as described in the article
> >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > And since testng 5.14 me and some other guys are facing with
> > duplicating problem:
> > PK
> > >If my test method A failed with soft assertion, the report shows two
> > instances of method A – one >failed and one succeeded. Both having the same
> > exception list.
> > >However, if method A succeeded, there is just one instance of A in the
> > report – the succeeded >one.
> > >Also, I am sure the test method is getting executed only once – it is only
> > the report that has the >duplicate exception list.
>
> > Any ideas how to resolve this issue?
>
> > Thx in advanced,
> > Yury
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "testng-users" group.
> > To post to this group, send email to [hidden email].
> > To unsubscribe from this group, send email to
> > [hidden email].
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Yury Kryazhev
In reply to this post by Cédric Beust ♔-2
Missprinted in one method:
    @Override
    public void beforeInvocation(IInvokedMethod iInvokedMethod,
ITestResult iTestResult) {
        ExtendedSeleneseTestBaseCommon.setPassed();
    }

On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:

> Hi Yury,
>
> Can you show a small code example that produces this problem so I can try
> it?
>
> Thanks.
>
> --
> Cédric
>
> On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]>wrote:
>
>
>
>
>
>
>
> > Hi,
>
> > I'm using soft asserts in my tests as described in the article
> >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > And since testng 5.14 me and some other guys are facing with
> > duplicating problem:
> > PK
> > >If my test method A failed with soft assertion, the report shows two
> > instances of method A – one >failed and one succeeded. Both having the same
> > exception list.
> > >However, if method A succeeded, there is just one instance of A in the
> > report – the succeeded >one.
> > >Also, I am sure the test method is getting executed only once – it is only
> > the report that has the >duplicate exception list.
>
> > Any ideas how to resolve this issue?
>
> > Thx in advanced,
> > Yury
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "testng-users" group.
> > To post to this group, send email to [hidden email].
> > To unsubscribe from this group, send email to
> > [hidden email].
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Re: Soft asserts

Cédric Beust ♔-2
I don't know what ExtendedSeleneseTestBaseCommon is, can you post a self-contained example that doesn't have any external dependencies?

Thanks.

-- 
Cédric




On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]> wrote:
Missprinted in one method:
   @Override
   public void beforeInvocation(IInvokedMethod iInvokedMethod,
ITestResult iTestResult) {
       ExtendedSeleneseTestBaseCommon.setPassed();
   }

On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> Hi Yury,
>
> Can you show a small code example that produces this problem so I can try
> it?
>
> Thanks.
>
> --
> Cédric
>
> On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]>wrote:
>
>
>
>
>
>
>
> > Hi,
>
> > I'm using soft asserts in my tests as described in the article
> >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > And since testng 5.14 me and some other guys are facing with
> > duplicating problem:
> > PK
> > >If my test method A failed with soft assertion, the report shows two
> > instances of method A – one >failed and one succeeded. Both having the same
> > exception list.
> > >However, if method A succeeded, there is just one instance of A in the
> > report – the succeeded >one.
> > >Also, I am sure the test method is getting executed only once – it is only
> > the report that has the >duplicate exception list.
>
> > Any ideas how to resolve this issue?
>
> > Thx in advanced,
> > Yury
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "testng-users" group.
> > To post to this group, send email to [hidden email].
> > To unsubscribe from this group, send email to
> > [hidden email].
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Yury Kryazhev
Sorry for bad example, here is a better one

java:
public class InvokedMethodListener implements IInvokedMethodListener {

    public void beforeInvocation(IInvokedMethod iInvokedMethod,
ITestResult iTestResult) {
        TestBase.setPassed();
    }

    public void afterInvocation(IInvokedMethod method, ITestResult
result) {
        Reporter.setCurrentTestResult(result);
        if (method.isTestMethod()) {
            if (!TestBase.getPassed()) {
                result.setStatus(ITestResult.FAILURE);
            }
        }
    }
}

public class TestBase {
    private static boolean passed = true;
    public static void setFailed() {
        passed = false;
    }
    public static void setPassed() {
        passed = true;
    }
    public static boolean getPassed() {
        return passed;
    }
}

test:
public class TestImpl extends TestBase{
    @Test
    public void test_1() throws Exception {
        TestBase.setFailed();
    }
}

You can run it as testng configuration using testng.xml:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite verbose="0" name="Bad" parallel="false">
    <test verbose="2" name="TestImpl" preserve-order="true">
        <classes>
            <class name="TestImpl"/>
        </classes>
    </test>
    <listeners>
      <listener class-name="InvokedMethodListener" />
    </listeners>
</suite>


On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:

> I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> self-contained example that doesn't have any external dependencies?
>
> Thanks.
>
> --
> Cédric
>
> On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]>wrote:
>
>
>
>
>
>
>
> > Missprinted in one method:
> >     @Override
> >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > ITestResult iTestResult) {
> >         ExtendedSeleneseTestBaseCommon.setPassed();
> >     }
>
> > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > Hi Yury,
>
> > > Can you show a small code example that produces this problem so I can try
> > > it?
>
> > > Thanks.
>
> > > --
> > > Cédric
>
> > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]
> > >wrote:
>
> > > > Hi,
>
> > > > I'm using soft asserts in my tests as described in the article
> > > >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > And since testng 5.14 me and some other guys are facing with
> > > > duplicating problem:
> > > > PK
> > > > >If my test method A failed with soft assertion, the report shows two
> > > > instances of method A – one >failed and one succeeded. Both having the
> > same
> > > > exception list.
> > > > >However, if method A succeeded, there is just one instance of A in the
> > > > report – the succeeded >one.
> > > > >Also, I am sure the test method is getting executed only once – it is
> > only
> > > > the report that has the >duplicate exception list.
>
> > > > Any ideas how to resolve this issue?
>
> > > > Thx in advanced,
> > > > Yury
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "testng-users" group.
> > > > To post to this group, send email to [hidden email].
> > > > To unsubscribe from this group, send email to
> > > > [hidden email].
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/testng-users?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "testng-users" group.
> > To post to this group, send email to [hidden email].
> > To unsubscribe from this group, send email to
> > [hidden email].
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Yury Kryazhev
Cédric, any news?
We've got more than 1k test in our production that are behaving this
way, it's really important for us.

Thx in advanced

On 5 авг, 12:28, Yury Kryazhev <[hidden email]> wrote:

> Sorry for bad example, here is a better one
>
> java:
> public class InvokedMethodListener implements IInvokedMethodListener {
>
>     public void beforeInvocation(IInvokedMethod iInvokedMethod,
> ITestResult iTestResult) {
>         TestBase.setPassed();
>     }
>
>     public void afterInvocation(IInvokedMethod method, ITestResult
> result) {
>         Reporter.setCurrentTestResult(result);
>         if (method.isTestMethod()) {
>             if (!TestBase.getPassed()) {
>                 result.setStatus(ITestResult.FAILURE);
>             }
>         }
>     }
>
> }
>
> public class TestBase {
>     private static boolean passed = true;
>     public static void setFailed() {
>         passed = false;
>     }
>     public static void setPassed() {
>         passed = true;
>     }
>     public static boolean getPassed() {
>         return passed;
>     }
>
> }
>
> test:
> public class TestImpl extends TestBase{
>     @Test
>     public void test_1() throws Exception {
>         TestBase.setFailed();
>     }
>
> }
>
> You can run it as testng configuration using testng.xml:
> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
> <suite verbose="0" name="Bad" parallel="false">
>     <test verbose="2" name="TestImpl" preserve-order="true">
>         <classes>
>             <class name="TestImpl"/>
>         </classes>
>     </test>
>     <listeners>
>       <listener class-name="InvokedMethodListener" />
>     </listeners>
> </suite>
>
> On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:
>
>
>
>
>
>
>
> > I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> > self-contained example that doesn't have any external dependencies?
>
> > Thanks.
>
> > --
> > Cédric
>
> > On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]>wrote:
>
> > > Missprinted in one method:
> > >     @Override
> > >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > ITestResult iTestResult) {
> > >         ExtendedSeleneseTestBaseCommon.setPassed();
> > >     }
>
> > > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > > Hi Yury,
>
> > > > Can you show a small code example that produces this problem so I can try
> > > > it?
>
> > > > Thanks.
>
> > > > --
> > > > Cédric
>
> > > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]
> > > >wrote:
>
> > > > > Hi,
>
> > > > > I'm using soft asserts in my tests as described in the article
> > > > >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > > And since testng 5.14 me and some other guys are facing with
> > > > > duplicating problem:
> > > > > PK
> > > > > >If my test method A failed with soft assertion, the report shows two
> > > > > instances of method A – one >failed and one succeeded. Both having the
> > > same
> > > > > exception list.
> > > > > >However, if method A succeeded, there is just one instance of A in the
> > > > > report – the succeeded >one.
> > > > > >Also, I am sure the test method is getting executed only once – it is
> > > only
> > > > > the report that has the >duplicate exception list.
>
> > > > > Any ideas how to resolve this issue?
>
> > > > > Thx in advanced,
> > > > > Yury
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "testng-users" group.
> > > > > To post to this group, send email to [hidden email].
> > > > > To unsubscribe from this group, send email to
> > > > > [hidden email].
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "testng-users" group.
> > > To post to this group, send email to [hidden email].
> > > To unsubscribe from this group, send email to
> > > [hidden email].
> > > For more options, visit this group at
> > >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Re: Soft asserts

François Reynaud
Hi Cedric,

I remember asking the same question some time ago, and I think the problem came from 
 result.setStatus(ITestResult.FAILURE);

setStatus exists but the variable behing it is not supposed to be mutable or something along those line.

Btw, that wasn't blocking as far as I'm concerned as it never hides a failure. It only create a bit of confusion as the total number of tests varies.

thanks
françois

On Tue, Aug 9, 2011 at 9:22 AM, Yury Kryazhev <[hidden email]> wrote:
Cédric, any news?
We've got more than 1k test in our production that are behaving this
way, it's really important for us.

Thx in advanced

On 5 авг, 12:28, Yury Kryazhev <[hidden email]> wrote:
> Sorry for bad example, here is a better one
>
> java:
> public class InvokedMethodListener implements IInvokedMethodListener {
>
>     public void beforeInvocation(IInvokedMethod iInvokedMethod,
> ITestResult iTestResult) {
>         TestBase.setPassed();
>     }
>
>     public void afterInvocation(IInvokedMethod method, ITestResult
> result) {
>         Reporter.setCurrentTestResult(result);
>         if (method.isTestMethod()) {
>             if (!TestBase.getPassed()) {
>                 result.setStatus(ITestResult.FAILURE);
>             }
>         }
>     }
>
> }
>
> public class TestBase {
>     private static boolean passed = true;
>     public static void setFailed() {
>         passed = false;
>     }
>     public static void setPassed() {
>         passed = true;
>     }
>     public static boolean getPassed() {
>         return passed;
>     }
>
> }
>
> test:
> public class TestImpl extends TestBase{
>     @Test
>     public void test_1() throws Exception {
>         TestBase.setFailed();
>     }
>
> }
>
> You can run it as testng configuration using testng.xml:
> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
> <suite verbose="0" name="Bad" parallel="false">
>     <test verbose="2" name="TestImpl" preserve-order="true">
>         <classes>
>             <class name="TestImpl"/>
>         </classes>
>     </test>
>     <listeners>
>       <listener class-name="InvokedMethodListener" />
>     </listeners>
> </suite>
>
> On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:
>
>
>
>
>
>
>
> > I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> > self-contained example that doesn't have any external dependencies?
>
> > Thanks.
>
> > --
> > Cédric
>
> > On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]>wrote:
>
> > > Missprinted in one method:
> > >     @Override
> > >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > ITestResult iTestResult) {
> > >         ExtendedSeleneseTestBaseCommon.setPassed();
> > >     }
>
> > > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > > Hi Yury,
>
> > > > Can you show a small code example that produces this problem so I can try
> > > > it?
>
> > > > Thanks.
>
> > > > --
> > > > Cédric
>
> > > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]
> > > >wrote:
>
> > > > > Hi,
>
> > > > > I'm using soft asserts in my tests as described in the article
> > > > >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > > And since testng 5.14 me and some other guys are facing with
> > > > > duplicating problem:
> > > > > PK
> > > > > >If my test method A failed with soft assertion, the report shows two
> > > > > instances of method A – one >failed and one succeeded. Both having the
> > > same
> > > > > exception list.
> > > > > >However, if method A succeeded, there is just one instance of A in the
> > > > > report – the succeeded >one.
> > > > > >Also, I am sure the test method is getting executed only once – it is
> > > only
> > > > > the report that has the >duplicate exception list.
>
> > > > > Any ideas how to resolve this issue?
>
> > > > > Thx in advanced,
> > > > > Yury
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "testng-users" group.
> > > > > To post to this group, send email to [hidden email].
> > > > > To unsubscribe from this group, send email to
> > > > > [hidden email].
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "testng-users" group.
> > > To post to this group, send email to [hidden email].
> > > To unsubscribe from this group, send email to
> > > [hidden email].
> > > For more options, visit this group at
> > >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Re: Soft asserts

niharika varshney
In reply to this post by Yury Kryazhev
I believe, before AfterInvocation is called and after the testmethod executes, the test result is added to the corresponding count for the report.

The test result is pass before afterInvocation is run, so the count gets added to Pass count and then in aferinvocation you set the testresult to fail which gets added to the failed count when the failures are analysed ... hence you see the 1 for pass and 1 for fail count.

We also ran into the issue as we were catching exceptions in our code and then markin case as passed or failed in the afterInvocation based on those exceptions...
we then started to set the throwable to the last caught failure, to let TestNG know at test method completion, that there was an exception in the testcase, which got the correct count in the report..

Regards,
Niharika


On Tue, Aug 9, 2011 at 1:52 PM, Yury Kryazhev <[hidden email]> wrote:
Cédric, any news?
We've got more than 1k test in our production that are behaving this
way, it's really important for us.

Thx in advanced

On 5 авг, 12:28, Yury Kryazhev <[hidden email]> wrote:
> Sorry for bad example, here is a better one
>
> java:
> public class InvokedMethodListener implements IInvokedMethodListener {
>
>     public void beforeInvocation(IInvokedMethod iInvokedMethod,
> ITestResult iTestResult) {
>         TestBase.setPassed();
>     }
>
>     public void afterInvocation(IInvokedMethod method, ITestResult
> result) {
>         Reporter.setCurrentTestResult(result);
>         if (method.isTestMethod()) {
>             if (!TestBase.getPassed()) {
>                 result.setStatus(ITestResult.FAILURE);
>             }
>         }
>     }
>
> }
>
> public class TestBase {
>     private static boolean passed = true;
>     public static void setFailed() {
>         passed = false;
>     }
>     public static void setPassed() {
>         passed = true;
>     }
>     public static boolean getPassed() {
>         return passed;
>     }
>
> }
>
> test:
> public class TestImpl extends TestBase{
>     @Test
>     public void test_1() throws Exception {
>         TestBase.setFailed();
>     }
>
> }
>
> You can run it as testng configuration using testng.xml:
> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
> <suite verbose="0" name="Bad" parallel="false">
>     <test verbose="2" name="TestImpl" preserve-order="true">
>         <classes>
>             <class name="TestImpl"/>
>         </classes>
>     </test>
>     <listeners>
>       <listener class-name="InvokedMethodListener" />
>     </listeners>
> </suite>
>
> On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:
>
>
>
>
>
>
>
> > I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> > self-contained example that doesn't have any external dependencies?
>
> > Thanks.
>
> > --
> > Cédric
>
> > On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]>wrote:
>
> > > Missprinted in one method:
> > >     @Override
> > >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > ITestResult iTestResult) {
> > >         ExtendedSeleneseTestBaseCommon.setPassed();
> > >     }
>
> > > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > > Hi Yury,
>
> > > > Can you show a small code example that produces this problem so I can try
> > > > it?
>
> > > > Thanks.
>
> > > > --
> > > > Cédric
>
> > > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]
> > > >wrote:
>
> > > > > Hi,
>
> > > > > I'm using soft asserts in my tests as described in the article
> > > > >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > > And since testng 5.14 me and some other guys are facing with
> > > > > duplicating problem:
> > > > > PK
> > > > > >If my test method A failed with soft assertion, the report shows two
> > > > > instances of method A – one >failed and one succeeded. Both having the
> > > same
> > > > > exception list.
> > > > > >However, if method A succeeded, there is just one instance of A in the
> > > > > report – the succeeded >one.
> > > > > >Also, I am sure the test method is getting executed only once – it is
> > > only
> > > > > the report that has the >duplicate exception list.
>
> > > > > Any ideas how to resolve this issue?
>
> > > > > Thx in advanced,
> > > > > Yury
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "testng-users" group.
> > > > > To post to this group, send email to [hidden email].
> > > > > To unsubscribe from this group, send email to
> > > > > [hidden email].
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "testng-users" group.
> > > To post to this group, send email to [hidden email].
> > > To unsubscribe from this group, send email to
> > > [hidden email].
> > > For more options, visit this group at
> > >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Re: Soft asserts

Cédric Beust ♔-2
In reply to this post by Yury Kryazhev
Hi Yury,

This should be fixed in the beta, can you try it?  http://testng.org/beta

-- 
Cédric




On Tue, Aug 9, 2011 at 1:22 AM, Yury Kryazhev <[hidden email]> wrote:
Cédric, any news?
We've got more than 1k test in our production that are behaving this
way, it's really important for us.

Thx in advanced

On 5 авг, 12:28, Yury Kryazhev <[hidden email]> wrote:
> Sorry for bad example, here is a better one
>
> java:
> public class InvokedMethodListener implements IInvokedMethodListener {
>
>     public void beforeInvocation(IInvokedMethod iInvokedMethod,
> ITestResult iTestResult) {
>         TestBase.setPassed();
>     }
>
>     public void afterInvocation(IInvokedMethod method, ITestResult
> result) {
>         Reporter.setCurrentTestResult(result);
>         if (method.isTestMethod()) {
>             if (!TestBase.getPassed()) {
>                 result.setStatus(ITestResult.FAILURE);
>             }
>         }
>     }
>
> }
>
> public class TestBase {
>     private static boolean passed = true;
>     public static void setFailed() {
>         passed = false;
>     }
>     public static void setPassed() {
>         passed = true;
>     }
>     public static boolean getPassed() {
>         return passed;
>     }
>
> }
>
> test:
> public class TestImpl extends TestBase{
>     @Test
>     public void test_1() throws Exception {
>         TestBase.setFailed();
>     }
>
> }
>
> You can run it as testng configuration using testng.xml:
> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
> <suite verbose="0" name="Bad" parallel="false">
>     <test verbose="2" name="TestImpl" preserve-order="true">
>         <classes>
>             <class name="TestImpl"/>
>         </classes>
>     </test>
>     <listeners>
>       <listener class-name="InvokedMethodListener" />
>     </listeners>
> </suite>
>
> On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:
>
>
>
>
>
>
>
> > I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> > self-contained example that doesn't have any external dependencies?
>
> > Thanks.
>
> > --
> > Cédric
>
> > On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <[hidden email]>wrote:
>
> > > Missprinted in one method:
> > >     @Override
> > >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > ITestResult iTestResult) {
> > >         ExtendedSeleneseTestBaseCommon.setPassed();
> > >     }
>
> > > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > > Hi Yury,
>
> > > > Can you show a small code example that produces this problem so I can try
> > > > it?
>
> > > > Thanks.
>
> > > > --
> > > > Cédric
>
> > > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <[hidden email]
> > > >wrote:
>
> > > > > Hi,
>
> > > > > I'm using soft asserts in my tests as described in the article
> > > > >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > > And since testng 5.14 me and some other guys are facing with
> > > > > duplicating problem:
> > > > > PK
> > > > > >If my test method A failed with soft assertion, the report shows two
> > > > > instances of method A – one >failed and one succeeded. Both having the
> > > same
> > > > > exception list.
> > > > > >However, if method A succeeded, there is just one instance of A in the
> > > > > report – the succeeded >one.
> > > > > >Also, I am sure the test method is getting executed only once – it is
> > > only
> > > > > the report that has the >duplicate exception list.
>
> > > > > Any ideas how to resolve this issue?
>
> > > > > Thx in advanced,
> > > > > Yury
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "testng-users" group.
> > > > > To post to this group, send email to [hidden email].
> > > > > To unsubscribe from this group, send email to
> > > > > [hidden email].
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "testng-users" group.
> > > To post to this group, send email to [hidden email].
> > > To unsubscribe from this group, send email to
> > > [hidden email].
> > > For more options, visit this group at
> > >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Soft asserts

Yury Kryazhev
Hi, Cédric!

With last beta report seems fine, thank you )

But maven output is quite strange:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.39
sec <<< FAILURE!

Results :

Failed tests:
  test_1(TestImpl)
  test_1(TestImpl)

Tests run: 2, Failures: 2, Errors: 0, Skipped: 0

Truly,
Yury

On 9 авг, 21:39, Cédric Beust ♔ <[hidden email]> wrote:

> Hi Yury,
>
> This should be fixed in the beta, can you try it?  http://testng.org/beta
>
> --
> Cédric
>
> On Tue, Aug 9, 2011 at 1:22 AM, Yury Kryazhev <[hidden email]>wrote:
>
>
>
>
>
>
>
> > Cédric, any news?
> > We've got more than 1k test in our production that are behaving this
> > way, it's really important for us.
>
> > Thx in advanced
>
> > On 5 авг, 12:28, Yury Kryazhev <[hidden email]> wrote:
> > > Sorry for bad example, here is a better one
>
> > > java:
> > > public class InvokedMethodListener implements IInvokedMethodListener {
>
> > >     public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > ITestResult iTestResult) {
> > >         TestBase.setPassed();
> > >     }
>
> > >     public void afterInvocation(IInvokedMethod method, ITestResult
> > > result) {
> > >         Reporter.setCurrentTestResult(result);
> > >         if (method.isTestMethod()) {
> > >             if (!TestBase.getPassed()) {
> > >                 result.setStatus(ITestResult.FAILURE);
> > >             }
> > >         }
> > >     }
>
> > > }
>
> > > public class TestBase {
> > >     private static boolean passed = true;
> > >     public static void setFailed() {
> > >         passed = false;
> > >     }
> > >     public static void setPassed() {
> > >         passed = true;
> > >     }
> > >     public static boolean getPassed() {
> > >         return passed;
> > >     }
>
> > > }
>
> > > test:
> > > public class TestImpl extends TestBase{
> > >     @Test
> > >     public void test_1() throws Exception {
> > >         TestBase.setFailed();
> > >     }
>
> > > }
>
> > > You can run it as testng configuration using testng.xml:
> > > <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
> > > <suite verbose="0" name="Bad" parallel="false">
> > >     <test verbose="2" name="TestImpl" preserve-order="true">
> > >         <classes>
> > >             <class name="TestImpl"/>
> > >         </classes>
> > >     </test>
> > >     <listeners>
> > >       <listener class-name="InvokedMethodListener" />
> > >     </listeners>
> > > </suite>
>
> > > On 4 авг, 11:01, Cédric Beust ♔ <[hidden email]> wrote:
>
> > > > I don't know what ExtendedSeleneseTestBaseCommon is, can you post a
> > > > self-contained example that doesn't have any external dependencies?
>
> > > > Thanks.
>
> > > > --
> > > > Cédric
>
> > > > On Thu, Aug 4, 2011 at 12:00 AM, Yury Kryazhev <
> > [hidden email]>wrote:
>
> > > > > Missprinted in one method:
> > > > >     @Override
> > > > >    public void beforeInvocation(IInvokedMethod iInvokedMethod,
> > > > > ITestResult iTestResult) {
> > > > >         ExtendedSeleneseTestBaseCommon.setPassed();
> > > > >     }
>
> > > > > On 3 авг, 20:39, Cédric Beust ♔ <[hidden email]> wrote:
> > > > > > Hi Yury,
>
> > > > > > Can you show a small code example that produces this problem so I
> > can try
> > > > > > it?
>
> > > > > > Thanks.
>
> > > > > > --
> > > > > > Cédric
>
> > > > > > On Wed, Aug 3, 2011 at 3:31 AM, Yury Kryazhev <
> > [hidden email]
> > > > > >wrote:
>
> > > > > > > Hi,
>
> > > > > > > I'm using soft asserts in my tests as described in the article
>
> >http://seleniumexamples.com/blog/guide/using-soft-assertions-in-testng/
>
> > > > > > > And since testng 5.14 me and some other guys are facing with
> > > > > > > duplicating problem:
> > > > > > > PK
> > > > > > > >If my test method A failed with soft assertion, the report shows
> > two
> > > > > > > instances of method A – one >failed and one succeeded. Both
> > having the
> > > > > same
> > > > > > > exception list.
> > > > > > > >However, if method A succeeded, there is just one instance of A
> > in the
> > > > > > > report – the succeeded >one.
> > > > > > > >Also, I am sure the test method is getting executed only once –
> > it is
> > > > > only
> > > > > > > the report that has the >duplicate exception list.
>
> > > > > > > Any ideas how to resolve this issue?
>
> > > > > > > Thx in advanced,
> > > > > > > Yury
>
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the
> > Google
> > > > > Groups
> > > > > > > "testng-users" group.
> > > > > > > To post to this group, send email to
> > [hidden email].
> > > > > > > To unsubscribe from this group, send email to
> > > > > > > [hidden email].
> > > > > > > For more options, visit this group at
> > > > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > Groups
> > > > > "testng-users" group.
> > > > > To post to this group, send email to [hidden email].
> > > > > To unsubscribe from this group, send email to
> > > > > [hidden email].
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/testng-users?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "testng-users" group.
> > To post to this group, send email to [hidden email].
> > To unsubscribe from this group, send email to
> > [hidden email].
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.