SkipException in the config method marks test as failed

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

SkipException in the config method marks test as failed

Bartek Florczak
Hello! 

I'm trying to fail the test suite after the first test failed. 

The code looks like this:

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class FailFastListener implements IInvokedMethodListener {
   private static final String FAIL_FAST = System.getProperty("failFast");
   private static final Boolean IS_FAIL_FAST = Boolean.parseBoolean(FAIL_FAST);
   private static boolean hasFailures = false;

   @Override
   public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
      synchronized (this) {
         if (!IS_FAIL_FAST || !method.isTestMethod()) {
            return;
         }
         if (hasFailures) {
            throw new SkipException(
                  String.format("Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
         }
      }
   }

   @Override
   public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
      synchronized (this) {
         if (!IS_FAIL_FAST && !method.isTestMethod()) {
            return;
         }
         if ((!hasFailures) && !testResult.isSuccess()) {
            hasFailures = true;
         }
      }
   }
}

The thing is, that I would like to not execute further configuration methods, like @BeforeClass and @BeforeMethod annotated methods. 
If in beforeInvocation method I change the condition to 
@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
   synchronized (this) {
      if (!IS_FAIL_FAST || (!method.isTestMethod() && !method.getTestMethod().isBeforeClassConfiguration()
 && !method.getTestMethod().isBeforeMethodConfiguration())) {
         return;
      }
      if (hasFailures) {
         throw new SkipException(
               String.format("Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
      }
   }
}

after the SkipException is thrown, the test will be set as failed, which is not what I'm expecting - I want to get only on failure from the tests run. 
Is there any way of skipping configuration method from the IInvokedMethodListener or IInvokedMethodListener2 ?

Or SkipException should also skip the configuration?

Thx in advance!

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Krishnan Mahadevan

Maybe you could try using : org.testng.IConfigurationListener2 and see if that helps ?

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 20, 2018 at 7:29 PM
To: testng-users <[hidden email]>
Subject: [testng-users] SkipException in the config method marks test as failed

 

Hello! 

 

I'm trying to fail the test suite after the first test failed. 

 

The code looks like this:

 

import org.testng.IInvokedMethod;
import
org.testng.IInvokedMethodListener;
import
org.testng.ITestResult;

public class
FailFastListener implements IInvokedMethodListener {
   
private static final String FAIL_FAST = System.getProperty("failFast");
   private static final
Boolean IS_FAIL_FAST = Boolean.parseBoolean(FAIL_FAST);
   private static boolean
hasFailures = false;

   
@Override
   
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST || !method.isTestMethod()) {
           
return;
         
}
         
if (hasFailures) {
           
throw new SkipException(
                  String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
         
}
      }
   }

   
@Override
   
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST && !method.isTestMethod()) {
           
return;
         
}
         
if ((!hasFailures) && !testResult.isSuccess()) {
           
hasFailures = true;
         
}
      }
   }
}


The thing is, that I would like to not execute further configuration methods, like @BeforeClass and @BeforeMethod annotated methods. 

If in beforeInvocation method I change the condition to 

@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
   
synchronized (this) {
     
if (!IS_FAIL_FAST || (!method.isTestMethod() && !method.getTestMethod().isBeforeClassConfiguration()
 && !method.getTestMethod().isBeforeMethodConfiguration())) {
         
return;
     
}
     
if (hasFailures) {
         
throw new SkipException(
               String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
     
}
   }
}

 

after the SkipException is thrown, the test will be set as failed, which is not what I'm expecting - I want to get only on failure from the tests run. 

Is there any way of skipping configuration method from the IInvokedMethodListener or IInvokedMethodListener2 ?

 

Or SkipException should also skip the configuration?

 

Thx in advance!

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Bartek Florczak
Thx for the quick reply (as You always do ;) ) 

Hm, I might have no idea how would it help there? 
If I throw SkipException before/after the config method, the ITestResult result is still SKIPPED. So it seems like it should be handled properly - test should be marked as skipped, not failed.

If I throw SkipException only to the test methods, the exception message is not printed out to the console. 
If I throw it before each configuration method - the tests are marked as failed, and the Exception is thrown in the console. 


W dniu środa, 20 czerwca 2018 16:21:37 UTC+2 użytkownik Krishnan Mahadevan napisał:

Maybe you could try using : org.testng.IConfigurationListener2 and see if that helps ?

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Wednesday, June 20, 2018 at 7:29 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: [testng-users] SkipException in the config method marks test as failed

 

Hello! 

 

I'm trying to fail the test suite after the first test failed. 

 

The code looks like this:

 

import org.testng.IInvokedMethod;
import
org.testng.IInvokedMethodListener;
import
org.testng.ITestResult;

public class
FailFastListener implements IInvokedMethodListener {
   
private static final String FAIL_FAST = System.getProperty("failFast");
   private static final
Boolean IS_FAIL_FAST = Boolean.parseBoolean(FAIL_FAST);
   private static boolean
hasFailures = false;

   
@Override
   
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST || !method.isTestMethod()) {
           
return;
         
}
         
if (hasFailures) {
           
throw new SkipException(
                  String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
         
}
      }
   }

   
@Override
   
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST && !method.isTestMethod()) {
           
return;
         
}
         
if ((!hasFailures) && !testResult.isSuccess()) {
           
hasFailures = true;
         
}
      }
   }
}


The thing is, that I would like to not execute further configuration methods, like @BeforeClass and @BeforeMethod annotated methods. 

If in beforeInvocation method I change the condition to 

@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
   
synchronized (this) {
     
if (!IS_FAIL_FAST || (!method.isTestMethod() && !method.getTestMethod().isBeforeClassConfiguration()
 && !method.getTestMethod().isBeforeMethodConfiguration())) {
         
return;
     
}
     
if (hasFailures) {
         
throw new SkipException(
               String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
     
}
   }
}

 

after the SkipException is thrown, the test will be set as failed, which is not what I'm expecting - I want to get only on failure from the tests run. 

Is there any way of skipping configuration method from the IInvokedMethodListener or IInvokedMethodListener2 ?

 

Or SkipException should also skip the configuration?

 

Thx in advance!

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="fOPjASXTAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Krishnan Mahadevan

Bartek,

>>> Thx for the quick reply (as You always do ;) ) 

Just trying to be of some help that’s all

 

Am hoping you are working with the latest released version of TestNG (6.14.3 at this point of time).

Failure in a @Test method and having further config methods skipped is perhaps applicable only to @BeforeClass and @BeforeMethod only. The others such as @BeforeSuite and @BeforeTest have already been executed.

Also it would be good if you could please help share a full fledged example along with the expected output, so that it elaborates your use case.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 20, 2018 at 8:32 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Thx for the quick reply (as You always do ;) ) 

 

Hm, I might have no idea how would it help there? 

If I throw SkipException before/after the config method, the ITestResult result is still SKIPPED. So it seems like it should be handled properly - test should be marked as skipped, not failed.

 

If I throw SkipException only to the test methods, the exception message is not printed out to the console. 

If I throw it before each configuration method - the tests are marked as failed, and the Exception is thrown in the console. 

 


W dniu środa, 20 czerwca 2018 16:21:37 UTC+2 użytkownik Krishnan Mahadevan napisał:

Maybe you could try using : org.testng.IConfigurationListener2 and see if that helps ?

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank">testng...@...>
Date: Wednesday, June 20, 2018 at 7:29 PM
To: testng-users <<a href="javascript:" target="_blank">testng...@...>
Subject: [testng-users] SkipException in the config method marks test as failed

 

Hello! 

 

I'm trying to fail the test suite after the first test failed. 

 

The code looks like this:

 

import org.testng.IInvokedMethod;
import
org.testng.IInvokedMethodListener;
import
org.testng.ITestResult;

public class
FailFastListener implements IInvokedMethodListener {
   
private static final String FAIL_FAST = System.getProperty("failFast");
   private static final
Boolean IS_FAIL_FAST = Boolean.parseBoolean(FAIL_FAST);
   private static boolean
hasFailures = false;

   
@Override
   
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST || !method.isTestMethod()) {
           
return;
         
}
         
if (hasFailures) {
           
throw new SkipException(
                  String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
         
}
      }
   }

   
@Override
   
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
     
synchronized (this) {
         
if (!IS_FAIL_FAST && !method.isTestMethod()) {
           
return;
         
}
         
if ((!hasFailures) && !testResult.isSuccess()) {
           
hasFailures = true;
         
}
      }
   }
}


The thing is, that I would like to not execute further configuration methods, like @BeforeClass and @BeforeMethod annotated methods. 

If in beforeInvocation method I change the condition to 

@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
   
synchronized (this) {
     
if (!IS_FAIL_FAST || (!method.isTestMethod() && !method.getTestMethod().isBeforeClassConfiguration()
 && !method.getTestMethod().isBeforeMethodConfiguration())) {
         
return;
     
}
     
if (hasFailures) {
         
throw new SkipException(
               String.format(
"Skipping \"%s\" test method.", method.getTestMethod().getMethodName()));
     
}
   }
}

 

after the SkipException is thrown, the test will be set as failed, which is not what I'm expecting - I want to get only on failure from the tests run. 

Is there any way of skipping configuration method from the IInvokedMethodListener or IInvokedMethodListener2 ?

 

Or SkipException should also skip the configuration?

 

Thx in advance!

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank">testng-users...@....
To post to this group, send email to <a href="javascript:" target="_blank">testng...@....
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Bartek Florczak
Yes, I'm working with the latest version available. 
BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Krishnan Mahadevan

Awesome. Link to a github repo would be great.

 

W.r.t TestNG 7.0, I don’t think a release has been planned yet. Since this new release is going to be a major release (with JDK8 as a minimum JDK needs), we are still working on addressing issues, internal code refactoring etc.,

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 20, 2018 at 8:43 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Yes, I'm working with the latest version available. 

BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Bartek Florczak
Hello Krishnan,

I've prepared some debug code.

https://github.com/baflQA/testNG_parallel_debug

If You run Class2, You should get this result:
1 failed, 11 ignore of total 12 tests. 

In the console, there will be a trace of all the config methods being executed. 

If in the line 20 of the FailFastListener You remove 
|| !method.isTestMethod()

You will get 4 failure, 3 skipped of total 7 tests... At least that's what IntelliJ shows :)

What I want to achieve there, is in case of the first failure, be able to skip not only the tests but also at least @Before and After methods config, where some time-consuming tasks are executed (like navigation to some page or refreshing the page using WebDriver). 

From my perspective, if SkipException class is thrown in the config, it should not cause the test to be marked as failed. Maybe my approach is wrong there?

W dniu środa, 20 czerwca 2018 17:15:51 UTC+2 użytkownik Krishnan Mahadevan napisał:

Awesome. Link to a github repo would be great.

 

W.r.t TestNG 7.0, I don’t think a release has been planned yet. Since this new release is going to be a major release (with JDK8 as a minimum JDK needs), we are still working on addressing issues, internal code refactoring etc.,

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Wednesday, June 20, 2018 at 8:43 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Yes, I'm working with the latest version available. 

BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="G3H2kxrWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Krishnan Mahadevan

Bartek,

My apologies for getting back late.

 

I was able to get this to work by moving the logic that you have housed within beforeInvocation() into beforeConfiguration() method.

 

I have attached a patch file that you can use. Please take a look at it.

 

You can try running the TestRunner.java from the patch.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Thursday, June 21, 2018 at 1:27 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Hello Krishnan,

 

I've prepared some debug code.

 

https://github.com/baflQA/testNG_parallel_debug

 

If You run Class2, You should get this result:

1 failed, 11 ignore of total 12 tests. 

 

In the console, there will be a trace of all the config methods being executed. 

 

If in the line 20 of the FailFastListener You remove 

|| !method.isTestMethod()


You will get 4 failure, 3 skipped of total 7 tests... At least that's what IntelliJ shows :)

 

What I want to achieve there, is in case of the first failure, be able to skip not only the tests but also at least @Before and After methods config, where some time-consuming tasks are executed (like navigation to some page or refreshing the page using WebDriver). 

 

From my perspective, if SkipException class is thrown in the config, it should not cause the test to be marked as failed. Maybe my approach is wrong there?


W dniu środa, 20 czerwca 2018 17:15:51 UTC+2 użytkownik Krishnan Mahadevan napisał:

Awesome. Link to a github repo would be great.

 

W.r.t TestNG 7.0, I don’t think a release has been planned yet. Since this new release is going to be a major release (with JDK8 as a minimum JDK needs), we are still working on addressing issues, internal code refactoring etc.,

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank">testng...@...>
Date: Wednesday, June 20, 2018 at 8:43 PM
To: testng-users <<a href="javascript:" target="_blank">testng...@...>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Yes, I'm working with the latest version available. 

BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank">testng-users...@....
To post to this group, send email to <a href="javascript:" target="_blank">testng...@....
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

0001-Changes-to-show-how-to-get-this-to-work.patch (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Bartek Florczak
Dear Krishnan

You simply ROCK!
This is exactly what I wanted to achieve. 
I've also implemented IDataProviderListener, so that no DataProvider is executed before skipped test methods.

So kudos!

But still, I find it a bit inconsistent that SkipException in before invocation cause the test marked as failed :)

W dniu środa, 27 czerwca 2018 06:32:05 UTC+2 użytkownik Krishnan Mahadevan napisał:

Bartek,

My apologies for getting back late.

 

I was able to get this to work by moving the logic that you have housed within beforeInvocation() into beforeConfiguration() method.

 

I have attached a patch file that you can use. Please take a look at it.

 

You can try running the TestRunner.java from the patch.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Thursday, June 21, 2018 at 1:27 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Hello Krishnan,

 

I've prepared some debug code.

 

<a href="https://github.com/baflQA/testNG_parallel_debug" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2FbaflQA%2FtestNG_parallel_debug\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGK-dz7TB4xyIU-mXelxJfF0CaaoA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2FbaflQA%2FtestNG_parallel_debug\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGK-dz7TB4xyIU-mXelxJfF0CaaoA&#39;;return true;">https://github.com/baflQA/testNG_parallel_debug

 

If You run Class2, You should get this result:

1 failed, 11 ignore of total 12 tests. 

 

In the console, there will be a trace of all the config methods being executed. 

 

If in the line 20 of the FailFastListener You remove 

|| !method.isTestMethod()


You will get 4 failure, 3 skipped of total 7 tests... At least that's what IntelliJ shows :)

 

What I want to achieve there, is in case of the first failure, be able to skip not only the tests but also at least @Before and After methods config, where some time-consuming tasks are executed (like navigation to some page or refreshing the page using WebDriver). 

 

From my perspective, if SkipException class is thrown in the config, it should not cause the test to be marked as failed. Maybe my approach is wrong there?


W dniu środa, 20 czerwca 2018 17:15:51 UTC+2 użytkownik Krishnan Mahadevan napisał:

Awesome. Link to a github repo would be great.

 

W.r.t TestNG 7.0, I don’t think a release has been planned yet. Since this new release is going to be a major release (with JDK8 as a minimum JDK needs), we are still working on addressing issues, internal code refactoring etc.,

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 20, 2018 at 8:43 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Yes, I'm working with the latest version available. 

BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to [hidden email].
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="E1Y1fJs9BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: SkipException in the config method marks test as failed

Krishnan Mahadevan

Glad that I was of some help Bartek

 

>>> But still, I find it a bit inconsistent that SkipException in before invocation cause the test marked as failed

 

Can you please help file a bug for this with a sample? We can take a look at it and get back.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 27, 2018 at 11:48 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Dear Krishnan

 

You simply ROCK!

This is exactly what I wanted to achieve. 

I've also implemented IDataProviderListener, so that no DataProvider is executed before skipped test methods.

 

So kudos!

 

But still, I find it a bit inconsistent that SkipException in before invocation cause the test marked as failed :)


W dniu środa, 27 czerwca 2018 06:32:05 UTC+2 użytkownik Krishnan Mahadevan napisał:

Bartek,

My apologies for getting back late.

 

I was able to get this to work by moving the logic that you have housed within beforeInvocation() into beforeConfiguration() method.

 

I have attached a patch file that you can use. Please take a look at it.

 

You can try running the TestRunner.java from the patch.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank">testng...@...> on behalf of Bartek Florczak <<a href="javascript:" target="_blank">zaw...@...>
Reply-To: <<a href="javascript:" target="_blank">testng...@...>
Date: Thursday, June 21, 2018 at 1:27 PM
To: testng-users <<a href="javascript:" target="_blank">testng...@...>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Hello Krishnan,

 

I've prepared some debug code.

 

 

If You run Class2, You should get this result:

1 failed, 11 ignore of total 12 tests. 

 

In the console, there will be a trace of all the config methods being executed. 

 

If in the line 20 of the FailFastListener You remove 

|| !method.isTestMethod()


You will get 4 failure, 3 skipped of total 7 tests... At least that's what IntelliJ shows :)

 

What I want to achieve there, is in case of the first failure, be able to skip not only the tests but also at least @Before and After methods config, where some time-consuming tasks are executed (like navigation to some page or refreshing the page using WebDriver). 

 

From my perspective, if SkipException class is thrown in the config, it should not cause the test to be marked as failed. Maybe my approach is wrong there?


W dniu środa, 20 czerwca 2018 17:15:51 UTC+2 użytkownik Krishnan Mahadevan napisał:

Awesome. Link to a github repo would be great.

 

W.r.t TestNG 7.0, I don’t think a release has been planned yet. Since this new release is going to be a major release (with JDK8 as a minimum JDK needs), we are still working on addressing issues, internal code refactoring etc.,

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Bartek Florczak <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, June 20, 2018 at 8:43 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] SkipException in the config method marks test as failed

 

Yes, I'm working with the latest version available. 

BTW - any plans for releasing a new one? A lot of commits since the last release ;) 

 

I'll provide some code later on, as a link to the GitHub repo.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank">testng-users...@....
To post to this group, send email to <a href="javascript:" target="_blank">testng...@....
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.