jUnit reports show false fails

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

jUnit reports show false fails

Milo Gertjejansen
I have recently moved from jUnit to TestNG.

We are getting some false fails in some of our jUnit reports. If setup or teardown fails for one method it gets reported on in another method (in addition to method it should be reported in).

For example, the method TestOne.testOne() fails. A number of minutes later the BeforeMethod for TestTwo.testTwo() fails. In the jUnit reports for TestOne, you would see TestTwo.testTwo's BeforeMethod failure. In TestTwo's report you would see TestTwo.testTwo's BeforeMethod failure like normal.

Looking at the logs, TestOne failed for a legitimate reason but seems to be overwritten by TestTwo's fail in the jUnit reports.

Has anybody experienced this and, if so, was there anything you could do to fix it?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jUnit reports show false fails

⇜Krishnan Mahadevan⇝

What version of TestNG are you working with ? Can you please try using TestNG 6.12 and see if the issue happens ?

If the issue happens, I would request you to please help file a bug in https://github.com/cbeust/testng/issues and add some sample that can be used to reproduce the problem.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Milo Gertjejansen <[hidden email]>
Reply-To: <[hidden email]>
Date: Thursday, September 21, 2017 at 6:54 PM
To: testng-users <[hidden email]>
Subject: [testng-users] jUnit reports show false fails

 

I have recently moved from jUnit to TestNG.

 

We are getting some false fails in some of our jUnit reports. If setup or teardown fails for one method it gets reported on in another method (in addition to method it should be reported in).

 

For example, the method TestOne.testOne() fails. A number of minutes later the BeforeMethod for TestTwo.testTwo() fails. In the jUnit reports for TestOne, you would see TestTwo.testTwo's BeforeMethod failure. In TestTwo's report you would see TestTwo.testTwo's BeforeMethod failure like normal.

 

Looking at the logs, TestOne failed for a legitimate reason but seems to be overwritten by TestTwo's fail in the jUnit reports.

 

Has anybody experienced this and, if so, was there anything you could do to fix it?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jUnit reports show false fails

Milo Gertjejansen
I am not positive the version. I am using the selenium-standalone-server.jar, which has TestNG packaged in with it. I can switch to TestNG 6.12 though and report back.

On Thursday, September 21, 2017 at 8:44:24 AM UTC-5, Krishnan Mahadevan wrote:

What version of TestNG are you working with ? Can you please try using TestNG 6.12 and see if the issue happens ?

If the issue happens, I would request you to please help file a bug in <a href="https://github.com/cbeust/testng/issues" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcbeust%2Ftestng%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEkZFwrR886j6_KMGsK-VN4fP6q5g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcbeust%2Ftestng%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEkZFwrR886j6_KMGsK-VN4fP6q5g&#39;;return true;">https://github.com/cbeust/testng/issues and add some sample that can be used to reproduce the problem.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...> on behalf of Milo Gertjejansen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">milo.ger...@gm.mediabeacon.com>
Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Thursday, September 21, 2017 at 6:54 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: [testng-users] jUnit reports show false fails

 

I have recently moved from jUnit to TestNG.

 

We are getting some false fails in some of our jUnit reports. If setup or teardown fails for one method it gets reported on in another method (in addition to method it should be reported in).

 

For example, the method TestOne.testOne() fails. A number of minutes later the BeforeMethod for TestTwo.testTwo() fails. In the jUnit reports for TestOne, you would see TestTwo.testTwo's BeforeMethod failure. In TestTwo's report you would see TestTwo.testTwo's BeforeMethod failure like normal.

 

Looking at the logs, TestOne failed for a legitimate reason but seems to be overwritten by TestTwo's fail in the jUnit reports.

 

Has anybody experienced this and, if so, was there anything you could do to fix it?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="k0RR6JUHAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jUnit reports show false fails

⇜Krishnan Mahadevan⇝

AFAIK, the selenium standalone uber jar doesn’t come bundled with TestNG (not atleast the latest ones). What version of Selenium standalone are you on ? 3.5.3 is the latest.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Milo Gertjejansen <[hidden email]>
Reply-To: <[hidden email]>
Date: Thursday, September 21, 2017 at 7:21 PM
To: testng-users <[hidden email]>
Subject: Re: [testng-users] jUnit reports show false fails

 

I am not positive the version. I am using the selenium-standalone-server.jar, which has TestNG packaged in with it. I can switch to TestNG 6.12 though and report back.

On Thursday, September 21, 2017 at 8:44:24 AM UTC-5, Krishnan Mahadevan wrote:

What version of TestNG are you working with ? Can you please try using TestNG 6.12 and see if the issue happens ?

If the issue happens, I would request you to please help file a bug in https://github.com/cbeust/testng/issues and add some sample that can be used to reproduce the problem.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank">testng...@...> on behalf of Milo Gertjejansen <<a href="javascript:" target="_blank">milo.ger...@...>
Reply-To: <<a href="javascript:" target="_blank">testng...@...>
Date: Thursday, September 21, 2017 at 6:54 PM
To: testng-users <<a href="javascript:" target="_blank">testng...@...>
Subject: [testng-users] jUnit reports show false fails

 

I have recently moved from jUnit to TestNG.

 

We are getting some false fails in some of our jUnit reports. If setup or teardown fails for one method it gets reported on in another method (in addition to method it should be reported in).

 

For example, the method TestOne.testOne() fails. A number of minutes later the BeforeMethod for TestTwo.testTwo() fails. In the jUnit reports for TestOne, you would see TestTwo.testTwo's BeforeMethod failure. In TestTwo's report you would see TestTwo.testTwo's BeforeMethod failure like normal.

 

Looking at the logs, TestOne failed for a legitimate reason but seems to be overwritten by TestTwo's fail in the jUnit reports.

 

Has anybody experienced this and, if so, was there anything you could do to fix it?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank">testng-users...@....
To post to this group, send email to <a href="javascript:" target="_blank">testng...@....
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jUnit reports show false fails

Milo Gertjejansen
Ah possibly 2.53. I am looking in the VERSION.txt in the root of the jar.

On Thursday, September 21, 2017 at 8:55:59 AM UTC-5, Krishnan Mahadevan wrote:

AFAIK, the selenium standalone uber jar doesn’t come bundled with TestNG (not atleast the latest ones). What version of Selenium standalone are you on ? 3.5.3 is the latest.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...> on behalf of Milo Gertjejansen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">milo.ger...@gm.mediabeacon.com>
Reply-To: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Date: Thursday, September 21, 2017 at 7:21 PM
To: testng-users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@...>
Subject: Re: [testng-users] jUnit reports show false fails

 

I am not positive the version. I am using the selenium-standalone-server.jar, which has TestNG packaged in with it. I can switch to TestNG 6.12 though and report back.

On Thursday, September 21, 2017 at 8:44:24 AM UTC-5, Krishnan Mahadevan wrote:

What version of TestNG are you working with ? Can you please try using TestNG 6.12 and see if the issue happens ?

If the issue happens, I would request you to please help file a bug in <a href="https://github.com/cbeust/testng/issues" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcbeust%2Ftestng%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEkZFwrR886j6_KMGsK-VN4fP6q5g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcbeust%2Ftestng%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEkZFwrR886j6_KMGsK-VN4fP6q5g&#39;;return true;">https://github.com/cbeust/testng/issues and add some sample that can be used to reproduce the problem.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ <a href="http://wakened-cognition.blogspot.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwakened-cognition.blogspot.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHzOdYJCASIDF_28vQtkp9gnKAYSQ&#39;;return true;">http://wakened-cognition.blogspot.com/

My Technical Scribbings @ <a href="http://rationaleemotions.wordpress.com/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Frationaleemotions.wordpress.com%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEcBOKyRn0lpL8LDbBuyAlKdwgXwQ&#39;;return true;">http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of Milo Gertjejansen <[hidden email].com>
Reply-To: <[hidden email]>
Date: Thursday, September 21, 2017 at 6:54 PM
To: testng-users <[hidden email]>
Subject: [testng-users] jUnit reports show false fails

 

I have recently moved from jUnit to TestNG.

 

We are getting some false fails in some of our jUnit reports. If setup or teardown fails for one method it gets reported on in another method (in addition to method it should be reported in).

 

For example, the method TestOne.testOne() fails. A number of minutes later the BeforeMethod for TestTwo.testTwo() fails. In the jUnit reports for TestOne, you would see TestTwo.testTwo's BeforeMethod failure. In TestTwo's report you would see TestTwo.testTwo's BeforeMethod failure like normal.

 

Looking at the logs, TestOne failed for a legitimate reason but seems to be overwritten by TestTwo's fail in the jUnit reports.

 

Has anybody experienced this and, if so, was there anything you could do to fix it?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to [hidden email].
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jcOhtTcIAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">testng...@....
Visit this group at <a href="https://groups.google.com/group/testng-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/testng-users&#39;;return true;">https://groups.google.com/group/testng-users.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.