Tranformer fail to modify the invocationCount when running tests in parallel

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 

On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to <a href="javascript:_e({}, &#39;cvml&#39;, &#39;testng-users@googlegroups.com&#39;);" target="_blank">testng-users@....
To unsubscribe from this group, send email to <a href="javascript:_e({}, &#39;cvml&#39;, &#39;testng-users%2Bunsubscribe@googlegroups.com&#39;);" target="_blank">testng-users+unsubscribe@....
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 
I also used the maven sure file plug-in at first but i don't know how to make my test suites run in parallel.
Here is the configuration in the pom file
<configuration>
           <suiteXmlFiles>
            <suiteXmlFile>testng1.xml</suiteXmlFile>
             <suiteXmlFile>testng2.xml</suiteXmlFile>
         </suiteXmlFiles>
        </configuration>
Using mvn test only make the test run sequentially. How can you make it run in parallel?
 
Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

No, in this case, i created 7 different class Transformer, and 7 files different files xml, each of them have only one suite as you can see in my last email.
 
are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 

I'm not sure about it right now, i will look into it and let you know tomorrow.
Thanks you so much.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
In reply to this post by ⇜Krishnan Mahadevan⇝
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

I tried to use the master suite as you said and only the listener in the master suite is invoked. Even using maven to run the all the suites sequentially, i got the same result, only the listener of the last suite being called
So there is no way to run the test-suites in parallel if each suite have its own listener? 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
In reply to this post by ⇜Krishnan Mahadevan⇝
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.


On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝
Take a look at the default EmailableReporter.java that comes bundled within TestNG


It should help you get an understanding.

Don't forget to add a reference to this implementation as a listener in your master suite file
using <listeners> tag.

Alternatively you could also use the ServiceLoader approach : http://beust.com/weblog/2011/07/01/announcing-testng-6-1/


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:37 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.



On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
Hi Krishnan,
I had a look to the  EmailableReporter.java, i think that to delete the skippedTest in the report, all i have to do is to delete the line containing the phrase "..SkippedTests.." . For example, for the file EmailableReporter.java, i can just delete these lines:
   resultSummary(testContext.getSkippedTests(), testName, "skipped", "");
   resultDetail(testContext.getSkippedTests(), "skipped");
   q = getMethodSet(overview.getSkippedTests()).size();
In the other hand, i think that i will also have to modify the TestHTMLReporter.java, TextReporter.java.
Can you give me a good start for this?
Thanks.

On Fri, Aug 24, 2012 at 5:15 PM, Krishnan Mahadevan <[hidden email]> wrote:
Take a look at the default EmailableReporter.java that comes bundled within TestNG


It should help you get an understanding.

Don't forget to add a reference to this implementation as a listener in your master suite file
using <listeners> tag.

Alternatively you could also use the ServiceLoader approach : http://beust.com/weblog/2011/07/01/announcing-testng-6-1/


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:37 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.



On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝
I haven't had the chance to take a detailed look at EmailableReporter.java so for now am not in a position to comment any further than I already have :( 


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Mon, Aug 27, 2012 at 7:24 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
I had a look to the  EmailableReporter.java, i think that to delete the skippedTest in the report, all i have to do is to delete the line containing the phrase "..SkippedTests.." . For example, for the file EmailableReporter.java, i can just delete these lines:
   resultSummary(testContext.getSkippedTests(), testName, "skipped", "");
   resultDetail(testContext.getSkippedTests(), "skipped");
   q = getMethodSet(overview.getSkippedTests()).size();
In the other hand, i think that i will also have to modify the TestHTMLReporter.java, TextReporter.java.
Can you give me a good start for this?
Thanks.

On Fri, Aug 24, 2012 at 5:15 PM, Krishnan Mahadevan <[hidden email]> wrote:
Take a look at the default EmailableReporter.java that comes bundled within TestNG


It should help you get an understanding.

Don't forget to add a reference to this implementation as a listener in your master suite file
using <listeners> tag.

Alternatively you could also use the ServiceLoader approach : http://beust.com/weblog/2011/07/01/announcing-testng-6-1/


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:37 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.



On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

nguyenki
Thanks Kishman.
I have written my own custom Reporter, and it work as expected.  
Now i just have a little trouble when adding these listeners in the MasterSuite.xml
Let me specify:
So i have 3 custom listeners :  FFEmailableReport.java,  FFSuiteHTMLReporter.java, FFTesstHTMLReporter.java
i added it in the MasterSuite.xml

<suite name="Suite1" parallel="methods" >
   <listeners>
    <listener class-name = "com.FiftyFive.app.FFTesstHTMLReporter"/>
    <listener class-name="com.FiftyFive.app.MasterTransformer"/>
    <listener class-name="com.FiftyFive.app.FFEmailableReport"/>
    <listener class-name = "com.FiftyFive.app.FFSuiteHTMLReporter"/>
   </listeners>
<suite-files name = "DASH BOARD TEST ALL CLIENT TEST" > 
 <suite-file path="test-suites/BNB.xml"></suite-file> 
 <suite-file path="test-suites/DACIA.xml"></suite-file>
 <suite-file path="test-suites/RENAULT.xml"></suite-file>
 </suite-files>

The two last test suites work like a charm (all the skipped test are deleted in the report) but not the first suite, i found the problem is that the first custom listener is not invoked for this suite. (BNB.xml)
So now i'm trying the second solution with SERVICELOADER as you suggested, but after reading the doc in the homepage of TestNG, i can not find the location META-INF/services/org.testng.ITestNGListener
I'm using Window 7.
Can you help me out?

Thanks a lot.


On Tue, Aug 28, 2012 at 11:31 AM, Krishnan Mahadevan <[hidden email]> wrote:
I haven't had the chance to take a detailed look at EmailableReporter.java so for now am not in a position to comment any further than I already have :( 


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Mon, Aug 27, 2012 at 7:24 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
I had a look to the  EmailableReporter.java, i think that to delete the skippedTest in the report, all i have to do is to delete the line containing the phrase "..SkippedTests.." . For example, for the file EmailableReporter.java, i can just delete these lines:
   resultSummary(testContext.getSkippedTests(), testName, "skipped", "");
   resultDetail(testContext.getSkippedTests(), "skipped");
   q = getMethodSet(overview.getSkippedTests()).size();
In the other hand, i think that i will also have to modify the TestHTMLReporter.java, TextReporter.java.
Can you give me a good start for this?
Thanks.

On Fri, Aug 24, 2012 at 5:15 PM, Krishnan Mahadevan <[hidden email]> wrote:
Take a look at the default EmailableReporter.java that comes bundled within TestNG


It should help you get an understanding.

Don't forget to add a reference to this implementation as a listener in your master suite file
using <listeners> tag.

Alternatively you could also use the ServiceLoader approach : http://beust.com/weblog/2011/07/01/announcing-testng-6-1/


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:37 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.



On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Tranformer fail to modify the invocationCount when running tests in parallel

⇜Krishnan Mahadevan⇝

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Tue, Aug 28, 2012 at 8:26 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks Kishman.
I have written my own custom Reporter, and it work as expected.  
Now i just have a little trouble when adding these listeners in the MasterSuite.xml
Let me specify:
So i have 3 custom listeners :  FFEmailableReport.java,  FFSuiteHTMLReporter.java, FFTesstHTMLReporter.java
i added it in the MasterSuite.xml

<suite name="Suite1" parallel="methods" >
   <listeners>
    <listener class-name = "com.FiftyFive.app.FFTesstHTMLReporter"/>
    <listener class-name="com.FiftyFive.app.MasterTransformer"/>
    <listener class-name="com.FiftyFive.app.FFEmailableReport"/>
    <listener class-name = "com.FiftyFive.app.FFSuiteHTMLReporter"/>
   </listeners>
<suite-files name = "DASH BOARD TEST ALL CLIENT TEST" > 
 <suite-file path="test-suites/BNB.xml"></suite-file> 
 <suite-file path="test-suites/DACIA.xml"></suite-file>
 <suite-file path="test-suites/RENAULT.xml"></suite-file>
 </suite-files>

The two last test suites work like a charm (all the skipped test are deleted in the report) but not the first suite, i found the problem is that the first custom listener is not invoked for this suite. (BNB.xml)
So now i'm trying the second solution with SERVICELOADER as you suggested, but after reading the doc in the homepage of TestNG, i can not find the location META-INF/services/org.testng.ITestNGListener
I'm using Window 7.
Can you help me out?

I use Maven as a build tool. So let me explain to you in its lingo.

Under src/main/resources
I created a folder named META-INF
Within that I created a subfolder named "services"
Within that I created a text file [No extensions] named org.testng.ITestNGListener and included all the listeners [along with their full package structure]

I then made sure that this resource would be added to the target and also bundled into the jar that I would be building [ I needed this, because this jar was essentially my automation framework, which my end users would add up as a dependency and leverage its capabilities]

Make sure you are using TestNG 6.2.1 or higher for you to use ServiceLoaders capability because I believe it was from this version that TestNG rolled out this feature.

I believe you were referring to this documentation that Cedric has put up : http://testng.org/doc/documentation-main.html#listeners-service-loader

I used the same as a reference point as well, to adopt it to the maven world.

 

Thanks a lot.


On Tue, Aug 28, 2012 at 11:31 AM, Krishnan Mahadevan <[hidden email]> wrote:
I haven't had the chance to take a detailed look at EmailableReporter.java so for now am not in a position to comment any further than I already have :( 


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Mon, Aug 27, 2012 at 7:24 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
I had a look to the  EmailableReporter.java, i think that to delete the skippedTest in the report, all i have to do is to delete the line containing the phrase "..SkippedTests.." . For example, for the file EmailableReporter.java, i can just delete these lines:
   resultSummary(testContext.getSkippedTests(), testName, "skipped", "");
   resultDetail(testContext.getSkippedTests(), "skipped");
   q = getMethodSet(overview.getSkippedTests()).size();
In the other hand, i think that i will also have to modify the TestHTMLReporter.java, TextReporter.java.
Can you give me a good start for this?
Thanks.

On Fri, Aug 24, 2012 at 5:15 PM, Krishnan Mahadevan <[hidden email]> wrote:
Take a look at the default EmailableReporter.java that comes bundled within TestNG


It should help you get an understanding.

Don't forget to add a reference to this implementation as a listener in your master suite file
using <listeners> tag.

Alternatively you could also use the ServiceLoader approach : http://beust.com/weblog/2011/07/01/announcing-testng-6-1/


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:37 PM, nguyenkim thuat <[hidden email]> wrote:
Thanks you so much, i'm not really familiar with the custom reporter.
But i will give it a try now and let you know.
Best regards.



On Fri, Aug 24, 2012 at 4:59 PM, Krishnan Mahadevan <[hidden email]> wrote:
I doubt if its possible to delete skipped tests in the report by default without your own custom reporter.

You can circumvent this by creating your own reporter (implementing IReporter) and then filter out the skipped tests based upon the condition :

If number of tests < invocation count, then 
from (number of tests) + 1 to invocation count, 
skip recording the results in the reporter.

I cant think of anything else that might help you here.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 8:22 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Krishnan,
Finally, i ended up with the solution that modify the invocationCount to be the max value between all the values came from the test suites. If in the case that the number of times of URITest which is ran over the number it needed, i will throw a SkippedExeption , it is the only solution to make my test suites run parallel for now.
It not too bad, but there is one thing which bothers me a lot.
For exemple, in the first test suite, i just need to run 27 tests, but the number modified by the Transformer Annotation is now 99. So i get the result in report like that:
               DashBord Test (13/27/59)
which mean that there is 59 skipped tests being printed in report, and it's really not nice at all.
Is there anyway to delete the skipped tests results in the report?
Thanks. 

On Fri, Aug 24, 2012 at 10:19 AM, Krishnan Mahadevan <[hidden email]> wrote:
Yeah, that was what I thought too but then I later realized that it might not be the case. Let me elaborate

Let us consider the below MasterSuite.xml which is being invoked via the maven-surefire plugin.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" parallel="methods" >
<listeners>
<listener class-name="com.test.listeners.ShowSuiteCount"></listener>
</listeners>

<suite-files >
<suite-file path="src/test/resources/suite1.xml" ></suite-file>
<suite-file path="src/test/resources/suite2.xml"></suite-file>
</suite-files>
</suite> 

Even though suite1.xml and suite2.xml have a reference to the listener "com.test.listeners.ShowSuiteCount"
TestNG seems to invoke the listener that is part of ONLY the MasterSuite.xml and not the listeners that part of suite1.xml and suite2.xml

I thought that this must be by design, but Cedric states that this is a bug per this thread : https://groups.google.com/d/msg/testng-users/-/AizYGVPaIH4J

and has stated that he will take a look at this.

So in your case, irrespective of what listeners you have hooked up to your sub-suites, ONLY the listener hooked on to your master suite [In your case the the AnnotationTransformer implementing listener] would stand to be invoked.

But I am not sure, if the running the above stated MasterSuite.xml and running multiple suites via the command line [java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ]

are both one and the same. My naive gut feeling is, they have to be the same but for the suitethreadpoolsize parameter, which I cant find an equivalent from within the suite xml concept.

All said and done, yet again, I guess I will have to hope for a clearer explanation from Cedric, since I have only now started to look at the TestNG code base and trying to understand how it is built.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"



On Fri, Aug 24, 2012 at 1:36 PM, nguyenkim thuat <[hidden email]> wrote:
Hi Kishnan,
You are right, there is only the last instance of Transformer Annotation that is getting created for the last suites file. But that sound weird to me, because i have seven test suites and seven tranformer class independants, normally every test suite must create its own instance for the tranformer, right?


On Thu, Aug 23, 2012 at 7:29 PM, Krishnan Mahadevan <[hidden email]> wrote:
I have never run suites from command prompt [the way you are running]. I have only used the maven surefire plug-in. 

Based upon my understanding when you run a suite (which contains suites within itself) only the listener of the master suite is run [ I dont know if the way you are running equates to this scenario]

are you sure that there are multiple instances of your AnnotationTransformer being created or is it that just one instance that is getting created for the last suite file?

Perhaps you could add a System.out.println() to your AnnotationTransformer's default constructor and confirm this as well. 


On Thursday, August 23, 2012, nguyenkim thuat wrote:
Hi,
I'm trying to run 7 test suites in parallel using TestNG with the command line
      java org.testng.TestNG -suitethreadpoolsize 7 testng1.xml testng2.xml testng3.xml ..................
Each file xml use a class of transformer
File testng1.xml
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="DACIA" verbose="1" >
  <listeners>
    <listener class-name="com.FF.app.Transformer1"/>
  </listeners>
  <test name="DashBord Test">
      <parameter name = "client" value = "DACIA"/>
  <groups>
      <run>
           <include name = "Login"/>
        <include name = "URI"/>  
  <include name = "other"/>
     </run>
  </groups>
   <classes>
       <class name="com.FF.app.AppTest"/>
     </classes>
  </test>
</suite>

Class Tranformer1

public class Transformer1 implements IAnnotationTransformer {
protected TestConfig config;
private int number;
public Transformer1() {
Parser parse = new Parser(System.getProperty("user.dir")+"\\test-suites\\testng1.xml");
try {
List<XmlSuite> suites = parse.parseToList();
XmlSuite suite = suites.get(0);
String clientName = suite.getName();
this.config = new TestConfig(new ConfigURLandTextFile(clientName));
this.number = this.config.getInfoFromFileURI().size();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
if ("URITest".equals(testMethod.getName())) {
annotation.setInvocationCount(number);  // The number will change for every file xml
}
}
}

The tests launched in parallel but i got a problem with the Transformer which is all the value of invocationCount are identicals with the value came from the last xml file which is testng7.xml.
So the transformer will not be ran in parallel but sequentially?
Any suggestions?
Thanks.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.


--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.