Best way, from a performance view point, to run tests

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Best way, from a performance view point, to run tests

maggra
Hi guys,

I'm having a slight problem with performance when running one of my test suites. The suite tries to execute 1901 unit tests and takes about 3 minutes to do so. I've observed that the majority of this time is spent by TestNG and spent on parsing and deciding on what test methods to include.

My current setup adds everything (external JARs, compiled src, compiled test resources and compiled test src) to the classpath and let TestNG use that to find tests. I understand that this is a lot of unnecessary stuff for TestNG to examine for test methods, but I haven't bothered until now. Now I wish to limit the pre processing time and have tried to use classfileset to do so. However, processing takes even more time with this approach and have the side effect that my tests start to fail.

So what I'm asking is, what is the best practice to keep TestNG pre processing down when the classpath contains a lot of classes?

Regards,

//Magnus

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

AW: [testng-users] Best way, from a performance view point, to run tests

Hubert Grzeskowiak
Hi Magnus,
you can describe in your XML suite file quite precisely which packages, classes or even test methods in particular you want to execute. The more precisely you describe your tests, the faster TestNG will find them. This way TestNG also won't search on the CLASSPATH or anything, but only for the defined scopes. You can also use multiple suites or test groups for more control over what tests are run.
I hope this helps a bit.

Regards
Hubert Grzeskowiak
________________________________________
Von: [hidden email] [[hidden email]] im Auftrag von maggra [[hidden email]]
Gesendet: Freitag, 10. Januar 2014 11:04
An: [hidden email]
Betreff: [testng-users] Best way, from a performance view point, to run tests

Hi guys,

I'm having a slight problem with performance when running one of my test suites. The suite tries to execute 1901 unit tests and takes about 3 minutes to do so. I've observed that the majority of this time is spent by TestNG and spent on parsing and deciding on what test methods to include.

My current setup adds everything (external JARs, compiled src, compiled test resources and compiled test src) to the classpath and let TestNG use that to find tests. I understand that this is a lot of unnecessary stuff for TestNG to examine for test methods, but I haven't bothered until now. Now I wish to limit the pre processing time and have tried to use classfileset to do so. However, processing takes even more time with this approach and have the side effect that my tests start to fail.

So what I'm asking is, what is the best practice to keep TestNG pre processing down when the classpath contains a lot of classes?

Regards,

//Magnus

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Best way, from a performance view point, to run tests

maggra
No, sorry. That didn't help me, I'm afraid.

The thing is that all classes are in the same base package. Defining each sub package is not feasible and not desired, since developers could add packages without the CM, me, knowing about it. We have a very free development strategy where everyone has the ability to change everything :)

I'm really missing one feature in TestNG XML file, the ability to use something like Ants fileset. I would like to define package and **/**Test.class, if you see what I mean.

Here is my current XML config:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="CommonUnitTestSuite" parallel="classes" thread-count="4">
  <listeners>
    <listener class-name="testng.annotation.NoTestGroupTransformer"/>
    <listener class-name="testng.singletons.UnitTestsSingletonDestroyer"/>
  </listeners>

  <test name="CommonUnitTests">
    <groups>
      <run>
        <include name="unit"/>
        <exclude name="integration"/>
        <exclude name="context"/>
        <exclude name="browser"/>
        <exclude name="ignore"/>
        <exclude name="database"/>
      </run>
    </groups>

    <packages>
      <package name="ibe.*"/>
    </packages>

  </test>
</suite>

where "ibe.*" is the base package.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/groups/opt_out.