Parallel execution not working

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Parallel execution not working

thirsch.epages
Hi, 

I have used TestNG in the past, mostly through Gradle.

Now I took over a legacy project, which runs TestNG programmatically.

This is the part where TestNG gets configured:

    private TestNG getConfiguredTestNG(final Parameters parameters) throws IOException, URISyntaxException {


        TestNG testNg = new TestNG();


        // Implement our quarantine mechanism.

        testNg.setAnnotationTransformer(new AnnotationTransformer());


        // prevents skipping of test cases if @Before-/@AfterClass methods throwing exception

        testNg.setConfigFailurePolicy(XmlSuite.CONTINUE);


        if (parameters.getParallelMode() != null) {

            configureParallelism(testNg, parameters);

        } else {

            // tests are grouped and executed by test class instance

            testNg.setGroupByInstances(true);

        }

        

        // Set test classes

        File[] testSuite = getTestSuite(parameters);

        Class<?>[] testClasses = getTestClasses(parameters.getTestClasses(), testSuite);

        testNg.setTestClasses(testClasses);

        

        // Set directory for TestNG output

        testNg.setOutputDirectory(getTestNgOutputDir());

        

        // The list of test groups the user wants to start.

        final String testGroups = parameters.getTestGroups();

        // The list of test groups the user wants to exclude.

        final String excludeGroups = parameters.getExcludedGroups();


        // By default all test groups are about to be started, if the user did not request specific groups.

        if (testGroups == null) {

            // Exclude those test groups which should not run by default as per specification.

            final String excludedGroups = getExcludedGroups(excludeGroups);

            testNg.setExcludedGroups(excludedGroups);

        } else {

            // Run only selected test groups.

            testNg.setGroups(testGroups);


            // Exclude those test groups which should not run by default as per specification.

            final String excludedGroups = getExcludedGroups(testGroups, excludeGroups);

            testNg.setExcludedGroups(excludedGroups);

        }


        return testNg;

    }



with:

    private TestNG configureParallelism(TestNG testNg, Parameters parameters) {

        setParallelMode(testNg, parameters);

        setThreadCount(testNg, parameters);

        return testNg;

    }


    private TestNG setParallelMode(TestNG testNg, EsfParameters esfParameters) {

        String mode = esfParameters.getParallelMode();

        switch(mode) {

            case "methods":

                System.out.println("OK");

                testNg.setParallel(ParallelMode.METHODS);

                break;

            case "tests":

                testNg.setParallel(ParallelMode.TESTS);

                break;

            case "classes":

                testNg.setParallel(ParallelMode.CLASSES);

                break;

            case "instances":

                testNg.setParallel(ParallelMode.INSTANCES);

                break;

            default:

                throw new RuntimeException(String.format("Parallelism setting '%s' not recognized, see TestNG docs.", mode));

        }

        return testNg;

    }

    

    private TestNG setThreadCount(TestNG testNg, EsfParameters esfParameters) {

        Integer count = esfParameters.getThreadCount();

        if (count != null && count > 0) {

            testNg.setThreadCount(count);

        }

        return testNg;

    }



The second bit I added to configure parallelism.
But when this code is run, TestNG still only runs a single test at a time, no matter how many test classes get passed to it.
I also verified that  m_useThreadCount and related variables have the expected values before run() gets called

How to debug this? What can I do to get more insight about why TestNG only runs one method at a time?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Parallel execution not working

⇜Krishnan Mahadevan⇝

How are you ascertaining that your tests are not running in parallel ? Have you tried printing the thread id to confirm this ?

 

Here’s a sample :

 

import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestclassSample {
   
@Test
   
public void a() {
        prettyPrint();
    }

   
@Test
   
public void b() {
        prettyPrint();
    }

   
@Test
   
public void c() {
        prettyPrint();
    }

   
@Test
   
public void d() {
        prettyPrint();
    }

   
private static void prettyPrint() {
        ITestResult result = Reporter.getCurrentTestResult();
        String msg =
"Running " + result.getMethod().getMethodName() + "() running on [" +
                Thread.currentThread().getId() +
"]";
        System.
err.println(msg);
    }
}

 

 

import org.testng.TestNG;
import org.testng.annotations.Test;
import org.testng.xml.XmlSuite;

public class TestclassTestRunner {
   
@Test
   
public void testMethod() {
        TestNG testng =
new TestNG();
        testng.setTestClasses(
new Class[]{TestclassSample.class});
        testng.setThreadCount(
2);
        testng.setParallel(XmlSuite.ParallelMode.
METHODS);
        testng.setVerbose(
2);
        testng.run();
    }
}

 

Output:

 

Running a() running on [11]

Running b() running on [12]

Running c() running on [11]

Running d() running on [12]

PASSED: a

PASSED: b

PASSED: c

PASSED: d

 

===============================================

    Command line test

    Tests run: 4, Failures: 0, Skips: 0

===============================================

 

 

===============================================

Command line suite

Total tests run: 4, Failures: 0, Skips: 0

===============================================

 

 

===============================================

Default Suite

Total tests run: 1, Failures: 0, Skips: 0

===============================================

 

 

Process finished with exit code 0

 

 

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <[hidden email]> on behalf of <[hidden email]>
Reply-To: <[hidden email]>
Date: Wednesday, September 27, 2017 at 9:32 PM
To: testng-users <[hidden email]>
Subject: [testng-users] Parallel execution not working

 

Hi, 

I have used TestNG in the past, mostly through Gradle.

Now I took over a legacy project, which runs TestNG programmatically.

This is the part where TestNG gets configured:

    private TestNG getConfiguredTestNG(final Parameters parameters) throws IOException, URISyntaxException {

 

        TestNG testNg = new TestNG();

 

        // Implement our quarantine mechanism.

        testNg.setAnnotationTransformer(new AnnotationTransformer());

 

        // prevents skipping of test cases if @Before-/@AfterClass methods throwing exception

        testNg.setConfigFailurePolicy(XmlSuite.CONTINUE);

 

        if (parameters.getParallelMode() != null) {

            configureParallelism(testNg, parameters);

        } else {

            // tests are grouped and executed by test class instance

            testNg.setGroupByInstances(true);

        }

        

        // Set test classes

        File[] testSuite = getTestSuite(parameters);

        Class<?>[] testClasses = getTestClasses(parameters.getTestClasses(), testSuite);

        testNg.setTestClasses(testClasses);

        

        // Set directory for TestNG output

        testNg.setOutputDirectory(getTestNgOutputDir());

        

        // The list of test groups the user wants to start.

        final String testGroups = parameters.getTestGroups();

        // The list of test groups the user wants to exclude.

        final String excludeGroups = parameters.getExcludedGroups();

 

        // By default all test groups are about to be started, if the user did not request specific groups.

        if (testGroups == null) {

            // Exclude those test groups which should not run by default as per specification.

            final String excludedGroups = getExcludedGroups(excludeGroups);

            testNg.setExcludedGroups(excludedGroups);

        } else {

            // Run only selected test groups.

            testNg.setGroups(testGroups);

 

            // Exclude those test groups which should not run by default as per specification.

            final String excludedGroups = getExcludedGroups(testGroups, excludeGroups);

            testNg.setExcludedGroups(excludedGroups);

        }

 

        return testNg;

    }



with:

    private TestNG configureParallelism(TestNG testNg, Parameters parameters) {

        setParallelMode(testNg, parameters);

        setThreadCount(testNg, parameters);

        return testNg;

    }

 

    private TestNG setParallelMode(TestNG testNg, EsfParameters esfParameters) {

        String mode = esfParameters.getParallelMode();

        switch(mode) {

            case "methods":

                System.out.println("OK");

                testNg.setParallel(ParallelMode.METHODS);

                break;

            case "tests":

                testNg.setParallel(ParallelMode.TESTS);

                break;

            case "classes":

                testNg.setParallel(ParallelMode.CLASSES);

                break;

            case "instances":

                testNg.setParallel(ParallelMode.INSTANCES);

                break;

            default:

                throw new RuntimeException(String.format("Parallelism setting '%s' not recognized, see TestNG docs.", mode));

        }

        return testNg;

    }

    

    private TestNG setThreadCount(TestNG testNg, EsfParameters esfParameters) {

        Integer count = esfParameters.getThreadCount();

        if (count != null && count > 0) {

            testNg.setThreadCount(count);

        }

        return testNg;

    }



The second bit I added to configure parallelism.
But when this code is run, TestNG still only runs a single test at a time, no matter how many test classes get passed to it.
I also verified that  m_useThreadCount and related variables have the expected values before run() gets called

How to debug this? What can I do to get more insight about why TestNG only runs one method at a time?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Parallel execution not working

thirsch.epages
Hello Krishnan, thanks for your response.
Is there any way I can see standard out or standard err in the TestNG output?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Parallel execution not working

thirsch.epages
Ah, nevermind, it seems that using stdout instead of stderr makes the logs appear in the TestNG output.

I can see now, that multiple threads are indeed started, when I run your Example test.

The problem lies in the way TestNG was implemented in this context.
There is some code somewhere that prevents normal Tests from running in parallel, it seems, however it does that...
At least I know now that TestNG is not broken!

Thanks and regards, 
Thomas

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.