Filtering test groups - Maven 2, surefire and testng.xml

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Filtering test groups - Maven 2, surefire and testng.xml

Erez Nahir

Hi,

We are using maven2, surefire and testng. Each test method is grouped
in one or more groups ("checkin", "integration", "feature",...).

I'm looking for a way to filter running groups based on the environment
running the test (developers run "checkin", integration server running
"checkin" and "integration", and some feature dedicated server run
"feature").
Here is the maven plug-in:
<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-surefire-plugin</artifactId>
         <configuration>
           <testFailureIgnore>true</testFailureIgnore>
           <suiteXmlFiles>
             <suiteXmlFile>testng.xml</suiteXmlFile>
           </suiteXmlFiles>
       </configuration>
</plugin>
In my testng.xml I have:
<test verbose="1" name="mgmt" annotations="JDK">
    <groups>
      <run>
        <include name="checkin"/>
      </run>
    </groups>
    <classes>

I wonder if there is a way to replace "checkin" with some kind of
runtime variable that will contain the set of groups to run.

Thanks,
Erez.


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Cédric Beust ♔
Hi Erez,

I can think of a couple of way to achieve this, but the easiest is probably using Annotation Transformers.

You could write your own transformer that looks up the environment variable and then transforms all the groups() annotations to being equal to the group you want to run.

Let me know if you'd like more details.

--
Cedric


On 12/16/06, Erez Nahir <[hidden email]> wrote:

Hi,

We are using maven2, surefire and testng. Each test method is grouped
in one or more groups ("checkin", "integration", "feature",...).

I'm looking for a way to filter running groups based on the environment
running the test (developers run "checkin", integration server running
"checkin" and "integration", and some feature dedicated server run
"feature").
Here is the maven plug-in:
<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-surefire-plugin</artifactId>
         <configuration>
           <testFailureIgnore>true</testFailureIgnore>
           <suiteXmlFiles>
             <suiteXmlFile> testng.xml</suiteXmlFile>
           </suiteXmlFiles>
       </configuration>
</plugin>
In my testng.xml I have:
<test verbose="1" name="mgmt" annotations="JDK">
    <groups>
      <run>
        <include name="checkin"/>
      </run>
    </groups>
    <classes>

I wonder if there is a way to replace "checkin" with some kind of
runtime variable that will contain the set of groups to run.

Thanks,
Erez.







--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Erez Nahir

Hi Cedric,

Thanks for the prompt reply, yes, this sounds like a nice idea,
additional details will be great.

Erez.

On Dec 16, 9:55 pm, "Cédric Beust ♔ " <[hidden email]> wrote:

> Hi Erez,
>
> I can think of a couple of way to achieve this, but the easiest is probably
> using Annotation Transformers.
>
> You could write your own transformer that looks up the environment variable
> and then transforms all the groups() annotations to being equal to the group
> you want to run.
>
> Let me know if you'd like more details.
>
> --
> Cedric
>
> On 12/16/06, Erez Nahir <[hidden email]> wrote:
>
>
>
>
>
> > Hi,
>
> > We are using maven2, surefire and testng. Each test method is grouped
> > in one or more groups ("checkin", "integration", "feature",...).
>
> > I'm looking for a way to filter running groups based on the environment
> > running the test (developers run "checkin", integration server running
> > "checkin" and "integration", and some feature dedicated server run
> > "feature").
> > Here is the maven plug-in:
> > <plugin>
> >        <groupId>org.apache.maven.plugins</groupId>
> >        <artifactId>maven-surefire-plugin</artifactId>
> >          <configuration>
> >            <testFailureIgnore>true</testFailureIgnore>
> >            <suiteXmlFiles>
> >              <suiteXmlFile>testng.xml</suiteXmlFile>
> >            </suiteXmlFiles>
> >        </configuration>
> > </plugin>
> > In my testng.xml I have:
> > <test verbose="1" name="mgmt" annotations="JDK">
> >     <groups>
> >       <run>
> >         <include name="checkin"/>
> >       </run>
> >     </groups>
> >     <classes>
>
> > I wonder if there is a way to replace "checkin" with some kind of
> > runtime variable that will contain the set of groups to run.
>
> > Thanks,
> > Erez.--
> Cédric


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Cédric Beust ♔


On 12/16/06, Erez Nahir <[hidden email]> wrote:

Hi Cedric,

Thanks for the prompt reply, yes, this sounds like a nice idea,
additional details will be great.

First of all, let me try to talk you out of doing it :-)

If I understood correctly, you want developers in the various groups of your organization to run the "right" tests automatically.  But at the very least, they will have to set an environment variable (say MY_GROUPS="integration").

I'm not sure this is very different from telling each of them how they are supposed to invoke the right tests, or adding a target for them to protect them from the TestNG details (so they can type "ant integration").

You could also suggest them to have an alias, or even write a little script for each of them.

If you want to keep going in this direction, please take a look at the documentation on annotation transformers and then let me know what additional information you need to get started.

--
Cedric

 

Erez.

On Dec 16, 9:55 pm, "Cédric Beust ♔ " <[hidden email]> wrote:

> Hi Erez,
>
> I can think of a couple of way to achieve this, but the easiest is probably
> using Annotation Transformers.
>
> You could write your own transformer that looks up the environment variable
> and then transforms all the groups() annotations to being equal to the group
> you want to run.
>
> Let me know if you'd like more details.
>
> --
> Cedric
>
> On 12/16/06, Erez Nahir <[hidden email]> wrote:
>
>
>

>
>
> > Hi,
>
> > We are using maven2, surefire and testng. Each test method is grouped
> > in one or more groups ("checkin", "integration", "feature",...).
>
> > I'm looking for a way to filter running groups based on the environment
> > running the test (developers run "checkin", integration server running
> > "checkin" and "integration", and some feature dedicated server run
> > "feature").
> > Here is the maven plug-in:
> > <plugin>
> >        <groupId>org.apache.maven.plugins</groupId>
> >        <artifactId>maven-surefire-plugin</artifactId>
> >          <configuration>
> >            <testFailureIgnore>true</testFailureIgnore>
> >            <suiteXmlFiles>
> >              <suiteXmlFile>testng.xml </suiteXmlFile>
> >            </suiteXmlFiles>
> >        </configuration>
> > </plugin>
> > In my testng.xml I have:
> > <test verbose="1" name="mgmt" annotations="JDK">
> >     <groups>
> >       <run>
> >         <include name="checkin"/>
> >       </run>
> >     </groups>
> >     <classes>
>
> > I wonder if there is a way to replace "checkin" with some kind of
> > runtime variable that will contain the set of groups to run.
>
> > Thanks,
> > Erez.--
> Cédric





--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Erez Nahir

The 'problem' I'm trying to solve is how to make it transparent to the
developers.
By default, if they run mvn test it should run only the "checkin" tests
group, on the integration environment, which is a dedicated environment
with specific setup (including 3rd party servers, simulators etc) I
would like to have a way to invoke also the "integration" tests.

Looking at the Annotation Transformers link, I'm also not sure this is
how I would like to do it (my major concern is how to integrate it with
maven surefire plug-in).

Do you have another idea?

Thanks,
Erez.

On Dec 16, 10:37 pm, "Cédric Beust ♔ " <[hidden email]> wrote:

> On 12/16/06, Erez Nahir <[hidden email]> wrote:
>
>
>
> > Hi Cedric,
>
> > Thanks for the prompt reply, yes, this sounds like a nice idea,
> > additional details will be great.First of all, let me try to talk you out of doing it :-)
>
> If I understood correctly, you want developers in the various groups of your
> organization to run the "right" tests automatically.  But at the very least,
> they will have to set an environment variable (say MY_GROUPS="integration").
>
> I'm not sure this is very different from telling each of them how they are
> supposed to invoke the right tests, or adding a target for them to protect
> them from the TestNG details (so they can type "ant integration").
>
> You could also suggest them to have an alias, or even write a little script
> for each of them.
>
> If you want to keep going in this direction, please take a look at the
> documentation
> on annotation transformers<http://testng.org/doc/documentation-main.html#annotationtransformers>and
> then let me know what additional information you need to get started.
>
> --
> Cedric
>
> Erez.
>
>
>
>
>
> > On Dec 16, 9:55 pm, "Cédric Beust ♔ " <[hidden email]> wrote:
> > > Hi Erez,
>
> > > I can think of a couple of way to achieve this, but the easiest is
> > probably
> > > using Annotation Transformers.
>
> > > You could write your own transformer that looks up the environment
> > variable
> > > and then transforms all the groups() annotations to being equal to the
> > group
> > > you want to run.
>
> > > Let me know if you'd like more details.
>
> > > --
> > > Cedric
>
> > > On 12/16/06, Erez Nahir <[hidden email]> wrote:
>
> > > > Hi,
>
> > > > We are using maven2, surefire and testng. Each test method is grouped
> > > > in one or more groups ("checkin", "integration", "feature",...).
>
> > > > I'm looking for a way to filter running groups based on the
> > environment
> > > > running the test (developers run "checkin", integration server running
> > > > "checkin" and "integration", and some feature dedicated server run
> > > > "feature").
> > > > Here is the maven plug-in:
> > > > <plugin>
> > > >        <groupId>org.apache.maven.plugins</groupId>
> > > >        <artifactId>maven-surefire-plugin</artifactId>
> > > >          <configuration>
> > > >            <testFailureIgnore>true</testFailureIgnore>
> > > >            <suiteXmlFiles>
> > > >              <suiteXmlFile>testng.xml</suiteXmlFile>
> > > >            </suiteXmlFiles>
> > > >        </configuration>
> > > > </plugin>
> > > > In my testng.xml I have:
> > > > <test verbose="1" name="mgmt" annotations="JDK">
> > > >     <groups>
> > > >       <run>
> > > >         <include name="checkin"/>
> > > >       </run>
> > > >     </groups>
> > > >     <classes>
>
> > > > I wonder if there is a way to replace "checkin" with some kind of
> > > > runtime variable that will contain the set of groups to run.
>
> > > > Thanks,
> > > > Erez.--
> > > Cédric--
> Cédric


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Thomas.M.Stahl@googlemail.com

I don't know much about Maven. But our strategy using Ant should be
possible with Maven, too:

We use a TestNG suite file as a *template*, which is dynamically
converted by XSLT to the real, running TestNG suite. Ant command line
parameters (or property files) control which test suite is generated.

Applied to your case, you could generate a "checkin suite" by default
for your developers. Your integration server would simply call "ant
integration" to generate a test suite for integration testing
(including the checkin tests).

This technique is very flexible: we use it to apply our tests to
different servers, selecting different groups, different test packages
and so on. Since the suite file is plain XML, anything can be generated
from it.

If you want, I could post our xslt script and the ant task (but both
are rather trivial).

Thomas - www.playchess.de


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Filtering test groups - Maven 2, surefire and testng.xml

Erez Nahir

Hi Thomas,

Please share your xslt script, it will give me more understanding of
how you do it.
How do you map tests to a method? Do you use TestNG's annotations or
with naming conventions?

Thanks,
Erez.


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---