How to get useful names from TestFactory tests?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get useful names from TestFactory tests?

Paul Cantrell-3

The TestFactory capability in TestNG is really lovely, but it has one
big problem: it renders the test report useless, by making all the
individual cases indistinguishable.

Here's my situation. I have a test factory which creates about 60
instances of this test class:

public class ConverterOutputTest
    {
    public ConverterOutputTest(
             String name,
             String input,
             String expectedOutput)
        { ... }

    @Test
    public void testConversion()
        { ... }
    }

Each instance is different - each has a different input and
expectedOutput, and a different name. However, when I run my tests
(either from the Eclipse plugin or from my build), I just see a huge
long line of identical test names:

        ConverterOutputTest.testConversion
        ConverterOutputTest.testConversion[2]
        ConverterOutputTest.testConversion[3]
        ConverterOutputTest.testConversion[4]
        ConverterOutputTest.testConversion[5]
        ConverterOutputTest.testConversion[6]
        ConverterOutputTest.testConversion[7]
        ConverterOutputTest.testConversion[8]
        ...etc...

When TestNG tells me that ConverterOutputTest.testConversion[44]
failed, well, that isn't very helpful!

I would really like TestNG to use that "name" parameter in its
reporting, instead of the usual Class.method format. That's what the
name is there for. How can I do that?

I tried implementing ITest, but that had no effect.

I searched the online documentation and the web for info on this, but
couldn't turn anything up. It seems like a very basic question. What
obvious thing am I missing?

Cheers,

Paul

_________________________________________________________________
Piano music podcast: http://inthehands.com
Other interesting stuff: http://innig.net


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀

On 4/16/07, Paul Cantrell <[hidden email]> wrote:

>
> The TestFactory capability in TestNG is really lovely, but it has one
> big problem: it renders the test report useless, by making all the
> individual cases indistinguishable.
>
> Here's my situation. I have a test factory which creates about 60
> instances of this test class:
>
> public class ConverterOutputTest
>     {
>     public ConverterOutputTest(
>              String name,
>              String input,
>              String expectedOutput)
>         { ... }
>
>     @Test
>     public void testConversion()
>         { ... }
>     }
>
> Each instance is different - each has a different input and
> expectedOutput, and a different name. However, when I run my tests
> (either from the Eclipse plugin or from my build), I just see a huge
> long line of identical test names:
>
>         ConverterOutputTest.testConversion
>         ConverterOutputTest.testConversion[2]
>         ConverterOutputTest.testConversion[3]
>         ConverterOutputTest.testConversion[4]
>         ConverterOutputTest.testConversion[5]
>         ConverterOutputTest.testConversion[6]
>         ConverterOutputTest.testConversion[7]
>         ConverterOutputTest.testConversion[8]
>         ...etc...
>
> When TestNG tells me that ConverterOutputTest.testConversion[44]
> failed, well, that isn't very helpful!
>
> I would really like TestNG to use that "name" parameter in its
> reporting, instead of the usual Class.method format. That's what the
> name is there for. How can I do that?
>
> I tried implementing ITest, but that had no effect.
>
> I searched the online documentation and the web for info on this, but
> couldn't turn anything up. It seems like a very basic question. What
> obvious thing am I missing?
>

Hi Paul!

I don't think you are missing anything. I thing this is a problem with
the plugin not being able to use that additional ITest functionality.
Still, I am wondering if you can run the same test (which implements
ITest interface) through the command line/Ant task and let me know if
the reports are more correct in this case. If so, I will try to
improve the plugin to provide more information when reporting.

bests,

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> Cheers,
>
> Paul
>
> _________________________________________________________________
> Piano music podcast: http://inthehands.com
> Other interesting stuff: http://innig.net
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Paul Cantrell-3

Actually, none of the tools seem to use the ITest functionality. I'm
using Maven for the current project (for better or for worse, but I
am), and the XML file it generates looks like this:

  <testcase time="0.009" name="testConversion "/>
  <testcase time="0.019" name="testConversion "/>
  <testcase time="0.019" name="testConversion "/>
  <testcase time="0.019" name="testConversion "/>

When a test fails, neither the text output not the XML contains the
name returned by getTestName().

I have verified that TestNG is calling my getTestName(). Here is the
trace:

        at
net.innig.sweetxml.ConverterTest.getTestName(ConverterTest.java:27)
        at org.testng.internal.TestResult.<init>(TestResult.java:56)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:372)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:
785)
        at
org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
        at org.testng.TestRunner.privateRun(TestRunner.java:693)
        at org.testng.TestRunner.run(TestRunner.java:574)
        at org.testng.SuiteRunner.privateRun(SuiteRunner.java:241)
        at org.testng.SuiteRunner.run(SuiteRunner.java:145)
        at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
        at org.testng.TestNG.runSuitesLocally(TestNG.java:863)

However, TestNG seems not to use that value anywhere after fetching
it.

Cheers, P

_________________________________________________________________
Piano music podcast: http://inthehands.com
Other interesting stuff: http://innig.net


On Apr 16, 4:14 am, "Alexandru Popescu ☀"
<[hidden email]> wrote:

> On 4/16/07, Paul Cantrell <[hidden email]> wrote:
>
>
>
>
>
> > The TestFactory capability in TestNG is really lovely, but it has one
> > big problem: it renders the test report useless, by making all the
> > individual cases indistinguishable.
>
> > Here's my situation. I have a test factory which creates about 60
> > instances of this test class:
>
> > public class ConverterOutputTest
> >     {
> >     public ConverterOutputTest(
> >              String name,
> >              String input,
> >              String expectedOutput)
> >         { ... }
>
> >     @Test
> >     public void testConversion()
> >         { ... }
> >     }
>
> > Each instance is different - each has a different input and
> > expectedOutput, and a different name. However, when I run my tests
> > (either from the Eclipse plugin or from my build), I just see a huge
> > long line of identical test names:
>
> >         ConverterOutputTest.testConversion
> >         ConverterOutputTest.testConversion[2]
> >         ConverterOutputTest.testConversion[3]
> >         ConverterOutputTest.testConversion[4]
> >         ConverterOutputTest.testConversion[5]
> >         ConverterOutputTest.testConversion[6]
> >         ConverterOutputTest.testConversion[7]
> >         ConverterOutputTest.testConversion[8]
> >         ...etc...
>
> > When TestNG tells me that ConverterOutputTest.testConversion[44]
> > failed, well, that isn't very helpful!
>
> > I would really like TestNG to use that "name" parameter in its
> > reporting, instead of the usual Class.method format. That's what the
> > name is there for. How can I do that?
>
> > I tried implementing ITest, but that had no effect.
>
> > I searched the online documentation and the web for info on this, but
> > couldn't turn anything up. It seems like a very basic question. What
> > obvious thing am I missing?
>
> Hi Paul!
>
> I don't think you are missing anything. I thing this is a problem with
> the plugin not being able to use that additional ITest functionality.
> Still, I am wondering if you can run the same test (which implements
> ITest interface) through the command line/Ant task and let me know if
> the reports are more correct in this case. If so, I will try to
> improve the plugin to provide more information when reporting.
>
> bests,
>
> ./alex
> --
> .w( the_mindstorm )p.
>   TestNG co-founder
> EclipseTestNG Creator
>
> > Cheers,
>
> > Paul
>
> > _________________________________________________________________
> > Piano music podcast:http://inthehands.com
> > Other interesting stuff:http://innig.net


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀
Thanks for checking this Paul. I will look into it and see if there is
anything I can do. Do you have any ideas on how the output should look
like? Something like:

ConverterOutputTest<ITest.getTestName()>.testConversion ?

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator


On 4/16/07, Paul Cantrell <[hidden email]> wrote:

>
> Actually, none of the tools seem to use the ITest functionality. I'm
> using Maven for the current project (for better or for worse, but I
> am), and the XML file it generates looks like this:
>
>   <testcase time="0.009" name="testConversion "/>
>   <testcase time="0.019" name="testConversion "/>
>   <testcase time="0.019" name="testConversion "/>
>   <testcase time="0.019" name="testConversion "/>
>
> When a test fails, neither the text output not the XML contains the
> name returned by getTestName().
>
> I have verified that TestNG is calling my getTestName(). Here is the
> trace:
>
>         at
> net.innig.sweetxml.ConverterTest.getTestName(ConverterTest.java:27)
>         at org.testng.internal.TestResult.<init>(TestResult.java:56)
>         at org.testng.internal.Invoker.invokeMethod(Invoker.java:372)
>         at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:
> 785)
>         at
> org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
>         at org.testng.TestRunner.privateRun(TestRunner.java:693)
>         at org.testng.TestRunner.run(TestRunner.java:574)
>         at org.testng.SuiteRunner.privateRun(SuiteRunner.java:241)
>         at org.testng.SuiteRunner.run(SuiteRunner.java:145)
>         at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
>         at org.testng.TestNG.runSuitesLocally(TestNG.java:863)
>
> However, TestNG seems not to use that value anywhere after fetching
> it.
>
> Cheers, P
>
> _________________________________________________________________
> Piano music podcast: http://inthehands.com
> Other interesting stuff: http://innig.net
>
>
> On Apr 16, 4:14am, "Alexandru Popescu ☀"
> <[hidden email]> wrote:
> > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> >
> >
> >
> >
> >
> > > The TestFactory capability in TestNG is really lovely, but it has one
> > > big problem: it renders the test report useless, by making all the
> > > individual cases indistinguishable.
> >
> > > Here's my situation. I have a test factory which creates about 60
> > > instances of this test class:
> >
> > > public class ConverterOutputTest
> > > {
> > > public ConverterOutputTest(
> > > String name,
> > > String input,
> > > String expectedOutput)
> > > { ... }
> >
> > > @Test
> > > public void testConversion()
> > > { ... }
> > > }
> >
> > > Each instance is different - each has a different input and
> > > expectedOutput, and a different name. However, when I run my tests
> > > (either from the Eclipse plugin or from my build), I just see a huge
> > > long line of identical test names:
> >
> > > ConverterOutputTest.testConversion
> > > ConverterOutputTest.testConversion[2]
> > > ConverterOutputTest.testConversion[3]
> > > ConverterOutputTest.testConversion[4]
> > > ConverterOutputTest.testConversion[5]
> > > ConverterOutputTest.testConversion[6]
> > > ConverterOutputTest.testConversion[7]
> > > ConverterOutputTest.testConversion[8]
> > > ...etc...
> >
> > > When TestNG tells me that ConverterOutputTest.testConversion[44]
> > > failed, well, that isn't very helpful!
> >
> > > I would really like TestNG to use that "name" parameter in its
> > > reporting, instead of the usual Class.method format. That's what the
> > > name is there for. How can I do that?
> >
> > > I tried implementing ITest, but that had no effect.
> >
> > > I searched the online documentation and the web for info on this, but
> > > couldn't turn anything up. It seems like a very basic question. What
> > > obvious thing am I missing?
> >
> > Hi Paul!
> >
> > I don't think you are missing anything. I thing this is a problem with
> > the plugin not being able to use that additional ITest functionality.
> > Still, I am wondering if you can run the same test (which implements
> > ITest interface) through the command line/Ant task and let me know if
> > the reports are more correct in this case. If so, I will try to
> > improve the plugin to provide more information when reporting.
> >
> > bests,
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> > TestNG co-founder
> > EclipseTestNG Creator
> >
> > > Cheers,
> >
> > > Paul
> >
> > > _________________________________________________________________
> > > Piano music podcast:http://inthehands.com
> > > Other interesting stuff:http://innig.net
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Paul Cantrell-3

Yes, this would make sense:

    ConverterOutputTest<My custom test name>.testConversion

One could also argue for this:

    My custom test name.testConversion

...because getTestName() can speak for the whole class, and can return
"ConverterOutputTest" as part of its name if it wants to.

I don't have strong feelings either way.

Would you like any help implementing this?

Cheers,

Paul


On Apr 16, 10:50 am, "Alexandru Popescu ☀"
<[hidden email]> wrote:

> Thanks for checking this Paul. I will look into it and see if there is
> anything I can do. Do you have any ideas on how the output should look
> like? Something like:
>
> ConverterOutputTest<ITest.getTestName()>.testConversion ?
>
> ./alex
> --
> .w( the_mindstorm )p.
>   TestNG co-founder
> EclipseTestNG Creator
>
> On 4/16/07, Paul Cantrell <[hidden email]> wrote:
>
>
>
> > Actually, none of the tools seem to use the ITest functionality. I'm
> > using Maven for the current project (for better or for worse, but I
> > am), and the XML file it generates looks like this:
>
> >   <testcase time="0.009" name="testConversion "/>
> >   <testcase time="0.019" name="testConversion "/>
> >   <testcase time="0.019" name="testConversion "/>
> >   <testcase time="0.019" name="testConversion "/>
>
> > When a test fails, neither the text output not the XML contains the
> > name returned by getTestName().
>
> > I have verified that TestNG is calling my getTestName(). Here is the
> > trace:
>
> >         at
> > net.innig.sweetxml.ConverterTest.getTestName(ConverterTest.java:27)
> >         at org.testng.internal.TestResult.<init>(TestResult.java:56)
> >         at org.testng.internal.Invoker.invokeMethod(Invoker.java:372)
> >         at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:
> > 785)
> >         at
> > org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
> >         at org.testng.TestRunner.privateRun(TestRunner.java:693)
> >         at org.testng.TestRunner.run(TestRunner.java:574)
> >         at org.testng.SuiteRunner.privateRun(SuiteRunner.java:241)
> >         at org.testng.SuiteRunner.run(SuiteRunner.java:145)
> >         at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
> >         at org.testng.TestNG.runSuitesLocally(TestNG.java:863)
>
> > However, TestNG seems not to use that value anywhere after fetching
> > it.
>
> > Cheers, P
>
> > _________________________________________________________________
> > Piano music podcast:http://inthehands.com
> > Other interesting stuff:http://innig.net
>
> > On Apr 16, 4:14am, "Alexandru Popescu ☀"
> > <[hidden email]> wrote:
> > > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
>
> > > > The TestFactory capability in TestNG is really lovely, but it has one
> > > > big problem: it renders the test report useless, by making all the
> > > > individual cases indistinguishable.
>
> > > > Here's my situation. I have a test factory which creates about 60
> > > > instances of this test class:
>
> > > > public class ConverterOutputTest
> > > > {
> > > > public ConverterOutputTest(
> > > > String name,
> > > > String input,
> > > > String expectedOutput)
> > > > { ... }
>
> > > > @Test
> > > > public void testConversion()
> > > > { ... }
> > > > }
>
> > > > Each instance is different - each has a different input and
> > > > expectedOutput, and a different name. However, when I run my tests
> > > > (either from the Eclipse plugin or from my build), I just see a huge
> > > > long line of identical test names:
>
> > > > ConverterOutputTest.testConversion
> > > > ConverterOutputTest.testConversion[2]
> > > > ConverterOutputTest.testConversion[3]
> > > > ConverterOutputTest.testConversion[4]
> > > > ConverterOutputTest.testConversion[5]
> > > > ConverterOutputTest.testConversion[6]
> > > > ConverterOutputTest.testConversion[7]
> > > > ConverterOutputTest.testConversion[8]
> > > > ...etc...
>
> > > > When TestNG tells me that ConverterOutputTest.testConversion[44]
> > > > failed, well, that isn't very helpful!
>
> > > > I would really like TestNG to use that "name" parameter in its
> > > > reporting, instead of the usual Class.method format. That's what the
> > > > name is there for. How can I do that?
>
> > > > I tried implementing ITest, but that had no effect.
>
> > > > I searched the online documentation and the web for info on this, but
> > > > couldn't turn anything up. It seems like a very basic question. What
> > > > obvious thing am I missing?
>
> > > Hi Paul!
>
> > > I don't think you are missing anything. I thing this is a problem with
> > > the plugin not being able to use that additional ITest functionality.
> > > Still, I am wondering if you can run the same test (which implements
> > > ITest interface) through the command line/Ant task and let me know if
> > > the reports are more correct in this case. If so, I will try to
> > > improve the plugin to provide more information when reporting.
>
> > > bests,
>
> > > ./alex
> > > --
> > > .w( the_mindstorm )p.
> > > TestNG co-founder
> > > EclipseTestNG Creator
>
> > > > Cheers,
>
> > > > Paul
>
> > > > _________________________________________________________________
> > > > Piano music podcast:http://inthehands.com
> > > > Other interesting stuff:http://innig.net


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀
On 4/16/07, Paul Cantrell <[hidden email]> wrote:

>
> Yes, this would make sense:
>
>     ConverterOutputTest<My custom test name>.testConversion
>
> One could also argue for this:
>
>     My custom test name.testConversion
>
> ...because getTestName() can speak for the whole class, and can return
> "ConverterOutputTest" as part of its name if it wants to.
>
> I don't have strong feelings either way.
>
> Would you like any help implementing this?
>


Well, by checking our code it looks ITest.getTestName() is used in
quite a few places, so I am starting to think that the problem you are
seeing is in fact caused by the fact that Maven plugin disables all
TestNG core reporters and uses its own one.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator


> Cheers,
>
> Paul
>
>
> On Apr 16, 10:50am, "Alexandru Popescu ☀"
> <[hidden email]> wrote:
> > Thanks for checking this Paul. I will look into it and see if there is
> > anything I can do. Do you have any ideas on how the output should look
> > like? Something like:
> >
> > ConverterOutputTest<ITest.getTestName()>.testConversion ?
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> > TestNG co-founder
> > EclipseTestNG Creator
> >
> > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> >
> >
> >
> > > Actually, none of the tools seem to use the ITest functionality. I'm
> > > using Maven for the current project (for better or for worse, but I
> > > am), and the XML file it generates looks like this:
> >
> > > <testcase time="0.009" name="testConversion "/>
> > > <testcase time="0.019" name="testConversion "/>
> > > <testcase time="0.019" name="testConversion "/>
> > > <testcase time="0.019" name="testConversion "/>
> >
> > > When a test fails, neither the text output not the XML contains the
> > > name returned by getTestName().
> >
> > > I have verified that TestNG is calling my getTestName(). Here is the
> > > trace:
> >
> > > at
> > > net.innig.sweetxml.ConverterTest.getTestName(ConverterTest.java:27)
> > > at org.testng.internal.TestResult.<init>(TestResult.java:56)
> > > at org.testng.internal.Invoker.invokeMethod(Invoker.java:372)
> > > at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:
> > > 785)
> > > at
> > > org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
> > > at org.testng.TestRunner.privateRun(TestRunner.java:693)
> > > at org.testng.TestRunner.run(TestRunner.java:574)
> > > at org.testng.SuiteRunner.privateRun(SuiteRunner.java:241)
> > > at org.testng.SuiteRunner.run(SuiteRunner.java:145)
> > > at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
> > > at org.testng.TestNG.runSuitesLocally(TestNG.java:863)
> >
> > > However, TestNG seems not to use that value anywhere after fetching
> > > it.
> >
> > > Cheers, P
> >
> > > _________________________________________________________________
> > > Piano music podcast:http://inthehands.com
> > > Other interesting stuff:http://innig.net
> >
> > > On Apr 16, 4:14am, "Alexandru Popescu ☀"
> > > <[hidden email]> wrote:
> > > > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> >
> > > > > The TestFactory capability in TestNG is really lovely, but it has one
> > > > > big problem: it renders the test report useless, by making all the
> > > > > individual cases indistinguishable.
> >
> > > > > Here's my situation. I have a test factory which creates about 60
> > > > > instances of this test class:
> >
> > > > > public class ConverterOutputTest
> > > > > {
> > > > > public ConverterOutputTest(
> > > > > String name,
> > > > > String input,
> > > > > String expectedOutput)
> > > > > { ... }
> >
> > > > > @Test
> > > > > public void testConversion()
> > > > > { ... }
> > > > > }
> >
> > > > > Each instance is different - each has a different input and
> > > > > expectedOutput, and a different name. However, when I run my tests
> > > > > (either from the Eclipse plugin or from my build), I just see a huge
> > > > > long line of identical test names:
> >
> > > > > ConverterOutputTest.testConversion
> > > > > ConverterOutputTest.testConversion[2]
> > > > > ConverterOutputTest.testConversion[3]
> > > > > ConverterOutputTest.testConversion[4]
> > > > > ConverterOutputTest.testConversion[5]
> > > > > ConverterOutputTest.testConversion[6]
> > > > > ConverterOutputTest.testConversion[7]
> > > > > ConverterOutputTest.testConversion[8]
> > > > > ...etc...
> >
> > > > > When TestNG tells me that ConverterOutputTest.testConversion[44]
> > > > > failed, well, that isn't very helpful!
> >
> > > > > I would really like TestNG to use that "name" parameter in its
> > > > > reporting, instead of the usual Class.method format. That's what the
> > > > > name is there for. How can I do that?
> >
> > > > > I tried implementing ITest, but that had no effect.
> >
> > > > > I searched the online documentation and the web for info on this, but
> > > > > couldn't turn anything up. It seems like a very basic question. What
> > > > > obvious thing am I missing?
> >
> > > > Hi Paul!
> >
> > > > I don't think you are missing anything. I thing this is a problem with
> > > > the plugin not being able to use that additional ITest functionality.
> > > > Still, I am wondering if you can run the same test (which implements
> > > > ITest interface) through the command line/Ant task and let me know if
> > > > the reports are more correct in this case. If so, I will try to
> > > > improve the plugin to provide more information when reporting.
> >
> > > > bests,
> >
> > > > ./alex
> > > > --
> > > > .w( the_mindstorm )p.
> > > > TestNG co-founder
> > > > EclipseTestNG Creator
> >
> > > > > Cheers,
> >
> > > > > Paul
> >
> > > > > _________________________________________________________________
> > > > > Piano music podcast:http://inthehands.com
> > > > > Other interesting stuff:http://innig.net
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀
On 4/16/07, Alexandru Popescu ☀ <[hidden email]> wrote:

> On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> >
> > Yes, this would make sense:
> >
> >     ConverterOutputTest<My custom test name>.testConversion
> >
> > One could also argue for this:
> >
> >     My custom test name.testConversion
> >
> > ...because getTestName() can speak for the whole class, and can return
> > "ConverterOutputTest" as part of its name if it wants to.
> >
> > I don't have strong feelings either way.
> >
> > Would you like any help implementing this?
> >
>
>
> Well, by checking our code it looks ITest.getTestName() is used in
> quite a few places, so I am starting to think that the problem you are
> seeing is in fact caused by the fact that Maven plugin disables all
> TestNG core reporters and uses its own one.
>

I have found a couple of reporters that are not using that
information. I will look what can be  improved there.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> ./alex
> --
> .w( the_mindstorm )p.
>   TestNG co-founder
> EclipseTestNG Creator
>
>
> > Cheers,
> >
> > Paul
> >
> >
> > On Apr 16, 10:50am, "Alexandru Popescu ☀"
> > <[hidden email]> wrote:
> > > Thanks for checking this Paul. I will look into it and see if there is
> > > anything I can do. Do you have any ideas on how the output should look
> > > like? Something like:
> > >
> > > ConverterOutputTest<ITest.getTestName()>.testConversion ?
> > >
> > > ./alex
> > > --
> > > .w( the_mindstorm )p.
> > > TestNG co-founder
> > > EclipseTestNG Creator
> > >
> > > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> > >
> > >
> > >
> > > > Actually, none of the tools seem to use the ITest functionality. I'm
> > > > using Maven for the current project (for better or for worse, but I
> > > > am), and the XML file it generates looks like this:
> > >
> > > > <testcase time="0.009" name="testConversion "/>
> > > > <testcase time="0.019" name="testConversion "/>
> > > > <testcase time="0.019" name="testConversion "/>
> > > > <testcase time="0.019" name="testConversion "/>
> > >
> > > > When a test fails, neither the text output not the XML contains the
> > > > name returned by getTestName().
> > >
> > > > I have verified that TestNG is calling my getTestName(). Here is the
> > > > trace:
> > >
> > > > at
> > > > net.innig.sweetxml.ConverterTest.getTestName(ConverterTest.java:27)
> > > > at org.testng.internal.TestResult.<init>(TestResult.java:56)
> > > > at org.testng.internal.Invoker.invokeMethod(Invoker.java:372)
> > > > at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:
> > > > 785)
> > > > at
> > > > org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
> > > > at org.testng.TestRunner.privateRun(TestRunner.java:693)
> > > > at org.testng.TestRunner.run(TestRunner.java:574)
> > > > at org.testng.SuiteRunner.privateRun(SuiteRunner.java:241)
> > > > at org.testng.SuiteRunner.run(SuiteRunner.java:145)
> > > > at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
> > > > at org.testng.TestNG.runSuitesLocally(TestNG.java:863)
> > >
> > > > However, TestNG seems not to use that value anywhere after fetching
> > > > it.
> > >
> > > > Cheers, P
> > >
> > > > _________________________________________________________________
> > > > Piano music podcast:http://inthehands.com
> > > > Other interesting stuff:http://innig.net
> > >
> > > > On Apr 16, 4:14am, "Alexandru Popescu ☀"
> > > > <[hidden email]> wrote:
> > > > > On 4/16/07, Paul Cantrell <[hidden email]> wrote:
> > >
> > > > > > The TestFactory capability in TestNG is really lovely, but it has one
> > > > > > big problem: it renders the test report useless, by making all the
> > > > > > individual cases indistinguishable.
> > >
> > > > > > Here's my situation. I have a test factory which creates about 60
> > > > > > instances of this test class:
> > >
> > > > > > public class ConverterOutputTest
> > > > > > {
> > > > > > public ConverterOutputTest(
> > > > > > String name,
> > > > > > String input,
> > > > > > String expectedOutput)
> > > > > > { ... }
> > >
> > > > > > @Test
> > > > > > public void testConversion()
> > > > > > { ... }
> > > > > > }
> > >
> > > > > > Each instance is different - each has a different input and
> > > > > > expectedOutput, and a different name. However, when I run my tests
> > > > > > (either from the Eclipse plugin or from my build), I just see a huge
> > > > > > long line of identical test names:
> > >
> > > > > > ConverterOutputTest.testConversion
> > > > > > ConverterOutputTest.testConversion[2]
> > > > > > ConverterOutputTest.testConversion[3]
> > > > > > ConverterOutputTest.testConversion[4]
> > > > > > ConverterOutputTest.testConversion[5]
> > > > > > ConverterOutputTest.testConversion[6]
> > > > > > ConverterOutputTest.testConversion[7]
> > > > > > ConverterOutputTest.testConversion[8]
> > > > > > ...etc...
> > >
> > > > > > When TestNG tells me that ConverterOutputTest.testConversion[44]
> > > > > > failed, well, that isn't very helpful!
> > >
> > > > > > I would really like TestNG to use that "name" parameter in its
> > > > > > reporting, instead of the usual Class.method format. That's what the
> > > > > > name is there for. How can I do that?
> > >
> > > > > > I tried implementing ITest, but that had no effect.
> > >
> > > > > > I searched the online documentation and the web for info on this, but
> > > > > > couldn't turn anything up. It seems like a very basic question. What
> > > > > > obvious thing am I missing?
> > >
> > > > > Hi Paul!
> > >
> > > > > I don't think you are missing anything. I thing this is a problem with
> > > > > the plugin not being able to use that additional ITest functionality.
> > > > > Still, I am wondering if you can run the same test (which implements
> > > > > ITest interface) through the command line/Ant task and let me know if
> > > > > the reports are more correct in this case. If so, I will try to
> > > > > improve the plugin to provide more information when reporting.
> > >
> > > > > bests,
> > >
> > > > > ./alex
> > > > > --
> > > > > .w( the_mindstorm )p.
> > > > > TestNG co-founder
> > > > > EclipseTestNG Creator
> > >
> > > > > > Cheers,
> > >
> > > > > > Paul
> > >
> > > > > > _________________________________________________________________
> > > > > > Piano music podcast:http://inthehands.com
> > > > > > Other interesting stuff:http://innig.net
> >
> >
> > > >
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Paul Cantrell-3
In reply to this post by Alexandru Popescu ☀

> Well, by checking our code it looks ITest.getTestName() is used in
> quite a few places, so I am starting to think that the problem you are
> seeing is in fact caused by the fact that Maven plugin disables all
> TestNG core reporters and uses its own one.

Well, it happens both with Maven *and* with the Eclipse plugin,
neither one of which relies on the other. I care more about the
Eclipse than than Maven, but both matter.

Cheers, P


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀

On 4/17/07, Paul Cantrell <[hidden email]> wrote:

>
> > Well, by checking our code it looks ITest.getTestName() is used in
> > quite a few places, so I am starting to think that the problem you are
> > seeing is in fact caused by the fact that Maven plugin disables all
> > TestNG core reporters and uses its own one.
>
> Well, it happens both with Maven *and* with the Eclipse plugin,
> neither one of which relies on the other. I care more about the
> Eclipse than than Maven, but both matter.
>

Yep, Eclipse is one of those that is not using the information :-(.
Now, I need to further investigate how this can be solved (the real
problem been the fact that not only the IDE relies on this code, but
all IDE support, and so changing the wiring "protocol" is not so
simple -- sometimes when considering backward compatibility even the
simplest ideas become really complex to be put in practice).

I'll keep you informed,

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> Cheers, P
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: How to get useful names from TestFactory tests?

Alexandru Popescu ☀
On 4/17/07, Alexandru Popescu ☀ <[hidden email]> wrote:

> On 4/17/07, Paul Cantrell <[hidden email]> wrote:
> >
> > > Well, by checking our code it looks ITest.getTestName() is used in
> > > quite a few places, so I am starting to think that the problem you are
> > > seeing is in fact caused by the fact that Maven plugin disables all
> > > TestNG core reporters and uses its own one.
> >
> > Well, it happens both with Maven *and* with the Eclipse plugin,
> > neither one of which relies on the other. I care more about the
> > Eclipse than than Maven, but both matter.
> >
>
> Yep, Eclipse is one of those that is not using the information :-(.
> Now, I need to further investigate how this can be solved (the real
> problem been the fact that not only the IDE relies on this code, but
> all IDE support, and so changing the wiring "protocol" is not so
> simple -- sometimes when considering backward compatibility even the
> simplest ideas become really complex to be put in practice).
>
> I'll keep you informed,
>

Paul I have committed changes that will allow passing the
ITest.getTestName() information to the Eclipse plugin (and for some
cases like the @Factory scenario it will be displayed). I still need
to make it work for other scenarios, but for the first phase I would
say it is good enough :-).

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> ./alex
> --
> .w( the_mindstorm )p.
>   TestNG co-founder
> EclipseTestNG Creator
>
> > Cheers, P
> >
> >
> > > >
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---