TestNG tests failures with JDK6

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

TestNG tests failures with JDK6

Alexandru Popescu ☀

Hi!

There is some method ordering problem I am seeing in Regression2 test
and Test order invocations and it is visible only with JDK6.

It looks like we have a couple of tests that depend on a fixed
ordering of unrelated methods
and between JDK5 and JDK6 there are some changes in the HashMap
implementation so that Map.values() returns differently this resulting
in failing tests.

I am not sure yet if there is a issue for TestNG or is just a problem
with our tests. I will further investigate.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Cédric Beust ♔
Interesting, keep us posted.

--
Cedric


On 12/12/06, Alexandru Popescu <[hidden email]> wrote:

Hi!

There is some method ordering problem I am seeing in Regression2 test
and Test order invocations and it is visible only with JDK6.

It looks like we have a couple of tests that depend on a fixed
ordering of unrelated methods
and between JDK5 and JDK6 there are some changes in the HashMap
implementation so that Map.values() returns differently this resulting
in failing tests.

I am not sure yet if there is a issue for TestNG or is just a problem
with our tests. I will further investigate.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator






--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Alexandru Popescu ☀
On 12/13/06, Cédric Beust ♔ <[hidden email]> wrote:
> Interesting, keep us posted.
>

Now, I am sure that it is something in the way HashMap-s are implemented:

[code]
public class HashMapValuesOrderTest {
   public static void main(String[] argv) {
        HashMap map= new HashMap();
        map.put("1", "1");
        map.put("2", "2");
        map.put("2.1", "2.1");
        map.put("2.2", "2.2");
        map.put("3", "3");
        map.put("abc", "abc");
        System.out.println("Values: " + map.values());
   }
}
[/code]

yields different results under JDK15.0_09 and JDK6 (agreed there
should not be a guaranteed order there). The only think I must check
is that our logic doesn't depend on a specific order and same for the
tests.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> --
> Cedric
>
>
>
> On 12/12/06, Alexandru Popescu
> <[hidden email] > wrote:
> >
> > Hi!
> >
> > There is some method ordering problem I am seeing in Regression2 test
> > and Test order invocations and it is visible only with JDK6.
> >
> > It looks like we have a couple of tests that depend on a fixed
> > ordering of unrelated methods
> > and between JDK5 and JDK6 there are some changes in the HashMap
> > implementation so that Map.values() returns differently this resulting
> > in failing tests.
> >
> > I am not sure yet if there is a issue for TestNG or is just a problem
> > with our tests. I will further investigate.
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >   TestNG co-founder
> > EclipseTestNG Creator
> >
> >
> >
> >
>
>
>
> --
> Cédric
>  >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Cedric-20

I'm not extremely surprised to hear this, but I am indeed surprised
that we have parts of TestNG that depends on this order (most likely my
fault since I wrote the dependency engine).  I'll track that one down,
although it might be a little while since my laptop is a Mac now (and
I'm guessing Sun didn't release a JDK6 for Mac...).

Thanks for pinpointing it, Alex.

--
Cedric


Alexandru Popescu wrote:

> On 12/13/06, Cédric Beust ♔ <[hidden email]> wrote:
> > Interesting, keep us posted.
> >
>
> Now, I am sure that it is something in the way HashMap-s are implemented:
>
> [code]
> public class HashMapValuesOrderTest {
>    public static void main(String[] argv) {
>         HashMap map= new HashMap();
>         map.put("1", "1");
>         map.put("2", "2");
>         map.put("2.1", "2.1");
>         map.put("2.2", "2.2");
>         map.put("3", "3");
>         map.put("abc", "abc");
>         System.out.println("Values: " + map.values());
>    }
> }
> [/code]
>
> yields different results under JDK15.0_09 and JDK6 (agreed there
> should not be a guaranteed order there). The only think I must check
> is that our logic doesn't depend on a specific order and same for the
> tests.
>
> ./alex
> --
> .w( the_mindstorm )p.
>   TestNG co-founder
> EclipseTestNG Creator
>
> > --
> > Cedric
> >
> >
> >
> > On 12/12/06, Alexandru Popescu
> > <[hidden email] > wrote:
> > >
> > > Hi!
> > >
> > > There is some method ordering problem I am seeing in Regression2 test
> > > and Test order invocations and it is visible only with JDK6.
> > >
> > > It looks like we have a couple of tests that depend on a fixed
> > > ordering of unrelated methods
> > > and between JDK5 and JDK6 there are some changes in the HashMap
> > > implementation so that Map.values() returns differently this resulting
> > > in failing tests.
> > >
> > > I am not sure yet if there is a issue for TestNG or is just a problem
> > > with our tests. I will further investigate.
> > >
> > > ./alex
> > > --
> > > .w( the_mindstorm )p.
> > >   TestNG co-founder
> > > EclipseTestNG Creator
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > Cédric
> >  >
> >


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Phillip Ross-2

Popping in a LinkedHashMap(1.4 compatible) in place of HashMap is an easy way to get deterministic ordering from maps. (even if using maps is determined to be bad)

>
> I'm not extremely surprised to hear this, but I am
> indeed surprised
> that we have parts of TestNG that depends on this
> order (most likely my
> fault since I wrote the dependency engine).  I'll
> track that one down,
> although it might be a little while since my laptop
> is a Mac now (and
> I'm guessing Sun didn't release a JDK6 for Mac...).
>
> Thanks for pinpointing it, Alex.
>
> --
> Cedric
>
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53279&messageID=107369#107369

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Cédric Beust ♔


On 12/13/06, Jesse Kuhnert <[hidden email]> wrote:

Popping in a LinkedHashMap(1.4 compatible) in place of HashMap is an easy way to get deterministic ordering from maps. (even if using maps is determined to be bad)

Nah, it's the wrong way to solve this problem.  If I need ordered values, I should be using a List.  If I'm using a Map, equality should be based on the content of the map, not on the ordering of the keys or values (which are stored as Collections).

--
Cédric
--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Alexandru Popescu ☀
On 12/13/06, Cédric Beust ♔ <[hidden email]> wrote:

>
>
> On 12/13/06, Jesse Kuhnert <[hidden email]> wrote:
> >
> > Popping in a LinkedHashMap(1.4 compatible) in place of HashMap is an easy
> way to get deterministic ordering from maps. (even if using maps is
> determined to be bad)
>
> Nah, it's the wrong way to solve this problem.  If I need ordered values, I
> should be using a List.  If I'm using a Map, equality should be based on the
> content of the map, not on the ordering of the keys or values (which are
> stored as Collections).
>

Cedric has well described why I have posted the message on the mailing
list and not jumped to the LinkedHashMap. I am almost done, but one
issue is bothering me as I am quite sure it is not due to Java6.

./alex
--
.w( the_mindstorm )p.
  TestNG co-founder
EclipseTestNG Creator

> --
> Cédric
>
>  >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Phillip Ross-2
In reply to this post by Cédric Beust ♔

I'm not going to argue algorithms with anyone who has them freshly in their head again, but from a naive (not knowing what you're doing/solving) point of view the linked approach might be beneficial in certain fringe conditions where hashing perf is more critical... ? ( maybe not .. whatever, doesn't matter..)

> Nah, it's the wrong way to solve this problem.  If I
> need ordered values, I
> should be using a List.  If I'm using a Map, equality
> should be based on the
> content of the map, not on the ordering of the keys
> or values (which are
> stored as Collections).
>
> --
> Cédric
>
>
> ---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53279&messageID=107450#107450

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

isterin
In reply to this post by Cedric-20
Cedric, JDK6 has been available on Mac for quite a while now.  All the
way through their beta program.  I'm guessing the GA release will be
available shortly, but for now you can download it from
https://connect.apple.com, you'll just have to sign up for a developer
connection membership, which is free.

Ilya

On 12/13/06, Cedric Beust <[hidden email]> wrote:

>
> I'm not extremely surprised to hear this, but I am indeed surprised
> that we have parts of TestNG that depends on this order (most likely my
> fault since I wrote the dependency engine).  I'll track that one down,
> although it might be a little while since my laptop is a Mac now (and
> I'm guessing Sun didn't release a JDK6 for Mac...).
>
> Thanks for pinpointing it, Alex.
>
> --
> Cedric
>
>
> Alexandru Popescu wrote:
> > On 12/13/06, Cédric Beust ♔ <[hidden email]> wrote:
> > > Interesting, keep us posted.
> > >
> >
> > Now, I am sure that it is something in the way HashMap-s are implemented:
> >
> > [code]
> > public class HashMapValuesOrderTest {
> >    public static void main(String[] argv) {
> >         HashMap map= new HashMap();
> >         map.put("1", "1");
> >         map.put("2", "2");
> >         map.put("2.1", "2.1");
> >         map.put("2.2", "2.2");
> >         map.put("3", "3");
> >         map.put("abc", "abc");
> >         System.out.println("Values: " + map.values());
> >    }
> > }
> > [/code]
> >
> > yields different results under JDK15.0_09 and JDK6 (agreed there
> > should not be a guaranteed order there). The only think I must check
> > is that our logic doesn't depend on a specific order and same for the
> > tests.
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >   TestNG co-founder
> > EclipseTestNG Creator
> >
> > > --
> > > Cedric
> > >
> > >
> > >
> > > On 12/12/06, Alexandru Popescu
> > > <[hidden email] > wrote:
> > > >
> > > > Hi!
> > > >
> > > > There is some method ordering problem I am seeing in Regression2 test
> > > > and Test order invocations and it is visible only with JDK6.
> > > >
> > > > It looks like we have a couple of tests that depend on a fixed
> > > > ordering of unrelated methods
> > > > and between JDK5 and JDK6 there are some changes in the HashMap
> > > > implementation so that Map.values() returns differently this resulting
> > > > in failing tests.
> > > >
> > > > I am not sure yet if there is a issue for TestNG or is just a problem
> > > > with our tests. I will further investigate.
> > > >
> > > > ./alex
> > > > --
> > > > .w( the_mindstorm )p.
> > > >   TestNG co-founder
> > > > EclipseTestNG Creator
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Cédric
> > >  >
> > >
>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: TestNG tests failures with JDK6

Cedric-20

Thanks, Ilya, Hani told me the same thing.

As for the bugs, Alexandru found them and fixed them, and while I take
full responsibility for them, everyone should be happy to hear they
were in our tests and not in the TestNG core :-)

--
Cedric


Ilya Sterin wrote:

> Cedric, JDK6 has been available on Mac for quite a while now.  All the
> way through their beta program.  I'm guessing the GA release will be
> available shortly, but for now you can download it from
> https://connect.apple.com, you'll just have to sign up for a developer
> connection membership, which is free.
>
> Ilya
>
> On 12/13/06, Cedric Beust <[hidden email]> wrote:
> >
> > I'm not extremely surprised to hear this, but I am indeed surprised
> > that we have parts of TestNG that depends on this order (most likely my
> > fault since I wrote the dependency engine).  I'll track that one down,
> > although it might be a little while since my laptop is a Mac now (and
> > I'm guessing Sun didn't release a JDK6 for Mac...).
> >
> > Thanks for pinpointing it, Alex.
> >
> > --
> > Cedric
> >
> >
> > Alexandru Popescu wrote:
> > > On 12/13/06, Cédric Beust ♔ <[hidden email]> wrote:
> > > > Interesting, keep us posted.
> > > >
> > >
> > > Now, I am sure that it is something in the way HashMap-s are implemented:
> > >
> > > [code]
> > > public class HashMapValuesOrderTest {
> > >    public static void main(String[] argv) {
> > >         HashMap map= new HashMap();
> > >         map.put("1", "1");
> > >         map.put("2", "2");
> > >         map.put("2.1", "2.1");
> > >         map.put("2.2", "2.2");
> > >         map.put("3", "3");
> > >         map.put("abc", "abc");
> > >         System.out.println("Values: " + map.values());
> > >    }
> > > }
> > > [/code]
> > >
> > > yields different results under JDK15.0_09 and JDK6 (agreed there
> > > should not be a guaranteed order there). The only think I must check
> > > is that our logic doesn't depend on a specific order and same for the
> > > tests.
> > >
> > > ./alex
> > > --
> > > .w( the_mindstorm )p.
> > >   TestNG co-founder
> > > EclipseTestNG Creator
> > >
> > > > --
> > > > Cedric
> > > >
> > > >
> > > >
> > > > On 12/12/06, Alexandru Popescu
> > > > <[hidden email] > wrote:
> > > > >
> > > > > Hi!
> > > > >
> > > > > There is some method ordering problem I am seeing in Regression2 test
> > > > > and Test order invocations and it is visible only with JDK6.
> > > > >
> > > > > It looks like we have a couple of tests that depend on a fixed
> > > > > ordering of unrelated methods
> > > > > and between JDK5 and JDK6 there are some changes in the HashMap
> > > > > implementation so that Map.values() returns differently this resulting
> > > > > in failing tests.
> > > > >
> > > > > I am not sure yet if there is a issue for TestNG or is just a problem
> > > > > with our tests. I will further investigate.
> > > > >
> > > > > ./alex
> > > > > --
> > > > > .w( the_mindstorm )p.
> > > > >   TestNG co-founder
> > > > > EclipseTestNG Creator
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Cédric
> > > >  >
> > > >
> >
> >
> > >
> >


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-dev" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en
-~----------~----~----~----~------~----~------~--~---