Review of "Developing Java Software"

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Review of "Developing Java Software"

Steve Loughran-7

On 17/12/06, Hani Suleiman <[hidden email]> wrote:

>
> >
> > This isnt code you are testing, its the configuration of the local
> > box.
> >
> Yes, which by its very nature is not easily testable, because tests
> if anything make the situation worse by pretending there's a known
> environment, when in fact you never know what sort of
> misconfiguration the live box has. There's nothing more evil than a
> test which passes because it just so happens to set up a fake
> unrealistic environment that just accidentally matches the nightly CI
> machine.
>
> > *** Any deployment without tests isn't replicable ***
> >
> Nonsense! I can tag a version in my source control system. Building
> that tag results in the same artifacts. In fact, if you want to be
> anal about it you could commit the artifacts themselves. The tests
> are completely orthogonal to deployment, unless I'm missing something!
>

Unless you are putting entire Xen/vmware images under SCM, you are
only tagging some of the artifacts you create, not the entire state of
the live system. its all those other bits that burn you.

So, instead of treating a misconfigured box as an accident, we should
treat it as the default state unless you do something about it. One
way is to automate deployment (what I work on). One way is to spell
out to the ops team in big letters what to do, and hope they read it.
Either way, you can include tests to verify system state.

These arent code unit tests, they are a form of 'configuration unit
tests', and they can be in whatever language is most appropriate.
Here's an example:
 -a multi tier system has a back end db for which the web front end gets a login
 -the web front ends are all connected by some fancy distributed cache
which works provided the clocks are roughly in sync
 -we use a temp filestore which must be of a certain size.
 -the boxes are all in an NT domain
 -some native app must be on the path

You can test these.
-database access-open up a jdbc connection using the login. No
connection: error.
web front end clocks. Hard to test, but if each node can be asked for
its clock, then you can -check that the other nodes are roughly in
sync and that their TZ setting is aligned.
-temp filestore: create a big file, read it.
-native apps. Exec it with bad args, and expect the right error message

These are not tests you run on some test system to say "yes, our app
is ready to ship". They are tests you run on the target systems as you
bring it up. I've done this with junit tests in an ant script. I've
done it in Cppunit on a COM thing, where we'd probe the registry for
the right settings.  I've done it in JSP pages. If you ever use Axis
and hit happyaxis.jsp, you are seeing some of my work, where I look
for the XML parser and warn if it is crimson.



 >  and the specifications come from

> > "the architects", who have a definite tendency to be developers who
> > stopped coding 5+ years ago, and wouldn't know a test case if it bit
> > them.
>
> Yes, the the testability of code is a luxury, not a necessity. It's
> nice to be able to test stuff, but it's not required for success or
> adoption. Want some examples? Servlets and even worse, JSP pages.
> It's impossible to test the latter for example, and JSF is even
> worse. The specification authors wouldn't know the difference between
> a spec and an implement if it painted itself purple and danced around
> them naked single kumbaya my lord, yet both are wildly successful.
> How do you explain that?

I dont know if JSF is a success. As to the others, JSP has a reference
implementation -tomcat/jasper- that effectively forms the
specification. Same for MS word, MS LanMan, and netscape/IE.  The main
implemenatation becomes the defacto specification, not the 'official'
piece of paper. Now, if there is just one implementation of a product,
that's fine. I dont care that there isnt a formal spec of Log4J
because it just works. Its when you want to switch implementations, or
have different implementations work together that you have problems.
In a world of reference implementations, you have to test against the
RI and go with what works. Samba have to test against LanMan, machine
generated .doc files have to be tested in MS Word, and web pages need
checking in IE6.

This can create a barrier to entry to competitors; how do you compete
with MSWord if you can't read its data? It also has the potential to
create a maintenance nightmare for the specific teams, as they have to
decide which parts of their app are bugs, and which are features which
need to be retained. Without a spec, without a test suite, there's no
way to be sure.

There's lots of products like this, but should they be considered
'standards', as opposed to one off implementations? That I don't know.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Review of "Developing Java Software"

Hani Suleiman

On Dec 17, 2006, at 4:52 PM, Steve Loughran wrote:

>>
>>> *** Any deployment without tests isn't replicable ***
>>>
>> Nonsense! I can tag a version in my source control system. Building
>> that tag results in the same artifacts. In fact, if you want to be
>> anal about it you could commit the artifacts themselves. The tests
>> are completely orthogonal to deployment, unless I'm missing  
>> something!
>>
>
> These arent code unit tests, they are a form of 'configuration unit
> tests', and they can be in whatever language is most appropriate.

Aha, yes, in this case, I agree with you 100% It's definitely a good  
idea to have a bunch of sanity check tests on any new environment,  
just to verify your assumptions (that the app can get to the db, that  
the filesystem has enough space, that the right bits are writable,  
etc). My argument was against 'code' type tests as being a  
requirement for deployment.

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

12