We recently wrote a software using Java 1.4, which prevented us from using some nice developer testing tools that required at least Java 5 (also known as 1.5). Namely these were JDave that would have underlined that we really wanted to do behavior-driven development (BDD), and WicketBenchTestCase of Wicket Bench, which allows easy testing of individual Apache Wicket components.
When discussing this, our coworker Ville remarked that often it makes sense to put the tests of the project in a separate module (something that would be a submodule in Maven 2 multiproject project, and a separate Eclipse project or another module in IntelliJ IDEA). This way it would be trivial to use a different JDK for test and production code. Nevertheless we decided to keep it simple by doing everything in a single Java 1.4 module; the software was small and we felt that the marginal hassle of going from n to n + 1 modules is highest when n == 1.
We wanted to do system testing on a production-resembling platform as soon in the project as possible, but due to various circumstances out of our control, we only got there rather late in the project. And when we finally had deployed the software in the real application server, we were baffled by unexpected errors in some LDAP searches done by our application.
After a quick screening of differencies between our development environments and the system test environment, we tried putting the (proprietary) production JDK also to our development machines. For our great satisfaction, some of the various developer tests (unit + integration tests covering about 90 % of all program code lines, ahem! :)) started failing with the exactly same errors as in the test environment. So this was a case of the JRE abstraction leaking.
Now that the problem could easily be reproduced on any development machine, fixing it was rather straightforward. We even changed our Continuum server to use the production JVM + JRE, and verified that after the fixes, the software was still working as expected on the Java implementation originally used in development.
And then a while later it occurred to me what a good idea, albeit for wrong resons, it had been to write our developer tests on the same Java version as the target environment. If not, reproducing and fixing our bug might have been a lot more difficult.
The Quest for Great Software 
26 August, 2007 at 15:13
I guess Ville proposed this idea because we used to use different JRE for our unit tests in our last project. JRE 6.0 was used for tests, while production server was JRE 1.4.2. Although, we had different JRE for unit tests, we did run all higher level tests with same JRE as used in production. So we actually had three different modules: for production code, for unit tests and for integration tests. I know that it is a bit complex setup, but for us it worked pretty well.
And I totally agree with your point. Project’s continuous integration environment should be very similar with production environment. So you should use same JRE if it is possible.
23 October, 2008 at 2:34
Nice article. Thanks. 🙂 Eugene