WebStart unit tests

I had a few bits of JNA functionality that were only active when in a web start environment, which made it a bit tricky to add tests for them that could be run at the same time as all my other tests. The functionality also showed up as a big splot of missing code coverage under clover, so I decided to tinker just a bit to see if I could somehow simulate a web start environment for the unit tests.

I managed to come up with a JUnit-based test fixture which ensures its test methods are all run in a web start environment. This works well under Windows and OSX; unfortunately *nix variations use NetX which needs some extra hand-holding when first run.

Finding the right hooks involved standard hacking to find whether the most configurability was offered by invoking the javaws executable or adding the javaws classes to my classpath and hooking directly into the Java. I ended up doing a Runtime.exec to ensure the running environment wouldn't interfere with that of Web Start.

The main obstacle to be overcome was signed code. You can get web start to run unsigned code by tweaking the local policy file to allow the code we're testing, but when you include any native code via the tag, JNLP requires the tag, which seems to re-trigger the requirement for signed code. Self-signing the code is no big deal, but that triggers web-starts authorization dialogs to allow the unknown CA to be used. Fortunately, the somewhat obscure deployment properties file can be used to temporarily allow the self-signed certs and bypass the dialogs. The tests can be run entirely without user input (except in the case of *nix, where NetX is not sufficiently configurable -- you have to dismiss the dialog on the first test run).

Unfortunately the exit codes for javaws don't correspond to the codes passed to System.exit by Java code, so I had to have the test fixture communicate to the running javaws instance via socket. Failures are transmitted by the fixture in such a way that you can't tell that the test is running in a separate process.

I encapsulated the deployment config, JNLP file creation, javaws launch, and test case execution into a single class, WebStartTest, which runs a few self tests and can be extended to add whatever other tests you need to run under web start. The JNLP is partly hard-coded to include files set up sepecifically for JNA testing, but should be trivial to change to accommodate a different project.

There are probably other ways to test WebStart code, but this fit nicely with the project's existing tests, allowing its WebStart features to be tested by test methods identical to those for the rest of the project.

Full source is here. This has been tested under Sun and IBM JDKs, on Windows, OSX, Solaris/sparc, and GNU/Linux.

JNA: increasing performance with large Structures

If you're using very large structures and using them often, here's a tip that can boost performance by several orders of magnitude. Note that you should follow this tip *only* if you really need the performance boost; otherwise you may wind up obfuscating your code.

By default, when JNA makes a native call it will copy the full contents of a Java Structure to native memory prior to the call and read it all back after the call. If your Structure is very large, this can result in significant overhead reflecting all the fields of the Structure. The reflection dwarfs the actual native communication time.

If you're only reading or writing a single field, it's much faster (although somewhat less elegant) to use the readField(String) and writeField(String) methods to access the data, while disabling the normal read and write. Depending on the size of your structure, you may see two orders of magnitude or more improvement in the native function call time.

Here's an example of performing the same operation two different ways:

class Big extends Structure {
    public int toNative;
    public int fromNative;
    // plus lots more fields
    // the more, the bigger the difference in performance
}

class FastBig extends Big {
    public void read() { }
    public void write() { }
}

Big big = new Big();
big.toNative = 42;
lib.callMyNativeFunction(big);
System.out.println("Got " + big.fromNative);

Big fast = new FastBig();
fast.toNative = 42;
fast.writeField("toNative");
lib.callMyNativeFunction(fast);
System.out.println("Got " + fast.readField("fromNative"));

If you wrap a loop and time these, you'll see what kind of difference it makes. On a test structure with 25 "int" fields, the fast version reduces time by a factor of 10.

Trivia: some other "struct" implementations (e.g. Javolution) use objects for all fields and require an explicit "write" or "set" on each. This reduces data transfer and/or reflection overhead, at the expense of simplicity of assignment and initialization.

JNA:
s.field = 1;
call(s.field);

Javolution:
s.field.set(1);
call(s.field.get());

JNA win64 port

JNA now supports 64-bit windows. The the build is a bit nonstandard, but it's automatic and reproducible. I was hoping to use mingw-64, but that project is not yet sufficient for JNA's purposes. Fortunately, the MS has a free 64-bit cross compiler which does work.

Thanks to the fine folks at Aquafold (actually mostly Niels G. the CTO) for making it possible.

Thanks also to T. Heller of python's ctypes for the initial glue bits.

Decorator Update

Thanks to some input from a few users, the AbstractComponentDecorator has been updated to fix a few visual and behavior artifacts.

Get the latest from Sourceforge SVN.

The decorator component allows you to apply arbitrary decorations to a component without affecting its layout or (necessarily) response to events.

This blog has many different examples of usage beyond simple decorations, including drag images and per-component input blocking.

Embedding DLL File Version Information with mingw32/cygwin

If you look at the properties window for a DLL file in Windows, you'll see a few bits about the file version and copyright info. I couldn't find any information on the web about how to embed that information into a DLL. Not terribly surprising, since it's probably something MSVC inserts automatically, so why would any one need to know how it's done?

Well, I'm building a DLL with cygwin/mingw32, so I do need to know how it's done. Fortunately, Mumit Khan, author of mingw32, wrote a utility called "windres", which compiles w32 resources for use in an executable or DLL. It'll compile directly to object format, which you can simply add to your final link.

For example:

% windres -i version.rc -o version.o
So the only thing I needed to figure out was the version resource format (I really didn't want to go grok the resource spec and write it from scratch). Luckily, Java's main DLL (java.dll, what else) has version information in it, and windres can also conveniently dump an exe or dll back into text format.

The java.dll resource info looks like this:

% windres -i java.dll -O rc

// Type: version
// Name: 1
LANGUAGE 0, 0
1 VERSIONINFO
 FILEVERSION 6, 0, 30, 5
 PRODUCTVERSION 6, 0, 30, 5
 FILEFLAGSMASK 0x3f
 FILEOS 0x4
 FILETYPE 0x2
BEGIN
  BLOCK "StringFileInfo"
  BEGIN
    BLOCK "000004b0"
    BEGIN
      VALUE "CompanyName", "Sun Microsystems, Inc."
      VALUE "FileDescription", "Java(TM) Platform SE binary"
      VALUE "FileVersion", "6.0.30.5"
      VALUE "Full Version", "1.6.0_03-b05"
      VALUE "InternalName", "java"
      VALUE "LegalCopyright", "Copyright \251 2004"
      VALUE "OriginalFilename", "java.dll"
      VALUE "ProductName", "Java(TM) Platform SE 6 U3"
      VALUE "ProductVersion", "6.0.30.5" 
  END 
END

With just a few tweaks to do a regexp replace on the new template and include the new obj in my build, my released dll now has visible version information (and in this case I'm perfectly happy having absolutely no clue what any of the resource file text actually means :).

Run your Java program as an NT service

Thomas Boerkel has contributed some mappings and code to the JNA project to facilitate access to Windows' user accounts, registry, and NT services. Included is an abstract class which is capable of installing, uninstalling, and running itself as an NT service. What sets this little gem apart from most available wrappers is that you don't need a Windows executable stub; the complete Windows service API is available to your Java program, including all system events and callbacks.

You can browse the source here. A NetBeans project is included.

I was working on an abstract service class as well but Thomas beat me to completion. Great work, Thomas!

Update: Make sure your classpath and classes are accessible by the local system account, or the service won't be able to start. To see if this is the issue, you can temporarily change the service to start using your user identity; if it works, then the problem is that the local system account doesn't have access.

Is final not final, or just not the final I wanted?

While adding some VM crash protection to JNA, I ran into this situation:

public class Pointer {
    public static final int SIZE = Native.POINTER_SIZE;
}

public class Native {
    public static final int POINTER_SIZE;
    static {
        initIDs();
        POINTER_SIZE = pointerSize();
    }
    private static native int pointerSize();
}

I had just moved the native initialization code from Pointer to Native. All the native methods were private, so the only changes were to update those two classes and rename the native methods. But when I reran the test suite, I started getting a host of errors.

There errors were all caused by Pointer.SIZE having a zero value. I figured that since Pointer.SIZE was final, the value it was assigned from must also have been zero. A quick inspection on Native.POINTER_SIZE, however, showed that it was propertly initialized and non-zero. How, then, could Pointer.SIZE be zero if Native.POINTER_SIZE was not?

Anyone who has done any JNI is probably aware that Java access modifiers don't mean anything to native code. My first thought was that perhaps "final" modifiers don't mean anything either. Only problem is that I was never touching that Pointer.SIZE field in native code. Never even accessed it.

Scanning though the native initialization code, I noticed that, among a host of standard JRE classes, a reference to the Pointer class was being created. This turns out to be the cause of my problem: during Native class initialization, the Pointer class was loaded from JNI. When the pointer final field was initialized, it got the value from a not-yet-initialized Native class.

Turns out, this can be duplicated with pure Java code:

public class LoadTest {
    public static final int VALUE;
    static {
        System.out.println("value=" + Secondary.VALUE);
        VALUE = 1;
        System.out.println("value=" + Secondary.VALUE);
    }
}

class Secondary {
    public static final int VALUE = LoadTest.VALUE;
}

When loaded, the class prints "value=0" twice, so final is final, just not the value I intended. Here, the problem is more apparent because the class dependency is clearly visible, which wasn't the case with the JNI initialization.

Maybe I'll call this antipattern something curiously obscure, like "Law of the Polygamist's Second Wife".