Thread termination/exit handlers

I'd been hunting for a while for a good solution to automatically call cleanup operations on exit from threads I didn't own.  Windows (XP onward) has a pretty straightforward solution if you're working with a DLL, but for pthreads on most other platforms, the solution is not as obvious.

Some folks have been using JNA to allow Java code to be invoked as callbacks from various streaming (video, sound) libraries.  If these callbacks come from threads instantiated from native code, the JVM has to (at least temporarily) map the thread into Java space for the duration of the callback.  If the callbacks are frequent, and always come in on the same native thread, we don't want to incur the mapping overhead on every invocation.  The solution, then is to avoid detaching the thread when the callback finishes.

Now we have a new problem.  When the native thread actually terminates, the JVM has no idea that the thread went away because it's only really got a placeholder Thread object, and hasn't hooked up all the plumbing it normally does to detect that the thread has gone away and clean up/GC the mapped Thread object.  Thus the need for a thread exit handler.  In some cases, you may not care about the minimal object overhead, for instance if you've just got a few native threads.  However, with a lot of threads coming and going (we can't force folks to thread-pool).

Windows

On Windows, if you have a DllMain function defined, you'll get notices when threads and processes attach/detach your DLL's code.  This works out nicely, we can make the VM detach the current thread when we get that message:

// Store thread-local information required to detach the thread
// (in my case, only a JVM reference was required)
// TlsSetValue is only set when we recognize that we need the 
// extra cleanup
static DWORD dwTlsIndex;
BOOL WINAPI DllMain(HINSTANCE hDLL, 
                    DWORD fdwReason, 
                    LPVOID lpvReserved) {
  switch (fdwReason) {
  case DLL_PROCESS_ATTACH:
    dwTlsIndex = TlsAlloc();
    if (dwTlsIndex == TLS_OUT_OF_INDEXES) {
      return FALSE;
    }
    break;
  case DLL_PROCESS_DETACH:
    TlsFree(dwTlsIndex);
    break;
  case DLL_THREAD_ATTACH:
    break;
  case DLL_THREAD_DETACH: {
    extern void detach_thread();
    detach_thread(TlsGetValue(dwTlsIndex));
    break;
  }
  default:
    break;
  }
  return TRUE;
}

 Easy enough.  Note that TlsSetValue is called elsewhere only when the callback decides it doesn't want to detach immediately (rather than every time a native thread attaches).  Callbacks normally detach on exit if they weren't attached to begin with.

POSIX Threads (pthreads)

The pthreads library is a different beast.  Searches for "thread termination handler" don't turn up much, except for the stack-based pthread_cancel_push/pop, which seems to do the right thing, but have to arrive in pairs.  It's actually in the pthreads implementation of thread-local storage that we find a way to attach a termination handler to a given thread.

When you define a given thread-local variable in pthreads (a "key" in pthreads lingo), you can provide a destructor function to be used to clean up the storage...on thread exit.  It was a little hard to find since it's not a thread termination handler per se, but rather a mechanism to clean up thread local storage.  I'd overlooked it several times because I wasn't looking for thread local storage solutions.

// Basic plumbing to create a unique key identifying
// specific thread-local storage
static pthread_key_t key;
static void make_key() {
  extern void detach_thread();
  pthread_key_create(&key, detach_thread);
}

    ...
    // This code gets called whenever we identify that
    // a thread needs detach-on-exit behavior
    static pthread_once_t key_once = PTHREAD_ONCE_INIT;
    pthread_once(&key_once, make_key);
    if (!jvm || pthread_getspecific(key) == NULL) {
      pthread_setspecific(key, jvm);
    }
    ...

Now detach_thread will be called (with the value of the thread-local storage as its single argument) when the thread exits.

And voilà, you have a POSIX thread termination handler.

Is final not final, or just not the final I wanted?

While adding some VM crash protection to JNA, I ran into this situation:

public class Pointer {
    public static final int SIZE = Native.POINTER_SIZE;
}

public class Native {
    public static final int POINTER_SIZE;
    static {
        initIDs();
        POINTER_SIZE = pointerSize();
    }
    private static native int pointerSize();
}

I had just moved the native initialization code from Pointer to Native. All the native methods were private, so the only changes were to update those two classes and rename the native methods. But when I reran the test suite, I started getting a host of errors.

There errors were all caused by Pointer.SIZE having a zero value. I figured that since Pointer.SIZE was final, the value it was assigned from must also have been zero. A quick inspection on Native.POINTER_SIZE, however, showed that it was propertly initialized and non-zero. How, then, could Pointer.SIZE be zero if Native.POINTER_SIZE was not?

Anyone who has done any JNI is probably aware that Java access modifiers don't mean anything to native code. My first thought was that perhaps "final" modifiers don't mean anything either. Only problem is that I was never touching that Pointer.SIZE field in native code. Never even accessed it.

Scanning though the native initialization code, I noticed that, among a host of standard JRE classes, a reference to the Pointer class was being created. This turns out to be the cause of my problem: during Native class initialization, the Pointer class was loaded from JNI. When the pointer final field was initialized, it got the value from a not-yet-initialized Native class.

Turns out, this can be duplicated with pure Java code:

public class LoadTest {
    public static final int VALUE;
    static {
        System.out.println("value=" + Secondary.VALUE);
        VALUE = 1;
        System.out.println("value=" + Secondary.VALUE);
    }
}

class Secondary {
    public static final int VALUE = LoadTest.VALUE;
}

When loaded, the class prints "value=0" twice, so final is final, just not the value I intended. Here, the problem is more apparent because the class dependency is clearly visible, which wasn't the case with the JNI initialization.

Maybe I'll call this antipattern something curiously obscure, like "Law of the Polygamist's Second Wife".

Building w32 JNI DLLs with gcc

After a bit of trial and error, I got JNA's native shared library to build (and run, and run tests) using gcc. Unfortunately the available documentation didn't quite work for me (might be a problem with my cygwin installation, but I wasn't the only one with an issue).

gcc-mingw is a version of cygwin gcc that can compile directly to the w32 api, without any of the cygwin emulation layer.


% gcc -dumpspecs | sed s%dllcrt2%/lib/mingw/dllcrt2%g > specs.new
% gcc -mno-cygwin -D_REENTRANT -D_GNU_SOURCE \
-D__int64="long long" -D_JNI_IMPLEMENTATION \
-o
-shared -Wl,--kill-at -specs specs.new -L /lib/mingw


The -mno-cygwin flag tells the compiler to run in mingw mode, i.e. omit all the cygwin unix compatibility layer and compile directly against the w32 api.

GCC's built-in 64-bit type is long long, so we map the type used in the JNI headers, __int64, to that type. Defining _JNI_IMPLEMENTATION ensures the JNI implementation exports its native method declarations.

Defining _REENTRANT and _GNU_SOURCE cause certain additional declarations of C library functions to be included. If you've never heard of them, most likely you don't need them.

The --kill-at flag to the linker ensures all symbols are exported undecorated, i.e. _my_func instead of _my_func@NN, which is the default when a function is declared as __stdcall. You could also use --add-stdcall-alias, which includes both versions of the symbol.

The GCC specs need tweaking or gcc-mingw doesn't find its initialization code for dlls, dllcrt2.o. We also have to nudge gcc-mingw to find the mingw libraries. Normally this is taken care of automatically by gcc, but for some reason my installation of cygwin gcc wouldn't find them, and no amount of -L or explicit object linkage would fix it.

Anyhow, with that out of the way, you don't have to have Microsoft's tools to build JNA or any other JNI library on w32.

I have noted that GCC apparently handles floating point a little differently, since the JNA tests that use FP return values/arguments are failing with the GCC-build dll.