If I were king: A proposal for fixing the Java programming language’s threading problems
Allen Holub suggests that the Java programming language’s threading model is possibly the weakest part of the language. It’s entirely inadequate for programs of realistic complexity and isn’t in the least bit object oriented. This article proposes significant changes and additions to the Java language that would address many of these problems.
The Java threading model is one of the least satisfactory aspects of the language. Though it’s a good thing that threading is built in, the syntactic and package-level support for threads is so minimal as to be worthless for all but the most trivial applications.
Most books on Java threading present a litany of everything wrong with Java threading model and a class library of Band-Aid solutions to those problems. I call the classes Band-Aids because the problems addressed by the classes should really be part of the syntax of the Java language. Using syntactic methods rather than libraries can give you better code in the long run since the compiler and JVM, working together, can perform optimizations that would be difficult or impossible with a library approach.
In my book Taming Java Threads (see Resources), and in this article, I go one step further and suggest a more positive approach to the threading problem by suggesting a few changes to the Java programming language that would provide real solutions. The main difference between this article and the book is that I’ve thought about things a bit more, so have improved the proposals a bit. These proposals are very tentative — they are just one person’s thoughts on the matter, and they would need a lot of work and peer review before being viable. But they’re a start. I intend to set up a working group on these issues; if you’re interested, send e-mail to firstname.lastname@example.org and I’ll drop you a note when something real starts happening.
The proposals presented here are also rather bold. Several people have suggested subtle, and minimal, changes to the Java Language Specification (JLS) (see Resources) to fix currently ambiguous JVM behavior, but I want more sweeping improvement.
On a practical note, many of my proposals involve the introduction of new keywords to the language. Though the usual requirement that you don’t want to break existing code is certainly valid, if the language is not to stagnate and thus become obsolete, it must be possible to introduce keywords. In order to introduce keywords that won’t conflict with existing identifiers, I’ve deliberately used a character (
$) that is illegal in an identifier. (For example,
$task rather than
task). A compiler command-line switch could perhaps enable variants on these keywords that would omit the dollar sign.
The concept of task
The fundamental problem with the Java threading model is the fact that it is not in the least bit object oriented. OO designers don’t think in terms of threads at all; rather, they think of synchronous messages (which are processed immediately — the message handler doesn’t return until the message is complete) and asynchronous messages (which are processed in the background some time after the message is received — the message handler returns immediately, long before the message is processed). The Java programming language’s
Toolkit.getImage() method is a good example of an asynchronous message. The
getImage() message handler returns immediately, before the entire image is fetched by a background thread.
That’s the OO approach, but as I said earlier, the Java threading model isn’t OO. A Java programming language thread is effectively nothing but a procedure (
run()) that calls other procedures. Notions of objects, asynchronous versus synchronous messages, and the like, are simply not addressed.
One solution to this problem, discussed in depth in my book, is an
Active_object. Active Objects are objects that can receive asynchronous requests, which are processed in the background some time after the request is received. In the Java programming language, a request can be encapsulated in an object — for example, you can pass the active object an instance of a
Runnable implementation whose
run() method encapsulates the work to do. This runnable object is queued up by the active object and then executed on a background thread when the object gets around to it.
The asynchronous messages running on an active object are effectively synchronous with respect to each other since they’re dequeued and executed one at a time by a single service thread. Consequently, you can eliminate much of the synchronization hassles required to program in a more procedural model by using an active object.
In a way, the Java programming language’s entire Swing/AWT subsystem is an active object. The only safe way to send a message to a Swing class is to call a method like
SwingUtilities.invokeLater(), which posts a runnable object on the Swing event queue to be handled by the Swing event-processing thread when it gets around to it.
My first proposal, then, is to incorporate active objects into the language itself by incorporating the notion of a task into the Java programming language. (The notion of a task is borrowed from Intel’s RMX operating system and the Ada programming language. Most real-time operating systems support similar concepts.)
A task has a built-in active-object dispatcher, and takes care of all the mechanics of handling asynchronous messages automatically.
You define a task exactly as you would a class, except that the
asynchronous modifier could be applied to methods of the task to indicate that those methods should execute in the background on the active-object dispatcher. To see the parallels with the class-based approach discussed in Chapter 9 of my book, consider the following file I/O class, which uses the
Active_object class discussed in Taming Java Threads to implement an asynchronous write operation:
All write requests are queued up on the active-object’s input queue with a
dispatch() call. Any exceptions that occur while processing the asynchronous message in the background are handled by the
Exception_handler object that’s passed into the
File_io_task‘s constructor. You would write to the file like this:
The main problem with this class-based approach is that it’s way too complicated — adding way too much clutter to the code to do a simple thing. Introducing the
$asynchronous keywords to the language lets you rewrite the previous code as follows:
Note that asynchronous methods don’t specify return values because the handler returns immediately, long before the requested operation completes. Consequently, there is not reasonable value that could be returned. The
$task keyword should work exactly like
class with respect to the derivation model: a
$task could implement interfaces, extend classes, and extend other tasks. Methods marked with the
asynchronous keyword are handled by the
$task in the background. Other methods would work synchronously, just as they do in classes.
$task keyword can be modified with an optional
$error clause (as shown), which specifies a default handler for any exceptions that are not caught by the asynchronous methods themselves. I’ve used
$ to represent the thrown exception object. If no
$error clause is specified, then a reasonable error message (and probably stack trace) is printed.
Note that the arguments to the asynchronous method must be immutable to be thread safe. The run-time system should take care of any semantics required to guarantee this immutability. (A simple copy is often not sufficient.)
All task objects would have to support a few pseudo-messages as well:
||Any asynchronous messages sent after this call is issued would throw a
||The caller blocks until the task is closed and all pending requests are processed.|
In addition to the usual modifiers (
public, etc.), the
task keyword would accept the
$pooled(n) modifier, which would cause the
task to use a thread pool rather than a single thread to run the asynchronous requests. The
n specifies the desired pool size; the pool can grow if necessary, but should deflate to the original size when the additional threads aren’t required. The pseudo-field $pool_size returns the original
n argument to the
I present a server-side socket handler as an example of thread-pool usage in Chapter 8 of Taming Java Threads, an implementation of which is a good example of a pooled task. The basic idea is to create a single object whose job is to monitor a server-side socket. Every time a client connects, the socket-server object would grab a thread from a pool of dormant pre-created threads and set that thread to work servicing the client connection. If more clients tried to connect than there were pre-created threads, the socket server would create additional client-service threads, but these extra threads would be discarded once the connection closed. You could implement a socket server in the proposed syntax as follows:
Socket_server object uses a single background thread to handle the asynchronous
listen() request, which encapsulates the socket "accept" loop. As each client connects,
listen() asks a
Client_handler to handle the request by calling
handle() request executes on its own thread (because this is a
Note that every asynchronous message sent to a
$pooled $task is effectively handled on its own thread. Since a
$pooled $task is typically used to implement an autonomous operation, probably the best way to handle the potentially hideous synchronization problems associated with the access of state variables is for the
this reference in the
$asynchronous method to reference a unique copy of the object. That is, when you send an asynchronous request to a
$pooled $task, a
clone() operation is performed and the method’s
this reference will reference the clone. Communication between threads can be done by means of synchronized access to
Improvements to synchronized
$task eliminates the need for synchronization in many situations, all multithreaded systems cannot be implemented solely in terms of tasks. Consequently, the existing threading model needs to be updated as well. The
synchronized keyword has several flaws:
- You cannot specify a timeout value.
- You cannot interrupt a thread that is waiting to acquire a lock.
- You cannot safely acquire multiple locks. (Multiple locks should always be acquired in the same order.)
You can solve these problems by extending the syntax of
synchronized both to support a list of multiple arguments and to accept a timeout specification (specified in brackets, below). Here’s the syntax that I’d like:
||Acquire the locks on the
||Acquire the locks on the
||Obvious extension of the previous code.|
||Acquire a single lock with a one-second timeout.|
||Acquire the lock for
TimeoutException is a
RuntimeException derivative that is thrown if the wait times out.
A timeout is necessary, but not sufficient, for making the code robust. You also need to be able to terminate the wait to acquire the lock externally. Consequently, the
interrupt() method, when sent to a thread that is waiting to acquire a lock, should break the waiting thread out of the acquisition block by tossing a
SynchronizationException object. This exception should be a
RuntimeException derivative so that it would not have to be handled explicitly.
The main problem with these proposed modifications to the
synchronized syntax is that they would require changes at the bytecode level, which currently implements
synchronized using enter-monitor and exit-monitor instructions. These instructions take no arguments, so the bytecode definition would have to be extended to support the acquisition of multiple locks. This change is no more serious than the changes added to the JVM in Java 2, however, and would be backward compatible with existing Java code.
Another solvable problem is the most common deadlock scenario, where two threads are both waiting for each other to do something. Consider the following, admittedly contrived, example:
Imagine that one thread calls
a(), but is preempted after acquiring
lock1 but before acquiring
lock2. A second thread comes along and calls
lock2, but can’t get
lock1 because the first thread has it. The first thread now wakes up and tries to acquire
lock2, but can’t get it because thread two has it. Deadlock. The synchronize-on-multiple-objects syntax can solve this problem as follows:
The compiler (or VM) will rearrange the acquisition order so that
lock1 is always acquired first, thereby eliminating the deadlock.
But, it’s not always possible to use multiple acquisition, so it would be nice to provide some way to break the deadlock automatically. One simple strategy is to release the lock occasionally if you’ve acquired one lock and are waiting for a second. That is, instead of waiting forever, wait like this:
If everybody who’s waiting for the lock uses a slightly different timeout value, the deadlock will be broken and one of the threads will get to run. I propose the following syntax to represent the preceding code:
synchronized statement waits forever, but gives up the lock occasionally to break a potential deadlock. Ideally, the timeout used for each iteration would differ from the previous iteration by some random amount.
Improvements to wait() and notify()
notify() system also has problems:
- There is no way to detect whether
wait()has returned normally or because of a timeout.
- There is no way to implement a traditional condition variable that remains in a "signaled" state.
- Nested-monitor lockout can happen too easily.
The timeout-detection problem is easily solved by redefining
wait() to return a
boolean (rather than
true return value would indicate a normal return,
false would indicate a timeout.
The notion of a state-based condition variable is an important one. The variable can be set to a
false state in which waiting threads will block until the variable enters a
true state; any thread that waits on a
true condition variable is released immediately. (The
wait() call won’t block in this case.) You can support this functionality by extending the syntax of
notify() as follows:
|notify();||Release all waiting threads without changing the state of the underlying condition variable.|
|notify(true);||Set the condition variable’s state to true and release any waiting threads. Subsequent calls to
|notify(false);||Set the condition variable’s state to false (subsequent calls to
The nested-monitor-lockout problem is thornier, and I don’t have an easy solution. Nested-monitor lockout is a form of deadlock that occurs when the holder of some lock doesn’t release it before suspending itself. Here’s an (again, contrived) example that demonstrates the problem, but real-world examples are legion:
The problem is that two locks are involved in both the
put() operations: one on the
Stack object and another on the
LinkedList. Consider what happens when a thread tries to pop from an empty stack. The thread acquires both locks, then calls
wait(), which releases the lock on the
Stack object, but not the one on the list. If a second thread tries to push an object, it will be suspended forever at the
synchronized(list) statement and never be permitted to actually push an object. Since a nonempty stack is the condition that the first thread is waiting for, this is a deadlock. That is, the first thread will never return from
wait(), because the second thread can never reach the
notify(), because the first thread’s holding of the lock is preventing the second thread from reaching that code.
There are lots of obvious solutions in the current example: do everything with method-level synchronization, for example. In the real world, the solution is not usually so simple, though.
One possible solution is for
wait() to release all the locks that the current thread has acquired in the opposite order of acquisition and then reacquire them in the original acquisition order when the wait is satisfied. I can imagine that code that leveraged this behavior would be almost impossible for a human being to figure out, however, so I don’t think that this is really a viable solution. If anybody has any ideas, send me e-mail.
I’d also like to be able to wait for complex conditions to become true. For example:
c are any
Fixing the Thread class
The ability to support both preemptive and cooperative threads is essential in some server applications, especially if you intend to squeeze maximum performance out of the system. I suggest that the Java programming language has gone too far in simplifying the threading model, and that the Posix/Solaris notion of a "green thread" and a "lightweight process" (discussed in Chapter 1 of Taming Java Threads) should be supported by the Java programming language. This means, of course, that some JVM implementations (such as NT implementations) would have to simulate cooperative threads internally, and other JVMs would have to simulate preemptive threading, but adding these extensions to the JVM is reasonably easy to do.
Thread, then, should always be preemptive. That is, a Java programming language thread should work much like a Solaris lightweight process. The
Runnable interface can be used to define a Solaris-like "green thread" that must explicitly yield control to other green threads running on the same lightweight process.
For example, the current syntax of:
would effectively create a green thread for the
Runnable object and bind that green thread to the lightweight process represented by the
Thread object. This change in implementation is transparent to existing code since the effective behavior is the same as at present.
By thinking of
Runnable objects as green threads, you can expand the Java programming language’s existing syntax to support multiple green threads bound to a single lightweight process simply by passing several
Runnable objects to the
Thread constructor. (The green threads would be cooperative with respect to each other, but could be preempted by other green threads (
Runnable objects) running on other lightweight processes (
Thread objects)). For example, the following code would create a green thread for each of the runnable objects, and these green threads will share the lightweight process represented by the
The existing idiom of overriding
Thread and implementing
run() should still work, but it should map to a single green thread bound to a lightweight process. (The default
run() method in the
Thread() class would effectively create a second
Runnable object internally.)
More facilities should be added to the language to support inter-thread communication. Right now, the
PipedOutputStream classes can be used for this purpose, but they are much too inefficient for most applications. I propose the following additions to the
- Add a
wait_for_start()method that blocks until a thread’s
run()method starts up. (It would be okay if the waiting thread was released just before run was called.) This way one thread could create one or more auxiliary threads, and then be assured that the auxiliary threads were running before the creating thread would continue with some operation.
- Add (to the
Object=$receive()methods that would use an internal blocking queue to pass objects between threads. The blocking queue would be created automatically as a side effect of the first
$send()call would enqueue the object; the
$receive()call would block until an object was enqueued, and then return that object. Variants on these methods would support timeouts on both the enqueue and dequeue operations:
$send(Object o, long timeout)and
Internal support for reader-writer locks
The notion of a reader-writer lock should be built into the Java programming language. Reader-writer locks are discussed in Taming Java Threads (and elsewhere), but to summarize: a reader-writer lock enforces the rule that multiple threads can simultaneously access an object, but only one thread at a time could modify the object, and modifications cannot go on while accesses are in progress. The syntax for a reader-writer lock can be borrowed from that of the
Multiple threads would be able to enter
$reading blocks only if no thread was in a
$writing block for a given object. A thread that tries to enter a
$writing block while reads are in progress will block until the reading threads exit the
$reading blocks. Threads that try to enter a
$writing block while another thread is in a
$writing block will block until the writing thread exits the
If both reading and writing threads are waiting for access, the reading threads will get access first by default, but this default can be changed by modifying the
class definition with the
$writer_priority attribute. For example:
Access to partially constructed objects should be illegal
The JLS currently permits access to a partially created object. For example, a thread created within a constructor can access the object being constructed, even though that object might not be fully constructed. The behavior of the following code is undefined:
The thread that sets
x to -1 can run in parallel to the thread that sets
x to 0. Consequently, the value of
x is unpredictable.
One possible solution to this problem is to require that the
run() methods of threads started from within a constructor not execute until that constructor returns, even if the thread created by the constructor is of higher priority than the one that called
That is, the
start() request must be deferred until the constructor returns.
Alternatively, the Java programming language could permit the synchronization of constructors. In other words, the following code (which is currently illegal) would work as expected:
I think that the first approach is cleaner than the second one, but it is admittedly harder to implement.
The volatile keyword should always work as expected
The JLS requires that the ordering of volatile operations be preserved. Most JVM implementations simply ignore this part of the spec and shouldn’t. There are hosts of similar problems that come up in multi-processor situations that also need to be addressed in the JLS. If you’re interested in this side of the threading mess, Bill Pugh at the University of Maryland is spearheading this effort (see Resources).
The lack of good access control makes threading more difficult than necessary. Often, methods don’t have to be thread safe if you can guarantee that they are called only from synchronized subsystems. I’d tighten up the Java programming language’s notion of access privilege as follows:
- Require explicit use of the
packagekeyword to specify package access. I think that the very existence of a default behavior is a flaw in any computer language, and am mystified that a default access privilege even exists (and am even more mystified that the default is "package" rather than "private"). The Java programming language doesn’t supply the equivalent of a default keyword anywhere else. Even though the requirement for an explicit
packagespecifier would break existing code, it would make that code a lot easier to read and could eliminate whole classes of potential bugs (if the access privilege had been omitted in error rather than deliberately, for example).
private protected, which works just like
protecteddoes now, but does not permit package access.
- Permit the syntax
private privateto specify "implementation access:"
privateto all outside objects, even objects of the same class as the current object. The only reference permitted to the left of the (implicit or explicit) dot is
- Extend the syntax of
publicto grant access to specific classes. For example, the following code would permit objects of class
some_method(), but the method would be private with respect to all other classes of objects.
public(Fred) void some_method()
This proposal is different from the C++ "friend" mechanism, which grants a class full access to all private parts of another class. Here, I’m suggesting a tightly controlled access to a limited set of methods. This way one class could define an interface to another class that would be invisible to the rest of the system. An obvious variant is:
public(Fred, Wilma) void some_method()
Require all field definitions to be
privateunless they reference truly immutable objects or define
static finalprimitive types. Directly accessing the fields of a class violates two basic principles of OO design: abstraction and encapsulation. From the threading perspective, allowing direct access to fields just makes it easier to inadvertently have unsynchronized access to a field.
$propertykeyword. Objects tagged in this way are accessible to a "bean box" application that is using the introspection APIs defined in the
Classclass, but otherwise work identically to
private private. The
$propertyattribute should be applicable to both fields and methods so that existing JavaBean getter/setter methods could be easily defined as properties.
The notion of immutability (an object whose value cannot change once it’s created) is invaluable in multithreaded situations since read-only access to immutable objects doesn’t have to be synchronized. The Java programming language’s implementation of immutability isn’t tight enough for two reasons:
- It is possible for an immutable object be accessed before it’s fully created, and this access might yield an incorrect value for some field.
- The definition of immutable (a class, all of whose fields are
final) is too loose. Objects addressed by
finalreferences can indeed change state, even though the reference itself cannot change state.
The first problem is solved by not permitting threads to be started from within constructors (or by delaying the start request until just before the constructor returns).
The second problem can be solved by requiring that
final references point to immutable objects. That is, an object is really immutable only if all of its fields are final and all of the fields of any referenced objects are final as well. In order not to break existing code, this definition could be enforced by the compiler only when a class is explicitly tagged as immutable as follows:
$immutable tag, the use of
final in the field definitions could be optional.
Finally, a bug in the Java compiler makes it impossible to reliably create immutable objects when inner classes are on the scene. When a class has nontrivial inner classes (as most of mine do), the compiler often incorrectly prints the error message:
This message can appear even though the blank final is indeed initialized in every constructor. This bug has been in the compiler since inner classes were first introduced in version 1.1, and at this writing (three years later) the bug is still there. It’s about time that this bug was fixed.
Instance-level access of class-level fields
In addition to access privileges, there is also the problem that both class-level (
static) methods and instance (non-
static) methods can directly access class-level (
static) fields. This access is dangerous because synchronizing the instance method doesn’t grab the class-level lock, so a
synchronized static method can access the class field at the same time as a
synchronized instance method. The obvious solution to this problem is to require that non-immutable
static fields be accessed from instance methods via
static accessor methods. This requirement would mandate both compile and run-time checks, of course. Under these guidelines, the following code would be illegal:
g() can run in parallel and modify
x simultaneously (with undefined results). Remember, there are two locks here: the
static method acquires the lock associated with the
Class object and the nonstatic method acquires the lock associated with the instance. The compiler should either require the following structure when accessing non-immutable
static fields from instance methods:
Or the compiler should require the use of a reader/writer lock:
Alternatively — and this is the ideal solution — the compiler should automatically synchronize access to non-immutable static fields with a reader/writer lock so that the programmer wouldn’t have to worry about it.
Abrupt shutdown of daemon threads
Daemon threads are shut down abruptly when all the non-daemon threads terminate. This is a problem when the daemon has created some sort of global resource (such as a database connection or a temporary file), but hasn’t closed or destroyed that resource when it is terminated.
I’d solve that problem by making the rule that the JVM will not shut down the application if:
- Any non-daemon threads are running, or
- Any daemon threads are executing a
synchronizedblock of code.
The daemon thread is subject to abrupt termination as soon as it leaves the
synchronized block or
Bring back the stop(), suspend(), and resume() methods
This one may not be possible for practical reasons, but I’d like
stop() not to be deprecated (in both
ThreadGroup). I would change the semantics so that calling
stop() wouldn’t break your code, however. The problem with
stop(), you’ll remember, is that
stop() gives up any locks when the thread terminates, thereby potentially leaving the objects that the thread is working on in an unstable (partially modified) state. These objects can nonetheless be accessed since the stopped thread has released its lock on the object.
This problem can be solved by redefining the behavior of
stop() such that the thread would terminate immediately only if it is holding no locks. If it is holding locks, I’d like the thread to be terminated immediately after releasing the last one. You could implement this behavior with a mechanism similar to an exception toss. The stopped thread would set a flag that would be tested immediately after exiting all synchronized blocks. If the flag was set, an implicit exception would be thrown, but the exception would not be catchable and would not cause any output to be generated when the thread terminated. Note that Microsoft’s NT operating system doesn’t handle an abrupt externally implied stop very well. (It doesn’t notify dynamic-link libraries of the stop, so system-wide resource leaks can develop.) That’s why I’m recommending an exception-like approach that simply causes
run() to return.
The practical problem with this exception-style approach is that you’d have to insert code to test the "stopped" flag at the end of every
synchronized block, and this extra code would both slow down the program and make it larger. Another approach that comes to mind is to make
stop() implement a "lazy" stop in which the thread terminates the next time it calls
yield(). I’d also add
stopped() methods to
Thread (which would work much like
interrupted(), but would detect the "stop-requested" state). This solution isn’t as universal as the first, but is probably workable and wouldn’t have the overhead.
resume() methods should just be put back into the Java programming language. They’re useful, and I don’t like being treated like a kindergartner. Removing them simply because they are potentially dangerous — a thread can be holding a lock when suspended — is insulting. Let me decide whether or not I want to use them. Sun could always make it a run-time exception for
suspend() to be called if the receiving thread is holding any locks, or better yet, defer the actual suspension until the thread gives up its locks.
Blocking I/O should work correctly
You should be able to interrupt any blocking operation, not just just
sleep(). I discussed this issue in the context of sockets in Chapter 2 of Taming Java Threads, but right now, the only way to interrupt a blocking I/O operation on a socket is to close the socket, and there’s no way to interrupt a blocking I/O operation on a file. Once you initiate a read request, for example, the thread is blocked until it actually reads something. Even closing the file handle does not break you out of the read operation.
Moreover, your program must be able to timeout of a blocking I/O operation. All objects on which blocking operations can occur (such as InputStream objects) should support a method like this:
This is the equivalent to the Socket class’s
setSoTimeout(time) method. Similarly, you should be able to pass a timeout into the blocking call as an argument.
The ThreadGroup class
ThreadGroup should implement all the methods of
Thread that can change a thread’s state. I particularly want it to implement
join() so that I can wait for all threads in the group to terminate.
So that’s my list. As I said in the title of this article, if I were king… (sigh). I’m hoping that some of these changes (or their equivalent) will eventually migrate into the language. I really think that the Java language is a great programming language; but I also think that the Java threading model simply wasn’t thought-out well enough, and that’s a pity. The Java programming language is evolving, however, so there is a path to follow to improve the situation.