Here’s an email with a question about my NIO article and my response:

Hi Brian,

I very much enjoyed your article in the Dr. Dobbs September issue.

Was wondering whether it made sense to combine multi-threading with the selector. More specifically, after method processKeys calls method selector.selectedKeys, it could then delegate the I/O to worker threads to improve performance.

Interesting question. This is something I’ve thought about, but never implemented and put into the production Orbitz.com site. This would be a very interesting experiment because it could turn out to reduce overall performance and increase load as using multiple Selectors does, or it might increase performance without extra load. My initial thoughts on this were along the lines of:

1. It really depends on how much work the Selector is doing. In my examples the Selector is doing very little in addition to just reading and writing from the socket. Once it has all the data from the socket and has verified the HTTP message, it returns the data to the caller and allows the caller to perform post-processing (the execute threads). I’m not certain if the work of reading and writing bytes to and from the socket is an expensive operation. I’d definitely have to put some monitoring into the processKeys method to know for sure. If it turns out to be expensive the trade-off between the complexity of thread management, making the server CPU bound, possibly increasing load and performance of having more IO threads would need to be carefully considered.
2. You would definitely want to have a thread pool since spawning and GCing threads is not ideal.
3. You can easily determine the size of the thread pool because of the limit to the number of keys allowed in the Selector. Any smaller sizing than the maximum length of the Selector’s keys would introduce contention issues for threads in the pool, although these might not be apparent under low load.
4. Sizing is definitely very important when adding threading to NIO. You want to stay away from managing a queue of IO work and a smaller sized thread pool, because the queue could end up overflowing (unlikely, but possible).
5. Idle threads don’t take up much load but lots of working threads do. You need to be very careful about the load on the server. One problem I could forsee, which occurs with many Selectors, is that all of the threads in the JVM are doing a lot of work and end up thrashing the box. It is unclear whether or not having threads do the reading, writing and post processing would increase load or not, but it would be something to watch very closely.
6. Perhaps a better strategy would be simply to increase the number of NIO worker threads to something inline with the number of CPUs and server characteristics. For example, if the NIO implementation with a single Selector is putting around 0.4 load average on a box with 2 CPUs, perhaps using 4 Selectors would be a better solution. This would increase load, but not cause the box to start thrashing.
7. Since the Selector is in a single thread, using a simple round-robin pool of these Selector threads should work fine. Or, a priority based solution such that the thread whose Selector has the fewest keys outstanding is used.

All that said, I would probably feel more comfortable attempting to use the multiple Selector threads approach instead of the reading, writing and post-processing threads. This seems to maintain the semantics of a single thread doing IO work (or stated better as very few threads doing IO work).