Aug 282005
 

I received this from a friend via email and I thought it was a very interesting comment. My thoughts on patterns has changed lately, but I still think they are valuable. Here’s the email:

I am unfortunately unable to make it to tonight’s CJUG so I just wanted to pass along some information. Since we’re all about reusable patterns, I found it odd that you didn’t mention Reactor or other related communications patterns in your Dr. Dobb’s article. I’ve attached some references for you on these patterns (along with the link http://www.cs.wustl.edu/~schmidt/patterns-ace.html )

Take care!

The attachments that came along are inline with the link given, so I won’t post them here. Here’s my response:

I have an interesting interaction with patterns. I use them all the time, but I rarely put a name to the face. I’ve written a few articles in the past and done a number of open source projects and my use of patterns in class naming, documentation and my writing is sporadic at best. It seems as though over time my staying up-to-date and well informed about patterns has dwindled and my communication about patterns has also tapered off. However, my use of patterns seems to be as consistent if not better as time progresses. I probably should be better at communicating and documenting my patterns considering that it helps people who haven’t encountered certain concepts more easily understand and get up to speed on things.

Aug 152005
 

Just debugging some Java over the weekend and one thing that floated around was that using STRACE on various Java threads produced output that had the error:

EAGAIN (Resource temporarily unavailable)

Poking around there is a bug open with Sun about how Java produces random numbers on JDK 1.4. Looks like it reads in the /jre/lib/security/java.security file to determine which file on the operating system to get random numbers from. On Linux this is /dev/random. /dev/random has the possibility of becoming empty because it is essentially a list of random numbers produced by watching various system resources. The solution appears to be to change the value in the security file to use urandom instead. The changes would look like this:

securerandom.source=file:/dev/urandom

/dev/urandom is will use /dev/random unless it is empty in which case it will use a pseudo-random number algorithm. Here is the bug for it:


http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4705093

Aug 142005
 

I received this on the Chicago Java User Group list and thought I would post my findings.

I’m not a Dr. Dobbs subscriber, so forgive me if this is redundant.

I tried to rewrite a lighweight, stateful IO-and-thread-based server as an NIO-and-selector-based one, but gave up when I ran into a hard limit on the number of channels that a selector can handle (I believe it was 64). I could’ve scaled this with a multi-threaded, multi-selector approach, but the result would have been even more complex than the original. Am I missing something? Thanks in advance.

This is an interesting comment. I’ve heard that NIO on indows has many problems. We have never seen these types of limits or other limitations with NIO running on Linux. Currently, the solution that the article covers is a single selector with a limit of 200 channels. I’ve done a little digging and seen that a 64 channel limit was a win32 library bug that only allowed 64 objects to be in a wait state during IO operations. The bug at Sun is:


http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4503092

This states that this issue was fixed in the 1.4.1 release. If this isn’t the case and you are running 1.5 or 1.4.2, you should re-open the issue with Sun, or if you are brave enough, with Microsoft (hehe). Many folks appear to have solved this using a multi-selector multi-threaded approach.

In terms of a multi-threaded approach, a general good rule of thumb is one selector per CPU. Therefore, if you need more bandwidth and have extra CPUs, you could implement a multi-selector multi-threaded solution. I’ve also seen people suggest that 2-3 selectors and threads per CPU is okay, but I’ve not tried any of these solutions in a high volume production application, so I can’t speak to performance, load and stability. I’d consider upgrading JDK versions to the latest of the line you are using (i.e. 1.4.2_08 for 1.4 and 1.5.0_04 for 1.5) to ensure that you have as many bug fixes in you JVM as possible. This might clear up the problems and not force you to use multiple threads.

Aug 072005
 

Here’s an email with a question about my NIO article and my response:

Hi Brian,

I very much enjoyed your article in the Dr. Dobbs September issue.

Was wondering whether it made sense to combine multi-threading with the selector. More specifically, after method processKeys calls method selector.selectedKeys, it could then delegate the I/O to worker threads to improve performance.


Interesting question. This is something I’ve thought about, but never implemented and put into the production Orbitz.com site. This would be a very interesting experiment because it could turn out to reduce overall performance and increase load as using multiple Selectors does, or it might increase performance without extra load. My initial thoughts on this were along the lines of:

1. It really depends on how much work the Selector is doing. In my examples the Selector is doing very little in addition to just reading and writing from the socket. Once it has all the data from the socket and has verified the HTTP message, it returns the data to the caller and allows the caller to perform post-processing (the execute threads). I’m not certain if the work of reading and writing bytes to and from the socket is an expensive operation. I’d definitely have to put some monitoring into the processKeys method to know for sure. If it turns out to be expensive the trade-off between the complexity of thread management, making the server CPU bound, possibly increasing load and performance of having more IO threads would need to be carefully considered.
2. You would definitely want to have a thread pool since spawning and GCing threads is not ideal.
3. You can easily determine the size of the thread pool because of the limit to the number of keys allowed in the Selector. Any smaller sizing than the maximum length of the Selector’s keys would introduce contention issues for threads in the pool, although these might not be apparent under low load.
4. Sizing is definitely very important when adding threading to NIO. You want to stay away from managing a queue of IO work and a smaller sized thread pool, because the queue could end up overflowing (unlikely, but possible).
5. Idle threads don’t take up much load but lots of working threads do. You need to be very careful about the load on the server. One problem I could forsee, which occurs with many Selectors, is that all of the threads in the JVM are doing a lot of work and end up thrashing the box. It is unclear whether or not having threads do the reading, writing and post processing would increase load or not, but it would be something to watch very closely.
6. Perhaps a better strategy would be simply to increase the number of NIO worker threads to something inline with the number of CPUs and server characteristics. For example, if the NIO implementation with a single Selector is putting around 0.4 load average on a box with 2 CPUs, perhaps using 4 Selectors would be a better solution. This would increase load, but not cause the box to start thrashing.
7. Since the Selector is in a single thread, using a simple round-robin pool of these Selector threads should work fine. Or, a priority based solution such that the thread whose Selector has the fewest keys outstanding is used.

All that said, I would probably feel more comfortable attempting to use the multiple Selector threads approach instead of the reading, writing and post-processing threads. This seems to maintain the semantics of a single thread doing IO work (or stated better as very few threads doing IO work).