Unit Testing Asynchronous Code

I try to avoid using code that instantiates threads from unit-tests. They're awkward to write, brittle and I would rather extract the controlling thread, using the test as the controller. However there are times when it's unavoidable.

Here's an example. A PriceFinder is a class that goes and retrieves a price for a symbol asyncronously, returning immediately. Sometime later it receives a response and performs a callback on a handler. I've left out the implementation of how it actually does this (maybe a web-service, database call or JMS message).

public class PriceFinder {
  public void findPrice(String symbol, PriceHandler handler) { ... }
}

public interface PriceHandler {
  void foundPrice(Price price);
}

To test this, a handler can be used from the test case that records what is passed in. The test can then wait for a specified time and assert that the correct result is received. If the asyncronous code does not complete within the specified time, the test will fail (suggesting the code is either running very slowly or is never going to complete).

public class PriceFinderTest extends TestCase {

  private PriceFinder finder = new PriceFinder();
  private Price receivedPrice;

  public void testRetrievesAValidPrice() throws Exception {
    finder.findPrice("MSFT", new PriceHandler() {
      public void foundPrice(Price price) {
        receivedPrice = price;
      }
    });

    // Smelly!
    Thread.sleep(2000); // Wait two seconds.
    assertNotNull("Expected a price", receivedPrice);
  }

}

However, this sucks as it will slow your test suite right down if you have loads of tests using Thread.sleep().

A less time consuming way to do it is by using wait() and notify() on a lock. The handler notifies the lock and the test waits for this notification. In case this notification never happens, a timeout is used when calling wait().

public class PriceFinderTest extends TestCase {

  private PriceFinder finder = new PriceFinder();
  private Price receivedPrice;
  private Object lock = new Object();

  public void testRetrievesAValidPrice() throws Exception {
    finder.findPrice("MSFT", new PriceHandler() {
      public void foundPrice(Price price) {
        receivedPrice = price;
        synchronized(lock) {
          lock.notify();
        }
      }
    });

    synchronized(lock) {
      lock.wait(2000); // Wait two seconds or until the 
                       // monitor has been notified.
                       // But there's still a problem...
    } 
    assertNotNull("Expected a price", receivedPrice);
  }

}

This optimistic approach results in fast running tests while all is good. If the PriceFinder is behaving well, the test will not wait any longer than the PriceFinder takes to complete its work. If PriceFinder has a bug in it and never calls the handler, the test will fail in at most two seconds.

However, there's still a subtle issue. In the case that the PriceFinder is really fast, it may call notify() before the test starts wait()ing. The test will still pass, but it will wait until the timeout occurs.

This is where threading and synchronization get messy and beyond me. Doug Lea has a nice little class called Latch in his concurrency library (available in JDK 1.5 as CountDownLatch). A latch can only be locked once, once released it will never lock again.

public class PriceFinderTest extends TestCase {

  private PriceFinder finder = new PriceFinder();
  private Price receivedPrice;
  private Latch latch = new Latch();

  public void testRetrievesAValidPrice() throws Exception {
    finder.findPrice("MSFT", new PriceHandler() {
      public void foundPrice(Price price) {
        receivedPrice = price;
        latch.release();
      }
    });

    latch.attempt(2000); // Wait until the latch is released
                          // or a timeout occurs.
    assertNotNull("Expected a price", receivedPrice);
  }

}

That'll do it.

• links for 2005-02-19

Comments

Jon Tirsen

Surprisingly I don't agree with you, Joe! :-)

Sure, in the case you are presenting it would certainly work and it is probably the type of test I would use. Especially if the code for PriceFinder is not under my control.

In the general case testing asynchronuos code this way is very, very tedious. Generally speaking using a latch sets up a synchronization barrier between the unit-test thread and the controller-thread. The number of sync barriers grows very fast, for example, to test a delayer (something that receives a signal, waits a specific time and then sends a signal) the number of barriers you need is at least five.

A more useful pattern (and it can probably be used to test your example too) is as you say have the unit-test thread be the controller-thread for the component. To this you refactor the code under test so that it's main loop would look something like:

while(!isStopped()) {
runOnce();
}

The runOnce method would do the async processing (such as polling a blocking queue, a JMS queue or similar, it could potentially be a blocking operation).

Now in the test set up the context, run the runOnce method, and check the results. Voila! Only one thread needed, no intermittent failures, no deadlocks, no blocking threads and so on.

Joe Walnes

Absolutely! I would always choose to use the main JUnit thread as the controller, wherever possible and use an approach like you've described.

However, when you do actually need to test to test the threading code itself, this is the approach I'd use. If you find yourself doing this more than a few times it's a sign that you need to separate your threading code out from other areas of the application.

Greg Vaughn

If you're using util.concurrent, why can't findPrice() return a FutureResult? That's much more unit test friendly in general. If unit testing is hard, then your design needs work.

Neil Swingler

Nice one Greg, I always used util.concurrent channel to "post" a result back to the main testing thread but the FutureResult looks like a closer match.

confused

Am i stupid or does this not compile?
How the hell does the anonymous inner class get a reference to latch?

confused

im really dumb before my first coffee of the day. really, really dumb

dreamhead

Cool!
But we must add concurrency library to our project. It's boring.

user-id: null

I copied this technique into a unit test last week. It was just what I needed and helped root out a race condition. Thanks for sharing this.

Greg

Is there anyway one can access the confluence page you linked to? ("TDD Practices and Patterns")

Name:
Email:
URL: