Comet and Java

By on May 22, 2008 6:04 am

One of the difficulties implementing Comet on Java is the lack of any acknowledgement in the current Servlet spec (v2.5) that any HTTP connection may be anything other than short-lived. Unlike many of the other components in the JavaEE stack, servlets are ubiquitous so we don't really have a choice to use an alternative.

Servlet version 3.0 is in the works, several of the people that blog at Comet Daily are on the Servlet spec expert group and want to see this oversight fixed, but it will be a while before the spec is done, and even longer before we can rely on it’s support everywhere.

In an ideal world you’d do something like this:

public void doPost() {
  // Some setup stuff
  while (holdConnectionOpen) {
    while (nothingToOutput) {
      passThreadBackToOS();
      doOutput();
    }
  }
  // Any close down stuff
}

The passThreadBackToOS() is the bit where we need to hand wave a bit – "this is not the thread you are looking for".

We need a way to get control back in several circumstances:

  • There is output (some Comet techniques require re-connection on output)
  • We've waited longer than 60 seconds (the 'why?' will have to wait for another post)
  • Something has gone wrong with the connection and we need to reset
  • The container wants to shut down

So we're going to have to register something to wake us up whenever we want to, but for now we're going to continue hand-waving.

This kind of code would be possible if we were to use full continuations – the kind that are available in RIFE. However full continuations don’t mix well with Comet. The whole point of wanting to reduce thread usage is to reduce resource consumption, and full continuations are fairly heavy on resources. (Interestingly continuations have been available at a JVM level for a long time, and there is talk of them being opened up at an API level before too long, however I suspect they'll still be the wrong tool for the job)

So given that we can't do passThreadBackToOS(), what are the options for Comet?

Just wait()

If you are only allowed to use the Servlet spec then you have no option but to call wait() and arrange for something to notify() us when it's time to do something.

public void doPost() {
  // Some setup stuff
  while (holdConnectionOpen) {
    while (nothingToOutput) {
      wakeThreadUsingNotify(lock);
      lock.wait();
      doOutput();
    }
  }
  // Any close down stuff
}

Clearly this doesn't scale well. DWR has a nice way to gradually fallback to polling if the server gets overloaded, but for many systems this won't be needed. What are the options if we can assume some support from the Servlet engine?

Throw back to the start of Servlet.service()

The expense with full continuations is in storing the stack. What if we drop the stack storage? We pass control back to the web container using a special exception (this is how continuations were implemented in RIFE under the covers), and then just re-call doPost() at a later date.

The API would then look something like this:

public void doPost() {
  if (!restartedConnection) {
    // Some setup stuff
  }
  while (holdConnectionOpen) {
    while (nothingToOutput) {
      continuation.suspend(); // throws an exception
      doOutput();
    }
  }
  // Any close down stuff
}

This is how Jetty supports asynchronous servlets in version 6.0. You can read about Ajax Continuations, but note that they are different from full continuations because they're not saving the stack, just restarting the Servlet.service() method. Grizzly has a similar API to the Jetty one, however it's implementation is closer to the idea below.

While this was the first available way to do async servlets, the general consensus now is that the implementation (using Exceptions to restart the method) is a little hacky, and better options are available.

Finish Servlet.service() and have something else signal end of the request

The most common pattern, and the one being adopted in Servlet Spec v3.0 is to have a ServletRequest.suspend() method. This says to the container: "When Servlet.service() ends, we're not done. Only end the request when something calls ServletRequest.complete() or ServletRequest.resume().

public void doPost() {
  if (!restartedConnection) {
    // Some setup stuff
  }
  while (holdConnectionOpen) {
    while (nothingToOutput) {
      if (outputPending) {
        doOutput();
      }
      request.suspend();
      return;
    }
  }
  // Any close down stuff
}

Putting it all together

 The tricky thing is that these 3 models do radically different things in order to wait. The stack essentially travels in 3 different directions, throwing unwinds it backwards, return unwinds it forwards and wait freezes it.

It's tricky to come up with a model for a system that can use all of these systems. If you are trying to unify them, then you should take a look at the PollHandler class in DWR, and how it deals with the Sleeper and Alarm interfaces. There is also a Getting Started guide.