Support for Servlet 3.0/3.1 asynchronous request processing [SPR-8517]

[spring-projects-issues]

Chris Beams opened SPR-8517 and commented

---

Sub-tasks:

• Documentation for the Servlet 3, async support [SPR-9400] #14036 Documentation for the Servlet 3, async support

Issue Links:

• Support Servlet 3.0 (JSR-315) [SPR-5587] #10258 Support Servlet 3.0 (JSR-315)
• Add Servlet 3 async support related config options [SPR-9399] #14035 Add Servlet 3 async support related config options
• Add context to StaleAsyncWebRequestException to allow for better exception handling [SPR-9369] #14006 Add context to StaleAsyncWebRequestException to allow for better exception handling

12 votes, 23 watchers

[spring-projects-issues]

Christopher Hunt commented

My belief is that developers are looking for a pain-free method of delivering web services; in particular RESTful web services delivering JSON payloads given the rise of JavaScript Rich Internet Applications.

Spring MVC is a great way to deliver RESTful/JSON web services with a great deal of flexibility. I personally think that our approach in terms of leveraging Spring MVC instead of adopting JAX-RS pays off.

However I believe that developers want the "thread-per-request" capability. Indeed, I even believe that this should be the default behaviour offered to the users of Spring MVC - why would you not want request-per-thread providing that their implementation is easy to achieve?

Take something like this:

@RequestMapping(value = "ids", method = RequestMethod.GET)
@ResponseBody
public List<?> getAvailableIDs(HttpServletResponse httpResponse)
    throws IOException {
  List<?> result = null;

  Message<String> request = MessageBuilder.withPayload("").build();
  Message<?> reply = template.sendAndReceive(idsRequestChannel, request);

  if (reply != null) {
    Object payload = reply.getPayload();
    if (payload instanceof List<?>) {
      result = (List<?>) reply.getPayload();
    }
  }

  if (result == null) {
    httpResponse.sendError(HttpServletResponse.SC_NOT_FOUND);
  }

  return result;
}

Perhaps, if a future is returned, Spring MVC could process the future accordingly e.g.

@RequestMapping(value = "ids", method = RequestMethod.GET)
public Future<?> getAvailableIDs(final HttpServletResponse httpResponse)
    throws IOException {
  Message<String> request = MessageBuilder.withPayload("").build();
  template.send(idsRequestChannel, request);

  Future<List<?>> futureReply = new FutureTemplateReceiveTask<List<?>>(
      new TemplateReceiveCallable<List<?>>(template) {
        @Override
        public Message<?> poll() {
          // template receive timeout set to 0 so will return
          // immediately if no reply available.
          return template.receive();
        }

        @Override
        public List<?> process(Message<?> reply) throws Exception {
          // We're here because a reply is now available. We can
          // now process it.
          List<?> result = null;

          if (reply != null) {
            Object payload = reply.getPayload();
            if (payload instanceof List<?>) {
              result = (List<?>) reply.getPayload();
            }
          }

          if (result == null) {
            httpResponse
                .sendError(HttpServletResponse.SC_NOT_FOUND);
          }

          return result;
        }
      });

  return futureReply;
}

The main intention here is to illustrate that if a future is returned then Spring MVC understands that a response is coming later. I'm not even sure if I'm doing the right thing with the Spring Integration template in terms of polling, but hopefully you'll get the idea.

[spring-projects-issues]

Chris Beams commented

Added Jon to the watchers list. @Jon, since you've been pretty deep in async lately, perhaps you have some comments here?

[spring-projects-issues]

Craig commented

Could Spring make @Async annotations in servlets use Servlet 3.0 asynchronous processing instead of its current implementation when available? If that's not possible (I'm not expert, so perhaps that idea doesn't even make sense!), perhaps a similar annotation-based approach that falls back to "hogging" the thread when Servlet 3.0 support isn't available could be done?

[spring-projects-issues]

Christopher Hunt commented

A better code example based on using an async gateway from Spring Integration. An async gateway will return a Future. Here's the original code without async gateway:

@RequestMapping(value = "ids", method = RequestMethod.GET)
@ResponseBody
public List<?> getAvailableIDs(HttpServletResponse httpResponse)
    throws IOException {

  List<?> result = timeZoneServiceGateway.getAvailableIDs("");

  // Pass back a 404 if we've got nothing to return.
  if (result == null) {
    httpResponse.sendError(HttpServletResponse.SC_NOT_FOUND);
  }

  return result;
}

...and here's how an async gateway may be handled:

@RequestMapping(value = "ids", method = RequestMethod.GET)
@ResponseBody
public Future<List<String>> getAvailableIDs(
    final HttpServletResponse httpResponse) throws IOException {

  final Future<List<String>> futureResult = timeZoneServiceGateway
      .getAvailableIDs("");

  return new FutureTask<List<String>>(new Callable<List<String>>() {
    @Override
    public List<String> call() throws Exception {
      List<String> result = futureResult.get();

      // Pass back a 404 if we've got nothing to return.
      if (result == null) {
        httpResponse.sendError(HttpServletResponse.SC_NOT_FOUND);
      }
      return result;
    }

  });
}

[spring-projects-issues]

Christopher Hunt commented

Slight flaw in the above code - returning a future task here makes no sense as it will never be invoked - MVC is just seeing the future and nothing else. We want something like what I had before where the future task can be event driven.

[spring-projects-issues]

Arjen Poutsma commented

Rather than using a Future, Jon and I think a Callable seems more suitable. So the MVC handler code will look something like:

@RequestMapping("/hotels/{id}")
public Callable<ModelAndView> handle(@PathVariable final long id) {
    return new Callable<ModelAndView>() {
        public ModelAndView call() throws Exception {
            Hotel hotel = hotelServive.getHotel(id);
            return new ModelAndView("hotel", "hotel", hotel);
        }
    };
}

Supporting this should be relatively trivial: we'd just add a CallableMethodReturnValueHandler, which would use the Servlet 3.0 async stuff, like so:

public void handleReturnValue(Object returnValue,
                              MethodParameter returnType,
                              ModelAndViewContainer mavContainer,
                              NativeWebRequest webRequest) throws Exception {
    final Callable callable = (Callable) returnValue;
    final HttpServletRequest request = webRequest.getNativeRequest(HttpServletRequest.class);
    final HttpServletResponse response = webRequest.getNativeResponse(HttpServletResponse.class);
    final AsyncContext asyncContext = request.startAsync(request, response);
    asyncContext.start(new Runnable() {
        public void run() {
            try {
                Object result = callable.call();
                // use HandlerMethodReturnValueHandlers to handle the result
                asyncContext.complete();
            }
            catch (Exception ex) {
                // do error handling
            }
        }
    });
}

This CallableMethodReturnValueHandler would itself need access to all other HandlerMethodReturnValueHandlers as configured on the RequestMappingHandlerAdapter, so that it can resolve the result of Callable.call().

[spring-projects-issues]

Christopher Hunt commented

Thanks for this. Please note that I suggested a Future mainly because Spring Integration's async gateway returns one. I have a feeling that calling a gateway method asynchronously would be a reasonably common use-case.

[spring-projects-issues]

Adam Skogman commented

@Arjen, invoking asynchContext.start() on a runnable wrapping the returned callable would just dispatch an(other) thread to handle the callable. What I'd like to see is something where the async handler developer can invoke the same processing that Spring usually does on a returned object, but asynchronously. In pseudo code:

@RequestMapping("/hotels/{id}")
@Async
public void handle(@PathVariable final long id, final SpringAsyncContext springAsyncContext) {

// invoke some async processing here, like an asynch http client
myAsynchService.doSomeStuff(id, new Callback() {
    public void onServiceHasDataForMe(DomainObject do) {

        springAsyncContext.getModel().add(do);
        springAsyncContext.view("same-string-that-I-can-usually-return-from-handler");

        doSomeMoreStuffHere();

    }
}

Excuse the ugly example, but you get the point, surely.

[spring-projects-issues]

Rossen Stoyanchev commented

It looks like you want to be in control of spawning a tread and notify Spring MVC when ready rather than the other way around, i.e. let Spring MVC invoke a Callable that encapsulates what you need to get done asynchronously. Is that correct? If so why do you prefer to have it done this way and where would asyncContext.start(..) be invoked from?

[spring-projects-issues]

Adam Skogman commented

Hmm, in the situation I'm thinking of, a thread that is working a NIO selector is invoking a callback. It would make sense for it not to invoke the rendering of the view in the same thread, but rather defer that to the servlet threadpool. So, when my code calls to the springAsyncContext.view(...) then spring could use asyncContext.start(...) to spawn the actual view handling etc to the servlet pool and let the calling thread proceed... Tricky semantics perhaps...

Still, my point is that at some point in time, some thread that is quite probably not a servlet container thread will determine that all the data needed for the response has been gathered. It should then somehow "trigger" Spring to render the view. Only then should a thread from the servlet threadpool be assigned again to render a view etc. It would be nice if that view rendering step could reuse much of the same view technology that Spring MVC has, and encapsulate the fact that start() and complete() has to be called.

[spring-projects-issues]

Christopher Hunt commented

My original suggestion was to defer the creation of a response via Spring Integration. In this instance, a controller calls upon an SI gateway to deliver a message which in turn returns a Future (SI's async gateway). Then, return the Future back from the controller. This takes the creation of a response off the NIO thread.

[spring-projects-issues]

Rossen Stoyanchev commented

Adam, i don't think it works that way. AsyncContext.start(Runnable) needs to be called from the main (Servlet) processing thread to ensure the response isn't committed when it exits the service() method. To invoke a SI chain couldn't you call a synchronous Gateway?

[spring-projects-issues]

Rossen Stoyanchev commented

I stand corrected. It looks like the creation of an AsyncContext is sufficient so the response is not committed and the thread can be created by all available means.

What's nice about returning Callable<ModelAndView>, Callable<Account>, etc. is it is very simple. The framework creates the thread and takes care of remaining processing (view rendering, exception handling, etc.). There is no departure from the existing programming model. Alternatively we can let the controller start the thread, which is more work but also requires a mechanism to hand control back to Spring MVC for remaining processing.

[spring-projects-issues]

Rossen Stoyanchev commented

A series of blog posts is in progress that explain what's available for M1. The main choices are returning a Callable or a DeferredResult. The latter should meet Adam Skogman's concern.

I'm resolving this ticket as complete since the support is fairly complete. Also see related tickets and subtasks for additional planned work. Feel free to continue to comment and if necessary open new tickets with more specific additional requests and issues.