异步请求

Web MVC 异步请求集成，如下：

DeferredResult

Once the asynchronous request processing feature is enabled in the Servlet container, controller methods can wrap any supported controller method return value with DeferredResult, as the following example shows:

@GetMapping("/quotes")
@ResponseBody
public DeferredResult<String> quotes() {
  DeferredResult<String> deferredResult = new DeferredResult<>();
  // Save the deferredResult somewhere..
  return deferredResult;
}

// From some other thread...
deferredResult.setResult(result);

The controller can produce the return value asynchronously, from a different thread — for example, in response to an external event (JMS message), a scheduled task, or other event.

Callable

A controller can wrap any supported return value with java.util.concurrent.Callable, as the following example shows:

@PostMapping
public Callable<String> processUpload(final MultipartFile file) {
  return () -> "someView";
}

The return value can then be obtained by running the given task through the configured AsyncTaskExecutor.

Processing

Here is a very concise overview of Servlet asynchronous request processing:

  • A MockRequest can be put in asynchronous mode by calling request.startAsync(). The main effect of doing so is that the Servlet (as well as any filters) can exit, but the response remains open to let processing complete later.

  • The call to request.startAsync() returns AsyncContext, which you can use for further control over asynchronous processing. For example, it provides the dispatch method, which is similar to a forward from the Servlet API, except that it lets an application resume request processing on a Servlet container thread.

  • The MockRequest provides access to the current DispatcherType, which you can use to distinguish between processing the initial request, an asynchronous dispatch, a forward, and other dispatcher types.

DeferredResult processing works as follows:

  • The controller returns a DeferredResult and saves it in some in-memory queue or list where it can be accessed.

  • Web MVC calls request.startAsync().

  • Meanwhile, the MockDispatcher and all configured filters exit the request processing thread, but the response remains open.

  • The application sets the DeferredResult from some thread, and Web MVC dispatches the request back to the Servlet container.

  • The MockDispatcher is invoked again, and processing resumes with the asynchronously produced return value.

Callable processing works as follows:

  • The controller returns a Callable.

  • Web MVC calls request.startAsync() and submits the Callable to an AsyncTaskExecutor for processing in a separate thread.

  • Meanwhile, the MockDispatcher and all filters exit the Servlet container thread, but the response remains open.

  • Eventually the Callable produces a result, and Web MVC dispatches the request back to the Servlet container to complete processing.

  • The MockDispatcher is invoked again, and processing resumes with the asynchronously produced return value from the Callable.

For further background and context, you can also read the blog posts that introduced asynchronous request processing support in Web MVC 3.2.

Exception Handling

When you use a DeferredResult, you can choose whether to call setResult or setErrorResult with an exception. In both cases, Web MVC dispatches the request back to the Servlet container to complete processing. It is then treated either as if the controller method returned the given value or as if it produced the given exception. The exception then goes through the regular exception handling mechanism (for example, invoking @ExceptionHandler methods).

When you use Callable, similar processing logic occurs, the main difference being that the result is returned from the Callable or an exception is raised by it.

Interception

HandlerInterceptor instances can be of type AsyncHandlerInterceptor, to receive the afterConcurrentHandlingStarted callback on the initial request that starts asynchronous processing (instead of postHandle and afterCompletion).

HandlerInterceptor implementations can also register a CallableProcessingInterceptor or a DeferredResultProcessingInterceptor, to integrate more deeply with the lifecycle of an asynchronous request (for example, to handle a timeout event). See AsyncHandlerInterceptor for more details.

DeferredResult provides onTimeout(Runnable) and onCompletion(Runnable) callbacks. See the javadoc of DeferredResult for more details. Callable can be substituted for WebAsyncTask that exposes additional methods for timeout and completion callbacks.

Async Web MVC compared to WebFlux

The Servlet API was originally built for making a single pass through the Filter-Servlet chain. Asynchronous request processing lets applications exit the Filter-Servlet chain but leave the response open for further processing. The Web MVC asynchronous support is built around that mechanism. When a controller returns a DeferredResult, the Filter-Servlet chain is exited, and the Servlet container thread is released. Later, when the DeferredResult is set, an ASYNC dispatch (to the same URL) is made, during which the controller is mapped again but, rather than invoking it, the DeferredResult value is used (as if the controller returned it) to resume processing.

By contrast, Infra WebFlux is neither built on the Servlet API, nor does it need such an asynchronous request processing feature, because it is asynchronous by design. Asynchronous handling is built into all framework contracts and is intrinsically supported through all stages of request processing.

From a programming model perspective, both Web MVC and Infra WebFlux support asynchronous and Reactive Types as return values in controller methods. Web MVC even supports streaming, including reactive back pressure. However, individual writes to the response remain blocking (and are performed on a separate thread), unlike WebFlux, which relies on non-blocking I/O and does not need an extra thread for each write.

Another fundamental difference is that Web MVC does not support asynchronous or reactive types in controller method arguments (for example, @RequestBody, @RequestPart, and others), nor does it have any explicit support for asynchronous and reactive types as model attributes. Infra WebFlux does support all that.

Finally, from a configuration perspective the asynchronous request processing feature must be enabled at the Servlet container level.

HTTP Streaming

You can use DeferredResult and Callable for a single asynchronous return value. What if you want to produce multiple asynchronous values and have those written to the response? This section describes how to do so.

Objects

You can use the ResponseBodyEmitter return value to produce a stream of objects, where each object is serialized with an HttpMessageConverter and written to the response, as the following example shows:

@GetMapping("/events")
public ResponseBodyEmitter handle() {
  ResponseBodyEmitter emitter = new ResponseBodyEmitter();
  // Save the emitter somewhere..
  return emitter;
}

// In some other thread
emitter.send("Hello once");

// and again later on
emitter.send("Hello again");

// and done at some point
emitter.complete();

You can also use ResponseBodyEmitter as the body in a ResponseEntity, letting you customize the status and headers of the response.

When an emitter throws an IOException (for example, if the remote client went away), applications are not responsible for cleaning up the connection and should not invoke emitter.complete or emitter.completeWithError. Instead, the mockApi container automatically initiates an AsyncListener error notification, in which Web MVC makes a completeWithError call. This call, in turn, performs one final ASYNC dispatch to the application, during which Web MVC invokes the configured exception resolvers and completes the request.

SSE

SseEmitter (a subclass of ResponseBodyEmitter) provides support for Server-Sent Events, where events sent from the server are formatted according to the W3C SSE specification. To produce an SSE stream from a controller, return SseEmitter, as the following example shows:

@GetMapping(path="/events", produces=MediaType.TEXT_EVENT_STREAM_VALUE)
public SseEmitter handle() {
  SseEmitter emitter = new SseEmitter();
  // Save the emitter somewhere..
  return emitter;
}

// In some other thread
emitter.send("Hello once");

// and again later on
emitter.send("Hello again");

// and done at some point
emitter.complete();

While SSE is the main option for streaming into browsers, note that Internet Explorer does not support Server-Sent Events. Consider using Infra WebSocket messaging with SockJS fallback transports (including SSE) that target a wide range of browsers.

See also previous section for notes on exception handling.

Raw Data

Sometimes, it is useful to bypass message conversion and stream directly to the response OutputStream (for example, for a file download). You can use the StreamingResponseBody return value type to do so, as the following example shows:

@GetMapping("/download")
public StreamingResponseBody handle() {
  return new StreamingResponseBody() {
    @Override
    public void writeTo(OutputStream outputStream) throws IOException {
      // write...
    }
  };
}

You can use StreamingResponseBody as the body in a ResponseEntity to customize the status and headers of the response.

Reactive Types

Web MVC supports use of reactive client libraries in a controller . This includes the WebClient from today-webflux and others, such as Infra Data reactive data repositories. In such scenarios, it is convenient to be able to return reactive types from the controller method.

Reactive return values are handled as follows:

  • A single-value promise is adapted to, similar to using DeferredResult. Examples include Mono (Reactor) or Single (RxJava).

  • A multi-value stream with a streaming media type (such as application/x-ndjson or text/event-stream) is adapted to, similar to using ResponseBodyEmitter or SseEmitter. Examples include Flux (Reactor) or Observable (RxJava). Applications can also return Flux<ServerSentEvent> or Observable<ServerSentEvent>.

  • A multi-value stream with any other media type (such as application/json) is adapted to, similar to using DeferredResult<List<?>>.

Web MVC supports Reactor and RxJava through the ReactiveAdapterRegistry from today-core, which lets it adapt from multiple reactive libraries.

For streaming to the response, reactive back pressure is supported, but writes to the response are still blocking and are run on a separate thread through the configured AsyncTaskExecutor, to avoid blocking the upstream source such as a Flux returned from WebClient.

Context Propagation

It is common to propagate context via java.lang.ThreadLocal. This works transparently for handling on the same thread, but requires additional work for asynchronous handling across multiple threads. The Micrometer Context Propagation library simplifies context propagation across threads, and across context mechanisms such as ThreadLocal values, Reactor context, GraphQL Java context, and others.

If Micrometer Context Propagation is present on the classpath, when a controller method returns a reactive type such as Flux or Mono, all ThreadLocal values, for which there is a registered io.micrometer.ThreadLocalAccessor, are written to the Reactor Context as key-value pairs, using the key assigned by the ThreadLocalAccessor.

For other asynchronous handling scenarios, you can use the Context Propagation library directly. For example:

Java
// Capture ThreadLocal values from the main thread ...
ContextSnapshot snapshot = ContextSnapshot.captureAll();

// On a different thread: restore ThreadLocal values
try (ContextSnapshot.Scope scope = snapshot.setThreadLocals()) {
  // ...
}

The following ThreadLocalAccessor implementations are provided out of the box:

  • LocaleContextThreadLocalAccessor — propagates LocaleContext via LocaleContextHolder

  • RequestAttributesThreadLocalAccessor — propagates RequestAttributes via RequestContextHolder

The above are not registered automatically. You need to register them via ContextRegistry.getInstance() on startup.

For more details, see the documentation of the Micrometer Context Propagation library.

Disconnects

The Servlet API does not provide any notification when a remote client goes away. Therefore, while streaming to the response, whether through SseEmitter or reactive types, it is important to send data periodically, since the write fails if the client has disconnected. The send could take the form of an empty (comment-only) SSE event or any other data that the other side would have to interpret as a heartbeat and ignore.

Alternatively, consider using web messaging solutions that have a built-in heartbeat mechanism.

Configuration

The asynchronous request processing feature must be enabled at the Servlet container level. The MVC configuration also exposes several options for asynchronous requests.

Web MVC

The MVC configuration exposes the following options for asynchronous request processing:

  • Java configuration: Use the configureAsyncSupport callback on WebMvcConfigurer.

  • XML namespace: Use the <async-support> element under <mvc:annotation-driven>.

You can configure the following:

  • The default timeout value for async requests depends on the underlying Servlet container, unless it is set explicitly.

  • AsyncTaskExecutor to use for blocking writes when streaming with Reactive Types and for executing Callable instances returned from controller methods. The one used by default is not suitable for production under load.

  • DeferredResultProcessingInterceptor implementations and CallableProcessingInterceptor implementations.

Note that you can also set the default timeout value on a DeferredResult, a ResponseBodyEmitter, and an SseEmitter. For a Callable, you can use WebAsyncTask to provide a timeout value.