Document DataBuffer/PooledDataBuffer and codecs

Issue: SPR-16156

spring-core/src/main/java/org/springframework/core/io/buffer/DataBuffer.java

@@ -40,9 +40,9 @@
  * <p>The {@linkplain #capacity() capacity} of a {@code DataBuffer} is expanded on demand,
  * similar to {@code StringBuilder}.
  *
- * <p>The main purpose of the {@code DataBuffer} abstraction is provide a convenient wrapper around
- * {@link ByteBuffer}, similar to Netty's {@link io.netty.buffer.ByteBuf}, that can also be used on
- * non-Netty platforms (i.e. Servlet).
+ * <p>The main purpose of the {@code DataBuffer} abstraction is to provide a convenient wrapper
+ * around {@link ByteBuffer} that is similar to Netty's {@link io.netty.buffer.ByteBuf}, but that
+ * can also be used on non-Netty platforms (i.e. Servlet).
  *
  * @author Arjen Poutsma
  * @since 5.0
@@ -237,14 +237,14 @@ public interface DataBuffer {
 	ByteBuffer asByteBuffer(int index, int length);
 
 	/**
-	 * Expose this buffer's data as an {@link InputStream}. Both data and position are
+	 * Expose this buffer's data as an {@link InputStream}. Both data and read position are
 	 * shared between the returned stream and this data buffer.
 	 * @return this data buffer as an input stream
 	 */
 	InputStream asInputStream();
 
 	/**
-	 * Expose this buffer's data as an {@link OutputStream}. Both data and position are
+	 * Expose this buffer's data as an {@link OutputStream}. Both data and write position are
 	 * shared between the returned stream and this data buffer.
 	 * @return this data buffer as an output stream
 	 */

spring-core/src/main/java/org/springframework/core/io/buffer/DataBufferFactory.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2016 the original author or authors.
+ * Copyright 2002-2017 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -19,7 +19,7 @@
 import java.nio.ByteBuffer;
 
 /**
- * A factory for {@link DataBuffer}s,allowing for allocation and wrapping of
+ * A factory for {@link DataBuffer}s, allowing for allocation and wrapping of
  * data buffers.
  *
  * @author Arjen Poutsma
@@ -46,14 +46,16 @@ public interface DataBufferFactory {
 	DataBuffer allocateBuffer(int initialCapacity);
 
 	/**
-	 * Wrap the given {@link ByteBuffer} in a {@code DataBuffer}.
+	 * Wrap the given {@link ByteBuffer} in a {@code DataBuffer}. Unlike
+	 * {@linkplain #allocateBuffer(int) allocating}, wrapping does not use new memory.
 	 * @param byteBuffer the NIO byte buffer to wrap
 	 * @return the wrapped buffer
 	 */
 	DataBuffer wrap(ByteBuffer byteBuffer);
 
 	/**
-	 * Wrap the given {@code byte} array in a {@code DataBuffer}.
+	 * Wrap the given {@code byte} array in a {@code DataBuffer}. Unlike
+ 	 * {@linkplain #allocateBuffer(int) allocating}, wrapping does not use new memory.
 	 * @param bytes the byte array to wrap
 	 * @return the wrapped buffer
 	 */

src/docs/asciidoc/core.adoc

@@ -34,4 +34,6 @@ include::core/core-aop-api.adoc[leveloffset=+1]
 
 include::core/core-null-safety.adoc[leveloffset=+1]
 
+include::core/core-databuffer-codec.adoc[leveloffset=+1]
+
 include::core/core-appendix.adoc[leveloffset=+1]

src/docs/asciidoc/core/core-databuffer-codec.adoc

@@ -0,0 +1,150 @@
+[[databuffers]]
+= Data Buffers and Codecs
+
+== Introduction
+
+The `DataBuffer` interface defines an abstraction over byte buffers.
+The main reason for introducing it, and not use the standard `java.nio.ByteBuffer` instead, is Netty.
+Netty does not use `ByteBuffer`, but instead offers `ByteBuf` as an alternative.
+Spring's `DataBuffer` is a simple abstraction over `ByteBuf` that can also be used on non-Netty
+platforms (i.e. Servlet 3.1+).
+
+== `DataBufferFactory`
+
+The `DataBufferFactory` offers functionality to allocate new data buffers, as well as to wrap
+existing data.
+The `allocate` methods allocate a new data buffer, with a default or given capacity.
+Though `DataBuffer` implementation grow and shrink on demand, it is more efficient to give the
+capacity upfront, if known.
+The `wrap` methods decorate an existing `ByteBuffer` or byte array.
+Wrapping does not involve allocation: it simply decorates the given data with a `DataBuffer`
+implementation.
+
+There are two implementation of `DataBufferFactory`: the `NettyDataBufferFactory` which is meant
+to be used on Netty platforms, such as Reactor Netty.
+The other implementation, the `DefaultDataBufferFactory`, is used on other platforms, such as
+Servlet 3.1+ servers.
+
+== The `DataBuffer` interface
+
+The `DataBuffer` interface is similar to `ByteBuffer`, but offers a number of advantages.
+Similar to Netty's `ByteBuf`, the `DataBuffer` abstraction offers independent read and write
+positions.
+This is different from the JDK's `ByteBuffer`, which only exposes one position for both reading and
+writing, and a separate `flip()` operation to switch between the two  I/O operations.
+In general, the following invariant holds for the read position, write position, and the capacity:
+
+--
+`0` <= _read position_ <= _write position_ <= _capacity_
+--
+
+When reading bytes from the `DataBuffer`, the read position is automatically updated in accordance with
+the amount of data read from the buffer.
+Similarly, when writing bytes to the `DataBuffer`, the write position is updated with the amount of
+data written to the buffer.
+Also, when writing data, the capacity of a `DataBuffer` is automatically expanded, just like `StringBuilder`,
+`ArrayList`, and similar types.
+
+Besides the reading and writing functionality mentioned above, the `DataBuffer` also has methods to
+view a (slice of a) buffer as `ByteBuffer`, `InputStream`, or `OutputStream`.
+Additionally, it offers methods to determine the index of a given byte.
+
+There are two implementation of `DataBuffer`: the `NettyDataBuffer` which is meant to be used on
+Netty platforms, such as Reactor Netty.
+The other implementation, the `DefaultDataBuffer`, is used on other platforms, such as Servlet 3.1+
+servers.
+
+=== `PooledDataBuffer`
+
+The `PooledDataBuffer` is an extension to `DataBuffer` that adds methods for reference counting.
+The `retain` method increases the reference count by one.
+The `release` method decreases the count by one, and releases the buffer's memory when the count
+reaches 0.
+Both of these methods are related to _reference counting_, a mechanism that is explained below.
+
+Note that `DataBufferUtils` offers useful utility methods for releasing and retaining pooled data
+buffers.
+These methods take a plain `DataBuffer` as parameter, but only call `retain` or `release` if the
+passed data buffer is an instance of `PooledDataBuffer`.
+
+[[databuffer-reference-counting]]
+==== Reference Counting
+
+Reference counting is not a common technique in Java; it is much more common in other programming
+languages such as Object C and C++.
+In and of itself, reference counting is not complex: it basically involves tracking the number of
+references that apply to an object.
+The reference count of a `PooledDataBuffer` starts at 1, is incremented by calling `retain`,
+and decremented by calling `release`.
+As long as the buffer's reference count is larger than 0 the buffer will not be released.
+When the number decreases to 0, the instance will be released.
+In practice, this means that the reserved memory captured by the buffer will be returned back to
+the memory pool, ready to be used for future allocations.
+
+In general, _the last component to access a `DataBuffer` is responsible for releasing it_.
+Withing Spring, there are two sorts of components that release buffers: decoders and transports.
+Decoders are responsible for transforming a stream of buffers into other types (see <<codecs>> below),
+ and transports are responsible for sending buffers across a network boundary, typically as an HTTP message.
+This means that if you allocate data buffers for the purpose of putting them into an outbound HTTP
+message (i.e. client-side request or server-side response), they do not have to be released.
+The other consequence of this rule is that if you allocate data buffers that do not end up in the
+body, for instance because of a thrown exception, you will have to release them yourself.
+The following snippet shows a typical `DataBuffer` usage scenario when dealing with methods that
+throw exceptions:
+
+[source,java,indent=0]
+[subs="verbatim,quotes"]
+----
+DataBufferFactory factory = ...
+DataBuffer buffer = factory.allocateBuffer(); <1>
+boolean release = true; <2>
+try {
+  writeDataToBuffer(buffer); <3>
+  putBufferInHttpBody(buffer);
+  release = false; <4>
+} finally {
+  if (release) {
+    DataBufferUtils.release(buffer); <5>
+  }
+}
+
+private void writeDataToBuffer(DataBuffer buffer) throws IOException { <3>
+  ...
+}
+----
+<1> A new buffer is allocated.
+<2> A boolean flag indicates whether the allocated buffer should be released.
+<3> This example method loads data into the buffer. Note that the method can throw an `IOException`,
+and therefore a `finally` block to release the buffer is required.
+<4> If no exception occurred, we switch the `release` flag to `false` as the buffer will now be
+released as part of sending the HTTP body across the wire.
+<5> If an exception did occur, the flag is still set to `true`, and the buffer will be released
+here.
+
+=== DataBufferUtils
+
+`DataBufferUtils` contains various utility methods that operate on data buffers.
+It contains methods for reading a `Flux` of `DataBuffer` objects from an `InputStream` or NIO
+`Channel`, and methods for writing a data buffer `Flux` to an `OutputStream` or `Channel`.
+`DataBufferUtils` also exposes `retain` and `release` methods that operate on plain `DataBuffer`
+instances (so that casting to a `PooledDataBuffer` is not required).
+
+[codecs]
+== Codecs
+
+The `org.springframework.core.codec` package contains the two main abstractions for converting a
+stream of bytes into a stream of objects, or vice-versa.
+The `Encoder` is a strategy interface that encodes a stream of objects into an output stream of
+data buffers.
+The `Decoder` does the reverse: it turns a stream of data buffers into a stream of objects.
+Note that a decoder instance needs to consider <<databuffer-reference-counting, reference counting>>.
+
+Spring comes with a wide array of default codecs, capable of converting from/to `String`,
+`ByteBuffer`, byte arrays, and also codecs that support marshalling libraries such as JAXB and
+Jackson.
+Withing the context of Spring WebFlux, codecs are used to convert the request body into a
+`@RequestMapping` parameter, or to convert the return type into the response body that is sent back
+to the client.
+The default codecs are configured in the `WebFluxConfigurationSupport` class, and can easily be
+changed by overriding the `configureHttpMessageCodecs` when inheriting from that class.
+For more information about using codecs in WebFlux, see <<web-reactive#webflux-codecs, this section>>.

src/docs/asciidoc/web/webflux.adoc

@@ -434,7 +434,8 @@ for encoding and decoding the HTTP request and response body with Reactive Strea
 It builds on lower level contracts from `spring-core`:
 
 * {api-spring-framework}/core/io/buffer/DataBuffer.html[DataBuffer] -- abstraction for
-byte buffers -- e.g. Netty `ByteBuf`, `java.nio.ByteBuffer`
+byte buffers -- e.g. Netty `ByteBuf`, `java.nio.ByteBuffer`, see
+<<core#databuffers, Data Buffers and Codecs>>.
 * {api-spring-framework}/core/codec/Encoder.html[Encoder] -- serialize a stream of Objects
 to a stream of data buffers
 * {api-spring-framework}/core/codec/Decoder.html[Decoder] -- deserialize a stream of data