RetainableByteBuffer as mutable #11801

gregw · 2024-05-16T08:09:03Z

Opening this PR against 12.1.x rather than 12.0.x, replacing #11598

I'm just going to summarize the objectives of this PR:

Homogenize the APIs used for buffer manipulation in utils, aggregators, accumulators and buffers. Currently we have several different naming conventions and semantics (e.g. is the buffer consumed or not).
Reduce the API impedance between key abstractions. Buffers from Content.Sources should be compatible with buffers from ByteBufferPools and both should be easily written to Content.Sinks
Separate the read-only APIs from read-write APIs. The API used should indicate if buffer mutation is permitted or not.
protect mutable APIs from being applied to retained buffers
Avoid or hide modal APIs. Abstract away from the BB fill/flush mode issues so that these are rarely a concern for most buffer manipulations.
Where possible, internal implementations should leave BBs in the appropriate mode to avoid excess flipping
Remove unnecessary copies and encourage retention where possible and desirable.
Create common mechanisms that could allow heuristics decisions regarding copy vs retain, so that common usage does not need to make a hard coded choice.
Compatible with current buffer and API usage. We want evolution not revolution.
Avoid the complaints about BufferUtil and flipToFill and flipToFlush

Small tweaks to the RBB API to make the concept more uniform throughout the codebase.

Too much noise in this PR. Will make another PR for it.

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/ByteBufferPool.java

lorban · 2024-05-21T08:29:15Z

...re/jetty-http2/jetty-http2-common/src/main/java/org/eclipse/jetty/http2/HTTP2Connection.java

            if (currentBuffer == null)
                throw new IllegalStateException();

            if (currentBuffer.hasRemaining())
                throw new IllegalStateException();

            currentBuffer.release();
-            networkBuffer = new NetworkBuffer();
+            networkBuffer = bufferPool.acquire(bufferSize, isUseInputDirectByteBuffers()).asMutable();


It looks odd that ByteBufferPool.acquire() returns non-mutable buffers.

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

...ty-http2/jetty-http2-common/src/main/java/org/eclipse/jetty/http2/internal/HTTP2Flusher.java

lorban · 2024-05-21T14:12:39Z

...ty-http2/jetty-http2-common/src/main/java/org/eclipse/jetty/http2/internal/HTTP2Flusher.java

@@ -339,6 +334,15 @@ private void finish()
        }
    }

+    private void release()
+    {
+        if (!released)


Shouldn't accumulator.release() be idempotent instead of using this released flag?

The underlying reference counter class is not idempotent, as it throws if you release beyond 0.
Hence we can't just call accumulator.release() without it throwing. This we need to boolean.

I think it is good that we check for over-releases

lorban · 2024-05-22T08:14:36Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/Retainable.java

+     */
+    default int getRetained()
+    {
+        return -1;


0 sounds like a more sensible default, doesn't it?

Maybe. or throw Unsupported operation exception, as it should only be used be real retainables

lorban · 2024-05-22T08:19:25Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+     * Creates a deep copy of this RetainableByteBuffer that is entirely independent
+     * @return A copy of this RetainableByteBuffer
+     */
+    default RetainableByteBuffer copy()


I'm wondering if that API isn't going to cause more harm than good as I fail to see when we'd like to make copies into newly allocated buffers, except in tests.

It is used in the take implementations.... which themselves are mostly used for tests..... but copy is something likely to be needed if usage of these APIs becomes more widespread, so I'd much rather we implement it and test it correctly.

If we don't use it, then it is not a problem. If we do, then it is needed.

I'd much rather have this API then see code doing BufferUtil.copy(rbb.getByteBuffer()), which could result in multiple copies

lorban · 2024-05-22T08:20:37Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+
+    /**
+     * @return the capacity
+     * @see #maxSize()


Isn't it the right moment to break backward compatibility and stop using int sizing?

The approach I've taken is to keep int usage for the APIs that are named the same as BB. So int capacity() and int remaining(). This means that existing code does not break.

But there are new equivalent methods that use long, so long maxSize() and long size() are the equivalent of capacity and remaining.

lorban · 2024-05-22T08:23:33Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+        if (!canRetain())
+            return new NonRetainableByteBuffer(slice);
+
+        retain();


It's good that you javadoc'ed that the originating RBB gets retained by slice(), but I remember having a lengthy discussion about slice() retaining or not with @sbordet, but I cannot remember the outcome, so we may need to revisit this later.

lorban · 2024-05-22T08:25:51Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+     * @param last true if this is the last write.
+     * @see org.eclipse.jetty.io.Content.Sink#write(boolean, ByteBuffer, Callback)
+     */
+    default void writeTo(Content.Sink sink, boolean last) throws IOException


I don't think we want to expose blocking APIs here.

It is just a convenience wrapper of the async API. I found myself passing Callback.NOOP too often, which I think is dangerous as you can get code that looks like it works on simple tests, but fails if something blocks. Far better to provide the API so that short cuts are not taken elsewhere.

Every call to buffer.writeTo(sink, false, Callback.NOOP) is a bug in waiting!

gregw · 2024-05-22T10:18:03Z

@lorban thanks for the review. I've answered a some of the questions and will work on some of the fixes/changes you suggest tomorrow.

lorban · 2024-05-22T15:21:51Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+        }
+
+        @Override
+        public Mutable slice(long length)


This implementation of slice() does not retain, unlike the version it overrides.

It doesn't need to. It retains the contained buffers and then returns an entirely new DynamicCapacity instance over those buffers. Thus it implements the contract. i.e it returns an independent set of pointers over shared data. When the nested buffers are consumed from either the original or the slice, they are released, so when both original and slice are consumed (or explicitly released), then so are the nested buffers

lorban · 2024-05-22T15:22:42Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+
+            if (shouldAggregate(bytes, length) && append(bytes))
+            {
+                bytes.release();


This breaks the Retainable contract: it's up to the caller to release.

No. See javadoc for add. It explicitly says that responsibility is passed. This is how add differs from append.

lorban · 2024-05-22T15:23:29Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/RetainableByteBuffer.java

+                return this;
+            }
+
+            _buffers.add(bytes);


This also breaks the Retainable contract: you're keeping a ref to the RBB after returning, so you should retain it.

Ditto, see javadoc for add

lorban · 2024-05-22T15:32:28Z

...tp2/jetty-http2-common/src/main/java/org/eclipse/jetty/http2/generator/HeadersGenerator.java

+            // generate first fragment with as HEADERS with possible priority
+            generateHeader(accumulator, FrameType.HEADERS, length, flags, streamId);
+            generatePriority(accumulator, priority);
+            accumulator.add(hpack.slice(maxHeaderBlockFragment));


If hpack.slice() retains (like the RBB impl does) then the RBB returned by slice() should be released here. If it does not retains (like the RBB.DynamicCapacity impl does) then this code is fine... but it's breaking the Retainable contract.

The slice is retained, but then added to the accumulator, so the contract is that the accumulator take over responsibility for releasing the slice. If I had appended, then I would need to release here.

@gregw that is the opposite of the documented behavior that Retainables must have.

@sbordet you explicitly asked me to provide the add semantic so we can "give" a RBB and not have to release it ourselves. The documented behaviour for Retainable is described as "typical" and as a "general rule", should I think is equivalent to a SHOULD rather than a MUST. The divergence from the norm is well documented in the signature of the add method.

We could instead do:

Suggested change

accumulator.add(hpack.slice(maxHeaderBlockFragment));

RetainableByteBuffer hpackSlice = hpack.slice(maxHeaderBlockFragment);

accumulator.append(hpackSlice);

hpackSlice.release();

But that is more verbose and means the reference counter is needlessly incremented and decremented, then we end up at the same place.

This is using exactly the same semantic as the existing org.eclipse.jetty.io.ByteBufferPool.Accumulator#append, which also deviates from the "general rule", but without any javadoc saying so!

gregw · 2024-05-22T23:42:57Z

@lorban can you go through my responses and resolve any that you are satisfied with. I still have a few TODOs I'm working on as a result of your feedback....

…1/10541/chunkAsRBB

gregw · 2024-05-27T02:50:13Z

@lorban Can you quickly go through and resolve any questions that you think are well enough answered?

lorban · 2024-05-28T13:17:59Z

@gregw the only point of contention I see is the behavior of accumulator.add() w.r.t the Retainable contract. I don't mind making an exception to the rules if that's properly javadoc'ed, as you did.

But this caught me by surprise, and I assume it could again in the future, more so if we multiply the exceptions. So I think there should be some hint in the method's name that it's not playing by the rules to suggest the caller of that API to go read the javadoc.

How about renaming add() to addPreRetained() or something like that? And maybe put it in bold at the top of the javadoc that an exception to the Retainable contract is being made?

gregw · 2024-05-29T21:19:21Z

But this caught me by surprise, and I assume it could again in the future, more so if we multiply the exceptions. So I think there should be some hint in the method's name that it's not playing by the rules to suggest the caller of that API to go read the javadoc.

@lorban I too was caught by surprise that we already have some accumulators that have this semantic behind a method called append, so this is not new behaviour.

How about renaming add() to addPreRetained() or something like that? And maybe put it in bold at the top of the javadoc that an exception to the Retainable contract is being made?

I'm open to change the name, but I don't like addPreRetained, @sbordet has referred to it as offer so we could use that name?

gregw added 30 commits May 16, 2024 17:41

Make chunk a RBB

4abb57b

Small tweaks to the RBB API to make the concept more uniform throughout the codebase.

Make chunk a RBB

974d4c4

Small tweaks to the RBB API to make the concept more uniform throughout the codebase.

revert EventsHandler change

74ce7c2

WIP

5fd2543

updates from review

6992609

updates from review

4776471

more tests

5723732

inline asReadOnly

67c302b

Added Appendable as a replacement for both Accumulator and Aggregator

010267e

Better slice implementation

d18c93a

protect from modification if retained

ecdc2de

avoid flip-flopping in append loops

d29fb05

long methods

b0fb6cd

use some appends

ecaf1b6

javadoc

f17395f

Fixed test

521dccd

appendable wrap

ec022ff

detailString

0d6a1c7

Revert NON_POOLING change

df3bf77

Too much noise in this PR. Will make another PR for it.

Testing Fixed

a5cd43f

Limit usage to buffer size requests

34620d7

Simplified hierarchy and naming

f7cf481

Simplified hierarchy and naming

4c1670d

more testing

4a726fb

improved test

eb1e17a

disable leaky tests

91e9561

removed redundant NetworkBuffer class

595323c

Better heuristic for retaining buffers

1cb541e

Updated BufferedResponseHandler

4442c49

Use RBB in BAEP

4481998

gregw mentioned this pull request May 16, 2024

Make Content.Chunk a RetainableByteBuffer #11798

Closed

gregw requested a review from lorban May 16, 2024 23:53

gregw added 12 commits May 17, 2024 10:46

update ee11

ad75635

take never retains

fda044d

take never retains

fefc233

work in progress

9012b73

fluent style

c50055b

javadoc

abeaf8e

javadoc

87e8f01

javadoc

1d8f0ac

Merge branch 'jetty-12.1.x' into fix/jetty-12.1/10541/chunkAsRBB

0012282

introduced take and takeFrom

293a225

take can return single buffer

8b112d7

Merge branch 'jetty-12.1.x' into fix/jetty-12.1/10541/chunkAsRBB

7fa0d3c

lorban reviewed May 22, 2024

View reviewed changes

gregw added 3 commits May 23, 2024 08:14

ByteBufferPool acquire returns Mutable

8630fb1

updates from review

077fd25

updates from review

cc9fe32

gregw added 5 commits May 23, 2024 11:52

Merge branch 'jetty-12.1.x' into fix/jetty-12.1/10541/chunkAsRBB

145684f

more updates from review

0a4c799

fixes for review updates

6b99ff7

Merge remote-tracking branch 'origin/jetty-12.1.x' into fix/jetty-12.…

ab59f9b

…1/10541/chunkAsRBB

fixes for review updates

022b83f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RetainableByteBuffer as mutable #11801

RetainableByteBuffer as mutable #11801

gregw commented May 16, 2024

lorban May 21, 2024

lorban May 21, 2024

gregw May 22, 2024

lorban May 22, 2024

gregw May 22, 2024

lorban May 22, 2024

gregw May 22, 2024 •

edited

lorban May 22, 2024

gregw May 22, 2024

lorban May 22, 2024

lorban May 22, 2024

gregw May 22, 2024 •

edited

gregw commented May 22, 2024

lorban May 22, 2024

gregw May 22, 2024

lorban May 22, 2024

gregw May 22, 2024

lorban May 22, 2024

gregw May 22, 2024

lorban May 22, 2024

gregw May 22, 2024

sbordet May 22, 2024

gregw May 23, 2024 •

edited

gregw commented May 22, 2024

gregw commented May 27, 2024

lorban commented May 28, 2024

gregw commented May 29, 2024

-            accumulator.add(hpack.slice(maxHeaderBlockFragment));
+            RetainableByteBuffer hpackSlice = hpack.slice(maxHeaderBlockFragment);
+            accumulator.append(hpackSlice);
+            hpackSlice.release();

RetainableByteBuffer as mutable #11801

Are you sure you want to change the base?

RetainableByteBuffer as mutable #11801

Conversation

gregw commented May 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw May 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw May 22, 2024 • edited

Choose a reason for hiding this comment

gregw commented May 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw May 23, 2024 • edited

Choose a reason for hiding this comment

gregw commented May 22, 2024

gregw commented May 27, 2024

lorban commented May 28, 2024

gregw commented May 29, 2024

gregw May 22, 2024 •

edited

gregw May 22, 2024 •

edited

gregw May 23, 2024 •

edited