Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

8296024: Usage of DirectBuffer::address should be guarded #11260

Closed
wants to merge 19 commits into from
Closed

8296024: Usage of DirectBuffer::address should be guarded #11260

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
23a6fd1
Guard usages of DirectBuffer::address
minborg Nov 21, 2022
9fafafa
Cleanup after comments
minborg Nov 21, 2022
9fcf2bb
Remove comments
minborg Nov 21, 2022
c081b4a
Rename methods
minborg Nov 21, 2022
eca7c38
Cleanup static declarations
minborg Nov 21, 2022
17a72e9
Update src/java.base/share/classes/sun/nio/ch/DirectBuffer.java
minborg Nov 21, 2022
a0de3bc
Update src/java.base/share/classes/jdk/internal/access/JavaNioAccess.…
minborg Nov 21, 2022
7097934
Remove comments
minborg Nov 21, 2022
88ed3aa
Merge branch 'guard-directbuffer-address' of https://github.com/minbo…
minborg Nov 21, 2022
0526e3d
Rework Acquisition
minborg Nov 22, 2022
06c764c
Remove redundant guard and fix comment permanently
minborg Nov 22, 2022
88ae68b
Refactor to try-finally handling
minborg Nov 23, 2022
682b6f5
Cleanup
minborg Nov 23, 2022
0546b23
Remove redundant method
minborg Nov 24, 2022
30aff11
Reformat and fix comment
minborg Nov 24, 2022
95c4058
Re-introduce address variables
minborg Nov 28, 2022
eedf41a
Re-introduce yet another address vairable
minborg Nov 28, 2022
619473e
Remove session exposure
minborg Nov 29, 2022
cd95bc8
Merge master
minborg Dec 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 44 additions & 32 deletions src/java.base/share/classes/com/sun/crypto/provider/GaloisCounterMode.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@

package com.sun.crypto.provider;

import jdk.internal.access.JavaNioAccess;
import jdk.internal.access.SharedSecrets;
import jdk.internal.misc.Unsafe;
import sun.nio.ch.DirectBuffer;
import sun.security.jca.JCAUtil;
Expand Down Expand Up @@ -92,6 +94,8 @@ abstract class GaloisCounterMode extends CipherSpi {

static final byte[] EMPTY_BUF = new byte[0];

private static final JavaNioAccess NIO_ACCESS = SharedSecrets.getJavaNioAccess();

private boolean initialized = false;

SymmetricCipher blockCipher;
Expand Down Expand Up @@ -907,44 +911,49 @@ int doLastBlock(GCMOperation op, ByteBuffer buffer, ByteBuffer src,
* and if dst will overwrite src data before src can be processed.
* If so, make a copy to put the dst data in.
*/
@SuppressWarnings("try")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After looking at the implementation some more, I'm not sure this need fixing? E.g. this method is just using the address to compute some overlap - and return a buffer sliced accordingly. There's no access to the buffer data (except for the last part which does a put). The access will fail if the session is closed from underneath. I don't think this can crash the VM (in fact this code did not have a reachability fence to begin with).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well spotted. I will remove the guarding here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This @SuppressWarnings annotation is no longer needed:

Suggested change
@SuppressWarnings("try")

ByteBuffer overlapDetection(ByteBuffer src, ByteBuffer dst) {
if (src.isDirect() && dst.isDirect()) {
DirectBuffer dsrc = (DirectBuffer) src;
DirectBuffer ddst = (DirectBuffer) dst;

// Get the current memory address for the given ByteBuffers
long srcaddr = dsrc.address();
long dstaddr = ddst.address();
try (var srcGuard = NIO_ACCESS.acquireSession(src);
var dstGuard = NIO_ACCESS.acquireSession(dst)) {

// Find the lowest attachment that is the base memory address
// of the shared memory for the src object
while (dsrc.attachment() != null) {
srcaddr = ((DirectBuffer) dsrc.attachment()).address();
dsrc = (DirectBuffer) dsrc.attachment();
}
// Get the current memory address for the given ByteBuffers
long srcaddr = dsrc.address();
long dstaddr = ddst.address();

// Find the lowest attachment that is the base memory address
// of the shared memory for the dst object
while (ddst.attachment() != null) {
dstaddr = ((DirectBuffer) ddst.attachment()).address();
ddst = (DirectBuffer) ddst.attachment();
}
// Find the lowest attachment that is the base memory address
// of the shared memory for the src object
while (dsrc.attachment() != null) {
srcaddr = ((DirectBuffer) dsrc.attachment()).address();
dsrc = (DirectBuffer) dsrc.attachment();
}

// If the base addresses are not the same, there is no overlap
if (srcaddr != dstaddr) {
return dst;
}
// At this point we know these objects share the same memory.
// This checks the starting position of the src and dst address
// for overlap.
// It uses the base address minus the passed object's address to
// get the offset from the base address, then add the position()
// from the passed object. That gives up the true offset from
// the base address. As long as the src side is >= the dst
// side, we are not in overlap.
if (((DirectBuffer) src).address() - srcaddr + src.position() >=
((DirectBuffer) dst).address() - dstaddr + dst.position()) {
return dst;
// Find the lowest attachment that is the base memory address
// of the shared memory for the dst object
while (ddst.attachment() != null) {
dstaddr = ((DirectBuffer) ddst.attachment()).address();
ddst = (DirectBuffer) ddst.attachment();
}

// If the base addresses are not the same, there is no overlap
if (srcaddr != dstaddr) {
return dst;
}
// At this point we know these objects share the same memory.
// This checks the starting position of the src and dst address
// for overlap.
// It uses the base address minus the passed object's address to
// get the offset from the base address, then add the position()
// from the passed object. That gives up the true offset from
// the base address. As long as the src side is >= the dst
// side, we are not in overlap.
if (((DirectBuffer) src).address() - srcaddr + src.position() >=
((DirectBuffer) dst).address() - dstaddr + dst.position()) {
return dst;
}
}

} else if (!src.isDirect() && !dst.isDirect()) {
Expand Down Expand Up @@ -1510,6 +1519,7 @@ public int doFinal(byte[] in, int inOfs, int inLen, byte[] out,
* data. If the verification fails, the 'dst' left to it's original
* values if crypto was in-place; otherwise 'dst' is zeroed
*/
@SuppressWarnings("try")
@Override
public int doFinal(ByteBuffer src, ByteBuffer dst)
throws IllegalBlockSizeException, AEADBadTagException,
Expand Down Expand Up @@ -1585,8 +1595,10 @@ public int doFinal(ByteBuffer src, ByteBuffer dst)
int ofs = dst.arrayOffset() + dst.position();
Arrays.fill(dst.array(), ofs , ofs + len, (byte)0);
} else {
Unsafe.getUnsafe().setMemory(((DirectBuffer)dst).address(),
len + dst.position(), (byte)0);
try (var guard = NIO_ACCESS.acquireSession(dst)) {
Unsafe.getUnsafe().setMemory(((DirectBuffer) dst).address(),
len + dst.position(), (byte) 0);
}
}
throw new AEADBadTagException("Tag mismatch");
}
Expand Down
20 changes: 19 additions & 1 deletion src/java.base/share/classes/java/nio/Buffer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -778,6 +778,14 @@ final void checkSession() {
// setup access to this package in SharedSecrets
SharedSecrets.setJavaNioAccess(
new JavaNioAccess() {

// We are not using a lambda here because this would create
// a circular dependency with lambda factories.
static final JavaNioAccess.SessionAcquisition NO_OP_CLOSE = new JavaNioAccess.SessionAcquisition() {
@Override
public void close() {}
};

@Override
public BufferPool getDirectBufferPool() {
return Bits.BUFFER_POOL;
Expand Down Expand Up @@ -823,7 +831,7 @@ public MemorySegment bufferSegment(Buffer buffer) {
}

@Override
public Runnable acquireSession(Buffer buffer, boolean async) {
public Runnable acquireSessionOrNull(Buffer buffer, boolean async) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If allocation/performance is an issue, a relatively straightforward way to adjust the code would be to let go of the TWR entirely. E.g. we have code that does this:

Buffer b = ...
try {
    // use buffer.address();
} finally {
    Reference.reachabilityFence(b);
}

We could transform to this:

Buffer b = ...
acquire(b); // calls MemorySessionImpl.acquire0 if session is not null
try {
    // use buffer.address();
} finally {
    release(b); // calls MemorySessionImpl.release0 if session is not null (and maybe adds a reachability fence, just in case)
}

This leads to a simpler patch that is probably easier to validate. The remaining IOUtil code will be using a different idiom, and perhaps we can, at some point, go back and make that consistent too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AutoCloseable experiment was interesting to try but I think you are right, acquire try { .. } finally release would be simpler and also remove the concerns about the performance critical code paths.

var session = buffer.session();
if (session == null) {
return null;
Expand All @@ -835,6 +843,16 @@ public Runnable acquireSession(Buffer buffer, boolean async) {
return session::release0;
}

@Override
public SessionAcquisition acquireSession(Buffer buffer) {
var session = buffer.session();
if (session == null) {
return NO_OP_CLOSE;
}
session.acquire0();
return session::release0;
}

@Override
public void force(FileDescriptor fd, long address, boolean isSync, long offset, long size) {
MappedMemoryUtils.force(fd, address, isSync, offset, size);
Expand Down
5 changes: 4 additions & 1 deletion src/java.base/share/classes/java/util/zip/Adler32.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
import jdk.internal.util.Preconditions;
import jdk.internal.vm.annotation.IntrinsicCandidate;

import static java.util.zip.ZipUtils.NIO_ACCESS;

/**
* A class that can be used to compute the Adler-32 checksum of a data
* stream. An Adler-32 checksum is almost as reliable as a CRC-32 but
Expand Down Expand Up @@ -87,6 +89,7 @@ public void update(byte[] b, int off, int len) {
*
* @since 1.8
*/
@SuppressWarnings("try")
@Override
public void update(ByteBuffer buffer) {
int pos = buffer.position();
Expand All @@ -96,7 +99,7 @@ public void update(ByteBuffer buffer) {
if (rem <= 0)
return;
if (buffer.isDirect()) {
try {
try (var guard = NIO_ACCESS.acquireSession(buffer)) {
adler = updateByteBuffer(adler, ((DirectBuffer)buffer).address(), pos, rem);
} finally {
Reference.reachabilityFence(buffer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated naming is a bit better but there are it still feels that that there are too many things going on at the use sites ("guard", "session", "reachability fence"). Ideally the acquire would return something with an accessor for the direct buffer address but that may not be possible. I wonder if changing NOP_CLOSE.close to do reachabilityFence(this) would work as expected and improve many of these use sites?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can certainly be done. We could, for example, add a new method to the JavaNioAccess interface:

AddressAcquisition acquireSession2(Buffer buffer); // to be renamed

This would allow us to go from:

            try (var guard = NIO_ACCESS.acquireSession(buffer)) {
                adler = updateByteBuffer(adler, ((DirectBuffer)buffer).address(), pos, rem);
            } finally {
                Reference.reachabilityFence(buffer);
            }

to

            try (var guard = NIO_ACCESS.acquireSession2(buffer)) {
                adler = updateByteBuffer(adler, guard.address(), pos, rem);
            }

Although this looks much simpler and concise, it means "a new object is created for each invocation" (*). This also allows us to remove the @SupressWarnings("try") annotations.

Here is how the undercarriage might look like:

                @Override
                public AddressAcquisition acquireSession2(Buffer buffer) {
                    var session = buffer.session();
                    if (session == null) {
                        return AddressAcquisition.create(buffer, () -> {});
                    }
                    session.acquire0();
                    return AddressAcquisition.create(buffer, session::release0);
                }

and

sealed interface AddressAcquisition extends AutoCloseable {

        long address();

        @Override
        void close();

        static AddressAcquisition create(Buffer buffer, Runnable closeActions) {
            return new AddressAcquisitionImpl(buffer, closeActions);
        }

        final class AddressAcquisitionImpl implements AddressAcquisition {

            private final DirectBuffer directBuffer;
            private final Runnable closeAction;

            public AddressAcquisitionImpl(Buffer buffer,
                                          Runnable closeAction) {
                this.directBuffer = (DirectBuffer) buffer;
                this.closeAction = closeAction;
            }

            @Override
            public long address() {
                return directBuffer.address();
            }

            @Override
            public void close() {
                try {
                    closeAction.run();
                } finally {
                    Reference.reachabilityFence(directBuffer);
                }
            }
        }

    }

(*) In reality, a representation of the object might be allocated on the stack instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this looks much simpler and concise, it means "a new object is created for each invocation"

My comment was actually to see if DirectBuffer could extend AutoCloseable so that the acquire returns "this" for the non-session case. Doing the equivalent for the session case might leak into MemorySessionImpl implementing DirectBuffer which you probably don't want to do. If NOP_CLOSE.close can do the Reference.reachabilityFence(this) then it would at least improve some of the use-sites.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this looks much simpler and concise, it means "a new object is created for each invocation"

My comment was actually to see if DirectBuffer could extend AutoCloseable so that the acquire returns "this" for the non-session case. Doing the equivalent for the session case might leak into MemorySessionImpl implementing DirectBuffer which you probably don't want to do. If NOP_CLOSE.close can do the Reference.reachabilityFence(this) then it would at least improve some of the use-sites.

Not sure that is simpler. ByteBuffer <: AutoCloseable doesn't seem to make sense to me. I'm also not sure how much object allocation (all this stuff will become value types) should be the driving factor in these code paths.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I didn't mean BB <: AC but to see if we avoid needing to wrap it because this PR is touching several low level and performance critical code paths. For the faraway places then having the close do a Reference.reachabilityFence(this) should avoid the surprise that the buffer has to kept alive even though it appears that the try-with-resources is doing it already.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reworked Acquisition. I think we could merge the old and new way in a separate PR.

Expand Down
5 changes: 4 additions & 1 deletion src/java.base/share/classes/java/util/zip/CRC32.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
import jdk.internal.util.Preconditions;
import jdk.internal.vm.annotation.IntrinsicCandidate;

import static java.util.zip.ZipUtils.NIO_ACCESS;

/**
* A class that can be used to compute the CRC-32 of a data stream.
*
Expand Down Expand Up @@ -87,6 +89,7 @@ public void update(byte[] b, int off, int len) {
*
* @since 1.8
*/
@SuppressWarnings("try")
@Override
public void update(ByteBuffer buffer) {
int pos = buffer.position();
Expand All @@ -96,7 +99,7 @@ public void update(ByteBuffer buffer) {
if (rem <= 0)
return;
if (buffer.isDirect()) {
try {
try (var guard = NIO_ACCESS.acquireSession(buffer)) {
crc = updateByteBuffer(crc, ((DirectBuffer)buffer).address(), pos, rem);
} finally {
Reference.reachabilityFence(buffer);
Expand Down
5 changes: 4 additions & 1 deletion src/java.base/share/classes/java/util/zip/CRC32C.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
import jdk.internal.vm.annotation.IntrinsicCandidate;
import sun.nio.ch.DirectBuffer;

import static java.util.zip.ZipUtils.NIO_ACCESS;

/**
* A class that can be used to compute the CRC-32C of a data stream.
*
Expand Down Expand Up @@ -160,6 +162,7 @@ public void update(byte[] b, int off, int len) {
* at the buffer's position. Upon return, the buffer's position will be
* updated to its limit; its limit will not have been changed.
*/
@SuppressWarnings("try")
@Override
public void update(ByteBuffer buffer) {
int pos = buffer.position();
Expand All @@ -171,7 +174,7 @@ public void update(ByteBuffer buffer) {
}

if (buffer.isDirect()) {
try {
try (var guard = NIO_ACCESS.acquireSession(buffer)) {
crc = updateDirectByteBuffer(crc, ((DirectBuffer) buffer).address(),
pos, limit);
} finally {
Expand Down
17 changes: 11 additions & 6 deletions src/java.base/share/classes/java/util/zip/Deflater.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
import jdk.internal.util.Preconditions;
import sun.nio.ch.DirectBuffer;

import static java.util.zip.ZipUtils.NIO_ACCESS;

/**
* This class provides support for general purpose compression using the
* popular ZLIB compression library. The ZLIB compression library was
Expand Down Expand Up @@ -331,14 +333,15 @@ public void setDictionary(byte[] dictionary) {
* @see Inflater#inflate
* @see Inflater#getAdler()
*/
@SuppressWarnings("try")
public void setDictionary(ByteBuffer dictionary) {
synchronized (zsRef) {
int position = dictionary.position();
int remaining = Math.max(dictionary.limit() - position, 0);
ensureOpen();
if (dictionary.isDirect()) {
long address = ((DirectBuffer) dictionary).address();
try {
try (var guard = NIO_ACCESS.acquireSession(dictionary)) {
setDictionaryBuffer(zsRef.address(), address + position, remaining);
} finally {
Reference.reachabilityFence(dictionary);
Expand Down Expand Up @@ -553,6 +556,7 @@ public int deflate(ByteBuffer output) {
* @throws IllegalArgumentException if the flush mode is invalid
* @since 1.7
*/
@SuppressWarnings("try")
public int deflate(byte[] output, int off, int len, int flush) {
Preconditions.checkFromIndexSize(len, off, output.length, Preconditions.AIOOBE_FORMATTER);
if (flush != NO_FLUSH && flush != SYNC_FLUSH && flush != FULL_FLUSH) {
Expand Down Expand Up @@ -587,7 +591,7 @@ public int deflate(byte[] output, int off, int len, int flush) {
inputPos = input.position();
int inputRem = Math.max(input.limit() - inputPos, 0);
if (input.isDirect()) {
try {
try (var guard = NIO_ACCESS.acquireSession(input)) {
long inputAddress = ((DirectBuffer) input).address();
result = deflateBufferBytes(zsRef.address(),
inputAddress + inputPos, inputRem,
Expand Down Expand Up @@ -678,6 +682,7 @@ public int deflate(byte[] output, int off, int len, int flush) {
* @throws ReadOnlyBufferException if the given output buffer is read-only
* @since 11
*/
@SuppressWarnings("try")
public int deflate(ByteBuffer output, int flush) {
if (output.isReadOnly()) {
throw new ReadOnlyBufferException();
Expand Down Expand Up @@ -710,7 +715,7 @@ public int deflate(ByteBuffer output, int flush) {
inputPos = this.inputPos;
if (output.isDirect()) {
long outputAddress = ((DirectBuffer) output).address();
try {
try (var guard = NIO_ACCESS.acquireSession(output)) {
result = deflateBytesBuffer(zsRef.address(),
inputArray, inputPos, inputLim - inputPos,
outputAddress + outputPos, outputRem,
Expand All @@ -731,10 +736,10 @@ public int deflate(ByteBuffer output, int flush) {
int inputRem = Math.max(input.limit() - inputPos, 0);
if (input.isDirect()) {
long inputAddress = ((DirectBuffer) input).address();
try {
try (var inGuard = NIO_ACCESS.acquireSession(output)) {
if (output.isDirect()) {
long outputAddress = outputPos + ((DirectBuffer) output).address();
try {
try (var outGuard = NIO_ACCESS.acquireSession(output)) {
result = deflateBufferBuffer(zsRef.address(),
inputAddress + inputPos, inputRem,
outputAddress, outputRem,
Expand All @@ -758,7 +763,7 @@ public int deflate(ByteBuffer output, int flush) {
int inputOffset = ZipUtils.getBufferOffset(input);
if (output.isDirect()) {
long outputAddress = ((DirectBuffer) output).address();
try {
try (var guard = NIO_ACCESS.acquireSession(output)) {
result = deflateBytesBuffer(zsRef.address(),
inputArray, inputOffset + inputPos, inputRem,
outputAddress + outputPos, outputRem,
Expand Down
Loading