Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8268673: Stack walk across optimized entry frame on fresh native thread fails #558

Closed

Conversation

JornVernee
Copy link
Member

@JornVernee JornVernee commented Jun 14, 2021

Hi,

When native code creates a new thread and calls a Panama upcall, and during that upcall a stack walk is triggered, getting the sender frame for the entry frame is not possible, and should not be attempted.

For JNI this case is handled already by indicating the end of the stack frame stream, but for Panama upcalls it is not, and the VM will either hit an assert or crash when trying to find the last Java frame before the entry frame (which does not exist in this case).

This patch adds handling for panama upcalls frames to frame::is_first_frame, which is used by the stack walking code to determine when to stop walking.

Thanks,
Jorn


Progress

  • Change must not contain extraneous whitespace
  • Change must be properly reviewed

Issue

  • JDK-8268673: Stack walk across optimized entry frame on fresh native thread fails

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/panama-foreign pull/558/head:pull/558
$ git checkout pull/558

Update a local copy of the PR:
$ git checkout pull/558
$ git pull https://git.openjdk.java.net/panama-foreign pull/558/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 558

View PR using the GUI difftool:
$ git pr show -t 558

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/panama-foreign/pull/558.diff

@@ -2306,6 +2306,7 @@ WB_ENTRY(void, WB_VerifyFrames(JNIEnv* env, jobject wb, jboolean log, jboolean u
tty_token = ttyLocker::hold_tty();
tty->print_cr("[WhiteBox::VerifyFrames] Walking Frames");
}
ResourceMark rm; // for verify
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran into an assert when testing because of a missing resource mark here, so I've added that as well.

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 14, 2021

👋 Welcome back jvernee! A progress list of the required criteria for merging this PR into foreign-memaccess+abi will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Ready for review label Jun 14, 2021
@mlbridge
Copy link

mlbridge bot commented Jun 14, 2021

Webrevs

Copy link
Collaborator

@mcimadamore mcimadamore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@openjdk
Copy link

openjdk bot commented Jun 14, 2021

@JornVernee This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268673: Stack walk across optimized entry frame on fresh native thread fails

Reviewed-by: mcimadamore

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the foreign-memaccess+abi branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the foreign-memaccess+abi branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Ready to be integrated label Jun 14, 2021
@JornVernee
Copy link
Member Author

/integrate

@openjdk
Copy link

openjdk bot commented Jun 14, 2021

Going to push as commit 2287ca5.

@openjdk openjdk bot closed this Jun 14, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Ready to be integrated rfr Ready for review labels Jun 14, 2021
@openjdk
Copy link

openjdk bot commented Jun 14, 2021

@JornVernee Pushed as commit 2287ca5.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227>
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112>

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227>
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112>

@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Jorn Vernee on panama-dev:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used
to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and can
confirm that the crashes are gone. However, copying bytes from heap to
native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the flag
it copies 13 null-bytes, though. 13 being the correct length of the
source buffer.

I guess this is not related to the stack walking problem. The only
thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227>
[2] method invoked during upcall, which copies bytes to the specified
segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112>

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Jorn Vernee on panama-dev:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used
to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and can
confirm that the crashes are gone. However, copying bytes from heap to
native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the flag
it copies 13 null-bytes, though. 13 being the correct length of the
source buffer.

I guess this is not related to the stack walking problem. The only
thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227>
[2] method invoked during upcall, which copies bytes to the specified
segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112>

@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

A reference to the first mentioned method is stored in a struct [1] (defined in fuse_operations.h) right here [2], which then gets passed to FUSE. FUSE then decides to call some of these methods when a corresponding file system access happens.

[1]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28>
[2]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227>
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 <https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112>

@mlbridge
Copy link

mlbridge bot commented Jun 15, 2021

Mailing list message from Jorn Vernee on panama-dev:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both invocation
modes should behave the same, but the implementations are completely
separate, so if setting the flag to false makes a difference there might
be a problem when passing one of the arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1]
(defined in fuse_operations.h) right here [2], which then gets passed
to FUSE. FUSE then decides to call some of these methods when a
corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only
used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and can
confirm that the crashes are gone. However, copying bytes from heap
to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the flag
it copies 13 null-bytes, though. 13 being the correct length of the
source buffer.

I guess this is not related to the stack walking problem. The only
thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

@mlbridge
Copy link

mlbridge bot commented Jul 8, 2021

Mailing list message from Jorn Vernee on panama-dev:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find
another problem with USE_INTRINSICS=true that seems to be a likely cause
of what you were seeing. Namely: when an upcall happens on a separate
native thread, the return value of the upcall was being corrupted when
detaching the thread from the JVM again. So, it might be the case that
the problem you were seeing was from an upcall returning an incorrect
result. (I did rule out that it's a problem with the way arguments are
passed).

I've just integrated a patch that fixes the issue:
https://github.com//pull/565

If you have an opportunity to test out the fix, that would be greatly
appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both
invocation modes should behave the same, but the implementations are
completely separate, so if setting the flag to false makes a
difference there might be a problem when passing one of the arguments
when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1]
(defined in fuse_operations.h) right here [2], which then gets passed
to FUSE. FUSE then decides to call some of these methods when a
corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only
used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and
can confirm that the crashes are gone. However, copying bytes from
heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the
flag it copies 13 null-bytes, though. 13 being the correct length
of the source buffer.

I guess this is not related to the stack walking problem. The only
thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jul 8, 2021

Mailing list message from Jorn Vernee on panama-dev:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find
another problem with USE_INTRINSICS=true that seems to be a likely cause
of what you were seeing. Namely: when an upcall happens on a separate
native thread, the return value of the upcall was being corrupted when
detaching the thread from the JVM again. So, it might be the case that
the problem you were seeing was from an upcall returning an incorrect
result. (I did rule out that it's a problem with the way arguments are
passed).

I've just integrated a patch that fixes the issue:
https://github.com//pull/565

If you have an opportunity to test out the fix, that would be greatly
appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both
invocation modes should behave the same, but the implementations are
completely separate, so if setting the flag to false makes a
difference there might be a problem when passing one of the arguments
when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1]
(defined in fuse_operations.h) right here [2], which then gets passed
to FUSE. FUSE then decides to call some of these methods when a
corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only
used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and
can confirm that the crashes are gone. However, copying bytes from
heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the
flag it copies 13 null-bytes, though. 13 being the correct length
of the source buffer.

I guess this is not related to the stack walking problem. The only
thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on foreign-jextract, where the issue was still present on 6f8f9e2 and fixed on 42e03fd.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find another problem with USE_INTRINSICS=true that seems to be a likely cause of what you were seeing. Namely: when an upcall happens on a separate native thread, the return value of the upcall was being corrupted when detaching the thread from the JVM again. So, it might be the case that the problem you were seeing was from an upcall returning an incorrect result. (I did rule out that it's a problem with the way arguments are passed).

I've just integrated a patch that fixes the issue: https://github.com//pull/565 <https://github.com//pull/565>

If you have an opportunity to test out the fix, that would be greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both invocation modes should behave the same, but the implementations are completely separate, so if setting the flag to false makes a difference there might be a problem when passing one of the arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1] (defined in fuse_operations.h) right here [2], which then gets passed to FUSE. FUSE then decides to call some of these methods when a corresponding file system access happens.

[1]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com <mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on foreign-jextract, where the issue was still present on 6f8f9e2 and fixed on 42e03fd.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find another problem with USE_INTRINSICS=true that seems to be a likely cause of what you were seeing. Namely: when an upcall happens on a separate native thread, the return value of the upcall was being corrupted when detaching the thread from the JVM again. So, it might be the case that the problem you were seeing was from an upcall returning an incorrect result. (I did rule out that it's a problem with the way arguments are passed).

I've just integrated a patch that fixes the issue: https://github.com//pull/565 <https://github.com//pull/565>

If you have an opportunity to test out the fix, that would be greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both invocation modes should behave the same, but the implementations are completely separate, so if setting the flag to false makes a difference there might be a problem when passing one of the arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1] (defined in fuse_operations.h) right here [2], which then gets passed to FUSE. FUSE then decides to call some of these methods when a corresponding file system access happens.

[1]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com <mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112 <https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Jorn Vernee on panama-dev:

Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on?foreign-jextract, where the issue was still present
on?6f8f9e28c54 and fixed on?42e03fd7c6a.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find
another problem with USE_INTRINSICS=true that seems to be a likely
cause of what you were seeing. Namely: when an upcall happens on a
separate native thread, the return value of the upcall was being
corrupted when detaching the thread from the JVM again. So, it might
be the case that the problem you were seeing was from an upcall
returning an incorrect result. (I did rule out that it's a problem
with the way arguments are passed).

I've just integrated a patch that fixes the issue:
https://github.com//pull/565

If you have an opportunity to test out the fix, that would be greatly
appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both
invocation modes should behave the same, but the implementations are
completely separate, so if setting the flag to false makes a
difference there might be a problem when passing one of the
arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1]
(defined in fuse_operations.h) right here [2], which then gets
passed to FUSE. FUSE then decides to call some of these methods
when a corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use
upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only
used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8) and
can confirm that the crashes are gone. However, copying bytes
from heap to native buffers still is a little odd. Take this
code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the
flag it copies 13 null-bytes, though. 13 being the correct length
of the source buffer.

I guess this is not related to the stack walking problem. The
only thing I can tell is that said flag has some influence on the
behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Btw: Any chance this will get backported toJDK-17?

Am 09.07.2021 um 11:39 schrieb Jorn Vernee <jorn.vernee at oracle.com>:

?
Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on foreign-jextract, where the issue was still present on 6f8f9e2 and fixed on 42e03fd.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find another problem with USE_INTRINSICS=true that seems to be a likely cause of what you were seeing. Namely: when an upcall happens on a separate native thread, the return value of the upcall was being corrupted when detaching the thread from the JVM again. So, it might be the case that the problem you were seeing was from an upcall returning an incorrect result. (I did rule out that it's a problem with the way arguments are passed).

I've just integrated a patch that fixes the issue: https://github.com//pull/565

If you have an opportunity to test out the fix, that would be greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both invocation modes should behave the same, but the implementations are completely separate, so if setting the flag to false makes a difference there might be a problem when passing one of the arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1] (defined in fuse_operations.h) right here [2], which then gets passed to FUSE. FUSE then decides to call some of these methods when a corresponding file system access happens.

[1]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
[2]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Sebastian Stenzel on panama-dev:

Btw: Any chance this will get backported toJDK-17?

Am 09.07.2021 um 11:39 schrieb Jorn Vernee <jorn.vernee at oracle.com>:

?
Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on foreign-jextract, where the issue was still present on 6f8f9e2 and fixed on 42e03fd.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find another problem with USE_INTRINSICS=true that seems to be a likely cause of what you were seeing. Namely: when an upcall happens on a separate native thread, the return value of the upcall was being corrupted when detaching the thread from the JVM again. So, it might be the case that the problem you were seeing was from an upcall returning an incorrect result. (I did rule out that it's a problem with the way arguments are passed).

I've just integrated a patch that fixes the issue: https://github.com//pull/565

If you have an opportunity to test out the fix, that would be greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both invocation modes should behave the same, but the implementations are completely separate, so if setting the flag to false makes a difference there might be a problem when passing one of the arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct [1] (defined in fuse_operations.h) right here [2], which then gets passed to FUSE. FUSE then decides to call some of these methods when a corresponding file system access happens.

[1]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
[2]: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit e4f89d7) and can confirm that the crashes are gone. However, copying bytes from heap to native buffers still is a little odd. Take this code: [1]; [2]

If I run the project with `-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`, it copies the contents of HELLO_STR to the segment. Without the flag it copies 13 null-bytes, though. 13 being the correct length of the source buffer.

I guess this is not related to the stack walking problem. The only thing I can tell is that said flag has some influence on the behaviour.

Cheers!
Sebastian

[1] my upcall method: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
[2] method invoked during upcall, which copies bytes to the specified segment: https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112

@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Jorn Vernee on panama-dev:

Yes, we are in the process of doing that, but currently a lot of the
reviewers are on vacation :)

Jorn

On 09/07/2021 14:27, Sebastian Stenzel wrote:

Btw: Any chance this will get backported toJDK-17?

Am 09.07.2021 um 11:39 schrieb Jorn Vernee <jorn.vernee at oracle.com>:

?

Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on?foreign-jextract, where the issue was still present
on?6f8f9e28c54 and fixed on?42e03fd7c6a.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find
another problem with USE_INTRINSICS=true that seems to be a likely
cause of what you were seeing. Namely: when an upcall happens on a
separate native thread, the return value of the upcall was being
corrupted when detaching the thread from the JVM again. So, it
might be the case that the problem you were seeing was from an
upcall returning an incorrect result. (I did rule out that it's a
problem with the way arguments are passed).

I've just integrated a patch that fixes the issue:
https://github.com//pull/565

If you have an opportunity to test out the fix, that would be
greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both
invocation modes should behave the same, but the implementations
are completely separate, so if setting the flag to false makes a
difference there might be a problem when passing one of the
arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct
[1] (defined in fuse_operations.h) right here [2], which then
gets passed to FUSE. FUSE then decides to call some of these
methods when a corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to
use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag
is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8)
and can confirm that the crashes are gone. However, copying
bytes from heap to native buffers still is a little odd. Take
this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the
flag it copies 13 null-bytes, though. 13 being the correct
length of the source buffer.

I guess this is not related to the stack walking problem. The
only thing I can tell is that said flag has some influence on
the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

1 similar comment
@mlbridge
Copy link

mlbridge bot commented Jul 9, 2021

Mailing list message from Jorn Vernee on panama-dev:

Yes, we are in the process of doing that, but currently a lot of the
reviewers are on vacation :)

Jorn

On 09/07/2021 14:27, Sebastian Stenzel wrote:

Btw: Any chance this will get backported toJDK-17?

Am 09.07.2021 um 11:39 schrieb Jorn Vernee <jorn.vernee at oracle.com>:

?

Great!

Thanks for testing,
Jorn

On 09/07/2021 09:58, Sebastian Stenzel wrote:

Hi Jorn,

yes, I can confirm this fixes the issue.

I tested on?foreign-jextract, where the issue was still present
on?6f8f9e28c54 and fixed on?42e03fd7c6a.

Cheers!
Sebastian

On 8. Jul 2021, at 17:05, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

I was not able to reproduce your specific problem, but I did find
another problem with USE_INTRINSICS=true that seems to be a likely
cause of what you were seeing. Namely: when an upcall happens on a
separate native thread, the return value of the upcall was being
corrupted when detaching the thread from the JVM again. So, it
might be the case that the problem you were seeing was from an
upcall returning an incorrect result. (I did rule out that it's a
problem with the way arguments are passed).

I've just integrated a patch that fixes the issue:
https://github.com//pull/565

If you have an opportunity to test out the fix, that would be
greatly appreciated.

Thanks,
Jorn

On 15/06/2021 17:46, Jorn Vernee wrote:

Ok thanks,

If upcalls are involved it makes sense then that the
ProgrammableUpcallHandler.USE_INTRINSICS has an effect. Both
invocation modes should behave the same, but the implementations
are completely separate, so if setting the flag to false makes a
difference there might be a problem when passing one of the
arguments when USE_INTRINSICS=true.

Thanks for testing, I'll try and reproduce the issue here as well.

Jorn

On 15/06/2021 17:10, Sebastian Stenzel wrote:

A reference to the first mentioned method is stored in a struct
[1] (defined in fuse_operations.h) right here [2], which then
gets passed to FUSE. FUSE then decides to call some of these
methods when a corresponding file system access happens.

[1]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java#L28
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/lowlevel/fuse_operations.java*L28__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5TIPESgu$>
[2]:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L25
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L25__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5f8dc5As$>

On 15. Jun 2021, at 16:52, Jorn Vernee <jorn.vernee at oracle.com
<mailto:jorn.vernee at oracle.com>> wrote:

Hi Sebastian,

This sounds very odd, as the code you link to doesn't seem to
use upcalls? The ProgrammableUpcallHandler.USE_INTRINSICS flag
is only used to control the invocation mode of upcalls.

May I ask how exactly you're diagnosing this problem?

Jorn

On 15/06/2021 15:48, Sebastian Stenzel wrote:

Hi,

I have pulled this fix (foreign-jextract commit?e4f89d7b0f8)
and can confirm that the crashes are gone. However, copying
bytes from heap to native buffers still is a little odd. Take
this code: [1]; [2]

If I run the project with
`-Djdk.internal.foreign.ProgrammableUpcallHandler.USE_INTRINSICS=false`,
it copies the contents of?HELLO_STR to the segment. Without the
flag it copies 13 null-bytes, though. 13 being the correct
length of the source buffer.

I guess this is not related to the stack walking problem. The
only thing I can tell is that said flag has some influence on
the behaviour.

Cheers!
Sebastian

[1] my upcall method:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java#L222-L227
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/main/java/de/skymatic/fusepanama/FuseOperations.java*L222-L227__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5SMpro9I$>
[2] method invoked during upcall, which copies bytes to the
specified segment:
https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java#L109-L112
<https://urldefense.com/v3/__https://github.com/skymatic/fuse-panama/blob/3e5dc43ec7a6ba7e2c39fcce0db48d4350fdc0b3/src/test/java/de/skymatic/fusepanama/examples/HelloPanamaFileSystem.java*L109-L112__;Iw!!GqivPVa7Brio!IACTB0TzhBi09Zn2FRLC-Lkt6VqU7CWFBtZDudXO2BDhgk1TYYFrljCX5ZFCsni2$>

@JornVernee JornVernee deleted the Async_Stack_Walk branch November 2, 2021 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated
2 participants