Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS) #3366

Closed
wants to merge 4 commits into from

Conversation

bplb
Copy link
Member

@bplb bplb commented Apr 7, 2021

Apparently fcntl(fd, F_FULLFSYNC) can fail with ENOTTY in addition to ENOTSUP although it is not so documented. The ioctl() case is documented as

[ENOTTY]       fildes is not associated with a character special
               device.
[ENOTTY]       The specified request does not apply to the kind of
               object that the descriptor fildes references.

This request proposes to catch ENOTTY as well and fall back to fsync(fd). The change was verified manually using a WebDAV server and no failures in CI test tiers 1-3 were observed. An alternative would be to fall back to fsync(fd) for all errors.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/3366/head:pull/3366
$ git checkout pull/3366

Update a local copy of the PR:
$ git checkout pull/3366
$ git pull https://git.openjdk.java.net/jdk pull/3366/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3366

View PR using the GUI difftool:
$ git pr show -t 3366

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/3366.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Apr 7, 2021

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Apr 7, 2021
@openjdk
Copy link

openjdk bot commented Apr 7, 2021

@bplb The following label will be automatically applied to this pull request:

  • nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the nio nio-dev@openjdk.org label Apr 7, 2021
@mlbridge
Copy link

mlbridge bot commented Apr 7, 2021

Webrevs

@AlanBateman
Copy link
Contributor

fcntl(fd, F_FULLFSYNC) is the semantics we want for local files. If it fails then the force method has to throw. The force method doesn't provide any guarantees when the file is remote so using fsync for the WebDAV case is okay. I can't find any man pages or other documentation that documents ENOTTY. I think I would be more comfortable with an approach that somehow detects the file is local or not, that would allow it to fallback to fsync when not local.

/* Try fsync() in case file is not local. */
result = fsync(fd);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to save/restore errno so that the original error from fcntl(F_FULLSYNC) is used in the error handling?

Also what you would think about dropping the check for ENOTSUP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about errno but didn't know whether it's common practice to cache it like that.

We had ENOTSUP I suppose in case the F_FULLSYNC is unsupported on a local file system. From the man page of fcntl() the doc of F_FULLSYNC states

This is currently implemented on HFS, MS-DOS (FAT), Universal Disk Format (UDF) and APFS file systems

although ENOTSUP itself is not actually documented here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good, I'm just wondering to myself if it is possible to get ENOTSUP from a local device.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to try to test this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was unable to get ENOTSUP from local FAT32 or NTFS devices. However from the review discussion [1] of [2] it looks like ENOTSUP has been observed and is handled in other packages [3]. Is there any harm to leaving it in?

[1] https://bugs.openjdk.java.net/browse/JDK-8080589
[2] http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003160.html
[3] https://community.oracle.com/tech/developers/discussion/453477/patch-when-fnctl-fd-f-fullsync-0-returns-enotsup

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The force method needs fcntl(F_FULLSYNC) when the file system is local so I think it would be safer to drop the special handle for ENOTSUP. This will mean fallback to fsync when the file is not local.

@openjdk
Copy link

openjdk bot commented Apr 8, 2021

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8232861: (fc) FileChannel.force fails on WebDAV file systems (macOS)

Reviewed-by: alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 295 new commits pushed to the master branch:

  • 5bd6c74: 8236127: Use value of --icon CLI option to set icon for exe installers
  • 81d35e4: 8264063: Outer Safepoint poll load should not reference the head of inner strip mined loop.
  • 04fa1ed: 8264848: [macos] libjvm.dylib linker warning due to macOS version mismatch
  • 214d6e2: 8263506: Make sun.net.httpserver.UnmodifiableHeaders unmodifiable
  • af13c64: 8264711: More runtime TRAPS cleanups
  • 3aec2d9: 8264718: Shenandoah: enable string deduplication during root scanning
  • 255afbe: 8264672: runtime/ParallelLoad/ParallelSuperTest.java timed out
  • ec599da: 8264633: Add missing logging to PlatformRecording#stop
  • e89542f: 8264352: AArch64: Optimize vector "not/andNot" for NEON and SVE
  • 016db40: 8263907: Specification of CellRendererPane::paintComponent(..Rectangle) should clearly mention which method it delegates the call to
  • ... and 285 more: https://git.openjdk.java.net/jdk/compare/6aa28b3bdb78eb041a963d659d61e1622bc43ef8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Apr 8, 2021
@bplb
Copy link
Member Author

bplb commented Apr 9, 2021

/integrate

@openjdk openjdk bot closed this Apr 9, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Apr 9, 2021
@openjdk
Copy link

openjdk bot commented Apr 9, 2021

@bplb Since your change was applied there have been 314 commits pushed to the master branch:

  • 1ca4abe: 8262881: port JVM/DI tests from JDK-4413752 to JVM/TI
  • 06e6b1f: 8259242: Remove ProtectionDomainSet_lock
  • 9bb1863: 8260923: Add more tests for SSLSocket input/output shutdown
  • 33fa855: 8264732: Clean up LinkResolver::vtable_index_of_interface_method()
  • b3782ea: 8264918: [JVMCI] getVtableIndexForInterfaceMethod doesn't check that type and method are related
  • f7a6c63: 8259822: [PPC64] Support the prefixed instruction format added in POWER10
  • a45733f: 8264779: Fix doclint warnings in java/nio
  • 3e57924: 8264885: Fix the code style of macro in aarch64_neon_ad.m4
  • 051c117: 8264923: PNGImageWriter.write_zTXt throws Exception with a typo
  • 1c6b113: 8264513: Cleanup CardTableBarrierSetC2::post_barrier
  • ... and 304 more: https://git.openjdk.java.net/jdk/compare/6aa28b3bdb78eb041a963d659d61e1622bc43ef8...master

Your commit was automatically rebased without conflicts.

Pushed as commit 6de0bb2.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated nio nio-dev@openjdk.org
2 participants