Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8283335 : Add exists and readAttributesIfExists methods to FileSystemProvider #9249

Closed
wants to merge 7 commits into from

Conversation

LanceAndersen
Copy link
Contributor

@LanceAndersen LanceAndersen commented Jun 22, 2022

Hi,

Please review the following patch which will:

  • Enhance the java.nio.file.spi.FileSystemProvider abstract class to include the methods

    • public boolean exists(Path path, LinkOption... options)
    • public A readAttributesIfExists(Path path, Class type, LinkOption... options)

This change allows for providers to provide optimizations when the file's attributes are not needed.

Mach5 tiers 1 - 3 run clean with this change

The CSR may be viewed at JDK-8283336

Best,
Lance


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change requires a CSR request to be approved

Issues

  • JDK-8283335: Add exists and readAttributesIfExists methods to FileSystemProvider
  • JDK-8283336: Add exists and readAttributesIfExists methods to FileSystemProvider (CSR)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/9249/head:pull/9249
$ git checkout pull/9249

Update a local copy of the PR:
$ git checkout pull/9249
$ git pull https://git.openjdk.org/jdk pull/9249/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 9249

View PR using the GUI difftool:
$ git pr show -t 9249

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/9249.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 22, 2022

👋 Welcome back lancea! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Jun 22, 2022
@openjdk
Copy link

openjdk bot commented Jun 22, 2022

@LanceAndersen The following labels will be automatically applied to this pull request:

  • core-libs
  • nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added nio nio-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Jun 22, 2022
@mlbridge
Copy link

mlbridge bot commented Jun 22, 2022

@openjdk openjdk bot added the csr Pull request needs approved CSR before integration label Jun 22, 2022
* are followed. If the option {@link LinkOption#NOFOLLOW_LINKS NOFOLLOW_LINKS}
* is present then symbolic links are not followed and the method
* {@link #readAttributes(Path, Class, LinkOption...)} is called
* to determine whether a file exists.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation is off at lines 1183-1184.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching that. Will be addressed in the next push

// get the UnixFileAttributes for a given file. Returns null if the file does not exist.
static UnixFileAttributes getIfExists(UnixPath path) throws UnixException {
UnixFileAttributes attrs = new UnixFileAttributes();
int errno = UnixNativeDispatcher.stat2(path, attrs);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is the only use of stat2(). It could be deleted if this method were replaced with this:

    static UnixFileAttributes getIfExists(UnixPath path) throws UnixException {
        UnixFileAttributes attrs = new UnixFileAttributes();
        try {
            UnixNativeDispatcher.stat(path, attrs);
            return attrs;
        } catch (UnixException e) {
            if (e.errno() == UnixConstants.ENOENT) {
                return null;
            }
            throw e;
        }
    }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is the only use of stat2(). It could be deleted if this method were replaced with this:

The purpose of these additions to the SPI is to improve the performance of user facing methods that don't throw an exception when the file doesn't exist. It needs a stat/equivalent that doesn't throw so this is the reason for stat2. The equivalent on Windows will need to added too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood.

* <p> It is implementation specific if all file attributes are read as an
* atomic operation with respect to other file system operations.
*
* @implSpec
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more bad indentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in next push

* @since 20
*/
public boolean exists(Path path, LinkOption... options) {
try {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think that this is a great change (avoiding the need for various parts of the system to communicate through exceptions).

Trivially, and easily, I often find it useful to add mock-like tests for these specified implementations (implSpec), i.e. just subclass FSP mocking out the abstract methods to ensure that the default implementations invoke the appropriate ones.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the suggestion Chris

@AlanBateman
Copy link
Contributor

This should be performance neutral for the default provider and positive for the zip provider. Do we have any micros that could be added?

case READ -> r = true;
case WRITE -> w = true;
case EXECUTE -> x = true;
default -> throw new AssertionError("Should not get here");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The switch should be exhaustive so I assume the default case is not required here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in next push

@AlanBateman
Copy link
Contributor

The end date in the copyright header will need updated a few files, I assume you'll do that before this change is integrated.

LinkOption... options)
throws IOException
{
return exists(path) ? readAttributes(path, type, options) : null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you add the following to ZipPath then it would allow you to implement it in one step rather than two:

ZipFileAttributes readAttributesIfExits() throws IOException {
    return zfs.getFileAttributes(getResolvedPath());
 }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the proposed change above, Mach5 tiers 1-3 continue to be clean

@openjdk openjdk bot removed the csr Pull request needs approved CSR before integration label Jun 24, 2022
@AlanBateman
Copy link
Contributor

The implementation changes in latest version address my previous points, thanks! I don't the test as it duplicates most of the existing of the TestProvider in this directory and the naming is inconsistent with the existing tests. I expect you should be able to just extent TestProvider and override the two methods to record that they can been called.

@LanceAndersen
Copy link
Contributor Author

The implementation changes in latest version address my previous points, thanks! I don't the test as it duplicates most of the existing of the TestProvider in this directory and the naming is inconsistent with the existing tests. I expect you should be able to just extent TestProvider and override the two methods to record that they can been called.

I thought about using TestProvider and then talked myself out of it. So yes, I can do that. If you have a preferred name other than TestOverRideFSPMethods please let me know

@AlanBateman
Copy link
Contributor

I thought about using TestProvider and then talked myself out of it. So yes, I can do that. If you have a preferred name other than TestOverRideFSPMethods please let me know

The only change that should be needed to TestProvider is to implement the checkAccess method with:

        Path delegate = theFileSystem.unwrap(file);
        defaultProvider.checkAccess(delegate, modes);

With that change it should be easy to extend and override exists/readAttributesIfExists to record that they are called. The need to run with the test with -Djava.nio.file.spi.DefaultFileSystemProvider=TestProvider goes away too and and should be much simpler to test that the expected delegation.

@openjdk openjdk bot removed the rfr Pull request is ready for review label Jul 4, 2022
@openjdk openjdk bot added the rfr Pull request is ready for review label Jul 4, 2022
@AlanBateman
Copy link
Contributor

The updated TestDelegation test is looking a bit better now but I think it would be simplified a lot more by getting rid of the data providers, just aren't needed in this test.

@LanceAndersen
Copy link
Contributor Author

The updated TestDelegation test is looking a bit better now but I think it would be simplified a lot more by getting rid of the data providers, just aren't needed in this test.

Unless you feel this is a must, I would prefer to keep the DataProviders. The benefit I see is the test code does not need to be duplicated per parameter, each test scenario can be run as an individual test so that you do not need extra plumbing to run each test scenario in the unlikely event of a failure.

@AlanBateman
Copy link
Contributor

Unless you feel this is a must, I would prefer to keep the DataProviders. The benefit I see is the test code does not need to be duplicated per parameter, each test scenario can be run as an individual test so that you do not need extra plumbing to run each test scenario in the unlikely event of a failure.

Okay, but there are a few other things to mention:

One issue is the reset method is called at the end of each test. I think it needs to be at the beginning of the method, moved to a finally block of a try-finally, or maybe @BeforeMethod to reset before each test. The reason is that one test failing will cause the tests that follow to fail too.

The fields aren't constants so looks a bit strange (to me anyway) to be in uppercase. If you rename them then I think the tests would be a bit more readable.

@LanceAndersen
Copy link
Contributor Author

Unless you feel this is a must, I would prefer to keep the DataProviders. The benefit I see is the test code does not need to be duplicated per parameter, each test scenario can be run as an individual test so that you do not need extra plumbing to run each test scenario in the unlikely event of a failure.

Okay, but there are a few other things to mention:

One issue is the reset method is called at the end of each test. I think it needs to be at the beginning of the method, moved to a finally block of a try-finally, or maybe @BeforeMethod to reset before each test. The reason is that one test failing will cause the tests that follow to fail too.

Good catch, added @BeforeMethod

The fields aren't constants so looks a bit strange (to me anyway) to be in uppercase. If you rename them then I think the tests would be a bit more readable.

Ah, sometimes you see what you want. I must have had fields on my mind when I did that as I completely agree with you.

Both of the above are addressed in the latest update to the PR.

Thank you for the feedback.

Copy link
Contributor

@AlanBateman AlanBateman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all my comments have been addressed so I think this is good to go.

@openjdk
Copy link

openjdk bot commented Jul 5, 2022

@LanceAndersen This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8283335: Add exists and readAttributesIfExists methods to FileSystemProvider

Reviewed-by: alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 150 new commits pushed to the master branch:

  • c45d613: 8289687: [JVMCI] bug in HotSpotResolvedJavaMethodImpl.equals
  • 77c3bbf: 8289617: Remove test/jdk/java/net/ServerSocket/ThreadStop.java
  • a5934cd: 8289698: AArch64: Need to relativize extended_sp in frame
  • fd1bb07: 8287603: Avoid redundant HashMap.containsKey calls in NimbusDefaults.getDerivedColor
  • 4c997ba: 8289520: G1: Remove duplicate checks in G1BarrierSetC1::post_barrier
  • 1b997db: 8289427: compiler/compilercontrol/jcmd/ClearDirectivesFileStackTest.java failed with null setting
  • 688712f: 8289633: Forbid raw C-heap allocation functions in hotspot and fix findings
  • df063f7: 8289484: Cleanup unnecessary null comparison before instanceof check in java.rmi
  • 9ccae70: 8287593: ShortResponseBody could be made more resilient to rogue connections
  • bad9ffe: 8288947: G1: Consolidate per-region is-humongous query in G1CollectedHeap
  • ... and 140 more: https://git.openjdk.org/jdk/compare/82c77ca807d62c25b9605c6c8164e42af6c3ce6e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jul 5, 2022
@LanceAndersen
Copy link
Contributor Author

/integrate

@openjdk
Copy link

openjdk bot commented Jul 5, 2022

Going to push as commit d48694d.
Since your change was applied there have been 150 commits pushed to the master branch:

  • c45d613: 8289687: [JVMCI] bug in HotSpotResolvedJavaMethodImpl.equals
  • 77c3bbf: 8289617: Remove test/jdk/java/net/ServerSocket/ThreadStop.java
  • a5934cd: 8289698: AArch64: Need to relativize extended_sp in frame
  • fd1bb07: 8287603: Avoid redundant HashMap.containsKey calls in NimbusDefaults.getDerivedColor
  • 4c997ba: 8289520: G1: Remove duplicate checks in G1BarrierSetC1::post_barrier
  • 1b997db: 8289427: compiler/compilercontrol/jcmd/ClearDirectivesFileStackTest.java failed with null setting
  • 688712f: 8289633: Forbid raw C-heap allocation functions in hotspot and fix findings
  • df063f7: 8289484: Cleanup unnecessary null comparison before instanceof check in java.rmi
  • 9ccae70: 8287593: ShortResponseBody could be made more resilient to rogue connections
  • bad9ffe: 8288947: G1: Consolidate per-region is-humongous query in G1CollectedHeap
  • ... and 140 more: https://git.openjdk.org/jdk/compare/82c77ca807d62c25b9605c6c8164e42af6c3ce6e...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Jul 5, 2022
@openjdk openjdk bot closed this Jul 5, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jul 5, 2022
@openjdk
Copy link

openjdk bot commented Jul 5, 2022

@LanceAndersen Pushed as commit d48694d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs core-libs-dev@openjdk.org integrated Pull request has been integrated nio nio-dev@openjdk.org
Development

Successfully merging this pull request may close these issues.

4 participants