Implement bare-bones blob listing #884

jean-philippe-martin · 2016-04-08T23:13:46Z

Update FakeStorageRPC to (barely) implement list(bucket).
Wire the NIO provider to take advantage of it so we can list the contents of folders.
Add the corresponding test case.

I notice my IDE helpfully changed some of the files: it added an explicit java version (I'm using 1.8 features, hopefully that's OK?), and it added some wildcard imports.

Let me know which of those (if any) I should revert!

Also I'm not entirely sure of the contract that list(String bucket, Map<Option, ?> options) is supposed to follow, beyond "Lists the bucket's blobs". I filled it in with my best guess but if there's a spec somewhere for me to follow, pointers would be appreciated!

jean-philippe-martin · 2016-04-08T23:55:08Z

OK, looks like Java 8 is a no-go, I'll change things to the old way.

jean-philippe-martin · 2016-04-12T19:03:55Z

Changed to Java 1.7, PTAL.

...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

+        return ret.iterator();
+      }
+      @Override
+      public void close() throws IOException {


jean-philippe-martin · 2016-04-13T18:28:33Z

This should address all the comments so far.

...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

@@ -79,6 +82,34 @@
 @AutoService(FileSystemProvider.class)
 public final class CloudStorageFileSystemProvider extends FileSystemProvider {

+  private static class LazyPathIterator extends AbstractIterator<Path> {
+    private Iterator<Blob> blobIterator;


jean-philippe-martin · 2016-04-13T21:31:33Z

mmh, the extra line is still there even though I thought I deleted it. Let me try to delete more and see what happens. Alternatively, we could decide we're OK with two blank lines at the end of a file.

jean-philippe-martin · 2016-04-13T21:39:24Z

The mystery deepens. On my disk there clearly is only a single newline at the end (which is what you want to see), and git believes that my files match what's been pushed.

gcloud-java$ xxd gcloud-java-contrib/gcloud-java-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java  | tail -n 4
00005960: 7265 7475 726e 206e 6577 2049 4f45 7863  return new IOExc
00005970: 6570 7469 6f6e 286f 6f70 732e 6765 744d  eption(oops.getM
00005980: 6573 7361 6765 2829 2c20 6f6f 7073 293b  essage(), oops);
00005990: 0a20 207d 0a7d 0a                        .  }.}.
gcloud-java$ git diff
gcloud-java$ git commit
nothing added to commit but untracked files present

jean-philippe-martin · 2016-04-13T21:40:21Z

We can come back to this later.

...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

+    }
+  }
+
+
  private final Storage storage;


...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

+
+      // Does nothing since there's nothing to close. Commenting this method to quiet codacy.
+      @Override
+      public void close() throws IOException {


mziccard · 2016-04-14T14:26:11Z

The mystery deepens. On my disk there clearly is only a single newline at the end (which is what you want to see), and git believes that my files match what's been pushed.

I cloned your branch and I can see the 2 empty lines. If I remove one git diff shows:

--- a/gcloud-java-contrib/gcloud-java-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java
+++ b/gcloud-java-contrib/gcloud-java-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java
@@ -109,7 +109,6 @@ public final class CloudStorageFileSystemProvider extends FileSystemProvider {
     }
   }

-
   private final Storage storage;

   // used only when we create a new instance of CloudStorageFileSystemProvider.

gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/FakeStorageRpc.java

-    return null;
+    String preprefix = "";
+    for (Map.Entry<Option, ?> e : options.entrySet()) {
+      switch (e.getKey()) {


jean-philippe-martin · 2016-04-14T17:46:37Z

I don't know what to say. Does git lie to us when it says that my branch's up-to-date? It looks like what's on my machine isn't the same as what's on the server. I'm going to push with the other fixes, let's see if after that the state on my machine matches the state on the server.

jean-philippe-martin · 2016-04-14T17:55:34Z

FWIW, when I look at the file on GitHub's viewer, it shows no empty line at all at the end of the file, so line 637 is the last line. When I look at it in my IDE it shows one more line (638), empty.

The hex dump still shows a single 0x0a at the end of the file. The diff for this commit indicates no change to this last line, nor does the previous one. Yet we seem to be seeing something different.

When you clone my branch, and before you open the file in any editor, what does the hex dump show?

mziccard · 2016-04-14T18:21:03Z

The extra line seems to be gone now, am I wrong?

This looks good to me feel free to merge after squashing the commits.

jean-philippe-martin · 2016-04-14T20:25:38Z

Squashed. Can we agree not to worry about extra blank lines next time?

🤖 I have created a release *beep* *boop* --- ## [0.123.3](googleapis/java-errorreporting@v0.123.2...v0.123.3) (2022-06-23) ### Dependencies * update dependency com.google.cloud:google-cloud-shared-dependencies to v2.13.0 ([#882](googleapis/java-errorreporting#882)) ([b0956b6](googleapis/java-errorreporting@b0956b6)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [2.3.1](googleapis/java-containeranalysis@v2.3.0...v2.3.1) (2022-06-24) ### Dependencies * update dependency com.google.cloud:google-cloud-shared-dependencies to v2.13.0 ([googleapis#891](googleapis/java-containeranalysis#891)) ([afd35ec](googleapis/java-containeranalysis@afd35ec)) * update dependency io.grafeas:grafeas to v2.3.0 ([googleapis#884](googleapis/java-containeranalysis#884)) ([14a551d](googleapis/java-containeranalysis@14a551d)) * update dependency io.grafeas:grafeas to v2.3.1 ([googleapis#893](googleapis/java-containeranalysis#893)) ([b0d2eb0](googleapis/java-containeranalysis@b0d2eb0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Source-Link: googleapis/synthtool@7336562 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:d4b80feffe1579818cdc39466152e9de95789a193408506cd4a1ffbe8804dc00

🤖 I have created a release *beep* *boop* --- ## [4.0.1](googleapis/java-dataproc@v4.0.0...v4.0.1) (2022-07-13) ### Bug Fixes * enable longpaths support for windows test ([#1485](https://github.com/googleapis/java-dataproc/issues/1485)) ([#884](googleapis/java-dataproc#884)) ([92b0573](googleapis/java-dataproc@92b0573)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [3.3.2](googleapis/java-monitoring@v3.3.1...v3.3.2) (2022-07-13) ### Bug Fixes * enable longpaths support for windows test ([#1485](https://github.com/googleapis/java-monitoring/issues/1485)) ([#883](googleapis/java-monitoring#883)) ([0aa8d72](googleapis/java-monitoring@0aa8d72)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [2.9.0](googleapis/java-securitycenter@v2.8.0...v2.9.0) (2022-08-06) ### Features * Added container field to findings attributes ([#875](googleapis/java-securitycenter#875)) ([d9b8007](googleapis/java-securitycenter@d9b8007)) ### Dependencies * update dependency com.google.cloud:google-cloud-bigquery to v2.14.0 ([#878](googleapis/java-securitycenter#878)) ([4c80791](googleapis/java-securitycenter@4c80791)) * update dependency com.google.cloud:google-cloud-bigquery to v2.14.1 ([#881](googleapis/java-securitycenter#881)) ([c7c68d3](googleapis/java-securitycenter@c7c68d3)) * update dependency com.google.cloud:google-cloud-bigquery to v2.14.2 ([#888](googleapis/java-securitycenter#888)) ([e786c87](googleapis/java-securitycenter@e786c87)) * update dependency com.google.cloud:google-cloud-bigquery to v2.14.3 ([#889](googleapis/java-securitycenter#889)) ([9a1437b](googleapis/java-securitycenter@9a1437b)) * update dependency com.google.cloud:google-cloud-pubsub to v1.120.10 ([#887](googleapis/java-securitycenter#887)) ([d5e013b](googleapis/java-securitycenter@d5e013b)) * update dependency com.google.cloud:google-cloud-pubsub to v1.120.7 ([#884](googleapis/java-securitycenter#884)) ([8392fb7](googleapis/java-securitycenter@8392fb7)) * update dependency com.google.cloud:google-cloud-pubsub to v1.120.9 ([#886](googleapis/java-securitycenter#886)) ([6db2c64](googleapis/java-securitycenter@6db2c64)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v3 ([#882](googleapis/java-securitycenter#882)) ([43d2ca4](googleapis/java-securitycenter@43d2ca4)) * update dependency com.google.protobuf:protobuf-java-util to v3.21.3 ([#877](googleapis/java-securitycenter#877)) ([b02a0d0](googleapis/java-securitycenter@b02a0d0)) * update dependency com.google.protobuf:protobuf-java-util to v3.21.4 ([#880](googleapis/java-securitycenter#880)) ([9373129](googleapis/java-securitycenter@9373129)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

…ator_java versions (#884) - [ ] Regenerate this pull request now. PiperOrigin-RevId: 472750037 Source-Link: googleapis/googleapis@88f2ea3 Source-Link: googleapis/googleapis-gen@230a558 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjMwYTU1ODgzMDZhYWUxOGZlOGYyYTU3ZjE0ZDQwMzlhZDcyYzkwMSJ9

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [protobuf](https://developers.google.com/protocol-buffers/) | `==3.19.5` -> `==3.20.3` | [![age](https://badges.renovateapi.com/packages/pypi/protobuf/3.20.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/protobuf/3.20.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/protobuf/3.20.3/compatibility-slim/3.19.5)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/protobuf/3.20.3/confidence-slim/3.19.5)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-datacatalog).

…nager to v2.3.10 (googleapis#884) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-secretmanager](https://togithub.com/googleapis/java-secretmanager) | `2.3.9` -> `2.3.10` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-secretmanager/2.3.10/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-secretmanager/2.3.10/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-secretmanager/2.3.10/compatibility-slim/2.3.9)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-secretmanager/2.3.10/confidence-slim/2.3.9)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-secretmanager).

🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:grpc-gcp](https://togithub.com/GoogleCloudPlatform/grpc-gcp-java/tree/master/grpc-gcp) ([source](https://togithub.com/GoogleCloudPlatform/grpc-gcp-java)) | `1.3.0` -> `1.3.1` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:grpc-gcp/1.3.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:grpc-gcp/1.3.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:grpc-gcp/1.3.1/compatibility-slim/1.3.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:grpc-gcp/1.3.1/confidence-slim/1.3.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>GoogleCloudPlatform/grpc-gcp-java</summary> ### [`v1.3.1`](https://togithub.com/GoogleCloudPlatform/grpc-gcp-java/compare/v1.3.0...v1.3.1) [Compare Source](https://togithub.com/GoogleCloudPlatform/grpc-gcp-java/compare/v1.3.0...v1.3.1) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-shared-dependencies).

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 8, 2016

aozarov added the contrib label Apr 11, 2016

jean-philippe-martin reviewed Apr 13, 2016
View reviewed changes

...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

return ret.iterator();

}

@Override

public void close() throws IOException {

This comment was marked as spam.

Sign in to view

mziccard reviewed Apr 13, 2016
View reviewed changes

mziccard reviewed Apr 14, 2016
View reviewed changes

...-nio/src/main/java/com/google/gcloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

}

}

private final Storage storage;

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

mziccard reviewed Apr 14, 2016
View reviewed changes

jean-philippe-martin reviewed Apr 14, 2016
View reviewed changes

gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/FakeStorageRpc.java

return null;

String preprefix = "";

for (Map.Entry<Option, ?> e : options.entrySet()) {

switch (e.getKey()) {

This comment was marked as spam.

Sign in to view

Implement bare-bones blob listing

9c9dc0d

jean-philippe-martin force-pushed the fake_list branch from fc0af69 to 9c9dc0d Compare April 14, 2016 20:25

jean-philippe-martin merged commit 9a8c51a into googleapis:gcs-nio Apr 14, 2016

jean-philippe-martin deleted the fake_list branch April 14, 2016 21:06

mziccard mentioned this pull request Apr 15, 2016

NIO: Implement directory listing #813

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement bare-bones blob listing #884

Implement bare-bones blob listing #884

jean-philippe-martin commented Apr 8, 2016

jean-philippe-martin commented Apr 8, 2016

jean-philippe-martin commented Apr 12, 2016

This comment was marked as spam.

jean-philippe-martin commented Apr 13, 2016

This comment was marked as spam.

This comment was marked as spam.

jean-philippe-martin commented Apr 13, 2016

jean-philippe-martin commented Apr 13, 2016

jean-philippe-martin commented Apr 13, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

mziccard commented Apr 14, 2016

This comment was marked as spam.

jean-philippe-martin commented Apr 14, 2016

jean-philippe-martin commented Apr 14, 2016

mziccard commented Apr 14, 2016

jean-philippe-martin commented Apr 14, 2016

Implement bare-bones blob listing #884

Implement bare-bones blob listing #884

Conversation

jean-philippe-martin commented Apr 8, 2016

jean-philippe-martin commented Apr 8, 2016

jean-philippe-martin commented Apr 12, 2016

This comment was marked as spam.

jean-philippe-martin commented Apr 13, 2016

This comment was marked as spam.

This comment was marked as spam.

jean-philippe-martin commented Apr 13, 2016

jean-philippe-martin commented Apr 13, 2016

jean-philippe-martin commented Apr 13, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

mziccard commented Apr 14, 2016

This comment was marked as spam.

jean-philippe-martin commented Apr 14, 2016

jean-philippe-martin commented Apr 14, 2016

mziccard commented Apr 14, 2016

jean-philippe-martin commented Apr 14, 2016