Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ability to create a File object from URL #2432

Merged
merged 3 commits into from
Apr 15, 2024

Conversation

ddelgrosso1
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #1898 馃

@ddelgrosso1 ddelgrosso1 requested review from a team as code owners March 26, 2024 17:22
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/nodejs-storage API. labels Mar 26, 2024
@ddelgrosso1 ddelgrosso1 added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
@ddelgrosso1 ddelgrosso1 added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
@ddelgrosso1 ddelgrosso1 added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 26, 2024
Copy link
Member

@danielbankhead danielbankhead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! One optional suggestion

Comment on lines +2372 to +2391
static from(
publicUrlOrGsUrl: string,
storageInstance: Storage,
options?: FileOptions
): File {
const gsMatches = [...publicUrlOrGsUrl.matchAll(GS_UTIL_URL_REGEX)];
const httpsMatches = [...publicUrlOrGsUrl.matchAll(HTTPS_PUBLIC_URL_REGEX)];

if (gsMatches.length > 0) {
const bucket = new Bucket(storageInstance, gsMatches[0][1]);
return new File(bucket, gsMatches[0][2], options);
} else if (httpsMatches.length > 0) {
const bucket = new Bucket(storageInstance, httpsMatches[0][2]);
return new File(bucket, httpsMatches[0][3], options);
} else {
throw new Error(
'URL string must be of format gs://bucket/file or https://storage.googleapis.com/bucket/file'
);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could parse the URL, then use protocol to determine the logic. Here's an example with an optional storage param:

Suggested change
static from(
publicUrlOrGsUrl: string,
storageInstance: Storage,
options?: FileOptions
): File {
const gsMatches = [...publicUrlOrGsUrl.matchAll(GS_UTIL_URL_REGEX)];
const httpsMatches = [...publicUrlOrGsUrl.matchAll(HTTPS_PUBLIC_URL_REGEX)];
if (gsMatches.length > 0) {
const bucket = new Bucket(storageInstance, gsMatches[0][1]);
return new File(bucket, gsMatches[0][2], options);
} else if (httpsMatches.length > 0) {
const bucket = new Bucket(storageInstance, httpsMatches[0][2]);
return new File(bucket, httpsMatches[0][3], options);
} else {
throw new Error(
'URL string must be of format gs://bucket/file or https://storage.googleapis.com/bucket/file'
);
}
}
static from(
publicUrlOrGsUrl: string | URL,
storage?: Storage,
options?: FileOptions
): File {
const url = new URL(publicUrlOrGsUrl);
if (url.protocol === 'gs:') {
const storageInstance = storage ?? new Storage();
const bucket = new Bucket(storageInstance, url.host);
return new File(bucket, url.pathname.slice(1), options);
} else if (url.protocol === 'http:' || url.protocol === 'https:') {
const {groups: {bucketName, fileName}} = /\/(?<bucket>[^/]*)\/(?<filename>.*)/.exec(u.pathname);
const storageInstance = storage ?? new Storage({apiEndpoint: url.hostname});
const bucket = new Bucket(storageInstance, bucketName);
return new File(bucket, fileName, options);
} else {
throw new Error(
'URL string must be of format gs://bucket/file or https://storage.googleapis.com/bucket/file'
);
}
}

If we do this we'll need to make a small change to storage.ts on line ~180 to prevent accidental custom apiEndpoint when provided.
From:

if (options.apiEndpoint) {

To:

if (options.apiEndpoint && options.apiEndpoint !== apiEndpoint) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion, I'm apt to leave this using the regexs to make it a bit more concise and avoid the apiEndpoint logic. It also matches how it was done in the python client (not that consistency is SUPER important).

@ddelgrosso1 ddelgrosso1 merged commit 1b71fcc into googleapis:main Apr 15, 2024
16 checks passed
@ddelgrosso1 ddelgrosso1 deleted the from-url branch April 15, 2024 13:21
@shaffeeullah
Copy link
Contributor

let's gooooo 馃殌 so excited to see this feature get implemented!
appreciate you @ddelgrosso1 & @danielbankhead !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/nodejs-storage API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: Instantiate a File using Public URL
3 participants