Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firebase Admin SDK: getDownloadUrl - Permission denied. No READ permission #2344

Open
bytewiz opened this issue Oct 19, 2023 · 14 comments
Open
Assignees

Comments

@bytewiz
Copy link

bytewiz commented Oct 19, 2023

Describe your environment

  • Operating System version: OSX (and in the cloud environment)
  • Firebase SDK version: ^11.11.0 <= firebase-admin
  • Firebase Product: firebase-admin / google-storage / firebase-admin/storage
  • Node.js version: 16.16.0
  • NPM version: 8.11.0

Describe the problem:

I have now tried for a very long time to follow these docs in order to get getDownloadURL to work. https://firebase.google.com/docs/storage/admin/start#use_a_default_bucket
https://firebase.google.com/docs/storage/admin/start#shareable_urls

Regardless of how I initialize my app, when trying to use getDownloadURL I get Error: Permission denied. No READ permission.

Here is how different ways I tried initializing:

initializeApp({
    credential: applicationDefault(),
    storageBucket: "my-bucket.appspot.com",
});
initializeApp();
initializeApp({
    credential: cert(serviceAcount), // loaded from .json file (directly downloaded from firebase console)
    storageBucket: "my-bucket.appspot.com",
});
initializeApp({
    credential: cert({
      projectId: "my-project-id",
      privateKey: "my-private-key",
      clientEmail: "my-client-email"
    }), // grabbed from .json file (directly downloaded from firebase console)
    storageBucket: "my-bucket.appspot.com",
});

Furthermore, I have tried adding IAM roles to the service account:
Screenshot 2023-10-19 at 16 54 45

What I am trying to accomplish is simply what is done in the before-mentioned docs:

    // Triggered from storage.object().onFinalize(generateThumbnail);
    const bucket = getStorage().bucket(object.bucket);
    ...
    // Cloud Storage files.
    const file = bucket.file(filePath);
    const url = await getDownloadURL(file);
    console.log({ url });

What is going wrong here, as the docs states clearly I firebase admin sdk should have access by default?

Stacktrace: (from emulator)

⚠  functions: Error: Permission denied. No READ permission.
    at new ApiError (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:80:15)
    at Util.parseHttpRespBody (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:215:38)
    at Util.handleResp (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:156:117)
    at /../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:538:22
    at onResponse (/../functions/node_modules/firebase-admin/node_modules/retry-request/index.js:240:7)
    at /../functions/node_modules/firebase-admin/node_modules/teeny-request/build/src/index.js:217:17
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

My service account file:

{
  "type":"..",
  "project_id":"..",
  "private_key_id":"..",
  "private_key":"..",
  "client_email":"..",
  "client_id":"..",
  "auth_uri":"..",
  "token_uri":"..",
  "auth_provider_x509_cert_url":"..",
  "client_x509_cert_url":"..",
  "universe_domain":"..",
}
@google-oss-bot
Copy link

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

@tonyjhuang
Copy link

This is the emulator code that's returning this particular error message: https://github.com/firebase/firebase-tools/blob/b7eea76c22816a0caf4e45e6bd0f072c066c5d44/src/emulator/storage/apis/firebase.ts#L111

Unfortunately, our admin credential vetting in the emulator is in a pretty poor state. We don't have full support of OAuth access tokens, which is what the admin SDK is sending along with its requests.

In short the validation flow for getDownloadUrl in the emulator is this:

  1. Does the Authorization header value equal the string literal, "owner"?
  2. If not, validate security rules.

Since there's no way to get the admin SDK to pass along the header "Authorization: Bearer owner", this will likely remain broken until we have a full OAuth validation, which we haven't needed until the introduction of the admin getDownloadURL method.

See this comment at https://github.com/firebase/firebase-tools/blob/master/src/emulator/storage/rules/utils.ts#L81.

This is a true bug and we will get around to fixing this eventually but it's hard to say when the team will have the time to tackle this.

@bytewiz
Copy link
Author

bytewiz commented Oct 20, 2023

This is the emulator code that's returning this particular error message: https://github.com/firebase/firebase-tools/blob/b7eea76c22816a0caf4e45e6bd0f072c066c5d44/src/emulator/storage/apis/firebase.ts#L111

Unfortunately, our admin credential vetting in the emulator is in a pretty poor state. We don't have full support of OAuth access tokens, which is what the admin SDK is sending along with its requests.

In short the validation flow for getDownloadUrl in the emulator is this:

  1. Does the Authorization header value equal the string literal, "owner"?
  2. If not, validate security rules.

Since there's no way to get the admin SDK to pass along the header "Authorization: Bearer owner", this will likely remain broken until we have a full OAuth validation, which we haven't needed until the introduction of the admin getDownloadURL method.

See this comment at https://github.com/firebase/firebase-tools/blob/master/src/emulator/storage/rules/utils.ts#L81.

This is a true bug and we will get around to fixing this eventually but it's hard to say when the team will have the time to tackle this.

@tonyjhuang thanks for taking the time to review it!

But can I rely on that it only persists for the emulator and not in the "real" / production environment when deploying the function? Would the simplest initializeApp(); work or does it require the credentials as of one of the other examples?

Appreciate the support on this!

Or is there any other workaround to get the download url using the firebase-admin sdk in case the above is not working?

@bytewiz
Copy link
Author

bytewiz commented Nov 2, 2023

So does it actually work in production?? @maneesht @tonyjhuang

@bytewiz
Copy link
Author

bytewiz commented Nov 17, 2023

No one here? 😅

@weilinzung
Copy link

weilinzung commented Dec 5, 2023

Same issue with the Firebase emulator, even with custom rules. Only working after deploying to Firebase

    "storage": {
      "port": 9199,
      "rules": "storage-emulator.rules"
    },
rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow read, write: if true;
    }
  }
}

@christianbauer1
Copy link

I have the same issue here and am bypassing the getDownloadUrl in the emulator for now. @tonyjhuang Please notify us if this is fixed 👍

@Shakahs
Copy link

Shakahs commented Dec 20, 2023

Here's my workaround:
Obtain a download token from the emulator's REST API and manually construct an emulator-compatible download URL.

import fetch from 'cross-fetch';
import { FullMetadata } from '@firebase/storage-types';

/**
 * Asynchronously generates and returns the download URL for a file in a specified Firebase Storage emulator bucket.
 * The generated URL can be used to download the file.
 *
 * @param {string} bucket - The name of the Firebase Storage emulator bucket.
 * @param {string} filePath - The path to the file inside the bucket.
 * @returns {Promise<string>} - A promise that resolves to the download URL as a string.
 */
export const getEmulatorDownloadURL = async (bucket: string, filePath: string) => {
    // fetch a new download token
    const tokenGenerationFetch = await fetch(
        `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
            filePath,
        )}?create_token=true`,
        {
            method: 'POST',
            headers: {
                Authorization: 'Bearer owner',
            },
        },
    );
    const tokenGenerationResponse: FullMetadata & { downloadTokens: string } = await tokenGenerationFetch.json();
    const downloadToken = tokenGenerationResponse.downloadTokens.split(',')[0];

    // manually construct the emulator download url
    return `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
        filePath,
    )}?alt=media&token=${downloadToken}`;
};

@actuallymentor
Copy link

Is there any debug info we can provide to help make a fix? This is very unfortunate DX.

@akselipalmer
Copy link

You can use this function to conditionally run getDownloadUrl() or getEmulatorDownloadURL() based on whether you are running using the firebase emulators or in production.

exports.getFileDownloadUrl = async (filePath) => {
  // Use 'process.env.FUNCTIONS_EMULATOR === "true"' to check your environment.
  // Make sure that "true" is surrounded by quotes because it is a string, not a boolean.
  if (process.env.FUNCTIONS_EMULATOR === "true") {
    // Running using emulators.
    // You can find the bucket in the storage emulator suite.
    // Your bucket name should look something like this: <gs://your-app-name.appspot.com/>.
    return await getEmulatorDownloadURL(bucket, filePath);
  } else {
    // Running in production.
    const fileRef = getStorage().bucket().file(filePath);
    const fileUri = await getDownloadURL(fileRef);
    return fileUri;
  }
};

Here's my workaround: Obtain a download token from the emulator's REST API and manually construct an emulator-compatible download URL.

import fetch from 'cross-fetch';
import { FullMetadata } from '@firebase/storage-types';

/**
 * Asynchronously generates and returns the download URL for a file in a specified Firebase Storage emulator bucket.
 * The generated URL can be used to download the file.
 *
 * @param {string} bucket - The name of the Firebase Storage emulator bucket.
 * @param {string} filePath - The path to the file inside the bucket.
 * @returns {Promise<string>} - A promise that resolves to the download URL as a string.
 */
export const getEmulatorDownloadURL = async (bucket: string, filePath: string) => {
    // fetch a new download token
    const tokenGenerationFetch = await fetch(
        `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
            filePath,
        )}?create_token=true`,
        {
            method: 'POST',
            headers: {
                Authorization: 'Bearer owner',
            },
        },
    );
    const tokenGenerationResponse: FullMetadata & { downloadTokens: string } = await tokenGenerationFetch.json();
    const downloadToken = tokenGenerationResponse.downloadTokens.split(',')[0];

    // manually construct the emulator download url
    return `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
        filePath,
    )}?alt=media&token=${downloadToken}`;
};

@kdawgwilk
Copy link

Is there a way to set a custom storage.rules just for the storage emulator? e.g.

{
  ...
  "emulators": {
    "storage": {
      "port": 9199,
      "rules": "storage.rules.emulator"
    },
  }
}

This could be an easy workaround to use open rules for local and then use the regular rules for deployment as a workaround for now.

@anonimitoraf
Copy link

@kdawgwilk I haven't tried myself, but you should be able to use a different Firebase config file (ala firebase --config firebase.emulator.json) which references a different storage rules file

@jcruzv-prog
Copy link

jcruzv-prog commented Apr 26, 2024

This solution actually works, is the only way I found out there thanks!

@anonimitoraf
Copy link

Btw (for my use case), I found that the download URL was accessible via .publicUrl()

    const uploadRef = storage.bucket().file('assets/' + filename)
    await uploadRef.save(buffer, {
      metadata: { cacheControl: 'public,max-age=86400' },
      public: true,
    })
    return uploadRef.publicUrl()

or via .metadata.mediaLink

    const uploadRef = storage.bucket().file('assets/' + filename)
    await uploadRef.save(file.buffer, {
      metadata: { cacheControl: 'public,max-age=86400' },
      public: true,
    })
    const [metadata] = await uploadRef.getMetadata()
    return metadata.mediaLink

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests