Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage File.download error #494

Closed
Marc477 opened this issue Apr 13, 2015 · 4 comments
Closed

Storage File.download error #494

Marc477 opened this issue Apr 13, 2015 · 4 comments
Assignees
Labels
api: storage Issues related to the Cloud Storage API. 🚨 This issue needs some love. triage me I really want to be triaged. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@Marc477
Copy link

Marc477 commented Apr 13, 2015

I think there is an error in this function (storage/file.js):

File.prototype.download = function(options, callback) {
 if (util.is(options, 'function')) {
 callback = options;
 options = {};
}

callback = once(callback);

var destination = options.destination;
delete options.destination;

var fileStream = this.createReadStream(options);

if (destination) {
 fileStream
  .on('error', callback)
  .pipe(fs.createWriteStream(destination))
  .on('error', callback)
  .on('finish', callback);
} else {
var fileContents = new Buffer('');

fileStream
  .on('error', callback)
  .on('data', function(chunk) {
    fileContents = Buffer.concat([fileContents, chunk]);
  })
  .on('complete', function() {
    callback(null, fileContents);
  });
}
};

I dont use the destination option, and the function works, but if I have multiple requests to the same file. Then somtimes the download is pending and never returns anything...

I don't know if it is the once(callback) because it is the same callback for multiple requests but this is strange, I replaced the function with my own code and now it is working:

var iStream = file.gFile.createReadStream();
var fileData = new Buffer('');

iStream.on('error', function(err){
   Log.error(err);
   loadCB(null);
});

iStream.on('data', function(data){
   fileData = Buffer.concat([fileData, data]);
});

iStream.on('complete', function() {
   loadCB(fileData);
});
@stephenplusplus stephenplusplus added api: storage Issues related to the Cloud Storage API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Apr 13, 2015
@stephenplusplus
Copy link
Contributor

Thanks for looking into this! Can you demonstrate how you are making multiple requests?

@Marc477
Copy link
Author

Marc477 commented Apr 13, 2015

I'm hosting videos on google storage, I use a xml file to retreive the info of the video but I have to edit it server side before sending it to the client (so I use .download to get the xml file from storage to compute engine). If I start 7-8 pages at the same time in my browser, then they all try to get the xml file but some of them will never get a response (pending....) and no errors are thrown server side.

With the second code below I don't get this error

@callmehiphop
Copy link
Contributor

@Marc477 Are you still experiencing this issue? If so, could you provide code snippets on how you were using File#download (options, etc.)?

once doesn't augment the actual function, so you should be able to re-use the same callback as many times as you like.

@stephenplusplus
Copy link
Contributor

We've made multiple changes to some core aspects of our library since this issue was opened. Without more details, it's hard for us to debug. I hope it's been resolved since you opened this issue @Marc477, but if not, please re-open with anything else you can share that will help us catch the bug.

@yoshi-automation yoshi-automation added 🚨 This issue needs some love. triage me I really want to be triaged. labels Apr 6, 2020
chingor13 pushed a commit that referenced this issue Aug 22, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
chingor13 pushed a commit that referenced this issue Aug 22, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
chingor13 pushed a commit that referenced this issue Aug 22, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Sep 27, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Oct 11, 2022
This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/b205fd33-200c-4298-88b8-18b0d1c79a3e/targets

- [ ] To automatically regenerate this PR, check this box.
sofisl pushed a commit that referenced this issue Oct 13, 2022
This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/b205fd33-200c-4298-88b8-18b0d1c79a3e/targets

- [ ] To automatically regenerate this PR, check this box.
sofisl pushed a commit that referenced this issue Nov 9, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 10, 2022
This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/fc1dd810-ea11-44c0-ab02-fa000827a405/targets

- [ ] To automatically regenerate this PR, check this box.

Source-Link: googleapis/synthtool@363fe30
sofisl pushed a commit that referenced this issue Sep 13, 2023
[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ts-loader](https://togithub.com/TypeStrong/ts-loader) | [`^8.0.0` -> `^9.0.0`](https://renovatebot.com/diffs/npm/ts-loader/8.1.0/9.0.0) | [![age](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/compatibility-slim/8.1.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/confidence-slim/8.1.0)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>TypeStrong/ts-loader</summary>

### [`v9.0.0`](https://togithub.com/TypeStrong/ts-loader/blob/master/CHANGELOG.md#v900)

[Compare Source](https://togithub.com/TypeStrong/ts-loader/compare/v8.1.0...v9.0.0)

Breaking changes:

-   minimum webpack version: 5
-   minimum node version: 12

Changes:

-   [webpack 5 migration](https://togithub.com/TypeStrong/ts-loader/pull/1251) - thanks [@&#8203;johnnyreilly](https://togithub.com/johnnyreilly), [@&#8203;jonwallsten](https://togithub.com/jonwallsten), [@&#8203;sokra](https://togithub.com/sokra), [@&#8203;appzuka](https://togithub.com/appzuka), [@&#8203;alexander-akait](https://togithub.com/alexander-akait)

</details>

---

### Configuration

:date: **Schedule**: "after 9am and before 3pm" (UTC).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-automl).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. 🚨 This issue needs some love. triage me I really want to be triaged. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

4 participants