Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add flag to allow disabling auto decompression by client #850

Merged
merged 14 commits into from
Oct 4, 2019
Merged

feat: add flag to allow disabling auto decompression by client #850

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
1d785af
feat: add flag to allow disabling auto decomression by client
AVaksman Sep 11, 2019
421569c
refactor: rename returnCompressed default false to decompress default…
AVaksman Sep 11, 2019
b51a0a2
docs: fix
AVaksman Sep 11, 2019
9c111dd
Merge branch 'master' into allow_return_compressed
AVaksman Sep 17, 2019
78642ea
Merge branch 'master' into allow_return_compressed
AVaksman Sep 17, 2019
8a3671b
Merge branch 'master' into allow_return_compressed
AVaksman Sep 18, 2019
7f442ea
Merge branch 'master' into allow_return_compressed
AVaksman Sep 18, 2019
a657606
Merge branch 'master' into allow_return_compressed
AVaksman Sep 19, 2019
579a053
Merge branch 'master' into allow_return_compressed
AVaksman Sep 20, 2019
33be3be
Merge branch 'master' into allow_return_compressed
AVaksman Sep 24, 2019
f463908
Merge branch 'master' into allow_return_compressed
AVaksman Oct 3, 2019
be01018
fix: docs
AVaksman Oct 3, 2019
23247c8
Merge branch 'master' into allow_return_compressed
AVaksman Oct 3, 2019
489b4cd
fix: test description
AVaksman Oct 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 7 additions & 1 deletion src/file.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,7 @@ export interface CreateReadStreamOptions {
validation?: 'md5' | 'crc32c' | false | true;
start?: number;
end?: number;
decompress?: boolean;
}

export interface SaveOptions extends CreateWriteStreamOptions {}
Expand Down Expand Up @@ -1120,6 +1121,10 @@ class File extends ServiceObject<File> {
* NOTE: Byte ranges are inclusive; that is, `options.start = 0` and
* `options.end = 999` represent the first 1000 bytes in a file or object.
* NOTE: when specifying a byte range, data integrity is not available.
* @property {boolean} [decompress=true] Disable auto decompression of the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this from the perspective of the GCS service?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, nodejs-sotage API always requests that the data should be served as is (compressed if applicable) i. e. sending appropriate headers

const headers = {
  'Accept-Encoding': 'gzip',
  'Cache-Control': 'no-store',
} as Headers;

This flag is only from the perspective of nodejs-storage client to allow this library to not transcode the data if received from the server in compressed form and return to user compressed..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frankyn ping!

* received data. By default this option set to `true`.
stephenplusplus marked this conversation as resolved.
Show resolved Hide resolved
* Applicable in cases where the data was uploaded with
* `gzip: true` option. See {@link File#createWriteStream}.
*/
/**
* Create a readable stream to read the contents of the remote file. It can be
Expand Down Expand Up @@ -1192,6 +1197,7 @@ class File extends ServiceObject<File> {
* .pipe(fs.createWriteStream('/Users/stephen/logfile.txt'));
*/
createReadStream(options: CreateReadStreamOptions = {}): Readable {
options = Object.assign({decompress: true}, options);
const rangeRequest =
typeof options.start === 'number' || typeof options.end === 'number';
const tailRequest = options.end! < 0;
Expand Down Expand Up @@ -1312,7 +1318,7 @@ class File extends ServiceObject<File> {
throughStreams.push(validateStream);
}

if (isCompressed) {
if (isCompressed && options.decompress) {
throughStreams.push(zlib.createGunzip());
}

Expand Down
11 changes: 11 additions & 0 deletions test/file.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1203,6 +1203,17 @@ describe('File', () => {
.resume();
});

it('should not gunzip the response if "retunCompressed: true" is passed', done => {
jkwlui marked this conversation as resolved.
Show resolved Hide resolved
file
.createReadStream({decompress: false})
.once('error', done)
.on('data', (data: {}) => {
assert.strictEqual(data, GZIPPED_DATA);
done();
})
.resume();
});

it('should emit errors from the gunzip stream', done => {
const error = new Error('Error.');
const createGunzipStream = through();
Expand Down