Skip to content

Commit

Permalink
fix(rosetta): flakey trim-cache test (#4043)
Browse files Browse the repository at this point in the history
It appears the trim-cache test using tablet compression occasionally fails in CI/CD. I suspect this is due to the `save` function returning before the GZip stream has bene closed, resulting in the subsequent read attempting to consume an incomplete GZip object.

This adds the necessary provisions to ensure the GZip stream is closed (or failed) before `save` returns.



---

By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license].

[Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0
  • Loading branch information
RomainMuller committed Apr 5, 2023
1 parent 6086b34 commit f70c0f7
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 7 deletions.
11 changes: 7 additions & 4 deletions packages/jsii-rosetta/lib/json.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Readable, Writable, pipeline } from 'node:stream';
import { Readable, pipeline } from 'node:stream';
import { promisify } from 'node:util';
import { parser } from 'stream-json';
import * as Assembler from 'stream-json/Assembler';
Expand Down Expand Up @@ -36,12 +36,15 @@ export async function parse(reader: Readable): Promise<any> {
* better performance.
*
* @param value the value to be serialized.
* @param writer the write in which to write the JSON text.
* @param writers the sequence of write streams to use to output the JSON text.
*/
export async function stringify(value: any, writer: Writable): Promise<void> {
export async function stringify(
value: any,
...writers: Array<NodeJS.ReadWriteStream | NodeJS.WritableStream>
): Promise<void> {
const reader = new Readable({ objectMode: true });
reader.push(value);
reader.push(null);

return asyncPipeline(reader, disassembler(), stringer(), writer);
return asyncPipeline(reader, disassembler(), stringer(), ...writers);
}
5 changes: 2 additions & 3 deletions packages/jsii-rosetta/lib/tablets/tablets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -182,14 +182,13 @@ export class LanguageTablet {
* Saves the tablet schema to a file. If the compress option is passed, then
* the schema will be gzipped before writing to the file.
*/
public async save(filename: string, compress = false) {
public async save(filename: string, compress = false): Promise<void> {
await fs.mkdir(path.dirname(filename), { recursive: true });

const writeStream: Writable = createWriteStream(filename, { flags: 'w' });
const gzip = compress ? zlib.createGzip() : undefined;
gzip?.pipe(writeStream, { end: true });

return stringify(this.toSchema(), gzip ?? writeStream);
return stringify(this.toSchema(), ...(gzip ? [gzip] : []), writeStream);
}

private toSchema(): TabletSchema {
Expand Down

0 comments on commit f70c0f7

Please sign in to comment.