Skip to content

Commit

Permalink
fix(lib-storage): cleanup stream listeners to prevent memory leak
Browse files Browse the repository at this point in the history
Clean up event listeners in lib-storage's to prevents memory leaks caused by streaming the data to a growing number of variables as the function runs on longer files.
  • Loading branch information
samsullivan committed Dec 23, 2020
1 parent 64d2210 commit 3d36682
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion lib/storage/src/data-chunk/readable-helper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,28 @@ export async function* chunkFromReadable(reader: Readable, chunkSize: number): A
function _chunkFromStream(stream: Readable, chunkSize: number, oldBuffer: Buffer): Promise<StreamChunk> {
let currentChunk = oldBuffer;
return new Promise((resolve, reject) => {
const cleanupListeners = () => {
stream.removeAllListeners("data");
stream.removeAllListeners("error");
stream.removeAllListeners("end");
};

stream.on("data", (chunk) => {
currentChunk = Buffer.concat([currentChunk, Buffer.from(chunk)]);
if (currentChunk.length >= chunkSize) {
cleanupListeners();
resolve({
Body: currentChunk,
ended: false,
});
}
});
stream.on("error", reject);
stream.on("error", (err) => {
cleanupListeners();
reject(err);
});
stream.on("end", () => {
cleanupListeners();
resolve({
Body: currentChunk,
ended: true,
Expand Down

0 comments on commit 3d36682

Please sign in to comment.