Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDBStreamReadable prematurely emits end #54

Open
asprouse opened this issue May 9, 2019 · 2 comments
Open

DynamoDBStreamReadable prematurely emits end #54

asprouse opened this issue May 9, 2019 · 2 comments

Comments

@asprouse
Copy link
Contributor

asprouse commented May 9, 2019

DynamoDBStreamReadable prematurely emits end which causes the serverless-offline-dynamodb-streams plugin to close it's writable. This means that when the plugin is used with TRIM_HORIZON it will only fire handlers on sls offline start.

Looks like this PR to fix LATEST starting point might have broken this:
#37

@godu
Copy link
Contributor

godu commented May 15, 2019

@asprouse Could you give me a way to reproduce ?

@asprouse
Copy link
Contributor Author

This happens in serverless-offline-dynamodb-streams on line 165 you can observe end being emitted by simply inserting an on call to the readable:

const readable = DynamoDBReadable(
        dynamodbStreamsClient,
        streamARN,
        assign(this.config, {
          shardId,
          limit: tableEvent.batchSize,
          iterator: tableEvent.startingPosition || 'TRIM_HORIZON'
        })
      );

      readable.on('end', () => console.log('Readable End!');

      readable.pipe(
        new Writable({
          objectMode: true,
          write: (chunk, encoding, cb) => {
            this.eventHandler(streamARN, functionName, shardId, chunk, cb);
          }
        })
      );
    }, shards);

Since you are implementing your own writable this isn't an issue here but write streams are supposed to close when end is emitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants