Record processor not shutdown on IO close

[pvatterott]

I received an uncaught exception that meant the consumer was no longer connected to Kinesis. In this case, I would imagine that the KCL would either continuously attempt to reconnect or die and call shutdown. However, neither happened, which got my consumer process stuck in an invalid and dangerous state: thinking it was connected when it wasn't.

Stack trace:

Error: Kinesis Client Library is in the invalid state. Cannot proceed further.
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/kcl_manager.js", line 399, col 11, in KCLManager._handleStateInput
    throw new Error('Kinesis Client Library is in the invalid state. Cannot proceed further.');
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/kcl_manager.js", line 265, col 8, in KCLManager._onActionEnd
    this._handleStateInput(this._context, 'cleanup');
  File "events.js", line 106, col 13, in emitNone
  File "events.js", line 208, col 7, in ActionHandler.emit
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/action_handler.js", line 82, col 8, in ActionHandler._onIOClose
    this.emit('end');
  File "events.js", line 106, col 13, in emitNone
  File "events.js", line 208, col 7, in IOHandler.emit
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/io_handler.js", line 93, col 8, in IOHandler._onInputClose
    this.emit('close');
  File "events.js", line 111, col 20, in emitNone
  File "events.js", line 208, col 7, in Interface.emit
  File "readline.js", line 370, col 8, in Interface.close
  File "readline.js", line 149, col 10, in Socket.onend
  File "events.js", line 111, col 20, in emitNone
  File "events.js", line 208, col 7, in Socket.emit
  File "_stream_readable.js", line 1055, col 12, in endReadableNT
  File "internal/process/next_tick.js", line 138, col 11, in _combinedTickCallback
  File "internal/process/next_tick.js", line 218, col 9, in process._tickDomainCallback

[sahas-]

Did this work for anyone? I'm getting exactly the same error..

[fkoner]

Same error here!

[BBonifield]

It's interesting that this was creating on May 14th but has had two comments in the past 24 hours. That being said, I'm running into the same problem.

[walshe-boylesoftware]

im getting it too

[ps185130]

I'm getting Same error . Please let me know if there is a workaround.

[20k-ultra]

This is not related to original bug report but I think some people here might be experience the error that is being logged.

I just want to point out that I was getting this error and spent quite some time debugging only to realize I was rushing and didn't read the docs entirely. Make sure you have the KCL MultiLangDaemon running. You can easily do this via the bootstrap script found in bin/.

I think returning "Kinesis Client Library is in the invalid state." misleads the developer to thinking it is installed instead of returning "Kinesis Client Library not running/installed". The invalid state implies the state could be fixed via some configuration changes but really you're missing the software entirely. Still my fault for not taking the time to read the docs....

[asadighi]

Did you find any solution for this? I have the same problem.

[BBonifield]

@asadighi Yes, the problem is that the example application is not really clear how to set things up. When you boot things, you need to boot it using the kcl-bootstrap. For instance, to run a consumer, my script definition looks like:

export JAVA_HOME=$(/usr/libexec/java_home -v 1.8) && bin/kcl-bootstrap -e -p ./eb-eval-ping-consumer.properties

And then the properties file defines the actual node script that should be ran:

# The script that abides by the multi-language protocol. This script will
# be executed by the MultiLangDaemon, which will communicate with this script
# over STDIN and STDOUT according to the multi-language protocol.
executableName = node consumer.js --stream eb-eval-ping

# The name of an Amazon Kinesis stream to process.
streamName = eb-eval-ping

# Used by the KCL as the name of this application. Will be used as the name
# of an Amazon DynamoDB table which will store the lease and checkpoint
# information for workers with this application name
applicationName = eb-eval-ping-test

It's been awhile since I did this, but the root problem I had was trying to execute the script directly instead of using the bootstrap.