New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module-ssh2: socket event polling is implemented only with select(2) and no bounds checking is performed on the socket descriptor leading to a crash #714

Closed
davidnich opened this Issue Apr 7, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@davidnich
Contributor

davidnich commented Apr 7, 2016

I got the following backtrace (processed with c++filt to be readable) from the OCI library on OSX on my laptop - where the network connections go up and down, so there's definitely a bug somewhere in the ssh2 module regarding connection handling:

kpedbg_dmp_stack()+584
kpeDbgCrash()+413
kpeDbgSignalHandler()+220
skgesig_sigactionHandler()+287
_sigtramp()+26
0x700000933ea0()
SSH2Client::sshConnectUnlocked(int, ExceptionSink*)+330
SFTPClient::sftpConnectUnlocked(int, ExceptionSink*)+92
SFTPClient::connect(int, ExceptionSink*)+49
SSH2Base_connect_Vt(QoreObject*, SSH2Client*, QoreValueList const*, unsigned long long, ExceptionSink*)+63
BuiltinNormalMethodValueVariant::evalImpl(QoreObject*, AbstractPrivateData*, QoreValueList const*, unsigned long long, ExceptionSink*) const+37
@davidnich

This comment has been minimized.

Show comment
Hide comment
@davidnich

davidnich Apr 12, 2016

Contributor

possibly related crash:

INVALID-ENCODING: invalid UTF-8 encoding encountered in string: list: (1 element)
  [0]=hash: (13 members)
    id : 21070
    time : 2016-04-08 20:32:35.712428 Fri +02:00 (CEST)
    timeus : 712428
    class : 106
    classstr : "ALERT"
    event : 5006
    eventstr : "ALERT_ONGOING_RAISED"
    severity : 3
    severitystr : "MAJOR"
    caller : hash: (2 members)
      user : "<internal>"
      api : "external API call"
    info : hash: (11 members)
      type : "USER-CONNECTION"
      id : "sftp-b4n"
      alert : "USER-CONNECTION-UNAVAILABLE"
      alertid : 10253
      reason : "user connection monitor: connection could not be acquired to \"sftp-b4n\" (IE B4N SFTP Polling / Delivery connection) url: \"sftp://david@localhost/Users/david/src/Qorus/test/FAKE_NAS/fake-sftp/b4n\"; Connections.qc:675: f?E?f=???: error waiting for network (timeout: 60000ms) in SSH2Client::connect(); closing connection: Invalid argument"
      who : "Qorus"
      source : "user service ebs-ie-opco-in-b4n_polling v2.0 (264) method: init"
      object : "USER-CONNECTION sftp-b4n"
      instance : "quark-1"
      name : "sftp-b4n"
      auditid : -1
    compositeseverity : 3
    compositeseveritystr : "MAJOR"
unhandled QORE System exception thrown in TID 25 at 2016-04-08 20:32:35.721483 Fri +02:00 (CEST) in make_json() (QorusRestApiHandler.qc:6872, builtin code)
INVALID-ENCODING: invalid UTF-8 encoding encountered in string
call stack:
  3: RETHROW at QorusRestApiHandler.qc:6876
  2: make_json() (QorusRestApiHandler.qc:6872, builtin code)
  1: WebAppSocketHandler::eventListener() (qorus.q:517, user code)
ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_addr=0x1230ac19d]
kpedbg_dmp_stack()+584<-kpeDbgCrash()+413<-kpeDbgSignalHandler()+220<-skgesig_sigactionHandler()+287<-_sigtramp()+26<-malloc_zone_calloc()+78<-_ZN13QoreExceptionC2EPKcP16AbstractQoreNodeS3_()+87<-_ZN13ExceptionSink19raiseErrnoExceptionEPKciP14QoreStringNode()+249<-_ZN13ExceptionSink19raiseErrnoExceptionEPKciS1_z()+325<-_ZN10SSH2Client18waitSocketUnlockedEP13ExceptionSinkPKcS3_S3_ibP27AbstractDisconnectionHelper()+90<-_ZN10SSH2Client18sshConnectUnlockedEiP13ExceptionSink()+330<-_ZN10SFTPClient19sftpConnectUnlockedEiP13ExceptionSink()+92<-_ZN10SFTPClient7connectEiP13ExceptionSink()+49<-_ZL19SSH2Base_connect_VtP10QoreObjectP10SSH2ClientPK13QoreValueListyP13ExceptionSink()+63<-_ZNK31BuiltinNormalMethodValueVariant8evalImplEP10QoreObjectP19AbstractPrivateDataPK13QoreValueListyP13ExceptionSink()+37<-_ZN19qore_object_private32evalBuiltinMethodWithPrivateDataERK10QoreMethodPK30BuiltinNormalMethodVariantBasePK13QoreValueListyP13ExceptionSink()+116<-_ZNK30BuiltinNormalMethodVariantBase10evalMethodEP10QoreObjectR20CodeEvaluationHelperP13ExceptionSink()+474<-_ZNK19qore_method_private17evalNormalVariantEP10QoreObjectPK25QoreExternalMethodVariantPK12QoreListNodeP13ExceptionSink()+219<-_ZNK22AbstractMethodCallNode4execEP10QoreObjectPKcP13ExceptionSink()+100<-_ZNK23QoreDotEvalOperatorNode13evalValueImplERbP13ExceptionSink()+291<-_ZN18ValueEvalRefHolderC2EPK16AbstractQoreNodeP13ExceptionSink()+87<-_ZNK9ParseNode14bigIntEvalImplEP13ExceptionSink()+32<-_ZN19ExpressionStatement8execImplER9QoreValueP13ExceptionSink()+44<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152
Contributor

davidnich commented Apr 12, 2016

possibly related crash:

INVALID-ENCODING: invalid UTF-8 encoding encountered in string: list: (1 element)
  [0]=hash: (13 members)
    id : 21070
    time : 2016-04-08 20:32:35.712428 Fri +02:00 (CEST)
    timeus : 712428
    class : 106
    classstr : "ALERT"
    event : 5006
    eventstr : "ALERT_ONGOING_RAISED"
    severity : 3
    severitystr : "MAJOR"
    caller : hash: (2 members)
      user : "<internal>"
      api : "external API call"
    info : hash: (11 members)
      type : "USER-CONNECTION"
      id : "sftp-b4n"
      alert : "USER-CONNECTION-UNAVAILABLE"
      alertid : 10253
      reason : "user connection monitor: connection could not be acquired to \"sftp-b4n\" (IE B4N SFTP Polling / Delivery connection) url: \"sftp://david@localhost/Users/david/src/Qorus/test/FAKE_NAS/fake-sftp/b4n\"; Connections.qc:675: f?E?f=???: error waiting for network (timeout: 60000ms) in SSH2Client::connect(); closing connection: Invalid argument"
      who : "Qorus"
      source : "user service ebs-ie-opco-in-b4n_polling v2.0 (264) method: init"
      object : "USER-CONNECTION sftp-b4n"
      instance : "quark-1"
      name : "sftp-b4n"
      auditid : -1
    compositeseverity : 3
    compositeseveritystr : "MAJOR"
unhandled QORE System exception thrown in TID 25 at 2016-04-08 20:32:35.721483 Fri +02:00 (CEST) in make_json() (QorusRestApiHandler.qc:6872, builtin code)
INVALID-ENCODING: invalid UTF-8 encoding encountered in string
call stack:
  3: RETHROW at QorusRestApiHandler.qc:6876
  2: make_json() (QorusRestApiHandler.qc:6872, builtin code)
  1: WebAppSocketHandler::eventListener() (qorus.q:517, user code)
ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_addr=0x1230ac19d]
kpedbg_dmp_stack()+584<-kpeDbgCrash()+413<-kpeDbgSignalHandler()+220<-skgesig_sigactionHandler()+287<-_sigtramp()+26<-malloc_zone_calloc()+78<-_ZN13QoreExceptionC2EPKcP16AbstractQoreNodeS3_()+87<-_ZN13ExceptionSink19raiseErrnoExceptionEPKciP14QoreStringNode()+249<-_ZN13ExceptionSink19raiseErrnoExceptionEPKciS1_z()+325<-_ZN10SSH2Client18waitSocketUnlockedEP13ExceptionSinkPKcS3_S3_ibP27AbstractDisconnectionHelper()+90<-_ZN10SSH2Client18sshConnectUnlockedEiP13ExceptionSink()+330<-_ZN10SFTPClient19sftpConnectUnlockedEiP13ExceptionSink()+92<-_ZN10SFTPClient7connectEiP13ExceptionSink()+49<-_ZL19SSH2Base_connect_VtP10QoreObjectP10SSH2ClientPK13QoreValueListyP13ExceptionSink()+63<-_ZNK31BuiltinNormalMethodValueVariant8evalImplEP10QoreObjectP19AbstractPrivateDataPK13QoreValueListyP13ExceptionSink()+37<-_ZN19qore_object_private32evalBuiltinMethodWithPrivateDataERK10QoreMethodPK30BuiltinNormalMethodVariantBasePK13QoreValueListyP13ExceptionSink()+116<-_ZNK30BuiltinNormalMethodVariantBase10evalMethodEP10QoreObjectR20CodeEvaluationHelperP13ExceptionSink()+474<-_ZNK19qore_method_private17evalNormalVariantEP10QoreObjectPK25QoreExternalMethodVariantPK12QoreListNodeP13ExceptionSink()+219<-_ZNK22AbstractMethodCallNode4execEP10QoreObjectPKcP13ExceptionSink()+100<-_ZNK23QoreDotEvalOperatorNode13evalValueImplERbP13ExceptionSink()+291<-_ZN18ValueEvalRefHolderC2EPK16AbstractQoreNodeP13ExceptionSink()+87<-_ZNK9ParseNode14bigIntEvalImplEP13ExceptionSink()+32<-_ZN19ExpressionStatement8execImplER9QoreValueP13ExceptionSink()+44<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152
@davidnich

This comment has been minimized.

Show comment
Hide comment
@davidnich

davidnich Apr 13, 2016

Contributor

another crash on OS/X (laptop with frequent network changes)

ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_addr=0x200000021]
kpedbg_dmp_stack()+584<-kpeDbgCrash()+413<-kpeDbgSignalHandler()+220<-skgesig_sigactionHandler()+287<-_sigtramp()+26<-0x70000157bea0()<-_ZN10SSH2Client18sshConnectUnlockedEiP13ExceptionSink()+330<-_ZN10SFTPClient19sftpConnectUnlockedEiP13ExceptionSink()+92<-_ZN10SFTPClient7connectEiP13ExceptionSink()+49<-_ZL19SSH2Base_connect_VtP10QoreObjectP10SSH2ClientPK13QoreValueListyP13ExceptionSink()+63<-_ZNK31BuiltinNormalMethodValueVariant8evalImplEP10QoreObjectP19AbstractPrivateDataPK13QoreValueListyP13ExceptionSink()+37<-_ZN19qore_object_private32evalBuiltinMethodWithPrivateDataERK10QoreMethodPK30BuiltinNormalMethodVariantBasePK13QoreValueListyP13ExceptionSink()+116<-_ZNK30BuiltinNormalMethodVariantBase10evalMethodEP10QoreObjectR20CodeEvaluationHelperP13ExceptionSink()+474<-_ZNK19qore_method_private17evalNormalVariantEP10QoreObjectPK25QoreExternalMethodVariantPK12QoreListNodeP13ExceptionSink()+219<-_ZNK22AbstractMethodCallNode4execEP10QoreObjectPKcP13ExceptionSink()+100<-_ZNK23QoreDotEvalOperatorNode13evalValueImplERbP13ExceptionSink()+291<-_ZN18ValueEvalRefHolderC2EPK16AbstractQoreNodeP13ExceptionSink()+87<-_ZNK9ParseNode14bigIntEvalImplEP13ExceptionSink()+32<-_ZN19ExpressionStatement8execImplER9QoreValueP13ExceptionSink()+44<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152<-_ZN14StatementBlock8execImplER9QoreValueP13ExceptionSink()+130<-_ZN11IfStatement8execImplER9QoreValueP13ExceptionSink()+225<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152
Contributor

davidnich commented Apr 13, 2016

another crash on OS/X (laptop with frequent network changes)

ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_addr=0x200000021]
kpedbg_dmp_stack()+584<-kpeDbgCrash()+413<-kpeDbgSignalHandler()+220<-skgesig_sigactionHandler()+287<-_sigtramp()+26<-0x70000157bea0()<-_ZN10SSH2Client18sshConnectUnlockedEiP13ExceptionSink()+330<-_ZN10SFTPClient19sftpConnectUnlockedEiP13ExceptionSink()+92<-_ZN10SFTPClient7connectEiP13ExceptionSink()+49<-_ZL19SSH2Base_connect_VtP10QoreObjectP10SSH2ClientPK13QoreValueListyP13ExceptionSink()+63<-_ZNK31BuiltinNormalMethodValueVariant8evalImplEP10QoreObjectP19AbstractPrivateDataPK13QoreValueListyP13ExceptionSink()+37<-_ZN19qore_object_private32evalBuiltinMethodWithPrivateDataERK10QoreMethodPK30BuiltinNormalMethodVariantBasePK13QoreValueListyP13ExceptionSink()+116<-_ZNK30BuiltinNormalMethodVariantBase10evalMethodEP10QoreObjectR20CodeEvaluationHelperP13ExceptionSink()+474<-_ZNK19qore_method_private17evalNormalVariantEP10QoreObjectPK25QoreExternalMethodVariantPK12QoreListNodeP13ExceptionSink()+219<-_ZNK22AbstractMethodCallNode4execEP10QoreObjectPKcP13ExceptionSink()+100<-_ZNK23QoreDotEvalOperatorNode13evalValueImplERbP13ExceptionSink()+291<-_ZN18ValueEvalRefHolderC2EPK16AbstractQoreNodeP13ExceptionSink()+87<-_ZNK9ParseNode14bigIntEvalImplEP13ExceptionSink()+32<-_ZN19ExpressionStatement8execImplER9QoreValueP13ExceptionSink()+44<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152<-_ZN14StatementBlock8execImplER9QoreValueP13ExceptionSink()+130<-_ZN11IfStatement8execImplER9QoreValueP13ExceptionSink()+225<-_ZN17AbstractStatement4execER9QoreValueP13ExceptionSink()+243<-_ZN14StatementBlock10execInternER9QoreValueP13ExceptionSink()+152

@davidnich davidnich changed the title from module-ssh2: there is a bug in connection handling somewhere to module-ssh2: socket event polling is implemented only with select(2) and no bounds checking is performed on the socket descriptor leading to a crash Apr 15, 2016

@tethal tethal added the fixed label Apr 15, 2016

@tethal tethal closed this Apr 15, 2016

davidnich added a commit to qorelanguage/module-ssh2 that referenced this issue Apr 16, 2016

tethal added a commit to qorelanguage/module-ssh2 that referenced this issue Apr 16, 2016

Merge pull request #19 from qorelanguage/bugfix/714_socket_polling_he…
…aders

refs qorelanguage/qore#714 removed unnecessary async socket I/O headers

@davidnich davidnich modified the milestone: 0.8.12 May 29, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment