Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Heavy use of beacon crashes joola.io #305

Closed
orweinberger opened this issue Mar 25, 2014 · 0 comments
Closed

Heavy use of beacon crashes joola.io #305

orweinberger opened this issue Mar 25, 2014 · 0 comments
Assignees
@itayw
Labels
beacon blocker bug core expedite
Milestone
v0.4.0

Comments

@orweinberger
Copy link
Member

When trying to push a high volume of documents into joola.io via beacon, the system crashes:

[orw@orw-office joola.io]$ node joola.io.js --verbose
12:53:18.654Z DEBUG joola.io: [config-redis] Connected to redis @ undefined:undefined#0
12:53:18.655Z DEBUG joola.io: State for [config-redis] changed to [working]: redis [config-redis] is up.
12:53:18.658Z  WARN joola.io: Found an empty configuration store, building initial...
12:53:18.682Z  WARN joola.io: File logging disabled. Could not resolve path to log file [/var/log/joola.io//log-19953.log]: Path missing
12:53:18.767Z DEBUG joola.io: [runtime-redis] Connected to redis @ localhost:6379#3
12:53:18.804Z DEBUG joola.io: Message dispatch online.
connect.multipart() will be removed in connect 3.0
visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives
connect.limit() will be removed in connect 3.0
12:53:18.845Z DEBUG joola.io: joola.io HTTP server listening on port 8080
12:53:18.845Z DEBUG joola.io: State for [webserver-http] changed to [working]: webserver-http is up.
12:53:18.895Z DEBUG joola.io: joola.io HTTPS server listening on port 8081
12:53:18.896Z DEBUG joola.io: State for [webserver-https] changed to [working]: webserver-https is up.
12:53:18.896Z  INFO joola.io: joola.io version 3.4.0 started [xi194t0bt].
12:53:18.897Z DEBUG joola.io: State for [core] changed to [online]: init complete
12:53:18.901Z DEBUG joola.io: [publisher] Connected to redis @ localhost:6379#1
12:53:18.901Z DEBUG joola.io: State for [publisher] changed to [working]: redis [publisher] is up.
12:53:18.901Z DEBUG joola.io: [subscriber] Connected to redis @ localhost:6379#1
12:53:18.901Z DEBUG joola.io: State for [subscriber] changed to [working]: redis [subscriber] is up.
12:53:19.769Z DEBUG joola.io: Status change, joola.io is now [online]
12:53:19.773Z DEBUG joola.io: State changed to [online]
12:53:33.541Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.542Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.543Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.545Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.979Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.980Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.981Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.981Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.982Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:33.982Z DEBUG joola.io: Cache store refreshed due to a change.
12:53:34.015Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: collection already exists
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: collection already exists",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:403:20
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:296:16
    at /home/orw/dev/joola.io/lib/common/mongo.js:133:14
    at /home/orw/dev/joola.io/lib/common/mongo.js:74:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1012:24
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/utils.js:157:46
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1806:9
    at Server.Base._callHandler (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)
12:53:34.017Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: collection already exists
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: collection already exists",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:403:20
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:296:16
    at /home/orw/dev/joola.io/lib/common/mongo.js:133:14
    at /home/orw/dev/joola.io/lib/common/mongo.js:74:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1012:24
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/utils.js:157:46
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1806:9
    at Server.Base._callHandler (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)
12:53:34.022Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: collection already exists
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: collection already exists",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:403:20
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:296:16
    at /home/orw/dev/joola.io/lib/common/mongo.js:133:14
    at /home/orw/dev/joola.io/lib/common/mongo.js:74:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1012:24
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/utils.js:157:46
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1806:9
    at Server.Base._callHandler (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)
ERROR! Cannot call method 'createCollection' of undefined
TypeError: Cannot call method 'createCollection' of undefined
    at /home/orw/dev/joola.io/lib/common/mongo.js:71:8
    at /home/orw/dev/joola.io/lib/common/mongo.js:38:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/mongo_client.js:102:20
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:273:18
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:351:18
    at Server.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:210:38)
    at Db.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:347:21)
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:271:21
    at null.<anonymous> (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:563:7)
    at EventEmitter.emit (events.js:106:17)
12:53:36.048Z ERROR joola.io: DOMAIN FATAL EXCEPTION! Cannot call method 'createCollection' of undefined
{"name":"joola.io","hostname":"orw-office","pid":19953,"level":50,"msg":"DOMAIN FATAL EXCEPTION! Cannot call method 'createCollection' of undefined","time":"2014-03-25T12:53:36.048Z","v":0}
12:53:36.049Z DEBUG joola.io:
  TypeError: Cannot call method 'createCollection' of undefined
      at /home/orw/dev/joola.io/lib/common/mongo.js:71:8
      at /home/orw/dev/joola.io/lib/common/mongo.js:38:16
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/mongo_client.js:102:20
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:273:18
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:351:18
      at Server.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:210:38)
      at Db.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:347:21)
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:271:21
      at null.<anonymous> (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:563:7)
      at EventEmitter.emit (events.js:106:17)
Aborted
[orw@orw-office joola.io]$ ^C
[orw@orw-office joola.io]$ redis-cli flushall
OK
[orw@orw-office joola.io]$ node joola.io.js --verbose
12:54:02.945Z DEBUG joola.io: [config-redis] Connected to redis @ undefined:undefined#0
12:54:02.946Z DEBUG joola.io: State for [config-redis] changed to [working]: redis [config-redis] is up.
12:54:02.949Z  WARN joola.io: Found an empty configuration store, building initial...
12:54:02.969Z  WARN joola.io: File logging disabled. Could not resolve path to log file [/var/log/joola.io//log-20976.log]: Path missing
12:54:03.052Z DEBUG joola.io: [runtime-redis] Connected to redis @ localhost:6379#3
12:54:03.090Z DEBUG joola.io: Message dispatch online.
connect.multipart() will be removed in connect 3.0
visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives
connect.limit() will be removed in connect 3.0
12:54:03.133Z DEBUG joola.io: joola.io HTTP server listening on port 8080
12:54:03.133Z DEBUG joola.io: State for [webserver-http] changed to [working]: webserver-http is up.
12:54:03.184Z DEBUG joola.io: joola.io HTTPS server listening on port 8081
12:54:03.184Z DEBUG joola.io: State for [webserver-https] changed to [working]: webserver-https is up.
12:54:03.185Z  INFO joola.io: joola.io version 3.4.0 started [c06cVLbds].
12:54:03.185Z DEBUG joola.io: State for [core] changed to [online]: init complete
12:54:03.189Z DEBUG joola.io: [publisher] Connected to redis @ localhost:6379#1
12:54:03.189Z DEBUG joola.io: State for [publisher] changed to [working]: redis [publisher] is up.
12:54:03.190Z DEBUG joola.io: [subscriber] Connected to redis @ localhost:6379#1
12:54:03.190Z DEBUG joola.io: State for [subscriber] changed to [working]: redis [subscriber] is up.
12:54:04.056Z DEBUG joola.io: Status change, joola.io is now [online]
12:54:04.060Z DEBUG joola.io: State changed to [online]
12:54:09.334Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: Collection already exist
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: Collection already exist",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:400:18
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:51:20
    at /home/orw/dev/joola.io/lib/dispatch/collections.js:174:16
    at /home/orw/dev/joola.io/lib/dispatch/collections.js:102:14
    at /home/orw/dev/joola.io/lib/common/config.js:115:16
    at /home/orw/dev/joola.io/node_modules/nconf/lib/nconf/provider.js:267:34
    at Object.async.whilst (/home/orw/dev/joola.io/node_modules/async/lib/async.js:624:13)
    at /home/orw/dev/joola.io/node_modules/async/lib/async.js:620:23
12:54:09.817Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:09.818Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:09.818Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:09.819Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.397Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.398Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.399Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.399Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.400Z DEBUG joola.io: Cache store refreshed due to a change.
12:54:10.428Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: collection already exists
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: collection already exists",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:403:20
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:296:16
    at /home/orw/dev/joola.io/lib/common/mongo.js:133:14
    at /home/orw/dev/joola.io/lib/common/mongo.js:74:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1012:24
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/utils.js:157:46
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1806:9
    at Server.Base._callHandler (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)
12:54:10.429Z  WARN joola.io: Error while processing route [api/beacon/insert]: Failed to route action [fetch]: collection already exists
  err: {
    "name": "joola.io.engine Exception",
    "code": 500,
    "message": "Failed to route action [fetch]: collection already exists",
    "stack": null
  }
Trace
    at Object.exports.responseError (/home/orw/dev/joola.io/lib/webserver/routes/index.js:45:13)
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:381:25
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:403:20
    at /home/orw/dev/joola.io/lib/dispatch/beacon.js:296:16
    at /home/orw/dev/joola.io/lib/common/mongo.js:133:14
    at /home/orw/dev/joola.io/lib/common/mongo.js:74:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1012:24
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/utils.js:157:46
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:1806:9
    at Server.Base._callHandler (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)
ERROR! Cannot call method 'createCollection' of undefined
TypeError: Cannot call method 'createCollection' of undefined
    at /home/orw/dev/joola.io/lib/common/mongo.js:71:8
    at /home/orw/dev/joola.io/lib/common/mongo.js:38:16
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/mongo_client.js:102:20
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:273:18
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:351:18
    at Server.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:210:38)
    at Db.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:347:21)
    at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:271:21
    at null.<anonymous> (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:563:7)
    at EventEmitter.emit (events.js:106:17)
12:54:12.672Z ERROR joola.io: DOMAIN FATAL EXCEPTION! Cannot call method 'createCollection' of undefined
{"name":"joola.io","hostname":"orw-office","pid":20976,"level":50,"msg":"DOMAIN FATAL EXCEPTION! Cannot call method 'createCollection' of undefined","time":"2014-03-25T12:54:12.672Z","v":0}
12:54:12.673Z DEBUG joola.io:
  TypeError: Cannot call method 'createCollection' of undefined
      at /home/orw/dev/joola.io/lib/common/mongo.js:71:8
      at /home/orw/dev/joola.io/lib/common/mongo.js:38:16
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/mongo_client.js:102:20
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:273:18
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:351:18
      at Server.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:210:38)
      at Db.close (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:347:21)
      at /home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/db.js:271:21
      at null.<anonymous> (/home/orw/dev/joola.io/node_modules/mongodb/lib/mongodb/connection/server.js:563:7)
      at EventEmitter.emit (events.js:106:17)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@itayw itayw

Labels
beacon blocker bug core expedite
Projects
None yet
Milestone
v0.4.0
Development

No branches or pull requests
2 participants
@orweinberger @itayw