New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Feature: GridFS / bson / Meteor 1.6 upgrade? #1333

Closed
xet7 opened this Issue Nov 14, 2017 · 32 comments

Comments

4 participants
@xet7
Copy link
Member

xet7 commented Nov 14, 2017

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 3, 2017

Standalone Wekan seems to work on Node 6.12 and Meteor 1.6, still using MongoDB 3.2.18.

This does not fix crash bug when uploading image or file when using MongoDB 3.4.x

Current code is at meteor-1.6 branch:
https://github.com/wekan/wekan/tree/meteor-1.6

And code build script for Meteor 1.6 version is here:
https://github.com/wekan/wekan-maintainer/blob/master/virtualbox/rebuild-wekan-meteor-1.6.sh

This Node 6.12/Meteor 1.6 version does not yet work on Sandstorm:
https://groups.google.com/forum/#!topic/sandstorm-dev/wzZkXlTnuQw

Please test and report any problems. I don't know how this affects performance etc, as there has been performance fixes, I merged 2nd one some hours ago #1373 by @mfshiu .

I probably also provide Docker version of this Meteor 1.6 Wekan in near future.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 3, 2017

I did update Dockerfile and add that meteor-1.6 tag to Docker Hub, automatic build could take one hour, and then it will appear to:
https://hub.docker.com/r/wekanteam/wekan/tags/

It can be tested with similar Docker Compose script like at https://github.com/wekan/wekan-mongodb , just change wekanteam/wekan:latest tag to wekanteam/wekan:meteor-1.6

@ocdtrekkie

This comment has been minimized.

Copy link
Contributor

ocdtrekkie commented Dec 11, 2017

Can we perhaps get a clear list of blocking issues on running with the latest Node and Mongo? I definitely don't want to lose the regular Sandstorm release cadence for long, and getting these issues sorted would go a long way to resolving other regular issues with Wekan like people using the wrong Mongo version.

I'd be happy to pitch in some bounty money to get these resolved.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

@ocdtrekkie

  1. MongoDB version is 3.2.18. So it's newest 3.2.x.

Problem when using newer 3.4.10 version (newest of 3.4.x) is that when files is
uploaded, Node crashes and exists.

This is error when trying to restore 3.2.18 database to 3.4.x:

$ mongorestore --drop
2017-12-11T17:45:54.508+0200	using default 'dump' directory
2017-12-11T17:45:54.509+0200	preparing collections to restore from
2017-12-11T17:45:54.549+0200	reading metadata for admin.activities from dump/admin/activities.metadata.json
2017-12-11T17:45:54.555+0200	reading metadata for admin.presences from dump/admin/presences.metadata.json
2017-12-11T17:45:54.577+0200	reading metadata for admin.boards from dump/admin/boards.metadata.json
2017-12-11T17:45:54.583+0200	reading metadata for admin.cfs_gridfs.attachments.chunks from dump/admin/cfs_gridfs.attachments.chunks.metadata.json
2017-12-11T17:45:54.613+0200	restoring admin.activities from dump/admin/activities.bson
2017-12-11T17:45:54.656+0200	restoring admin.presences from dump/admin/presences.bson
2017-12-11T17:45:54.678+0200	restoring admin.boards from dump/admin/boards.bson
2017-12-11T17:45:54.708+0200	restoring admin.cfs_gridfs.attachments.chunks from dump/admin/cfs_gridfs.attachments.chunks.bson
2017-12-11T17:45:54.716+0200	no indexes to restore
2017-12-11T17:45:54.717+0200	finished restoring admin.presences (38 documents)
2017-12-11T17:45:54.718+0200	restoring indexes for collection admin.activities from metadata
2017-12-11T17:45:54.722+0200	restoring indexes for collection admin.boards from metadata
2017-12-11T17:45:54.734+0200	reading metadata for admin.cards from dump/admin/cards.metadata.json
2017-12-11T17:45:54.819+0200	restoring indexes for collection admin.cfs_gridfs.attachments.chunks from metadata
2017-12-11T17:45:54.822+0200	Failed: admin.cfs_gridfs.attachments.chunks: error creating indexes for admin.cfs_gridfs.attachments.chunks: createIndex error: The field 'w' is not valid for an index specification. Specification: { name: "files_id_1_n_1", ns: "admin.cfs_gridfs.attachments.chunks", w: 1, key: { files_id: 1, n: 1 } }

When I try to upload file (for example .png), I get this error:

SA GridFS - ERROR! { MongoError: Can't get executor for query { files_id: ObjectId('5a2eaa5f2d6e2f5059b1c3fa'), n: { $gte: 0 } }
    at Object.toError (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/utils.js:114:11)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1194:31
    at /home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
    at Server.Base._callHandler (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
    at MongoReply.parseBody (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
    at .<anonymous> (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:446:20)
    at emitOne (events.js:96:13)
    at emit (events.js:188:7)
    at .<anonymous> (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:207:13)
    at emitTwo (events.js:106:13)
    at emit (events.js:191:7)
    at Socket.<anonymous> (/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/connection.js:440:22)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
  name: 'MongoError',
  ok: 0,
  errmsg: 'Can\'t get executor for query { files_id: ObjectId(\'5a2eaa5f2d6e2f5059b1c3fa\'), n: { $gte: 0 } }',
  code: 17241,
  codeName: 'Location17241' }
/home/wekan/repos/wekan/.build/bundle/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:246
        throw message;      
        ^

In Meteor 1.6 Node versions seems to be 3.2.x:

$ meteor mongo
MongoDB shell version: 3.2.15                 
connecting to: 127.0.0.1:3001/meteor
Server has startup warnings: 
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] 
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] 
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-12-11T18:01:50.443+0200 I CONTROL  [initandlisten]

I will continue in another issue comment about Node version.

@ocdtrekkie

This comment has been minimized.

Copy link
Contributor

ocdtrekkie commented Dec 11, 2017

Interesting note from Meteor's repo: meteor/meteor#9396

3.4.x doesn't support 32-bit, and Meteor is going to support 3.2.x for 32-bit and 3.4.x for 64-bit. This change was merged 19 days ago and is presumably scheduled for Meteor 1.7.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

Standalone Wekan at meteor-1.6 branch is using Node 6.2, I'm currently testing using Node 6.12.2.
I did try newer Node 7.x/8.x/9.x but when I start Wekan the web interface does not load.

For Sandstorm I use meteor-spk 0.4.0 that maybe uses Node 8.8.1 and Mongo 3.2.15:

$ which meteor-spk
/home/wekan/repos/meteor-spk/meteor-spk-0.4.0/meteor-spk

When I start Wekan with meteor-spk dev then Wekan web interface does not load, it keeps spinning, and here are debug logs:

Debug log: Untitled Wekan board

...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2017-12-11T16:26:35.165+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=nTSTziHPa2L84fPjG
sandstorm/sandstorm-http-bridge.c++:2479: warning: App isn't listening for TCP connections after 30 seconds. Continuing to attempt to connect; address->toString() = 127.0.0.1:4000
** SANDSTORM SUPERVISOR: Grain still in use; staying up for now.
@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

Upgrading Node from 6.12 to 6.12.2 did not work. WIth 6.12.2 I did get this error:

$ ./start-wekan.sh 
Presence started serverId=p8uvSGudqe85bY8az
Exception in onConnection callback: TypeError: Object.getOwnPropertyDescriptors is not a function
    at insert (packages/mongo/collection.js:435:12)
    at Mongo.Collection.(anonymous function) [as insert] (packages/aldeed_collection2-core.js:232:19)
    at packages/3stack_presence/lib/server/presence.coffee:7:13
    at runWithEnvironment (packages/meteor.js:1240:24)
    at packages/meteor.js:1253:14
    at packages/ddp-server/livedata_server.js:1478:9
    at Hook.each (packages/callback-hook.js:128:15)
    at Server._handleConnect (packages/ddp-server/livedata_server.js:1476:27)
    at exposeLivedata (packages/meteorhacks_meteorx.js:26:25)
    at packages/meteorhacks_meteorx.js:175:1
    at packages/meteorhacks_meteorx.js:200:4
    at packages/meteorhacks_meteorx.js:212:3
    at infos.forEach.info (/home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:414:13)
    at Array.forEach (native)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:413:9
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:463:5
/home/wekan/repos/wekan/.build/bundle/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

TypeError: Object.getOwnPropertyDescriptors is not a function
    at insert (packages/mongo/collection.js:435:12)
    at Mongo.Collection.(anonymous function) [as insert] (packages/aldeed_collection2-core.js:232:19)
    at updateVersions (packages/autoupdate.js:125:20)
    at packages/autoupdate.js:189:3
    at Function.time (/home/wekan/repos/wekan/.build/bundle/programs/server/profile.js:309:28)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:423:13
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:464:5
    at Function.run (/home/wekan/repos/wekan/.build/bundle/programs/server/profile.js:510:12)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:462:11
@ocdtrekkie

This comment has been minimized.

Copy link
Contributor

ocdtrekkie commented Dec 11, 2017

It would probably be best to figure out what's wrong with newer Node versions outside of Sandstorm, so you have the full suite of debugging options you're used to. At the end of the day, all Sandstorm's log is telling you there is it can't load the web interface, and hence, after 30 seconds, is assuming something's wrong.

@ocdtrekkie

This comment has been minimized.

Copy link
Contributor

ocdtrekkie commented Dec 11, 2017

Given the changes, I'm guessing it's in 6.12.1 you found an issue, not 6.12.2, which just updates OpenSSL.

https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V6.md#6.12.2

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

With 6.12.1 I get same error:

$ ./start-wekan.sh 
Presence started serverId=GDbYfuSY4p7mEg7jQ
Exception in onConnection callback: TypeError: Object.getOwnPropertyDescriptors is not a function
    at insert (packages/mongo/collection.js:435:12)
    at Mongo.Collection.(anonymous function) [as insert] (packages/aldeed_collection2-core.js:232:19)
    at packages/3stack_presence/lib/server/presence.coffee:7:13
    at runWithEnvironment (packages/meteor.js:1240:24)
    at packages/meteor.js:1253:14
    at packages/ddp-server/livedata_server.js:1478:9
    at Hook.each (packages/callback-hook.js:128:15)
    at Server._handleConnect (packages/ddp-server/livedata_server.js:1476:27)
    at exposeLivedata (packages/meteorhacks_meteorx.js:26:25)
    at packages/meteorhacks_meteorx.js:175:1
    at packages/meteorhacks_meteorx.js:200:4
    at packages/meteorhacks_meteorx.js:212:3
    at infos.forEach.info (/home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:414:13)
    at Array.forEach (native)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:413:9
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:463:5
/home/wekan/repos/wekan/.build/bundle/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

TypeError: Object.getOwnPropertyDescriptors is not a function
    at insert (packages/mongo/collection.js:435:12)
    at Mongo.Collection.(anonymous function) [as insert] (packages/aldeed_collection2-core.js:232:19)
    at updateVersions (packages/autoupdate.js:125:20)
    at packages/autoupdate.js:189:3
    at Function.time (/home/wekan/repos/wekan/.build/bundle/programs/server/profile.js:309:28)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:423:13
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:464:5
    at Function.run (/home/wekan/repos/wekan/.build/bundle/programs/server/profile.js:510:12)
    at /home/wekan/repos/wekan/.build/bundle/programs/server/boot.js:462:11
@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

Could this be related? auth0/lock#112

@ocdtrekkie

This comment has been minimized.

Copy link
Contributor

ocdtrekkie commented Dec 11, 2017

Whatever is in that issue worked on 6.12.1, and not on 7.x. So that can't be your issue with upgrading to 6.12.1.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 11, 2017

The problem was that Meteor 1.6.1 upgrade broke Wekan. So when I went back to Meteor 1.6.0 then I was able to upgrade to Node 6.12.2. I did test 8.8.1 but it did not work yet. Continuing upgrades...

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Dec 13, 2017

Sorry I needed to go back to Meteor 1.4 and Node 4.8.4 because upgrades broke Docker/Snap/Sandstorm builds.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 6, 2018

@snowyurik found the code at #1409 that has problems on Meteor 1.6 and Node 8.8.1. This error is with Wekan in general, it's not specific to FreeBSD.

At wekan/models/attachments.js with Node 6.12.2 console shows both console log messages I added before and after attachment problem, and Wekan works. With Node 8.8.1 there is only first console log message "before attachment problem", and then Wekan web interface does not load at all.

https://github.com/wekan/wekan/blob/devel/models/attachments.js#L1-L26

console.log('before attachment problem');

Attachments = new FS.Collection('attachments', {
  stores: [

    // XXX Add a new store for cover thumbnails so we don't load big images in
    // the general board view
    new FS.Store.GridFS('attachments', {
      // If the uploaded document is not an image we need to enforce browser
      // download instead of execution. This is particularly important for HTML
      // files that the browser will just execute if we don't serve them with the
      // appropriate `application/octet-stream` MIME header which can lead to user
      // data leaks. I imagine other formats (like PDF) can also be attack vectors.
      // See https://github.com/wekan/wekan/issues/99
      // XXX Should we use `beforeWrite` option of CollectionFS instead of
      // collection-hooks?
      // We should use `beforeWrite`.
      beforeWrite: (fileObj) => {
        if (!fileObj.isImage()) {
          return {
            type: 'application/octet-stream',
          };
        }
        return {};
      },
    }),
  ],
});

console.log('after attachment problem');

@kentonv Do you have ideas about this?

@kentonv

This comment has been minimized.

Copy link

kentonv commented Jan 7, 2018

Doesn't ring any bells for me. Sandstorm's own upgrade to node 8 went pretty smoothly. But I guess your problem indicates that GridFS has a compatibility issue.

Did you try wrapping it in try/catch? Or does the process just exit/crash outright?

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 7, 2018

@kentonv

How would I wrap it with try/catch? Can you give an example?

Process keeps running and it does not give any error messages.

@kentonv

This comment has been minimized.

Copy link

kentonv commented Jan 7, 2018

I mean put try { where you have the first console.log and } catch (err) { console.log(err); throw err; } at the second. Since you know execution stops somewhere in between there.

If that doesn't catch anything -- and the process doesn't exit, as you said -- then it must be getting stuck.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 21, 2018

@kentonv

I did get Standalone Wekan working with Meteor 1.6.0.1, Node 8.9.3, NPM 5.5.1 and fibers 2.0.0.
I have your sandstorm.js capnp path fix applied.

I did try to add capnp also manually because I got errors at Sandstorm with meteor-spk 4.0.0:

> capnp@0.2.0 install /home/wekan/repos/wekan/node_modules/capnp
> node ./build.js

(node:10280) [DEP0006] DeprecationWarning: child_process: options.customFds option is deprecated. Use options.stdio instead.
make: Entering directory `/home/wekan/repos/wekan/node_modules/capnp/build'
  CXX(target) Release/obj.target/capnp/src/node-capnp/capnp.o
../src/node-capnp/capnp.cc:31:27: fatal error: capnp/dynamic.h: No such file or directory
 #include <capnp/dynamic.h>
                           ^
compilation terminated.
make: *** [Release/obj.target/capnp/src/node-capnp/capnp.o] Error 1
make: Leaving directory `/home/wekan/repos/wekan/node_modules/capnp/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/wekan/.meteor/packages/meteor-tool/.1.6.0_1.obe9u1++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Linux 4.4.14-040414-generic
gyp ERR! command "/home/wekan/.meteor/packages/meteor-tool/.1.6.0_1.obe9u1++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node" "/home/wekan/.meteor/packages/meteor-tool/.1.6.0_1.obe9u1++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/wekan/repos/wekan/node_modules/capnp
gyp ERR! node -v v8.9.3
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
Build failed
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! capnp@0.2.0 install: `node ./build.js`
npm ERR! Exit status 1

When I start Wekan in Sandstorm local dev, I get error:

Error: remote exception: Peer disconnected.
C++ location:(remote):??
type: disconnected

Sandstorm console log:

...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2018-01-21T22:14:26.362+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=zFAkbbmd4izSeNkgo
/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

Error: Cannot find module 'capnp'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9215:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2018-01-21T22:14:37.419+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=abwTnBgenCifJ8Ze7
/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

Error: Cannot find module 'capnp'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9215:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns

How to add capnp correctly?

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 22, 2018

I did build again, this time on js console:

...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2018-01-22T14:13:56.379+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=AvNt5c6atQYdpzKiZ
/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

Error: Cannot find module '/node_modules/capnp.js'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9215:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2018-01-22T14:14:11.682+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=NSnXdxjs5Qwb5nkjX
/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

Error: Cannot find module '/node_modules/capnp.js'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9215:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
@HLFH

This comment has been minimized.

Copy link

HLFH commented Jan 25, 2018

FYI, Meteor 1.6.2 supports MongoDB 3.6.
And ArchLinux has packaged MongoDB to version 3.6.0 on 25/12/2017.

https://github.com/meteor/meteor/archive/release/METEOR@1.6.2-beta.3.tar.gz

I had to remove MongoDB 3.6 and then downgrade to MongoDB 3.4. And connect to mongo, and do:

db.adminCommand( { setFeatureCompatibilityVersion: "3.4" } )

And then remove MongoDB 3.4 and install MongoDB 3.6 again...to make Wekan work again.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 25, 2018

@HLFH

Thanks! I will test it.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Jan 27, 2018

@HLFH

Did you get METEOR@1.6.2-beta.3.tar.gz release working with Wekan?

I have problems with ClassyTestCase errors peerlibrary/meteor-computed-field#13

@kentonv

This comment has been minimized.

Copy link

kentonv commented Jan 28, 2018

Hi @xet7,

You may want to check out: sandstorm-io/sandstorm#3006

Meteor changed the way require() looks up modules, and so it will no longer find the capnp module that is bundled by meteor-spk. I think you can fix this by doing require("/node_modules/capnp.js") instead of require("capnp").

Note that from the errors you posted, it looks like you may have declared the capnp npm module as a dependency in your package.json. You actually should not do this, because it causes Meteor to try to download and build the capnp module from npm, but it will get the wrong version, and it probably won't build unless you install capnp and other things. meteor-spk already bundles in a precompiled copy of the capnp module, so treat it like a built-in module -- don't declare a dependency.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 18, 2018

@kentonv

I don't understand where that capnp is anywhere:

dpkg -l | grep capnp
dpkg -l | grep capnproto

gives nothing.

At sandstorm.js there is:

const Capnp = require('/node_modules/capnp.js');

there is no mention of capnp in these files:

wekan/package.json
wekan/.meteor/packages
wekan/.meteor/versions
wekan/node_modules/...

I did install meteor-spk-4.0 package from:
https://github.com/sandstorm-io/meteor-spk

With these:

 mkdir -p ~/projects/meteor-spk
 cd ~/projects/meteor-spk
 curl https://dl.sandstorm.io/meteor-spk-0.4.0.tar.xz | tar Jxf -
 cd meteor-spk-0.4.0

When I start Wekan with meteor-spk dev and start Wekan grain it says:

Error: remote exception: Peer disconnected.
C++ location:(remote):??
type: disconnected

And console log says:

Error: Cannot find module '/node_modules/capnp.js'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9636:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: App exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
** Starting Mongo...
2018-02-17T23:51:40.867+0000 I STORAGE  Engine custom option: log=(prealloc=false,file_max=200KB)
about to fork child process, waiting until server is ready for connections.
forked process: 10
child process started successfully, parent exiting
** Starting Meteor...
Presence started serverId=Y3mLesyLc9rDEqrvs
/programs/server/node_modules/fibers/future.js:280
						throw(ex);
						^

Error: Cannot find module '/node_modules/capnp.js'
    at makeMissingError (packages/modules-runtime.js:231:12)
    at require (packages/modules-runtime.js:241:19)
    at sandstorm.js (sandstorm.js:25:17)
    at fileEvaluate (packages/modules-runtime.js:343:9)
    at require (packages/modules-runtime.js:238:16)
    at /programs/server/app/app.js:9636:1
    at infos.forEach.info (/programs/server/boot.js:414:13)
    at Array.forEach (<anonymous>)
    at /programs/server/boot.js:413:9
    at /programs/server/boot.js:463:5
    at Function.run (/programs/server/profile.js:510:12)
    at /programs/server/boot.js:462:11
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns

It seems that sandstorm did auto-update while I was testing compiled from source version, so I did get these:

added 136 packages in 21.063s
App is now available from Sandstorm server. Ctrl+C to disconnect.
^CRequesting shutdown due to signal: Interrupt
sandstorm/run-bundle.c++:3340: failed: mkstemp(commandFile): No such file or directory
stack: 0x5e39c5 0x5dfc7e

This is on Debian 10 buster:

$ uname -a
Linux pc 4.14.0-3-amd64 #1 SMP Debian 4.14.17-1 (2018-02-14) x86_64 GNU/Linux
$ cat /etc/issue
Debian GNU/Linux buster/sid \n \l

Are you really sure node-capnp is included in meteor-spk-4.0 ? How can I verify it?

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 18, 2018

I did not install that capnproto apt package that is mentioned at:
https://github.com/wekan/wekan-maintainer/blob/master/virtualbox/rebuild-wekan-meteor-1.6.sh

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 18, 2018

@kentonv

So this file should be there:

const Capnp = require('/node_modules/capnp.js');

but when I go to that directory, there is no such file:

~/projects/meteor-spk/meteor-spk-0.4.0/meteor-spk.deps/node_modules$ ls capnp
c++.capnp  persistent.capnp  rpc.capnp  rpc-twoparty.capnp  schema.capnp
@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 18, 2018

hmm, capnp.js seems to be there:

~/projects/meteor-spk/meteor-spk-0.4.0/meteor-spk.deps/node_modules$ ls
bson   capnp.js    core-util-is  inherits  mongodb       readable-stream  string_decoder
capnp  capnp.node  es6-promise   isarray   mongodb-core  sandstorm

But still Wekan does not find it.

@kentonv

This comment has been minimized.

Copy link

kentonv commented Feb 19, 2018

@xet7 Right, the capnp module is included in meteor-spk. You don't need to install it nor declare any dependency on it, it's just there.

I don't understand either why it isn't working... :/

@xet7 xet7 added this to In progress: by xet7. Order: from top to bottom of cards. in Wekan Roadmap Feb 20, 2018

@xet7 xet7 moved this from 1) In progress: by xet7, paid Wekan and unrelated work. Help welcome. Order: from top to bottom of cards. to 2) In progress: by xet7, not paid Wekan and unrelated work. Help welcome. Order: from top to bottom of cards. in Wekan Roadmap Feb 21, 2018

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 21, 2018

@kentonv

I hacked Wekan working on Sandstorm.

It seems that Swimlane view does not work properly. Anyway I'll push some crappy release to @ocdtrekkie for testing soon.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 21, 2018

Sandstorm is now using Meteor 1.6.0.1 version, with Wekan v0.76 released today for Sandstorm and other platforms.

Upgrades to Meteor 1.6.1 is separate issue meteor/meteor#9609 and would require a lot more work currently.

@xet7 xet7 closed this Feb 21, 2018

Wekan Roadmap automation moved this from 2) In progress: by xet7, not paid Open Source Wekan and other hobbies. Help welcome. Order: from top to bottom of cards. to Done Feb 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment