MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) #2977

mmasdoria · 2020-03-30T12:30:17Z

Issue

I have tried last version of Wekan with Mongodb replicaset 4.2.3 (mongodb Atlas).

MONGO_URL: mongodb://mongodb-atlas:xxxxxx@host1:27017,host2:27017,host3:27017/wekan-prod?ssl=true&replicaSet=wekan-replicaset&authSource=admin&retryWrites=true&w=majority

Wekan never connect to mongodb saying member of replicaset can't be found

Searching solution on google I found this post : https://github.com/meanware/gridfs

So I replace meteor gridfs default module (cfs:gridfs) by this one (meanware/gridf) and it WORKS !

For the moment, I found only one issue, I can't upload image into mongodb but this problem is blocking for us ...
Error : Cannot read property 'start' of undefined at Object.createReadStream (packages/meanware_gridfs.js:94:26))
So there is a problem with the module (meanware/gridfs) I decided to use earlier.

So I have 2 questions :

How is it possible that the default cfs:gridfs module in recent version of meteor(1.10) is not up to date (it seems to only works with old mongodb version when using with replicaset).
Can Wekan be used with recent version of mongodb with replicaset and how make it work ?

I'm not a developper, just a cloud architecture guy so my knowledge in development are very limited.

Thanks a lot.
BR
Mathieu Mesnil

xet7 · 2020-03-30T12:41:05Z

That's interesting, I have not yet tried Wekan on Atlas. I'll look at it.

mmasdoria · 2020-03-30T14:17:32Z

I found a solution for the image upload/download with mongodb.
After adding meteor package meanware/grid, I've modified file gridfs.server.js by adding the following code at the begining of fonction createReadStream as it was done in package cfs:gridfs:

options = options || {};

Now uploading/downloading an image seems to work.

Regards.

juliensl · 2020-11-17T17:00:42Z

Hi !

I am trying to connect my wekan app to mongoDB atlas and I still have a similar issue :

WARNING: failed loading: 
Error: ENOENT: no such file or directory, open
    at Object.openSync (fs.js:462:3)
    at Object.readFileSync (fs.js:364:35)
    at packages/wekan-oidc.js:30:14
    at packages/wekan-oidc.js:277:4
    at packages/wekan-oidc.js:285:3
    at /build/programs/server/boot.js:401:38
    at Array.forEach (<anonymous>)
    at /build/programs/server/boot.js:226:21
    at /build/programs/server/boot.js:464:7
    at Function.run (/build/programs/server/profile.js:280:14)
    at /build/programs/server/boot.js:463:13 {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT'
}
Presence started serverId=nDxjNZgkb8efT53gg
/build/programs/server/node_modules/fibers/future.js:280
                                                throw(ex);
                                                ^

Error: invalid schema, expected mongodb
    at module.exports (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/url_parser.js:20:11)
    at connect (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongo_client.js:290:16)
    at Function.MongoClient.connect (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongo_client.js:114:3)
    at FS.StorageAdapter.init (packages/cfs:gridfs/gridfs.server.js:164:27)
    at packages/meteor.js:306:21
    at new FS.StorageAdapter (packages/cfs_storage-adapter.js:285:42)
    at new FS.Store.GridFS (packages/cfs:gridfs/gridfs.server.js:46:10)
    at module (models/attachments.js:172:11)
    at fileEvaluate (packages/modules-runtime.js:336:7)
    at Module.require (packages/modules-runtime.js:238:14)
    at require (packages/modules-runtime.js:258:21)
    at /build/programs/server/app/app.js:65622:1
    at /build/programs/server/boot.js:401:38
    at Array.forEach (<anonymous>)
    at /build/programs/server/boot.js:226:21
    at /build/programs/server/boot.js:464:7
    at Function.run (/build/programs/server/profile.js:280:14)
    at /build/programs/server/boot.js:463:13

That's what I put in my docker-compose :
- MONGO_URL=mongodb+srv://wekan:MDP@wekan-test-emp....../wekan-test-empty

Using the latest version of wekan 4.53 and mongo version 4.2.10

Where are we on this problem ?
Do I have to try the meanware/gridfs workaround or you think it will be fixed soon ? 😃

Thank you for your work !

xet7 · 2020-11-17T17:33:01Z

@juliensl

That meanware/gridfs is old fork of gridfs. Current plans are to change from current Wekan cfs to ostrio:files #3273 https://github.com/veliovgroup/Meteor-Files . Although I don't know does ostrio:files work with MongoDB Atlas - if not, maybe that MongoDB Atlas specific changes could be sent as pull request to ostrio:files.

Well, "soon" depends on definition of soon:

Very soon: You try fixing it and you send pull request
Soon: Someone pays me to prioritize this at Commercial Support
Sometime: After I get most urgent other paid features done and have time to try add all other required code, and test would all those changed code work without breaking everyones existing MongoDB installs.

(Psst, I have crazy secret plan to change from MongoDB to other databases #787 , but all those urgent feature and bugfix requests asking "when this and that will be ready" for current Wekan disturb a little)

xet7 · 2020-11-28T19:39:16Z

Atlas is mentioned here:

Could someone test does Atlas work?

Nightreaver · 2023-01-09T14:43:15Z

Hello, this seems to be an issue I'm hitting currently.

Im having this kind of connection string mongodb+srv://username:password@myserver.home.net/wekan?ssl=false and it fails with

Error: invalid schema, expected mongodb
    at module.exports (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/url_parser.js:20:11)
    at connect (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongo_client.js:290:16)
    at Function.MongoClient.connect (/build/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongo_client.js:114:3)
    at FS.StorageAdapter.init (packages/cfs:gridfs/gridfs.server.js:164:27)
    at packages/meteor.js:343:21
    at new FS.StorageAdapter (packages/cfs_storage-adapter.js:285:42)
    at new FS.Store.GridFS (packages/cfs:gridfs/gridfs.server.js:46:10)
    at module (models/attachments_old.js:13:9)
    at fileEvaluate (packages/modules-runtime.js:336:7)
    at Module.require (packages/modules-runtime.js:238:14)
    at require (packages/modules-runtime.js:258:21)
    at /build/programs/server/app/app.js:167732:1
    at /build/programs/server/boot.js:369:38
    at Array.forEach (<anonymous>)
    at /build/programs/server/boot.js:210:21
    at /build/programs/server/boot.js:423:7
    at Function.run (/build/programs/server/profile.js:256:14)
    at /build/programs/server/boot.js:422:13

From what I could figure out searching, its probably this: https://stackoverflow.com/a/56766466

The error `invalid schema, expected mongodb` means that you're using an outdated node driver version. The old driver cannot parse the new mongodb+srv URI scheme.

Support for the mongodb+srv scheme was added in the node driver version 3.0 in this ticket: [NODE-1145](https://jira.mongodb.org/browse/NODE-1145).

I am not using Atlas, just the plain dns seedlist connection string as described here: https://www.mongodb.com/docs/v5.2/reference/connection-string/#dns-seed-list-connection-format

So is this related or doest it come from somwhere else?

xet7 · 2023-01-09T14:54:29Z

@Nightreaver

What version of MongoDB are you using?

Nightreaver · 2023-01-09T15:01:09Z

Having mongodb 5.x here Lauri Ojansivu ***@***.***> schrieb am Mo. 9. Jan. 2023 um 15:54:

…

@Nightreaver <https://github.com/Nightreaver> What version of MongoDB are you using? — Reply to this email directly, view it on GitHub <#2977 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY6MDBHVBSJ2EAUKJOI76DWRQRDBANCNFSM4LWROXPA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

xet7 · 2023-01-09T15:02:34Z

@Nightreaver

What version of WeKan? Is it Docker/Snap/Bundle/other?

Nightreaver · 2023-01-09T15:05:52Z

Sorry for lack of information … Wekan Docker in Kubernetes v6.67. with the helm chart we edited before new-year Lauri Ojansivu ***@***.***> schrieb am Mo. 9. Jan. 2023 um 16:02:

…

@Nightreaver <https://github.com/Nightreaver> What version of WeKan? Is it Docker/Snap/Bundle/other? — Reply to this email directly, view it on GitHub <#2977 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY6MDDVXQEZECMFEGHUDXLWRQSBNANCNFSM4LWROXPA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

xet7 · 2023-01-09T15:29:36Z

@Nightreaver

Ok. I'm In Progress of releasing WeKan v6.69, you can then test does it work better.

xet7 · 2023-01-09T15:38:22Z

In that new version, I'm testing does it help that I removed older MongoDB Node.js driver from package.json 17a06ad , because Meteor also has dependency to newer version of driver at https://github.com/meteor/meteor/blob/master/packages/npm-mongo/package.js

Newest WeKan currently uses MongoDB 6.x, versions are listed at https://wekan.github.io Download section, I'm looking are there any errors with using it.

xet7 · 2023-01-09T16:05:40Z

@Nightreaver

Please try WeKan v6.69 with Helm Chart version 1.1.6 and MongoDB 6.0.3:
https://artifacthub.io/packages/helm/wekan/wekan

Nightreaver · 2023-02-01T12:47:15Z

Hello,

I tried the 6.69 / 1.1.6 without success, the same error came up. I cannot test with mongodb 6.x as our IT only provides 5.x atm
Is there anything else that can be done?

xet7 · 2023-02-02T14:06:25Z

@Nightreaver

Try 6.72 / 1.1.9 with MongoDB 5.x.

Nightreaver · 2023-02-13T12:51:42Z

I've tested with v6.74 now and still getting Error: invalid schema, expected mongodb, when I switch back to the old string, wekan works well. so still just connection string issue.

mzch · 2023-05-03T05:57:03Z

When adding tls=true to MONGO_URL or setting MONGO_URL=mongodb+srv://..., wekan can't connect to mongodb with TLS.

MongoDB: 6.0.5
Wekan: v6.86

May  3 14:48:20 ls start.sh[197844]: Can not set createdAt index on "__pre_attachments" collection {
May  3 14:48:20 ls start.sh[197844]:   keys: { createdAt: 1 },
May  3 14:48:20 ls start.sh[197844]:   opts: { expireAfterSeconds: 10800, background: true },
May  3 14:48:20 ls start.sh[197844]:   details: MongoServerSelectionError: Hostname/IP does not match certificate's altnames: IP: 45.42.200.86 is not in the cert's list:
May  3 14:48:20 ls start.sh[197844]:       at Timeout._onTimeout (/srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/sdam/topology.js:292:38)
May  3 14:48:20 ls start.sh[197844]:       at listOnTimeout (internal/timers.js:557:17)
May  3 14:48:20 ls start.sh[197844]:       at processTimers (internal/timers.js:500:7)
May  3 14:48:20 ls start.sh[197844]:    => awaited here:
May  3 14:48:20 ls start.sh[197844]:       at Function.Promise.await (/srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:56:12)
May  3 14:48:20 ls start.sh[197844]:       at packages/ostrio:files/server.js:37:5
May  3 14:48:20 ls start.sh[197844]:       at /srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40 {
May  3 14:48:20 ls start.sh[197844]:     reason: TopologyDescription {
May  3 14:48:20 ls start.sh[197844]:       type: 'ReplicaSetNoPrimary',
May  3 14:48:20 ls start.sh[197844]:       servers: [Map],
May  3 14:48:20 ls start.sh[197844]:       stale: false,
May  3 14:48:20 ls start.sh[197844]:       compatible: true,
May  3 14:48:20 ls start.sh[197844]:       heartbeatFrequencyMS: 10000,
May  3 14:48:20 ls start.sh[197844]:       localThresholdMS: 15,
May  3 14:48:20 ls start.sh[197844]:       setName: 'rs01',
May  3 14:48:20 ls start.sh[197844]:       maxElectionId: new ObjectId("7fffffff000000000000000a"),
May  3 14:48:20 ls start.sh[197844]:       maxSetVersion: 2,
May  3 14:48:20 ls start.sh[197844]:       commonWireVersion: 0,
May  3 14:48:20 ls start.sh[197844]:       logicalSessionTimeoutMinutes: null
May  3 14:48:20 ls start.sh[197844]:     },
May  3 14:48:20 ls start.sh[197844]:     code: undefined,
May  3 14:48:20 ls start.sh[197844]:     [Symbol(errorLabels)]: Set(0) {}
May  3 14:48:20 ls start.sh[197844]:   }
May  3 14:48:20 ls start.sh[197844]: }
May  3 14:48:20 ls start.sh[197844]: Exception while polling query {"collectionName":"__pre_attachments","selector":{},"options":{"transform":null,"projection":{"_id":1,"isFinished":1}}} MongoServerSelectionError: Hostname/IP does not match certificate's altnames: IP: 45.42.200.86 is not in the cert's list:
May  3 14:48:20 ls start.sh[197844]:     at Timeout._onTimeout (/srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/sdam/topology.js:292:38)
May  3 14:48:20 ls start.sh[197844]:     at listOnTimeout (internal/timers.js:557:17)
May  3 14:48:20 ls start.sh[197844]:     at processTimers (internal/timers.js:500:7)
May  3 14:48:20 ls start.sh[197844]:  => awaited here:
May  3 14:48:20 ls start.sh[197844]:     at Function.Promise.await (/srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:56:12)
May  3 14:48:20 ls start.sh[197844]:     at packages/mongo/mongo_driver.js:1134:14
May  3 14:48:20 ls start.sh[197844]:     at /srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
May  3 14:48:20 ls start.sh[197844]:  => awaited here:
May  3 14:48:20 ls start.sh[197844]:     at Promise.await (/srv/wekan/.build/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:60:12)
May  3 14:48:20 ls start.sh[197844]:     at SynchronousCursor._nextObject (packages/mongo/mongo_driver.js:1183:38)
May  3 14:48:20 ls start.sh[197844]:     at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1198:22)
May  3 14:48:20 ls start.sh[197844]:     at SynchronousCursor.getRawObjects (packages/mongo/mongo_driver.js:1248:12)
May  3 14:48:20 ls start.sh[197844]:     at PollingObserveDriver._pollMongo (packages/mongo/polling_observe_driver.js:156:44)
May  3 14:48:20 ls start.sh[197844]:     at Object.task (packages/mongo/polling_observe_driver.js:93:12)
May  3 14:48:20 ls start.sh[197844]:     at Meteor._SynchronousQueue.SQp._run (packages/meteor.js:1026:16)
May  3 14:48:20 ls start.sh[197844]:     at packages/meteor.js:1003:12 {
May  3 14:48:20 ls start.sh[197844]:   reason: TopologyDescription {
May  3 14:48:20 ls start.sh[197844]:     type: 'ReplicaSetNoPrimary',
May  3 14:48:20 ls start.sh[197844]:     servers: Map(1) { '45.42.200.86:27017' => [ServerDescription] },
May  3 14:48:20 ls start.sh[197844]:     stale: false,
May  3 14:48:20 ls start.sh[197844]:     compatible: true,
May  3 14:48:20 ls start.sh[197844]:     heartbeatFrequencyMS: 10000,
May  3 14:48:20 ls start.sh[197844]:     localThresholdMS: 15,
May  3 14:48:20 ls start.sh[197844]:     setName: 'rs01',
May  3 14:48:20 ls start.sh[197844]:     maxElectionId: new ObjectId("7fffffff000000000000000a"),
May  3 14:48:20 ls start.sh[197844]:     maxSetVersion: 2,
May  3 14:48:20 ls start.sh[197844]:     commonWireVersion: 0,
May  3 14:48:20 ls start.sh[197844]:     logicalSessionTimeoutMinutes: null
May  3 14:48:20 ls start.sh[197844]:   },
May  3 14:48:20 ls start.sh[197844]:   code: undefined,
May  3 14:48:20 ls start.sh[197844]:   [Symbol(errorLabels)]: Set(0) {}
May  3 14:48:20 ls start.sh[197844]: }

xet7 changed the title ~~Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work~~ MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) Apr 15, 2020

xet7 mentioned this issue Apr 15, 2020

Does this work with CosmosDB or DynamoDB? #2852

Closed

xet7 mentioned this issue Jul 21, 2020

Question: back end other than Mongo? #787

Open

xet7 mentioned this issue Sep 13, 2020

cfs:gridfs deprecated #3272

Closed

xet7 added 3rd-parties-integration Bug labels Feb 20, 2021

xet7 mentioned this issue Jan 9, 2023

Try WeKan Snap Manual upgrade to newest Candidate? #4780

Closed

xet7 mentioned this issue Jul 10, 2023

WeKan v6.99.2 bug about pre_attachments MongoTopologyClosedError #5002

Closed

xet7 closed this as completed Nov 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) #2977

MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) #2977

mmasdoria commented Mar 30, 2020 •

edited

xet7 commented Mar 30, 2020

mmasdoria commented Mar 30, 2020

juliensl commented Nov 17, 2020

xet7 commented Nov 17, 2020

xet7 commented Nov 28, 2020

Nightreaver commented Jan 9, 2023

xet7 commented Jan 9, 2023

Nightreaver commented Jan 9, 2023 via email

xet7 commented Jan 9, 2023

Nightreaver commented Jan 9, 2023 via email

xet7 commented Jan 9, 2023

xet7 commented Jan 9, 2023

xet7 commented Jan 9, 2023

Nightreaver commented Feb 1, 2023

xet7 commented Feb 2, 2023

Nightreaver commented Feb 13, 2023

mzch commented May 3, 2023

MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) #2977

MongoDB Atlas (was: Wekan 3.86 et mongodb 4.2.3 replicaset doesn't work) #2977

Comments

mmasdoria commented Mar 30, 2020 • edited

Issue

xet7 commented Mar 30, 2020

mmasdoria commented Mar 30, 2020

juliensl commented Nov 17, 2020

xet7 commented Nov 17, 2020

xet7 commented Nov 28, 2020

Nightreaver commented Jan 9, 2023

xet7 commented Jan 9, 2023

Nightreaver commented Jan 9, 2023 via email

xet7 commented Jan 9, 2023

Nightreaver commented Jan 9, 2023 via email

xet7 commented Jan 9, 2023

xet7 commented Jan 9, 2023

xet7 commented Jan 9, 2023

Nightreaver commented Feb 1, 2023

xet7 commented Feb 2, 2023

Nightreaver commented Feb 13, 2023

mzch commented May 3, 2023

mmasdoria commented Mar 30, 2020 •

edited