New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update MongoDB to 4.0 #10058

Merged
merged 1 commit into from Aug 9, 2018

Conversation

Projects
None yet
4 participants
@klaussner
Collaborator

klaussner commented Jul 7, 2018

This PR updates MongoDB to version 4.0 and the driver to version 3.1.
Fixes meteor/meteor-feature-requests#269.

ignoreUndefined: true
ignoreUndefined: true,
// Use new URL parser to avoid deprecation warning.
useNewUrlParser: true

This comment has been minimized.

@klaussner

klaussner Jul 7, 2018

Collaborator

The driver shows this warning by default:

DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
@hwillson

This is awesome @klaussner - thanks very much!

@hwillson

This comment has been minimized.

Member

hwillson commented Jul 11, 2018

@klaussner One small thing - it looks like 3.6.4 is still being used for meteor mongo?

screenshot 2018-07-11 09 53 31

@klaussner

This comment has been minimized.

Collaborator

klaussner commented Jul 12, 2018

@hwillson Did you rebuild the dev bundle? This is the output I get from the mongo shell:

$ meteor-dev mongo
MongoDB shell version v4.0.0
connecting to: mongodb://127.0.0.1:3001/meteor
MongoDB server version: 4.0.0
Server has startup warnings:
2018-07-12T21:40:42.422+0200 I STORAGE  [initandlisten]
2018-07-12T21:40:42.422+0200 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-07-12T21:40:42.422+0200 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
meteor:PRIMARY>
@hwillson

This comment has been minimized.

Member

hwillson commented Jul 14, 2018

Hi @klaussner - I did rebuild the dev_bundle, but when I just tried again I'm now seeing bizarre pipelineOperator errors whenever I try to run the new build. I tested this out on a new laptop (with this as the first Meteor test), so I'm going to assume something is up on my end. The code changes look good, so my earlier 👍 still stands. I'll post back shortly after I reconfigure things, but if you're seeing the proper Mongo shell, then I'm happy. Thanks again for working on this!

@hwillson

This comment has been minimized.

Member

hwillson commented Jul 24, 2018

Quick update relating to #10058 (comment) - the pipelineOperator issue was fixed in benjamn/reify@9985a49, and will be addressed in devel shortly (which will make testing this PR much easier).

@hwillson

This comment has been minimized.

Member

hwillson commented Jul 24, 2018

After running with the new reify version, I now see:

➜  pr-10058 /Users/hwillson/Documents/git/meteor/forks/klaussner/meteor/meteor mongo
MongoDB shell version v4.0.0                  
connecting to: mongodb://127.0.0.1:3001/meteor
MongoDB server version: 4.0.0
---
Enable MongoDB's free cloud-based monitoring service to collect and display
metrics about your deployment (disk utilization, CPU, operation statistics,
etc).

The monitoring data will be available on a MongoDB website with a unique
URL created for you. Anyone you share the URL with will also be able to
view this page. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command:
db.enableFreeMonitoring()
---

meteor:PRIMARY> 

Should we suppress Mongo's free monitoring message (is that dynamically pulled in?), or leave it as is? Either way, it's working - thanks @klaussner!

@klaussner

This comment has been minimized.

Collaborator

klaussner commented Jul 25, 2018

Interesting, I didn't get the free monitoring message, just the filesystem warning. 🤷‍♂️ The only way I found to suppress them is to pass the --quiet option to the mongo shell (here) but then the version info and URI also disappear:

$ meteor-dev mongo
meteor:PRIMARY>

On the other hand, they can easily be found using the shell, so I think I'm in favor of using the --quiet option:

$ meteor-dev mongo
meteor:PRIMARY> version()
4.0.0
meteor:PRIMARY> db.serverCmdLineOpts().parsed.net
{ "bindIp" : "127.0.0.1", "port" : 3001 }
meteor:PRIMARY>
@hwillson

This comment has been minimized.

Member

hwillson commented Jul 25, 2018

I like how nice and clean things look using --quiet so I'm all for it. Mongo newcomers might not know how to get Mongo version information, but it is pretty easy to find out. So my vote is for --quiet; @benjamn @abernix what do you guys think?

@hwillson

This comment has been minimized.

Member

hwillson commented Aug 8, 2018

Hi @klaussner - 👍 on the --quiet flag. If you don't mind making that change, we'll get this merged (and if you're busy just let me know and I'll make the necessary changes). Thanks!

@klaussner klaussner force-pushed the klaussner:mongo-4.0-update branch from 29708e8 to bab35dc Aug 8, 2018

@klaussner

This comment has been minimized.

Collaborator

klaussner commented Aug 8, 2018

Thanks, @hwillson. I've added the --quiet flag and rebased onto the current devel. 🙂

@benjamn

This comment has been minimized.

Member

benjamn commented Aug 9, 2018

I've rebased these changes onto release-1.7.1 and built a new dev bundle (8.11.3.13): a875ac5

I think there may still be some test failures, judging from local meteor self-test attempts. Whenever you have a chance to look into that, just rebase against release-1.7.1, change the target branch of this PR, and add any additional commits as you see fit.

Thanks for working on this @klaussner!

@klaussner klaussner changed the base branch from devel to release-1.7.1 Aug 9, 2018

Update `mongo` test to match MongoDB 4.0 output
The `mongo` shell is run with the `--quiet` option (7f7a987), so there's no more output to match.

@klaussner klaussner force-pushed the klaussner:mongo-4.0-update branch from bab35dc to 2d75cc6 Aug 9, 2018

@benjamn

This comment has been minimized.

Member

benjamn commented Aug 9, 2018

I went digging for another way to disable the free monitoring advertisement, and I found this command-line option: --enableFreeMonitoring off

However, when I add that option to the mongod arguments, the process crashes with a segmentation fault!

At least, that's what happens on OSX. On Linux, I get

Error parsing command line: unrecognised option '--enableFreeMonitoring'
try '/home/ben/meteor/dev_bundle/mongodb/bin/mongod --help' for more information

I wrote up these findings in their JIRA issue pertaining to the problem: https://jira.mongodb.org/browse/SERVER-36474

It's hard to know how worried we should be about the general stability of Mongo 4.0.0. Should we be concerned about merely silencing (rather than disabling) this poorly-tested feature?

@benjamn benjamn merged commit 340c7d9 into meteor:release-1.7.1 Aug 9, 2018

18 checks passed

CLA Author has signed the Meteor CLA.
Details
ci/circleci: Clean Up Your tests passed on CircleCI!
Details
ci/circleci: Docs Your tests passed on CircleCI!
Details
ci/circleci: Get Ready Your tests passed on CircleCI!
Details
ci/circleci: Isolated Tests Your tests passed on CircleCI!
Details
ci/circleci: Test Group 0 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 1 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 10 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 2 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 3 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 4 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 5 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 6 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 7 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 8 Your tests passed on CircleCI!
Details
ci/circleci: Test Group 9 Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@mitar

This comment has been minimized.

Collaborator

mitar commented Sep 3, 2018

@klaussner Have you checked how the new transactions feature interact with Meteor's pub/sub and oplog tailing? We should make sure that if you do a transaction and modify data, that this is not leaked through oplog to pub/sub. Probably pub/sub should push updates to the client only after the transaction commits.

@klaussner

This comment has been minimized.

Collaborator

klaussner commented Sep 4, 2018

@mitar I haven't tried it but I assume that transactions don't work with pub/sub at the moment. Each transaction has its own oplog entry, which includes a list of operations (inserts, updates, etc.) that were committed successfully but Meteor doesn't handle this type of oplog entry yet, I think.

@mitar

This comment has been minimized.

Collaborator

mitar commented Sep 4, 2018

I think oplog entries just have a transaction ID associated with it. What I worry is that if Meteor oplog tailing ignores that entry, it might update based on some operations happening inside the transaction, which should not yet be visible outside. Or are you saying that operations inside transactions are not send over the oplog until committed? I do not know details here though. Maybe you are right. I just see a potential issue here.

@klaussner

This comment has been minimized.

Collaborator

klaussner commented Sep 5, 2018

Exactly, only operations of successful transactions will appear on the oplog. I opened an issue with more details: #10195.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment