Support for PostgreSQL as alternative to MongoDB

[engelgabriel]

Elephants don't fly, most of the time:
http://meteor-postgres.readthedocs.org
https://github.com/meteor/postgres-packages

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[dosire]

More context in https://about.gitlab.com/2015/08/22/gitlab-7-14-released/
To ship in 8.0 we need something to integrate around September 7 I think (@maxlazio knows more) so that will be really tight. Alternatively we can ship in 8.1. Please keep us posted.

[engelgabriel]

http://info.meteor.com/blog/an-early-look-at-sql-in-meteor

[engelgabriel]

Some approach to consider
http://www.postgresql.org/docs/9.4/static/datatype-json.html

[engelgabriel]

An early look at SQL in Meteor
https://forums.meteor.com/t/an-early-look-at-sql-in-meteor/8736

[engelgabriel]

More people wanting this to happen
#639

[jclabaugh]

+1

Mongo can be a pain and can be very expensive to scale in the cloud. At least allowing to swap the data store to AWS DynamoDb (also noSQL as a scalable Db-as-a-service) would alleviate some of this for large deployments or in an enterprise setting. But PostgreSQL 9.3 would work too.

[engelgabriel]

We found a few temporary road blocks that the our team is having to deal with.

1. Meteor is yet to release the official support to SQL, so we are playing only with alpha implementations at this point. That means that some of the core packages, like the accounts, don't work 100% with SQL yet. We are patching what we can to make it work, and hopefully can contribute something back to the to meteor master branch. https://github.com/RocketChat/Rocket.Chat/compare/postgresql
2. The third party packages we use, also do not support SQL yet, for obvious reasons. We are creating an abstraction layer that overwrites some meteor APIs to make they work without having to fork them (at least most of them) using https://github.com/goodybag/mongo-sql
3. We are refactoring the all the source code to isolate all DB calls into a package. The idea is to turn it into a replaceable package with the DB of choice. This should be finished this week.
   https://github.com/RocketChat/Rocket.Chat/compare/models

Still early days to commit to a date.. but I'll update this issue more regularly on our progress.

[dosire]

@engelgabriel thanks for the update!

[liamdawson]

Any updates?

[sammcj]

Any update on this?

[engelgabriel]

We finished to isolate all DB calls into a package, and now it is helping with the multi tenant support. Once that's done, will jump back into this issue with MDG to get the work done.

[sammcj]

Hi @engelgabriel what is the ticket with the multi-tenant support? Is that #404?

[engelgabriel]

no @sammcj it is #658

[sammcj]

Thanks @engelgabriel , great work 👍

[opairdrop]

I agree that meteor has the same decision, regarding which Rocket.Chat is just a prominent example. But this isn't something that can be punted to someone else in terms of responsibility.

If Mongo persists with the above licensing change then every framework that relies on them will suffer for it.

[sandys]

im actually hopeful about this because of a comment made in the rocketchat forums.
https://forums.rocket.chat/t/legal-implications-of-mongodb-licensing-change/2332/4?u=sandys

Support for postgresql however is quite orthogonal to this license change - more of a technical topic - and is likely to arrive with our maturing support for GraphQL / Apollo.

if the rocketchat team can be convinced that Postgres support is not an orthogonal thing and solves multiple problems in one shot... then they would pay more attention.

[piotrkochan]

@mathiasblc this is not a bug

[ChrissW-R1]

@mathiasblc this is not a bug

You're right. It is not a bug. But e.g. in our environment this tool is the only one which uses it's own database. So we couldn't migrated it in our backup routines or load balancers. So for installations with more than 600 users it makes sense to integrate it in the existing infrastructure.

[menturion]

+1 for Postgres.

[mnlbox]

Guys you can use nice realtime graphql on postgres with https://hasura.io and zero setup with just 15 mb docker image. It's an awesome tools and have a good mix with react.
Why you still use Meteor and Mongo?
It's 2019.

[hypery2k]

any progress on this?

[ghost]

distros dropping Mongo from their repos like hot potatoes...

https://wiki.archlinux.org/index.php/MongoDB

MongoDB has been removed from the official repositories due to its re-licensing issues

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8-beta/html/8.0_beta_release_notes/new-features#web_servers_databases_dynamic_languages_2

Note that the NoSQL MongoDB database server is not included in RHEL 8.0 Beta because it uses the Server Side Public License (SSPL).

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916107

MongoDB should not be part of (at least) Buster for the following
reasons:

• MongoDB 3.6 and 4.0 will be supported longer, but upstream's switch
  to SSPLv1 complicates matters. As discussed in #915537, we will not
  be distributing any SSPL-licensed software

---

It would seem prudent that alternatives are provided or else Rocket.Chat may end up in a ditch together with MongoDB

[Rauserch]

any Update yet?

[rodrigok]

I'm closing this since it's not in our roadmap.

Support for multiple databases is not feasible right now, due to performance issues and different features each database has. Migrate existant clients to a new database would be problematic.

This item may rise again in the future, but right now it's not possible.

[pierreozoux]

Thanks for closing this. Even though we are not all necessarily happy, we have to understand that you have to run a product, and this means making choices.

As RocketChat is based on meteor, people don't get the complexity to add this support now.

I think adding mongo support would be pretty equivalent to refactoring everything.

Thanks again and keep up the good work 👍

[apjapj]

Any news on PostgreSQL integration? The ticket was closed almost a year earlier and its very hard to justify using MongoDB with an opensource project nowadays. Thanks in advance

[piotrkochan]

@apjapj I don't believe that rocket chat will support postgresql ever

[seand88]

postgres has had a json/jsonb datatype for a while. it should be fairly easy to just change a document store to a postgres table jsonb store. each table would be a "collection" in mongo with one jsonb field per table.
If there is a decent repository layer this shouldnt be too much work.

Mongo has way less hosting options and is much more expensive to host.
Going this route you could virtually use any database like mysql etc.. as they have json fields as well.

[kohenkatz]

@apjapj @seand88 If you want a similar platform that supports PostgreSQL, look at Mattermost. As noted in #533 (comment), there's no way this is going to happen.

[pierreozoux]

it should be fairly easy to just change a document store to a postgres table jsonb store. each table would be a "collection" in mongo with one jsonb field per table.
If there is a decent repository layer this shouldnt be too much work.

Feel free to try, I'd love to see that you are right :) I think it is not that trivial :)

[loid345]

Are there any plans to support postgresql? Postgresql can successfully replace mongodb and has higher performance
https://www.enterprisedb.com/news/new-benchmarks-show-postgres-dominating-mongodb-varied-workloads

[rodrigok]

@loid345 we don't have any plans to replace MongoDB right now. But we are always open to contributions if figure out an easy way to make this replacement.

[pierreozoux]

There is maybe hope over there:
https://www.mangodb.io/

[yourhomelab]

Is this still on the list and work in progress?