MongoDB is dead. Long live Postgresql :)

[realmyst]

Hi!
Our company use Errbit very active, Errbit connect to more than one hundred projects that generate tens of thousands of errors per day.
And it would be great, except for one "but" - mongodb. 98 % of the problems that arise with errbit are problems with the database. Its volume on disk is growing 3-4 times faster than the real volume of data it store; it eats up all the memory without the possibility to limit this; it begins to slow down the application because of frequent disk access. Errbit works as long as it fits in memory. We don’t want to set a server with more than 100Gb of memory to Errbit.

As a result, the decision was made - to migrate Errbit on posgtresql. We did it in the beginning of this year. We wrote data’s migrator, added ActiveRecord, removed mongoid. In general, it works and moved into production successfully. Finally we sleep quietly, and don’t fear that mongodb will drive out redis to swap once again.

We suggest to put errbit on PG. For those who want to try - the code here: https://github.com/Undev/errbit/tree/pg-upstream
Work on the postgresql errbit will be continue.

[dmitry-a-l]

👍

[kirs]

Congrats. I was waiting for this feature since 2012. Thanks Undev team for the implementation!

[PlugIN73]

Good job! 👍

[vassilevsky]

We have been using the Undev's fork in production for quite some time and it has proved to be stable and well-working.

[davetoxa]

👍

[Valve]

Excellent!

[itsNikolay]

👍

[pencil]

👍

[calmyournerves]

👍

[luxflux]

👍

[cr0t]

👍

[gumayunov]

👍

Thanks to Undev team for this epic effort! Our project uses errbit for last several months. And we are not forced to maintain this piece of infrastructure - MongoDB. Well done! :)

It would be awesome if project maintainers migrate the project to PG.

[zavyrylin]

We all need it!

[artempartos]

👍

[akirill0v]

👍

[eychu]

👍

[av-ast]

👍

[mongrelion]

👍

[arthurnn]

@realmyst Thanks a lot for your suggestion.
Could you provide some benchmarks showing the actual improvements after migrating the database?
We are using errbit in a really big production environment without any problems so far. I would like to see some data behind those arguments.

Thanks

[evserykh]

👍

[ariejan]

👍

[thachck]

Vote for Postgresql!

[shingara]

It's not an issue. Not even a Pull Request.

So do a Pull Request if you want of create you own fork.

Speak on Mailing list if you want too.

[goodwink]

👍 switching to the postgresql fork

[zapnap]

[raphaelcosta]

👍

[rimenes]

👍

[realmyst]

@arthurnn Sure, I'll do some benchmarks asap.

[elmacnifico]

👍

[atipugin]

👍

[tab]

[kingink]

👍

[gnrfan]

[brndnblck]

Well... what might have turned into a constructive conversation with a real analysis of the problem, appears to have devolved into a gif-party. Boooooo. ಠ_ಠ

[Netherdrake]

@brandonblack There is no problem, it has been solved. This thread is just a celebration for the solution.

[churchofthought]

[brndnblck]

@Netherdrake funny, but are you sure about that? last I checked, still no numbers/benchmarks, no evidence this actually solved a real issue beyond mis-configuration/deployment issues, no code has changed and no pull request has been sent. I don't think I'd chalk it up as solved/settled yet despite the premature celebration.

[Netherdrake]

@brandonblack I've no idea, I'm here just for the drama.

[bf4]

@realmyst For goodness sake, make a PR, or offer to help with one. Otherwise, this is a pointless issue. It's not a bug report. It's not a pull request. It doesn't have any metrics. It's just an announcement of your fork with no offer to do anything about it. If this issue should exist anywhere, it should be on the errbit mailing list as a starting point for a discussion.

@shingara closed this issue 12 days ago as unproductive. All it's produced is a bunch of plus ones and gifs, I suppose to celebrate that you forked the project and didn't contribute back? Even though I have absolutely nothing to do with the work done in your fork, and haven't tried it, I'm tempted to make a PR myself with your code. It would make more sense for you to do that though, no?

Update: I've submitted this issue to the mailing list for discussion.

Update: There is now a PR work in progress: #623

[ty4z2008]

mongoDB and pgsql different with the applicable environment

[ghost]

PostgreSQL was written with c, lead us small memory footprint. C proud of by it's effective and performance. so i thought PostgreSQL will long live for ever.

[ammrwu]

nice

[haio]

[gnemoug]

不明觉厉

[zhangwenmao]

[TomDeMille]

When we interview engineers for our dev team we always ask them a variation of the following question: 'Whats the best database".

If their answer is anything other than "it depends on what problem you are trying to solve" or " tell me what you are using it for" then we know they are extremely inexperienced and we generally don't hire them.

You can't choose a tool without knowing what you are going to use it for.

[n1ghtmare]

Well Tom I don't disagree with you, but IMHO asking 'What's the best database ?' is a stupid question to begin with.

[TomDeMille]

I guess that's the point. If they don't recognize that it's a fallacy to try and answer that (stupid) question, at least without more information, then they are obviously not very bright or not very experienced. So it's actually a pretty smart question to ask in an interview.

[n1ghtmare]

@TomDeMille :) Good point. Agreed !

[orendon]

A document database seems to be a good fit for Errbit use case, I would love to see a blogpost or something from Undev regarding metrics, benchmarks, actual gain and etc after using it in production for a while.

[yinyangxu]

Awesome !

[schneiderderek]

Very nice!

[darkleaf]

I'm full-time errbit developer of the Undev company.
I added some cool features.
See change log, try it, send feedback and PR.

[darkleaf]

Hi!

Our fork is actively developing and during this time we did:

• Upgraded to rails 4.1.1
• Moved commits from errbit/errbit. Last commit was Apr 23, 2014
• Big refactoring for controllers
• Added ransack
• Added authority(ACL)
• Counter cache by counter_culture gem
• Fixed some N+1 problems
• Added sidekiq
• Added ability to customize the hosting repository links (see Errbit::Config.repository_hostings)
• Showed code changes(with git submodules) between deploys in deploys feed and page
• Updateed hoptoad_notifier to airbrake

Full changelog

The Undev has a lot of highload applications which send tons of errors. Sometimes they drop an errbit server. We've wrote erl_proxy which adds an errors in redis and sends them to errbit gradually. It helps distribute the load evenly, and not to miss a notification when we make erbbit's deploy.

I've added approximate plan of work in the readme. If someone wants to help - write to m.kuzmin@darkleaf.ru, because maybe I've already started to do a feature that you are going to do.
Besides of this, I offer think of a new name for this fork.

[dronov]

[artempartos]

👍

[seliverstov-maxim]

👍