Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoDB is dead. Long live Postgresql :) #614

Closed
realmyst opened this issue Nov 26, 2013 · 109 comments
Closed

MongoDB is dead. Long live Postgresql :) #614

realmyst opened this issue Nov 26, 2013 · 109 comments

Comments

@realmyst
Copy link

@realmyst realmyst commented Nov 26, 2013

Hi!
Our company use Errbit very active, Errbit connect to more than one hundred projects that generate tens of thousands of errors per day.
And it would be great, except for one "but" - mongodb. 98 % of the problems that arise with errbit are problems with the database. Its volume on disk is growing 3-4 times faster than the real volume of data it store; it eats up all the memory without the possibility to limit this; it begins to slow down the application because of frequent disk access. Errbit works as long as it fits in memory. We don’t want to set a server with more than 100Gb of memory to Errbit.

As a result, the decision was made - to migrate Errbit on posgtresql. We did it in the beginning of this year. We wrote data’s migrator, added ActiveRecord, removed mongoid. In general, it works and moved into production successfully. Finally we sleep quietly, and don’t fear that mongodb will drive out redis to swap once again.

We suggest to put errbit on PG. For those who want to try - the code here: https://github.com/Undev/errbit/tree/pg-upstream
Work on the postgresql errbit will be continue.

@lda
Copy link

@lda lda commented Nov 26, 2013

👍

@kirs
Copy link

@kirs kirs commented Nov 26, 2013

Congrats. I was waiting for this feature since 2012. Thanks Undev team for the implementation!

@PlugIN73
Copy link

@PlugIN73 PlugIN73 commented Nov 26, 2013

Good job! 👍

@vassilevsky
Copy link

@vassilevsky vassilevsky commented Nov 26, 2013

We have been using the Undev's fork in production for quite some time and it has proved to be stable and well-working.

@davetoxa
Copy link

@davetoxa davetoxa commented Nov 26, 2013

👍

@Valve
Copy link

@Valve Valve commented Nov 26, 2013

Excellent!

@itsNikolay
Copy link

@itsNikolay itsNikolay commented Nov 26, 2013

👍

4 similar comments
@pencil
Copy link

@pencil pencil commented Nov 26, 2013

👍

@calmyournerves
Copy link

@calmyournerves calmyournerves commented Nov 26, 2013

👍

@luxflux
Copy link
Contributor

@luxflux luxflux commented Nov 26, 2013

👍

@cr0t
Copy link
Contributor

@cr0t cr0t commented Nov 26, 2013

👍

@gumayunov
Copy link

@gumayunov gumayunov commented Nov 26, 2013

👍

Thanks to Undev team for this epic effort! Our project uses errbit for last several months. And we are not forced to maintain this piece of infrastructure - MongoDB. Well done! :)

It would be awesome if project maintainers migrate the project to PG.

@zavyrylin
Copy link

@zavyrylin zavyrylin commented Nov 26, 2013

We all need it!

@artempartos
Copy link

@artempartos artempartos commented Nov 26, 2013

👍

4 similar comments
@akirill0v
Copy link

@akirill0v akirill0v commented Nov 26, 2013

👍

@eychu
Copy link

@eychu eychu commented Nov 26, 2013

👍

@av-ast
Copy link

@av-ast av-ast commented Nov 26, 2013

👍

@mongrelion
Copy link

@mongrelion mongrelion commented Nov 26, 2013

👍

@arthurnn
Copy link
Member

@arthurnn arthurnn commented Nov 26, 2013

@realmyst Thanks a lot for your suggestion.
Could you provide some benchmarks showing the actual improvements after migrating the database?
We are using errbit in a really big production environment without any problems so far. I would like to see some data behind those arguments.

Thanks

@evserykh
Copy link

@evserykh evserykh commented Nov 26, 2013

👍

@ariejan
Copy link

@ariejan ariejan commented Nov 26, 2013

👍 :shipit:

@chautoni
Copy link

@chautoni chautoni commented Nov 26, 2013

Vote for Postgresql!

@shingara
Copy link
Member

@shingara shingara commented Nov 26, 2013

It's not an issue. Not even a Pull Request.

So do a Pull Request if you want of create you own fork.

Speak on Mailing list if you want too.

@shingara shingara closed this Nov 26, 2013
@goodwink
Copy link

@goodwink goodwink commented Nov 26, 2013

👍 switching to the postgresql fork

@zapnap
Copy link

@zapnap zapnap commented Nov 26, 2013

@raphaelcosta
Copy link

@raphaelcosta raphaelcosta commented Nov 26, 2013

👍

1 similar comment
@rimenes
Copy link

@rimenes rimenes commented Nov 26, 2013

👍

@realmyst
Copy link
Author

@realmyst realmyst commented Nov 26, 2013

@arthurnn Sure, I'll do some benchmarks asap.

@elmacnifico
Copy link

@elmacnifico elmacnifico commented Nov 26, 2013

👍

@atipugin
Copy link

@atipugin atipugin commented Nov 28, 2013

👍

@tab
Copy link

@tab tab commented Nov 28, 2013

bender_rodrigez

@kingink
Copy link

@kingink kingink commented Nov 28, 2013

👍

@gnrfan
Copy link

@gnrfan gnrfan commented Dec 1, 2013

0225

@brblck
Copy link

@brblck brblck commented Dec 2, 2013

Well... what might have turned into a constructive conversation with a real analysis of the problem, appears to have devolved into a gif-party. Boooooo. ಠ_ಠ

@Netherdrake
Copy link

@Netherdrake Netherdrake commented Dec 2, 2013

@brandonblack There is no problem, it has been solved. This thread is just a celebration for the solution.

@churchofthought
Copy link

@churchofthought churchofthought commented Dec 2, 2013

llxeq

@brblck
Copy link

@brblck brblck commented Dec 2, 2013

@Netherdrake funny, but are you sure about that? last I checked, still no numbers/benchmarks, no evidence this actually solved a real issue beyond mis-configuration/deployment issues, no code has changed and no pull request has been sent. I don't think I'd chalk it up as solved/settled yet despite the premature celebration.

@Netherdrake
Copy link

@Netherdrake Netherdrake commented Dec 2, 2013

@brandonblack I've no idea, I'm here just for the drama.
drama

@bf4
Copy link

@bf4 bf4 commented Dec 8, 2013

@realmyst For goodness sake, make a PR, or offer to help with one. Otherwise, this is a pointless issue. It's not a bug report. It's not a pull request. It doesn't have any metrics. It's just an announcement of your fork with no offer to do anything about it. If this issue should exist anywhere, it should be on the errbit mailing list as a starting point for a discussion.

@shingara closed this issue 12 days ago as unproductive. All it's produced is a bunch of plus ones and gifs, I suppose to celebrate that you forked the project and didn't contribute back? Even though I have absolutely nothing to do with the work done in your fork, and haven't tried it, I'm tempted to make a PR myself with your code. It would make more sense for you to do that though, no?

Update: I've submitted this issue to the mailing list for discussion.

Update: There is now a PR work in progress: #623

@ty4z2008
Copy link

@ty4z2008 ty4z2008 commented Dec 9, 2013

mongoDB and pgsql different with the applicable environment

@29e7e280-0d1c-4bba-98fe-f7cd3ca7500a

PostgreSQL was written with c, lead us small memory footprint. C proud of by it's effective and performance. so i thought PostgreSQL will long live for ever.

@ammrwu
Copy link

@ammrwu ammrwu commented Dec 9, 2013

nice

@haio
Copy link

@haio haio commented Dec 9, 2013

zan

@gnemoug
Copy link

@gnemoug gnemoug commented Dec 9, 2013

不明觉厉

@zhangwenmao
Copy link

@zhangwenmao zhangwenmao commented Dec 9, 2013

85hfyw_5ri89 r0 0 jzs u

@TomDeMille
Copy link

@TomDeMille TomDeMille commented Dec 9, 2013

When we interview engineers for our dev team we always ask them a variation of the following question: 'Whats the best database".

If their answer is anything other than "it depends on what problem you are trying to solve" or " tell me what you are using it for" then we know they are extremely inexperienced and we generally don't hire them.

You can't choose a tool without knowing what you are going to use it for.

@n1ghtmare
Copy link

@n1ghtmare n1ghtmare commented Dec 9, 2013

Well Tom I don't disagree with you, but IMHO asking 'What's the best database ?' is a stupid question to begin with.

@TomDeMille
Copy link

@TomDeMille TomDeMille commented Dec 9, 2013

I guess that's the point. If they don't recognize that it's a fallacy to try and answer that (stupid) question, at least without more information, then they are obviously not very bright or not very experienced. So it's actually a pretty smart question to ask in an interview.

@n1ghtmare
Copy link

@n1ghtmare n1ghtmare commented Dec 9, 2013

@TomDeMille :) Good point. Agreed !

@orendon
Copy link

@orendon orendon commented Jan 9, 2014

A document database seems to be a good fit for Errbit use case, I would love to see a blogpost or something from Undev regarding metrics, benchmarks, actual gain and etc after using it in production for a while.

@yinyangxu
Copy link

@yinyangxu yinyangxu commented Jan 22, 2014

Awesome !

@schneiderderek
Copy link

@schneiderderek schneiderderek commented Feb 19, 2014

Very nice!

@darkleaf
Copy link

@darkleaf darkleaf commented Feb 28, 2014

I'm full-time errbit developer of the Undev company.
I added some cool features.
See change log, try it, send feedback and PR.

@darkleaf
Copy link

@darkleaf darkleaf commented Jun 3, 2014

Hi!

Our fork is actively developing and during this time we did:

  • Upgraded to rails 4.1.1
  • Moved commits from errbit/errbit. Last commit was Apr 23, 2014
  • Big refactoring for controllers
  • Added ransack
  • Added authority(ACL)
  • Counter cache by counter_culture gem
  • Fixed some N+1 problems
  • Added sidekiq
  • Added ability to customize the hosting repository links (see Errbit::Config.repository_hostings)
  • Showed code changes(with git submodules) between deploys in deploys feed and page
  • Updateed hoptoad_notifier to airbrake

Full changelog

The Undev has a lot of highload applications which send tons of errors. Sometimes they drop an errbit server. We've wrote erl_proxy which adds an errors in redis and sends them to errbit gradually. It helps distribute the load evenly, and not to miss a notification when we make erbbit's deploy.

I've added approximate plan of work in the readme. If someone wants to help - write to m.kuzmin@darkleaf.ru, because maybe I've already started to do a feature that you are going to do.
Besides of this, I offer think of a new name for this fork.

@dronov
Copy link

@dronov dronov commented Jun 3, 2014

@artempartos
Copy link

@artempartos artempartos commented Jun 3, 2014

👍

1 similar comment
@seliverstov-maxim
Copy link

@seliverstov-maxim seliverstov-maxim commented Jun 4, 2014

👍

@errbit errbit locked and limited conversation to collaborators Jun 10, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.