New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoDB is dead. Long live Postgresql :) #614

Closed
realmyst opened this Issue Nov 26, 2013 · 109 comments

Comments

Projects
None yet
@realmyst

Hi!
Our company use Errbit very active, Errbit connect to more than one hundred projects that generate tens of thousands of errors per day.
And it would be great, except for one "but" - mongodb. 98 % of the problems that arise with errbit are problems with the database. Its volume on disk is growing 3-4 times faster than the real volume of data it store; it eats up all the memory without the possibility to limit this; it begins to slow down the application because of frequent disk access. Errbit works as long as it fits in memory. We don’t want to set a server with more than 100Gb of memory to Errbit.

As a result, the decision was made - to migrate Errbit on posgtresql. We did it in the beginning of this year. We wrote data’s migrator, added ActiveRecord, removed mongoid. In general, it works and moved into production successfully. Finally we sleep quietly, and don’t fear that mongodb will drive out redis to swap once again.

We suggest to put errbit on PG. For those who want to try - the code here: https://github.com/Undev/errbit/tree/pg-upstream
Work on the postgresql errbit will be continue.

@lda

This comment has been minimized.

Show comment
Hide comment

lda commented Nov 26, 2013

👍

@kirs

This comment has been minimized.

Show comment
Hide comment
@kirs

kirs Nov 26, 2013

Congrats. I was waiting for this feature since 2012. Thanks Undev team for the implementation!

kirs commented Nov 26, 2013

Congrats. I was waiting for this feature since 2012. Thanks Undev team for the implementation!

@PlugIN73

This comment has been minimized.

Show comment
Hide comment
@PlugIN73

PlugIN73 Nov 26, 2013

Good job! 👍

Good job! 👍

@vassilevsky

This comment has been minimized.

Show comment
Hide comment
@vassilevsky

vassilevsky Nov 26, 2013

We have been using the Undev's fork in production for quite some time and it has proved to be stable and well-working.

We have been using the Undev's fork in production for quite some time and it has proved to be stable and well-working.

@davetoxa

This comment has been minimized.

Show comment
Hide comment

👍

@Valve

This comment has been minimized.

Show comment
Hide comment
@Valve

Valve Nov 26, 2013

Excellent!

Valve commented Nov 26, 2013

Excellent!

@itsNikolay

This comment has been minimized.

Show comment
Hide comment

👍

@pencil

This comment has been minimized.

Show comment
Hide comment

pencil commented Nov 26, 2013

👍

@calmyournerves

This comment has been minimized.

Show comment
Hide comment

👍

@luxflux

This comment has been minimized.

Show comment
Hide comment
@luxflux

luxflux Nov 26, 2013

Contributor

👍

Contributor

luxflux commented Nov 26, 2013

👍

@cr0t

This comment has been minimized.

Show comment
Hide comment
@cr0t

cr0t Nov 26, 2013

Contributor

👍

Contributor

cr0t commented Nov 26, 2013

👍

@gumayunov

This comment has been minimized.

Show comment
Hide comment
@gumayunov

gumayunov Nov 26, 2013

👍

Thanks to Undev team for this epic effort! Our project uses errbit for last several months. And we are not forced to maintain this piece of infrastructure - MongoDB. Well done! :)

It would be awesome if project maintainers migrate the project to PG.

👍

Thanks to Undev team for this epic effort! Our project uses errbit for last several months. And we are not forced to maintain this piece of infrastructure - MongoDB. Well done! :)

It would be awesome if project maintainers migrate the project to PG.

@zavyrylin

This comment has been minimized.

Show comment
Hide comment
@zavyrylin

zavyrylin Nov 26, 2013

We all need it!

We all need it!

@artempartos

This comment has been minimized.

Show comment
Hide comment

👍

@saratovsource

This comment has been minimized.

Show comment
Hide comment

👍

@eychu

This comment has been minimized.

Show comment
Hide comment

eychu commented Nov 26, 2013

👍

@av-ast

This comment has been minimized.

Show comment
Hide comment

av-ast commented Nov 26, 2013

👍

@mongrelion

This comment has been minimized.

Show comment
Hide comment

👍

@arthurnn

This comment has been minimized.

Show comment
Hide comment
@arthurnn

arthurnn Nov 26, 2013

Member

@realmyst Thanks a lot for your suggestion.
Could you provide some benchmarks showing the actual improvements after migrating the database?
We are using errbit in a really big production environment without any problems so far. I would like to see some data behind those arguments.

Thanks

Member

arthurnn commented Nov 26, 2013

@realmyst Thanks a lot for your suggestion.
Could you provide some benchmarks showing the actual improvements after migrating the database?
We are using errbit in a really big production environment without any problems so far. I would like to see some data behind those arguments.

Thanks

@evserykh

This comment has been minimized.

Show comment
Hide comment

👍

@ariejan

This comment has been minimized.

Show comment
Hide comment

ariejan commented Nov 26, 2013

👍 :shipit:

@chautoni

This comment has been minimized.

Show comment
Hide comment
@chautoni

chautoni Nov 26, 2013

Vote for Postgresql!

Vote for Postgresql!

@shingara

This comment has been minimized.

Show comment
Hide comment
@shingara

shingara Nov 26, 2013

Member

It's not an issue. Not even a Pull Request.

So do a Pull Request if you want of create you own fork.

Speak on Mailing list if you want too.

Member

shingara commented Nov 26, 2013

It's not an issue. Not even a Pull Request.

So do a Pull Request if you want of create you own fork.

Speak on Mailing list if you want too.

@shingara shingara closed this Nov 26, 2013

@goodwink

This comment has been minimized.

Show comment
Hide comment
@goodwink

goodwink Nov 26, 2013

👍 switching to the postgresql fork

👍 switching to the postgresql fork

@zapnap

This comment has been minimized.

Show comment
Hide comment

zapnap commented Nov 26, 2013

@raphaelcosta

This comment has been minimized.

Show comment
Hide comment

👍

@rimenes

This comment has been minimized.

Show comment
Hide comment

rimenes commented Nov 26, 2013

👍

@realmyst

This comment has been minimized.

Show comment
Hide comment
@realmyst

realmyst Nov 26, 2013

@arthurnn Sure, I'll do some benchmarks asap.

@arthurnn Sure, I'll do some benchmarks asap.

@elmacnifico

This comment has been minimized.

Show comment
Hide comment

👍

@slava-vishnyakov

This comment has been minimized.

Show comment
Hide comment
@Andrew8xx8

This comment has been minimized.

Show comment
Hide comment

👍

@MaerF0x0

This comment has been minimized.

Show comment
Hide comment
@MaerF0x0

MaerF0x0 Nov 26, 2013

" it eats up all the memory without the possibility to limit this; " -- Sounds like massive indexes. Maybe you're indexing something big, like a string that has a very long average length ?

"ts volume on disk is growing 3-4 times faster than the real volume of data it store; it eats up all the memory without the possibility to limit this;" -- The key names are stored with the data, so if you have a keyname of "MyReallyLongKeyNameToPointToAnInteger" w/ an integer value. the key will store way more than the data. Some people have created mapping algorithms that interface between the codebase's objects and the objects in mongo .. This saves a number of bytes per document, potentially times millions of documents... It can also help if your mongod is not with the application (eg, you use mongolab or mongohq)

Eg:

// In code base
user =  {
name:'Bob Smith',
phone:'999-999-9999'
}

// same record In mongo
{
  a:'Bob Smith',
  b:'999-999-9999'
}

" it eats up all the memory without the possibility to limit this; " -- Sounds like massive indexes. Maybe you're indexing something big, like a string that has a very long average length ?

"ts volume on disk is growing 3-4 times faster than the real volume of data it store; it eats up all the memory without the possibility to limit this;" -- The key names are stored with the data, so if you have a keyname of "MyReallyLongKeyNameToPointToAnInteger" w/ an integer value. the key will store way more than the data. Some people have created mapping algorithms that interface between the codebase's objects and the objects in mongo .. This saves a number of bytes per document, potentially times millions of documents... It can also help if your mongod is not with the application (eg, you use mongolab or mongohq)

Eg:

// In code base
user =  {
name:'Bob Smith',
phone:'999-999-9999'
}

// same record In mongo
{
  a:'Bob Smith',
  b:'999-999-9999'
}
@itsNikolay

This comment has been minimized.

Show comment
Hide comment
@itsNikolay

itsNikolay Nov 26, 2013

dance
here coooomes

dance
here coooomes

@chytreg

This comment has been minimized.

Show comment
Hide comment

chytreg commented Nov 26, 2013

👍

@slackhappy

This comment has been minimized.

Show comment
Hide comment
@slackhappy

slackhappy Nov 26, 2013

did you try usePowerOf2Sizes?
http://docs.mongodb.org/manual/reference/command/collMod/
It would likely prevent your stoageSize from growing > 2 x dataSize, regardless of deletes

did you try usePowerOf2Sizes?
http://docs.mongodb.org/manual/reference/command/collMod/
It would likely prevent your stoageSize from growing > 2 x dataSize, regardless of deletes

@nickmartini

This comment has been minimized.

Show comment
Hide comment
@nickmartini

nickmartini Nov 26, 2013

I love postgres because I'm not a DBA and don't need to deal with it!!!

I love postgres because I'm not a DBA and don't need to deal with it!!!

@realmyst

This comment has been minimized.

Show comment
Hide comment
@realmyst

realmyst Nov 26, 2013

@slackhappy no, we didn't. It would not solve the problem with a lack of memory and swapping. Now, use <2Gb memory we don't have problems (mongodb lacked 16Gb). Data size little more 100 gb. And most importantly - It working fine without our participation.

@slackhappy no, we didn't. It would not solve the problem with a lack of memory and swapping. Now, use <2Gb memory we don't have problems (mongodb lacked 16Gb). Data size little more 100 gb. And most importantly - It working fine without our participation.

@MaerF0x0

This comment has been minimized.

Show comment
Hide comment
@MaerF0x0

MaerF0x0 Nov 26, 2013

@realmyst can you talk about postgres index sizes and if its doing a lot of disk work now that you've brought it down to 2GB ? Im curious if performance has degraded due to the lack of RAM. Also would be interested in seeing a mongo vs postgres on the same hardware comparison.

@realmyst can you talk about postgres index sizes and if its doing a lot of disk work now that you've brought it down to 2GB ? Im curious if performance has degraded due to the lack of RAM. Also would be interested in seeing a mongo vs postgres on the same hardware comparison.

@ranman

This comment has been minimized.

Show comment
Hide comment
@ranman

ranman Nov 26, 2013

It may be prudent to note that you're running multiple in memory databases on a single node: "mongodb will drive out redis to swap once again." -- which is, in general, a BAD idea for any database system.

ranman commented Nov 26, 2013

It may be prudent to note that you're running multiple in memory databases on a single node: "mongodb will drive out redis to swap once again." -- which is, in general, a BAD idea for any database system.

@simonc

This comment has been minimized.

Show comment
Hide comment

simonc commented Nov 27, 2013

👍

@message

This comment has been minimized.

Show comment
Hide comment

message commented Nov 27, 2013

@dpnl87

This comment has been minimized.

Show comment
Hide comment

dpnl87 commented Nov 27, 2013

👍

@drakmail

This comment has been minimized.

Show comment
Hide comment

👍

@dustalov

This comment has been minimized.

Show comment
Hide comment

👍

@josiahcarlson

This comment has been minimized.

Show comment
Hide comment
@josiahcarlson

josiahcarlson Nov 27, 2013

@TomDeMille Actually, MongoDB is horrible for ad-hoc queries compared to any of the legitimate players in the relational database space (MySQL, Postgres, MSSQL, Oracle, ...). This is simply because MongoDB knows how to perform queries of two different types: 1) query on a single index, 2) table scans. That's it, that's all.

On the other hand, the players in the relational space all know how to use multiple indexes in the same query, offer subqueries, usually offer covering indexes (so you don't need to hit your table if your index has all of the data you need), and are actually able to support legitimately simultaneous reads and writes. These all are what make ad-hoc queries viable when you are looking at data in any database.

@TomDeMille Actually, MongoDB is horrible for ad-hoc queries compared to any of the legitimate players in the relational database space (MySQL, Postgres, MSSQL, Oracle, ...). This is simply because MongoDB knows how to perform queries of two different types: 1) query on a single index, 2) table scans. That's it, that's all.

On the other hand, the players in the relational space all know how to use multiple indexes in the same query, offer subqueries, usually offer covering indexes (so you don't need to hit your table if your index has all of the data you need), and are actually able to support legitimately simultaneous reads and writes. These all are what make ad-hoc queries viable when you are looking at data in any database.

@TomDeMille

This comment has been minimized.

Show comment
Hide comment
@TomDeMille

TomDeMille Nov 27, 2013

True about multiple indexes, but not true that SQLServer can efficiently
process and save query plans for ad-hoc queries built using string
concatenation and executed using exec sql. On a table with 250 ad-hoc
columns and millions of rows, you can't create in practice create indexes
that would be efficient for all possible combinations of queries so the
ability is a moot point. The way the data is stored in MongoDB makes a
huge a difference of course, use of the index on an array is a trick that
works well... be happy to get into it further if anyone needs help but was
about 3x faster than a well indexes very fast SQLServer installation on
some seriously fast SAN storage. Using commodity hardware for MongoDB.

True about multiple indexes, but not true that SQLServer can efficiently
process and save query plans for ad-hoc queries built using string
concatenation and executed using exec sql. On a table with 250 ad-hoc
columns and millions of rows, you can't create in practice create indexes
that would be efficient for all possible combinations of queries so the
ability is a moot point. The way the data is stored in MongoDB makes a
huge a difference of course, use of the index on an array is a trick that
works well... be happy to get into it further if anyone needs help but was
about 3x faster than a well indexes very fast SQLServer installation on
some seriously fast SAN storage. Using commodity hardware for MongoDB.

@vohof

This comment has been minimized.

Show comment
Hide comment

vohof commented Nov 28, 2013

@becomingGuru

This comment has been minimized.

Show comment
Hide comment

👍

@jrdi

This comment has been minimized.

Show comment
Hide comment

jrdi commented Nov 28, 2013

@atipugin

This comment has been minimized.

Show comment
Hide comment

👍

@tab

This comment has been minimized.

Show comment
Hide comment

tab commented Nov 28, 2013

bender_rodrigez

@kingink

This comment has been minimized.

Show comment
Hide comment

kingink commented Nov 28, 2013

👍

@gnrfan

This comment has been minimized.

Show comment
Hide comment

gnrfan commented Dec 1, 2013

0225

@brblck

This comment has been minimized.

Show comment
Hide comment
@brblck

brblck Dec 2, 2013

Well... what might have turned into a constructive conversation with a real analysis of the problem, appears to have devolved into a gif-party. Boooooo. ಠ_ಠ

brblck commented Dec 2, 2013

Well... what might have turned into a constructive conversation with a real analysis of the problem, appears to have devolved into a gif-party. Boooooo. ಠ_ಠ

@Netherdrake

This comment has been minimized.

Show comment
Hide comment
@Netherdrake

Netherdrake Dec 2, 2013

@brandonblack There is no problem, it has been solved. This thread is just a celebration for the solution.

@brandonblack There is no problem, it has been solved. This thread is just a celebration for the solution.

@xibxor

This comment has been minimized.

Show comment
Hide comment

xibxor commented Dec 2, 2013

llxeq

@brblck

This comment has been minimized.

Show comment
Hide comment
@brblck

brblck Dec 2, 2013

@Netherdrake funny, but are you sure about that? last I checked, still no numbers/benchmarks, no evidence this actually solved a real issue beyond mis-configuration/deployment issues, no code has changed and no pull request has been sent. I don't think I'd chalk it up as solved/settled yet despite the premature celebration.

brblck commented Dec 2, 2013

@Netherdrake funny, but are you sure about that? last I checked, still no numbers/benchmarks, no evidence this actually solved a real issue beyond mis-configuration/deployment issues, no code has changed and no pull request has been sent. I don't think I'd chalk it up as solved/settled yet despite the premature celebration.

@Netherdrake

This comment has been minimized.

Show comment
Hide comment
@Netherdrake

Netherdrake Dec 2, 2013

@brandonblack I've no idea, I'm here just for the drama.
drama

@brandonblack I've no idea, I'm here just for the drama.
drama

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Dec 8, 2013

@realmyst For goodness sake, make a PR, or offer to help with one. Otherwise, this is a pointless issue. It's not a bug report. It's not a pull request. It doesn't have any metrics. It's just an announcement of your fork with no offer to do anything about it. If this issue should exist anywhere, it should be on the errbit mailing list as a starting point for a discussion.

@shingara closed this issue 12 days ago as unproductive. All it's produced is a bunch of plus ones and gifs, I suppose to celebrate that you forked the project and didn't contribute back? Even though I have absolutely nothing to do with the work done in your fork, and haven't tried it, I'm tempted to make a PR myself with your code. It would make more sense for you to do that though, no?

Update: I've submitted this issue to the mailing list for discussion.

Update: There is now a PR work in progress: #623

bf4 commented Dec 8, 2013

@realmyst For goodness sake, make a PR, or offer to help with one. Otherwise, this is a pointless issue. It's not a bug report. It's not a pull request. It doesn't have any metrics. It's just an announcement of your fork with no offer to do anything about it. If this issue should exist anywhere, it should be on the errbit mailing list as a starting point for a discussion.

@shingara closed this issue 12 days ago as unproductive. All it's produced is a bunch of plus ones and gifs, I suppose to celebrate that you forked the project and didn't contribute back? Even though I have absolutely nothing to do with the work done in your fork, and haven't tried it, I'm tempted to make a PR myself with your code. It would make more sense for you to do that though, no?

Update: I've submitted this issue to the mailing list for discussion.

Update: There is now a PR work in progress: #623

@ty4z2008

This comment has been minimized.

Show comment
Hide comment
@ty4z2008

ty4z2008 Dec 9, 2013

mongoDB and pgsql different with the applicable environment

ty4z2008 commented Dec 9, 2013

mongoDB and pgsql different with the applicable environment

@netroby

This comment has been minimized.

Show comment
Hide comment
@netroby

netroby Dec 9, 2013

PostgreSQL was written with c, lead us small memory footprint. C proud of by it's effective and performance. so i thought PostgreSQL will long live for ever.

netroby commented Dec 9, 2013

PostgreSQL was written with c, lead us small memory footprint. C proud of by it's effective and performance. so i thought PostgreSQL will long live for ever.

@ammrwu

This comment has been minimized.

Show comment
Hide comment

ammrwu commented Dec 9, 2013

nice

@haio

This comment has been minimized.

Show comment
Hide comment

haio commented Dec 9, 2013

zan

@gnemoug

This comment has been minimized.

Show comment
Hide comment
@gnemoug

gnemoug Dec 9, 2013

不明觉厉

gnemoug commented Dec 9, 2013

不明觉厉

@zhangwenmao

This comment has been minimized.

Show comment
Hide comment
@TomDeMille

This comment has been minimized.

Show comment
Hide comment
@TomDeMille

TomDeMille Dec 9, 2013

When we interview engineers for our dev team we always ask them a variation of the following question: 'Whats the best database".

If their answer is anything other than "it depends on what problem you are trying to solve" or " tell me what you are using it for" then we know they are extremely inexperienced and we generally don't hire them.

You can't choose a tool without knowing what you are going to use it for.

When we interview engineers for our dev team we always ask them a variation of the following question: 'Whats the best database".

If their answer is anything other than "it depends on what problem you are trying to solve" or " tell me what you are using it for" then we know they are extremely inexperienced and we generally don't hire them.

You can't choose a tool without knowing what you are going to use it for.

@n1ghtmare

This comment has been minimized.

Show comment
Hide comment
@n1ghtmare

n1ghtmare Dec 9, 2013

Well Tom I don't disagree with you, but IMHO asking 'What's the best database ?' is a stupid question to begin with.

Well Tom I don't disagree with you, but IMHO asking 'What's the best database ?' is a stupid question to begin with.

@TomDeMille

This comment has been minimized.

Show comment
Hide comment
@TomDeMille

TomDeMille Dec 9, 2013

I guess that's the point. If they don't recognize that it's a fallacy to try and answer that (stupid) question, at least without more information, then they are obviously not very bright or not very experienced. So it's actually a pretty smart question to ask in an interview.

I guess that's the point. If they don't recognize that it's a fallacy to try and answer that (stupid) question, at least without more information, then they are obviously not very bright or not very experienced. So it's actually a pretty smart question to ask in an interview.

@n1ghtmare

This comment has been minimized.

Show comment
Hide comment
@n1ghtmare

n1ghtmare Dec 9, 2013

@TomDeMille :) Good point. Agreed !

@TomDeMille :) Good point. Agreed !

@orendon

This comment has been minimized.

Show comment
Hide comment
@orendon

orendon Jan 9, 2014

A document database seems to be a good fit for Errbit use case, I would love to see a blogpost or something from Undev regarding metrics, benchmarks, actual gain and etc after using it in production for a while.

orendon commented Jan 9, 2014

A document database seems to be a good fit for Errbit use case, I would love to see a blogpost or something from Undev regarding metrics, benchmarks, actual gain and etc after using it in production for a while.

@yinyangxu

This comment has been minimized.

Show comment
Hide comment

Awesome !

@schneiderderek

This comment has been minimized.

Show comment
Hide comment

Very nice!

@darkleaf

This comment has been minimized.

Show comment
Hide comment
@darkleaf

darkleaf Feb 28, 2014

I'm full-time errbit developer of the Undev company.
I added some cool features.
See change log, try it, send feedback and PR.

I'm full-time errbit developer of the Undev company.
I added some cool features.
See change log, try it, send feedback and PR.

@darkleaf

This comment has been minimized.

Show comment
Hide comment
@darkleaf

darkleaf Jun 3, 2014

Hi!

Our fork is actively developing and during this time we did:

  • Upgraded to rails 4.1.1
  • Moved commits from errbit/errbit. Last commit was Apr 23, 2014
  • Big refactoring for controllers
  • Added ransack
  • Added authority(ACL)
  • Counter cache by counter_culture gem
  • Fixed some N+1 problems
  • Added sidekiq
  • Added ability to customize the hosting repository links (see Errbit::Config.repository_hostings)
  • Showed code changes(with git submodules) between deploys in deploys feed and page
  • Updateed hoptoad_notifier to airbrake

Full changelog

The Undev has a lot of highload applications which send tons of errors. Sometimes they drop an errbit server. We've wrote erl_proxy which adds an errors in redis and sends them to errbit gradually. It helps distribute the load evenly, and not to miss a notification when we make erbbit's deploy.

I've added approximate plan of work in the readme. If someone wants to help - write to m.kuzmin@darkleaf.ru, because maybe I've already started to do a feature that you are going to do.
Besides of this, I offer think of a new name for this fork.

darkleaf commented Jun 3, 2014

Hi!

Our fork is actively developing and during this time we did:

  • Upgraded to rails 4.1.1
  • Moved commits from errbit/errbit. Last commit was Apr 23, 2014
  • Big refactoring for controllers
  • Added ransack
  • Added authority(ACL)
  • Counter cache by counter_culture gem
  • Fixed some N+1 problems
  • Added sidekiq
  • Added ability to customize the hosting repository links (see Errbit::Config.repository_hostings)
  • Showed code changes(with git submodules) between deploys in deploys feed and page
  • Updateed hoptoad_notifier to airbrake

Full changelog

The Undev has a lot of highload applications which send tons of errors. Sometimes they drop an errbit server. We've wrote erl_proxy which adds an errors in redis and sends them to errbit gradually. It helps distribute the load evenly, and not to miss a notification when we make erbbit's deploy.

I've added approximate plan of work in the readme. If someone wants to help - write to m.kuzmin@darkleaf.ru, because maybe I've already started to do a feature that you are going to do.
Besides of this, I offer think of a new name for this fork.

@dronov

This comment has been minimized.

Show comment
Hide comment

dronov commented Jun 3, 2014

@artempartos

This comment has been minimized.

Show comment
Hide comment

👍

@seliverstov-maxim

This comment has been minimized.

Show comment
Hide comment

@errbit errbit locked and limited conversation to collaborators Jun 10, 2014

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.