Sentry database (postgresql) nodestore_node table size is growing huge.

[amitesh2181763]

Body

Hi Team,

From our initial Sentry setup, this "nodestore_node" table size is growing, and as per the sentry support side, we provided sentry database table (nodestore_node) table cleanup, which we are performing till now.

Cleanup procedure: 1) Deleting greater than 30 days of data from table "nodestore_node".
2) Use the pg_repack command to vacuum space.

Initial, we have seen our sentry database grow every day (5–6 GB), but now, in the last few months, we have seen the database size grow every day (10–12 GB), and due to this, we need to frequently do the cleanup to stop our application.

Can you please suggest if there is any other option apart from database cleanup to optimize our database and any way to set a limit on the amount of event data that is stored in our "nodestore_node"?

Also, what type of data is stored in this table, "nodestore_node," and is there any way to separate this data from the table., "nodestore_node"?

[amitesh2181763]

@hubertdeng123 @azaslavsky @BYK need your valuable suggestion in this case.

[BYK]

As far as I'm aware this table holds the raw event data. I've seen an S3 backend for this on GitHub somewhere but don't know how well it works.

Sentry.io itself uses Google BigTable for this (at least it used to) which makes it much less of a problem (except maybe a money pit).

Why is vacuuming not working for you?

[praseeb]

@BYK Thanks for the information, raw event data means it's kind of audit logs for each and every activity which is performed in the sentry application through API, User activity, every log analyses etc.. is it? Just trying to elaborate what does "raw event data" means.
Coming to the Vacuuming, yes that's working for us, but just checking if we can avoid performing this on weekly basis, since it's required a stop & start of the Application.

@amitesh2181763

[praseeb]

@BYK Also to reduce the incoming flow of data to this table, can we do the following?

turn off organizations:performance-view basically remove this entry from sentry.conf.py file? Will this help us in stopping the all-event data to store in that table?

[hubertdeng123]

How much your database grows per day really depends on how many events your Sentry instance is ingesting, which seems like a lot. What does your event volume look like? If you'd like to reduce incoming flow of data to the table, it may be useful to reduce the sample rate if you're sampling transactions, or rate limit the number of events that come in.