Skip to content

Conversation

dcramer
Copy link
Member

@dcramer dcramer commented Oct 18, 2013

This allows us to split off the (very heavy) storage costs of Event.data and rely on SQL for what it's good at: indexing schemas.

  • A new NodeField is included which handles the transitionary phase automatically.
  • Event.data is now using NodeField
  • SENTRY_NODESTORE options are added, which default to the Django backend (a new simple key/value table)
  • A Riak backend is available, and will be the recommended storage backend for large scale deployments.

This isn't ready to merge. Need to add some tests for the Riak backend, higher level integration, as well as do some manual QA

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.63%) when pulling 5360c7c on node-store into b83610a on master.

@dcramer
Copy link
Member Author

dcramer commented Oct 18, 2013

Need some Riak experts in here to make sure I'm not abusing the store for the "overwrite this value always" operation

/cc @dialtone

@dialtone
Copy link

I'm not an expert in Riak but... This probably depends on the storage backend: bitcask, eleveldb or even straight up memory.

The other issue is that you need to make sure that keys are balanced, and no one key is more accessed than others. The issue with that is that the scaling happens across the hash range but if you only use a few keys for everything you can't really aggregate write/read performance and the benefits disappear.

A partial way to solve the key hotness is to buffer writes for a short time before flushing them to the store.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 0517ecc on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.02%) when pulling 0517ecc on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.38%) when pulling 0517ecc on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.18%) when pulling 1b25a92 on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.56%) when pulling b0af57b on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.5%) when pulling ac59037 on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.52%) when pulling ac59037 on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) when pulling f85e9eb on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) when pulling 5d15a3e on node-store into b83610a on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.47%) when pulling 7f81b00 on node-store into b83610a on master.

@dcramer dcramer merged commit ce2c909 into master Oct 26, 2013
@github-actions github-actions bot locked and limited conversation to collaborators Dec 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants