Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache vizjson in redis to avoid hitting DB #2324 #2194

Merged
merged 20 commits into from
Feb 23, 2015
Merged

Conversation

rafatower
Copy link
Contributor

Ready to be merged

@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@javisantana javisantana changed the title Vizjson redis Vizjson redis [no merge] Feb 17, 2015
@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@rafatower rafatower changed the title Vizjson redis [no merge] Cache vizjson in redis to avoid hitting DB #2324 Feb 19, 2015
@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@rafatower
Copy link
Contributor Author

@javisantana review please.

A few notes:

  • Some of the tests use the actual redis, some of them mock it. That's intentional
  • It invalidates the redis_cache in every place it was done for varnish. A new method invalidate_cache replaces the old one invalidate_varnish_cache.
  • IMHO the invalidate_cache shouldn't be exposed outside of the Member object, but that would require a bigger refactor.

end

describe '#redis_cached' do
it "Uses the block given to calculate a hash if there's a cache miss" do
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kartones as a curiosity, this is how I solved the called block problem

@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@rafatower
Copy link
Contributor Author

@rochoa can you please review?

@rafatower
Copy link
Contributor Author

retest this, please

@@ -20,7 +20,8 @@
tables_metadata: 0,
api_credentials: 3,
users_metadata: 5,
redis_migrator_logs: 6
redis_migrator_logs: 6,
visualizations: 7,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we using another database for this? 😈

I understand how this can be useful, but unfortunately I consider
Redis multiple database errors my worst decision in Redis design at
all... without any kind of real gain, it makes the internals a lot
more complex. The reality is that databases don't scale well for a
number of reason, like active expire of keys and VM. If the DB
selection can be performed with a string I can see this feature being
used as a scalable O(1) dictionary layer, that instead it is not.

https://groups.google.com/forum/#!msg/redis-db/vS5wX8X4Cjg/8ounBXitG4sJ

@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

@rochoa
Copy link
Contributor

rochoa commented Feb 23, 2015

OK for me 👍 although I would try to avoid using an extra database for this.

@rafatower
Copy link
Contributor Author

Ok, I'll move it to $tables_metadata and tweak the key prefix though the naming is not the best :)

@rochoa
Copy link
Contributor

rochoa commented Feb 23, 2015

I agree with you on the naming issue but that's easier to fix than having to migrate another database in the future (yes, I know probably we won't migrate to just one database anytime soon but better to avoid making this the standard).

As discussed in peer review, using a new redis db should not be the
standard for new code. See
https://groups.google.com/forum/#!msg/redis-db/vS5wX8X4Cjg/8ounBXitG4sJ
@Cartofante
Copy link
Collaborator

Frontend tests were OK 👍 (details)

rafatower pushed a commit that referenced this pull request Feb 23, 2015
Cache vizjson in redis to avoid hitting DB #2324
@rafatower rafatower merged commit 324e6ea into master Feb 23, 2015
@rafatower rafatower deleted the vizjson-redis branch February 23, 2015 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants