Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
0 additions
and
158 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,158 +0,0 @@ | ||
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> | ||
| <html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> | ||
| <head> | ||
| <meta content="text/html; charset=utf-8" http-equiv="Content-type" /> | ||
| <link href="../../../favico.ico" rel="shortcut icon" /> | ||
| <script src="../../../javascripts/shjs-0.6/sh_main.min.js" type="text/javascript"></script> | ||
| <script src="http://platform.twitter.com/widgets.js" type="text/javascript"></script> | ||
| <link href="../../../javascripts/shjs-0.6/css/sh_nedit.min.css" rel="stylesheet" type="text/css" /> | ||
| <link href="../../../stylesheets/basic.css" rel="stylesheet" type="text/css" /> | ||
| <link href="../../../stylesheets/specific.css" rel="stylesheet" type="text/css" /> | ||
| <title>Searching with Picky: Redis</title> | ||
| <script type="text/javascript"> | ||
| //<![CDATA[ | ||
| var _gaq = _gaq || []; | ||
| _gaq.push(['_setAccount', 'UA-20991642-1']); | ||
| _gaq.push(['_trackPageview']); | ||
|
|
||
| (function() { | ||
| var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; | ||
| ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; | ||
| var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); | ||
| })(); | ||
| //]]> | ||
| </script> | ||
| </head> | ||
| <body onload='sh_highlightDocument("../../../javascripts/shjs-0.6/lang/", ".min.js");'> | ||
| <ol class="nav"> | ||
| <li> | ||
| <a href="./../../../../">home</a> | ||
| • | ||
| </li> | ||
| <li> | ||
| <a href="./../../../">blog</a> | ||
| • | ||
| </li> | ||
| <li> | ||
| <a href="./../../../../picky/">picky</a> | ||
| • | ||
| </li> | ||
| <li> | ||
| <a href="./../../../../phd/">phd</a> | ||
| • | ||
| </li> | ||
| <li> | ||
| <a href="./../../../../phony/">phony</a> | ||
| • | ||
| </li> | ||
| <li> | ||
| <a href="./../../../../view_models/">view models</a> | ||
| </li> | ||
| </ol> | ||
| <div class="post"> | ||
| <h1> | ||
| Searching with Picky: Redis | ||
| <a class="twitter-share-button" data-count="none" data-text="Searching with Picky&#58; Redis" data-url="http://florianhanke.com/blog/2011/03/02/searching-with-picky-redis.html" data-via="hanke" data-width="55px" href="http://twitter.com/share">Tweet</a> | ||
| </h1> | ||
| <div class="categories"> | ||
| ruby / picky / gems / redis | ||
| </div> | ||
| <!-- / - page.categories.each do |category| --> | ||
| <!-- / %a{ :href => "/blog/category/#{category}" }= category --> | ||
| <p>This is a post in the <a href="http://florianhanke.com/picky/index.html">Picky</a> series on its workings. If you haven’t tried it yet, do so in the <a href="http://florianhanke.com/picky/getting_started.html">Getting Started</a> section. It’s quick and painless :)</p> | ||
| <p>This post will be a very short introduction on Redis index backends and Picky, and how to configure your indexes to run on Redis.</p> | ||
| <p>I intended to do a massive writeup, but since all you do is change 6 characters <code>Memory</code> into 5 different characters <code>Redis</code> it just seemed like a massive overkill.</p> | ||
| <p>I admit though that many massive writeups have been done on even smaller changes, like “1.8” → “1.9” ;)</p> | ||
| <p>Ok, so what am I talking about?</p> | ||
| <h2>tl;dr</h2> | ||
| <ol> | ||
| <li><a href="http://redis.io/">Redis</a> can now be used in Picky as an index backend.</li> | ||
| <li>In your config, do <code>Index::Memory.new</code> → <code>Index::Redis.new</code> and you’re set :)</li> | ||
| <li>Memory and Redis indexes cannot (yet) be mixed and matched.</li> | ||
| <li>In 1.5.0, Picky uses Redis database 15.</li> | ||
| </ol> | ||
| <h2>What is Redis?</h2> | ||
| <p>Redis is – taken from the website – an “<em>open source, advanced key-value store</em>”. But this is not all. It also is a “<em>data structure server</em>”. Check it out <a href="http://redis.io/">on its very nicely done website</a>.</p> | ||
| <p>“But we already have the massively fast in-memory backend. Why Redis?”, you scream, indignantly.</p> | ||
| <h2>Why Redis?</h2> | ||
| <p>Granted, in-memory indexes in Picky are really fast. But they have a few drawbacks:</p> | ||
| <ol> | ||
| <li>Relatively slow search engine startup, as the <span class="caps">JSON</span> index files need to be loaded into memory. This is especially noticeable if the index is around 12 GB.</li> | ||
| <li>To restart Unicorn without a hitch you need double the space the in-memory index needs, since Unicorn starts up a second master in parallel to the old one.</li> | ||
| <li>They need to be reloaded to be updated (see last blog post).</li> | ||
| </ol> | ||
| <p>I haven’t had any problems with that, but I can see the problem. Hence, Redis.</p> | ||
| <h2>How do you use Redis indexes?</h2> | ||
| <p>Looking at the configuration that the scaffolding generates, you see that it uses an <code>Index::Memory</code> called books:</p> | ||
| <pre class="sh_ruby"><code>books_index = Index::Memory.new :books, Sources::CSV.new(:title, :author, file: 'app/library.csv')</code></pre> | ||
| <p>If you’d like to use the Redis backend instead, you’ll have to change <code>Memory</code> into <code>Redis</code>.</p> | ||
| <pre class="sh_ruby"><code>books_index = Index::Redis.new :books, Sources::CSV.new(:title, :author, file: 'app/library.csv')</code></pre> | ||
| <p>I know. Picky is hard on the typing hand ;)</p> | ||
| <p>Uh. That’s already it. Welcome Redis. Bye bye, Memory.</p> | ||
| <p>What you have to do now is re-index and start Picky:</p> | ||
| <pre class="sh_bash"><code>$ rake index
... indexing output ...
$ rake start</code></pre> | ||
| <p>Or, start Picky, re-index and search while it is indexing, to get some added fun value.</p> | ||
| <h2>What is the impact of Redis indexes?</h2> | ||
| <p>Compared to the in-memory index, what are the advantages and disadvantages?</p> | ||
| <p>Advantages:</p> | ||
| <ul> | ||
| <li>Faster startup time, especially with a large index.</li> | ||
| <li>Indexing as-you-search. (No index reloading)</li> | ||
| </ul> | ||
| <p>Drawbacks:</p> | ||
| <ul> | ||
| <li>Factors slower.</li> | ||
| </ul> | ||
| <h2>Caveats / Next Versions</h2> | ||
| <p>The Redis backend implementation in Picky is not yet customizable. This means that:</p> | ||
| <ol> | ||
| <li>It uses Redis database 15.</li> | ||
| <li>Returned entry ids are always strings, even when they were integers going in. You’ll have to convert them back.</li> | ||
| <li>Redis and Memory indexes cannot (yet) be mixed and matched. So this isn’t possible: <code>Query::Full.new(redis_index, memory_index)</code>. Picky will notify you if you try to do so, so no worries.</li> | ||
| </ol> | ||
| <p>I am focusing on these points in the upcoming 1.5.* versions.</p> | ||
| <h2>Outlook</h2> | ||
| <p>One of the next blog posts will look at the performance differences between the Redis backend and the memory backend.</p> | ||
| <p>I can already reveal that the memory backend will be faster. Surprise! ;) The question is: Is Redis so much slower as to be unbearable?</p> | ||
| <p><em>Music, pregnant with suspense, fills the room: Dun dun <span class="caps">DUNNN</span>.</em></p> | ||
| <h2>Conclusion</h2> | ||
| <p>So we’ve seen</p> | ||
| <ol> | ||
| <li>what Redis is.</li> | ||
| <li>that Picky offers two different index backends: In-Memory and Redis.</li> | ||
| <li>how you use/implement the Redis index backend in your search.</li> | ||
| </ol> | ||
| <p>Hope you learnt something new :)</p> | ||
| Next | ||
| <a href="/blog/2011/03/13/searching-with-picky-rake-tasks.html" title="Next post: Searching with Picky&#58; Rake Tasks">Searching with Picky: Rake Tasks</a> | ||
| <h2>Share</h2> | ||
| <p> | ||
| <a class="twitter-share-button" data-count="none" data-text="Searching with Picky&#58; Redis" data-url="http://florianhanke.com/blog/2011/03/02/searching-with-picky-redis.html" data-via="hanke" data-width="55px" href="http://twitter.com/share">Tweet</a> | ||
| </p> | ||
| <br /> | ||
| Previous | ||
| <a class="previous" href="../../../2011/02/20/searching-with-picky-live-reloading-indexes.html" title="Previous post: Searching with Picky&#58; Live&nbsp;reloading&nbsp;indexes">Searching with Picky: Live reloading indexes</a> | ||
| <h2>Comments?</h2> | ||
| <div id="disqus_thread"></div> | ||
| <script type="text/javascript"> | ||
| //<![CDATA[ | ||
| var disqus_shortname = 'florianhanke'; | ||
| var disqus_developer = location.host.match(/\.dev$|^localhost/) ? 1 : 0; | ||
| var disqus_identifier = '/2011/03/02/searching-with-picky-redis'; | ||
| var disqus_url = 'http://florianhanke.com/blog/2011/03/02/searching-with-picky-redis.html'; | ||
|
|
||
| /* * * DON'T EDIT BELOW THIS LINE * * */ | ||
| (function() { | ||
| var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; | ||
| dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js'; | ||
| (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); | ||
| })(); | ||
| //]]> | ||
| </script> | ||
| <noscript> | ||
| Please enable JavaScript to view the | ||
| <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a> | ||
| </noscript> | ||
| </div> | ||
| </body> | ||
| </html> | ||