Permalink
Browse files

avoid rerunning the redis commands for duplicate prefixes (our use ca…

…se for aliases will involve a lot of stuff like ['New York Yankees', 'Yankees Stadium'])
  • Loading branch information...
erwaller committed May 25, 2011
1 parent f5c1d21 commit 66a6a7e0b0f3afed9b45cd53481bc83a478eb90f
Showing with 4 additions and 5 deletions.
  1. +4 −5 lib/soulmate/loader.rb
View
@@ -29,11 +29,10 @@ def load(items)
# store the raw data in a separate key to reduce memory usage
Soulmate.redis.hset(database, id, JSON.dump(item))
- ([term] + (item["aliases"] || [])).each do |term|
- prefixes_for_phrase(term).each do |p|
- Soulmate.redis.sadd(base, p) # remember this prefix in a master set
- Soulmate.redis.zadd("#{base}:#{p}", score, id) # store the id of this term in the index
- end
+ phrase = ([term] + (item["aliases"] || [])).join(' ')
+ prefixes_for_phrase(phrase).uniq.each do |p|
+ Soulmate.redis.sadd(base, p) # remember this prefix in a master set
+ Soulmate.redis.zadd("#{base}:#{p}", score, id) # store the id of this term in the index
end
items_loaded += 1
end

0 comments on commit 66a6a7e

Please sign in to comment.