Clone this wiki locally
Glad you’d like to add another
tentacle source to Picky!
(to use for your index)
How to do it
- Check the available Sources if the source already exists.
- If yes, use (and improve) it :)
- If not, fork the repository and follow the instructions below.
- Add a new load statement to lib/picky/loader.rb
Every source derives from the base class
Sources::Base, which contains 3 methods. These methods are all called by the indexer:
connect_backend# Connect to the backend here if your source needs that.
take_snapshot(type)# If you have fast moving data, you might want to get a snapshot here (into a hash, or a temp table etc.)
harvest(type, category)# This is the mandatory method. Yield each id and data for that id.
By default, the methods are empty. So if you don’t need to connect to a backend, just don’t override the method.
This is how the CSV source implements the harvest method. It’s currently not optimized.
def harvest _, category # We don't care about the type, just about the category. index = field_names.index category.name # Get the right column from the data. ::CSV.foreach file_name do |ary| # A helper method that reads a line from the csv file. indexed_id = ary.shift text = ary[index] next unless text yield indexed_id, text # <- the most important part, yields the indexed_id and the corresponding text to the indexer. end end