59667324 use elasticsearch gem #138

merged 20 commits into from Jan 21, 2014

2 participants


This change is for https://www.pivotaltracker.com/story/show/59667324

Since Tire is being retired we moved to official Ruby Elasticsearch client.

As a result we have less complex code. Indexing code isn't so tightly coupled to Sequel model code. We can also run any Elasticsearch query or their combination - no need to wait for Tire add new query types. We now use multi query search endpoint to perform our search and save 4 http requests.

This also makes section search work, sections aren't being found at the moment: https://www.gov.uk/trade-tariff/search?utf8=%E2%9C%93&q=trade_tariff&t=animal+products&button=&day=20&month=1&year=2014&country= doesn't contain Section 1 "Live animals; animal products".

Depends on:

saulius added some commits Jan 15, 2014
@saulius saulius Remove Tire, Add custom search client
Client is based on official elasticsearch gem
@saulius saulius Bulk serialize indexed models
Extracts serialization from Sequel::Model descendant classes to separate
serializer classes.

Removed Sequel pagination compatibility extension because it will not be
required any more since we don't use Tire.
@saulius saulius Support for Elasticsearch in specs 7bccad4
@saulius saulius Extract Serializer and specs to app/serializers
Separate concerns from persistance (Sequel::Model)

Also added index and delete methods to SearchClient. Refactored
interactors to use proper interface.
@saulius saulius Remove references to Tire and comments 21f504f
@saulius saulius Remove Tariff indexer
Refactor reindexing code
@saulius saulius Make searching work with new Elasticsearch client fdcd5af
@saulius saulius FuzzySearch uses Elasticsearch Multi-Query API
Reduces HTTP requests from 5 to 1, searches happen much faster.
@saulius saulius Tag specs dependent on Elasticsearch
These are integration level specs, they require Elasticsearch test cluster
to be up in order to pass
@saulius saulius Add a simple Elasticsearch indexing Sequel plugin
For indexable models
@saulius saulius Drop elasticsearch search tag
Spawn server for each spec run
@saulius saulius Refresh search index after each index/delete operation
Because specs require instant visibility of indexed content unlike

Added more configuration options on TradeTariffBackend.
@saulius saulius Fix GoodsNomenclature factory and related specs 5b058bc
@saulius saulius Pass command to search_options on test environment a84f4e4
@saulius saulius Make sure elasticsearh binary is in PATH 832c9f4
@saulius saulius Remove SearchReference interactors
We now use simple callbacks with our own Sequel plugin elasticsearch
@saulius saulius Make sure search does not break with malformed query
Uses simple match Elasticsearch query instead of query_string which
supports custom syntax which is not necessary and can result in an
exception sometimes
@saulius saulius Raise QueryException if one of search queries contains an error key 4cfbbcb
@saulius saulius Extract #search_index_for method, it is being used more broadly
Added additional specs
@saulius saulius Remove vcr and its mock files
Specs run in integration with locally spawned Elasticsearch client.
@saulius saulius referenced this pull request in alphagov/trade-tariff-frontend Jan 20, 2014

Update Search Result parsing #114

@matthewford matthewford merged commit a7d61f2 into master Jan 21, 2014

1 check failed

Details default The Travis CI build could not complete due to an error

@sauliusg can we remove 1.9 from travis as it's failing

@jabley jabley deleted the 59667324-use-elasticsearch-gem branch Jan 21, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment