Bonsai ElasticSearch and Tire

[rahilsondhi]

I'm trying to get Bonsai ElasticSearch (cc: @nz) to work with Tire on Heroku but I'm getting this error:

2012-04-21T23:18:00+00:00 app[web.1]: Started GET "/search.json?q=janice" for 67.224.81.78 at 2012-04-21 19:18:00 -0400
2012-04-21T23:18:01+00:00 app[web.1]: Processing by SearchController#index as JSON
2012-04-21T23:18:01+00:00 app[web.1]:   Parameters: {"q"=>"janice"}
2012-04-21T23:18:01+00:00 app[web.1]: [REQUEST FAILED] curl -X GET "http://index.bonsai.io/artists,users/_search?pretty=true" -d '{"query":{"query_string":{"query":"janice"}}}'
2012-04-21T23:18:01+00:00 app[web.1]: Completed 500 Internal Server Error in 25ms
2012-04-21T23:18:01+00:00 app[web.1]: 
2012-04-21T23:18:01+00:00 app[web.1]: Tire::Search::SearchRequestFailed (401 : {"error": "Not authorized: Some endpoints are admin-only, ask support@onemorecloud.com."}
2012-04-21T23:18:01+00:00 app[web.1]: ):
2012-04-21T23:18:01+00:00 app[web.1]:   app/models/user.rb:141:in `search_for'
2012-04-21T23:18:01+00:00 app[web.1]:   app/controllers/search_controller.rb:6:in `index'

My tire.rb initializer has:

if ENV['BONSAI_INDEX_URL']
  Tire.configure do
    url "http://index.bonsai.io"
  end
  BONSAI_INDEX_NAME = ENV['BONSAI_INDEX_URL'][/[^\/]+$/]
end

And my model has:

include Tire::Model::Search
include Tire::Model::Callbacks
index_name BONSAI_INDEX_NAME

@nz says it's because Tire is omitting the index name from the BONSAI_INDEX_URL environment variable. He'll be able to offer more insight into this.

[nz]

I'm traveling and can't comment in too much detail at the moment.

The basic idea is that a search is being constructed as a multi-index search on the cluster rather than a multi-type search on the index. This is related to configuration conventions I've chatted with @karmi about in the past.

Rahil, can you also include an example of the code you're calling to invoke the search itself?

Nick Zadrozny

On Tuesday, April 24, 2012 at 7:25, Rahil Sondhi wrote:

I'm trying to get Bonsai ElasticSearch (cc: @nz) to work with Tire on Heroku but I'm getting this error:

2012-04-21T23:18:00+00:00 app[web.1]: Started GET "/search.json?q=janice" for 67.224.81.78 at 2012-04-21 19:18:00 -0400
2012-04-21T23:18:01+00:00 app[web.1]: Processing by SearchController#index as JSON
2012-04-21T23:18:01+00:00 app[web.1]: Parameters: {"q"=>"janice"}
2012-04-21T23:18:01+00:00 app[web.1]: [REQUEST FAILED] curl -X GET "http://index.bonsai.io/artists,users/_search?pretty=true" -d '{"query":{"query_string":{"query":"janice"}}}'
2012-04-21T23:18:01+00:00 app[web.1]: Completed 500 Internal Server Error in 25ms
2012-04-21T23:18:01+00:00 app[web.1]: 
2012-04-21T23:18:01+00:00 app[web.1]: Tire::Search::SearchRequestFailed (401 : {"error": "Not authorized: Some endpoints are admin-only, ask support@onemorecloud.com."}
2012-04-21T23:18:01+00:00 app[web.1]: ):
2012-04-21T23:18:01+00:00 app[web.1]: app/models/user.rb:141:in `search_for'
2012-04-21T23:18:01+00:00 app[web.1]: app/controllers/search_controller.rb:6:in `index'

My tire.rb initializer has:

if ENV['BONSAI_INDEX_URL']
Tire.configure do
url "http://index.bonsai.io"
end
BONSAI_INDEX_NAME = ENV['BONSAI_INDEX_URL'][/[^\/]+$/]
end

And my model has:

include Tire::Model::Search
include Tire::Model::Callbacks
index_name BONSAI_INDEX_NAME

@nz says it's because Tire is omitting the index name from the BONSAI_INDEX_URL environment variable. He'll be able to offer more insight into this.

---

Reply to this email directly or view it on GitHub:
#322

[karmi]

Right now, Tire uses separate indices for different models (ie. document types). In coordination with @nz, I'd like to change the defaults to use separate types; the user will still be able to use different indices, if she wishes, of course.

I plan to have a look into this in the near future. I'm not sure how multi-model searches -- since that is what you seem to be doing -- work right now with Tire and Bonsai.

[nz]

Fortunately, fwiw, multi-type searches within a single index are pretty straightforward…

curl 'http://localhost:9200/test/foo,bar/_search?q=*:*'

[karmi]

Yes, of course, they are very easy on the ES level. I have meant: I don't know how they work right now with Tire and Bonsai.

[hale]

@karmi Do you have a rough timeline for when the defaults will use types instead of indices for the different models? I understand this might not be a priority for you right now, I was just wondering if it was 'within the next month' or 'within the next 6 months'.

[karmi]

@hale, "within the next three months" sounds most plausible... Since it's a breaking change, I'd like to sync with other similar changes...

[nz]

@karmi So about those multi-index vs. multi-type searches…? are we due for another chat about that? :)

[mculp]

I'm also here through Bonsai research on multi-index vs. multi-type. Any word?

[nz]

@mculp, can't speak for @karmi wrt Tire's index usage conventions, but I'm working on some stuff on the Bonsai end of things to allow many indices through the Heroku addon. Would be happy to follow up with you on your use case in more detail over email.

[karmi]

All: Please direct all Bonsai questions to Bonsai support channels.

Do notice there's some new instructions for Bonsai/Heroku available: https://gist.github.com/2041121

[nz]

@karmi, any further comment on whether the single index per app per environment paradigm is on your roadmap?

I've been rolling out better multi-index support over at Bonsai for Tire users lately, but I still think a single index per app per environment is a useful paradigm for most apps. Maybe Tire's current conventions are too well entrenched to be worth the effort for an arguably marginal difference? Seems like a pretty fair position, if that's the one you've arrived at.

[karmi]

@nz It is still on the roadmap, though it is still a low priority.

(...) a single index per app per environment is a useful paradigm for most apps.

Disagree. I understand the argument “index is like a database and you have one database per app” can be made -- but it's applicable to small, basic apps only. All the awesome and powerful features of elasticsearch, such as index aliases, time-based indices, re-indexing without downtime, etc. go out of the window.

(...) Tire's current conventions are too well entrenched to be worth the effort (...)

Not sure I follow 100%, but don't think so -- time & energy are scarce resources, and there's still a lot of pull requests to process, bugs and inconsistencies to fix. This one issue arguably only affects Bonsai-based deployments, where many of those affect the majority of Tire/elasticsearch users.

[nz]

Thanks for the clarifications. My preferred conventions aside, the original question posed in this Issue has since been addressed on our end. But I appreciate your update and thoughts wrt mapping models to indexes vs. types.

[Will-Sommers]

Hello guys. Its been four months and I'm curious where this issue stands, specifically re: multi_search

[nz]

@Will-Sommers: Proper multi-search in Bonsai is still forthcoming. Consolidating into a single index is a good quick fix for today. I've made some updates to https://gist.github.com/nz/2041121 toward that end.

[nz]

@Will-Sommers, @hale, @rahilsondhi, @mculp — basic multi-index searching is supported in Bonsai; see this blog post for more details.

[karmi]

@nz Cool! 👍