Support Elasticsearch 7.0

[angristan]

Elasticsearch 7 was released about a month ago: https://www.elastic.co/blog/elasticsearch-7-0-0-released.

Currently the elasticsearch gems are pinned at 6.0.2, so Mastodon is not compatible with ES 7.

The gems have been updated for 7.0.0 support: https://github.com/elastic/elasticsearch-ruby/releases

[angristan]

I guess more work is needed, since mapping types have been removed.

mastodon~/live> env RAILS_ENV=production ./bin/tootctl search deploy
Resetting StatusesIndex
Traceback (most recent call last):
	28: from ./bin/tootctl:5:in `<main>'
	27: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	26: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	25: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	24: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	23: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:238:in `block in subcommand'
	22: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:115:in `invoke'
	21: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	20: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	19: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	18: from /home/mastodon/live/lib/mastodon/search_cli.rb:18:in `deploy'
	17: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:63:in `upgrade'
	16: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:218:in `subscribed_task_stats'
	15: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:180:in `subscribed'
	14: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:219:in `block in subscribed_task_stats'
	13: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:180:in `subscribed'
	12: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:220:in `block (2 levels) in subscribed_task_stats'
	11: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:71:in `block in upgrade'
	10: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:71:in `each'
	 9: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:73:in `block (2 levels) in upgrade'
	 8: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/rake_helper.rb:299:in `reset_one'
	 7: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/index/actions.rb:176:in `reset!'
	 6: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/chewy-5.0.0/lib/chewy/index/actions.rb:62:in `create!'
	 5: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-api-7.0.0/lib/elasticsearch/api/actions/indices/create.rb:95:in `create'
	 4: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-api-7.0.0/lib/elasticsearch/api/namespace/common.rb:38:in `perform_request'
	 3: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.0.0/lib/elasticsearch/transport/client.rb:160:in `perform_request'
	 2: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.0.0/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request'
	 1: from /home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.0.0/lib/elasticsearch/transport/transport/base.rb:338:in `perform_request'
/home/mastodon/live/vendor/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.0.0/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error': [400] {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters:  [status : {date_detection=false, properties={account_id={type=long}, id={type=long}, text={type=text, fields={stemmed={analyzer=content, type=text}}}, searchable_by={type=long}}}]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [status : {date_detection=false, properties={account_id={type=long}, id={type=long}, text={type=text, fields={stemmed={analyzer=content, type=text}}}, searchable_by={type=long}}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters:  [status : {date_detection=false, properties={account_id={type=long}, id={type=long}, text={type=text, fields={stemmed={analyzer=content, type=text}}}, searchable_by={type=long}}}]"}},"status":400} (Elasticsearch::Transport::Transport::Errors::BadRequest)

[noellabo]

I replied to chewy's issue dealing with the same issue.
toptal/chewy#673 (comment)

To solve this problem, we need to send a pull request to chewy to enable the compatibility option. Here is an example of a fix.
noellabo/chewy@94e8a6c

Correctly, we need to rewrite chewy without using the deprecated Type.

And there is another easy solution. In nginx, rewrite compatibility options to url parameters.
https://gist.github.com/Vincanote/2b18ba8aa33d068a9cc012b9a4d891af by @Vincanote

[Dee-2015]

Thank you for the update.

The include_type_name parameter will be removed in 8.X.

Any idea how much work it would be to rewrite chewy without the Type?

[dimlau]

Mastodon v3.0.0 rc1
The nginx side solution doesn't work any more!

[subelsky]

FYI there is a chewy PR that should fix this: toptal/chewy#673