How to set null_value in mapping?

[Draqun]

Hi.

When I read documentation I found parameter null_value

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html

When there is a (JSON) null value for the field, use the null_value as the field value. Defaults to not adding the field at all.

I want use it because sort option always put records with null on the end of list. I tried do it in this way, but it didn't work.

  settings do
    mapping _all: {} do
      indexes :source_server, type: :string, null_value: ""
      indexes :source_server_agent_number, type: :integer, null_value: 0
    end
  end

Any help would be appreciated.
Best regards.

[zaziemo]

@karmi I have the same problem. How can we deal with null_values using this gem?

[Draqun]

Sorry but I don't remember, and cannot check it in source code. Is possible I never found solution. Cannot help so again sorry and good luck.

[BriocheBerlin]

Hi @karmi, do you think passing null_value will be implemented any time soon? Thanks!

[karmi]

Hi, I did some experimenting with it, and this does work:

mapping do
    indexes :title, type: 'keyword', null_value: 'NOTHING'
  end

But it doesn't work for the text type (an analyzed type in Elasticsearch), which is the default for the indexes method -- see this issue in Elasticsearch for more context: elastic/elasticsearch#23839.

So, @zaziemo and @BriocheBerlin, it would work, if you'd explicitely set the field to be the keyword type, like in that example.

But the larger question is, in what case do you actually need this? Isn't it better to supply "null" values in the property, and then use the "missing" query/filter to get those documents?

---

@Draqun, many apologies for completely missing your question :(, I can see you've sent it in August, when I was offline, and it got lost in the fray of dozens of questions opened over the month... I hope you have solved it somehow. Sorry again!

[Draqun]

@karmi

No problem. I finished with RoR. I hate languages with single inheritance. I prefer Python and C++ :)

Best regards.

[zaziemo]

Hi @karmi,

thanks for your quick response.
We are still working with v 2.4 and are using the type: ‚completion’ because we use it in the context of a suggest field. Probably it doesn’t work with this type?

To explain the larger context: we have a database of speakers who can list their profile details in different languages, e.g. their main topic of expertise. Most of them only have one language so some fields remain empty. On indexing things break: not only does the empty field not get indexed, instead the whole profile it belongs to doesn’t get indexed and I still get the error
MapperParsingException[completion field [main_topic_en.suggest] does not support null values]
So probably, null_values might not be the way to go but the "missing" query/filter is only a solution in the process of querying. But we already have to fix our problem during indexing. Do we miss something? Thanks for your help!

[karmi]

We are still working with v 2.4 and are using the type: ‚completion’ because we use it in the context of a suggest field. Probably it doesn’t work with this type?

Yes, @zaziemo, that is what I see as well now on 2.4.0 when I try it.

Thanks for the details — I understand now how the null_value would be attractive there. I was playing with the activerecord_mapping_completion.rb example, and when the title attribute of the model is empty, I'm indeed getting the completion field [title.suggest] does not support null values error.

I think the only solution is to put the "protective" logic inside the indexing layer, and remove the empty property in the MyModel#as_indexed_json method, even something simple like should work:

mapping do
  indexes :title_suggest, type: 'completion'
end

# ...

def as_indexed_json(options={})
  output = as_json
  output.delete 'suggest' unless self.suggest.present?
  output.merge 'url' => "/articles/#{id}"
end

(If the project is open source, don't hesitate to ping me there!)

[zaziemo]

@karmi thanks a lot. With your hint and a little help of @svenfuchs I got it working using the select method output.select{ |key, value| value.present? }

[estolfo]

Closing as the issue seems to have been resolved.