Should we allow fields with * in their name?!

[nik9000]

Should we let field names contain a *? This gets confusing when you send the fields to us because we usually interpret * as a glob pattern.

Story:
I'm helping some folks debug a fun issue where their kibana discover tab stops working. It looks like they have fields with * in their name. Say c*g.

Kibana, entirely reasonably, reads those fields from field caps and send them back in the fields section of the search request. So:

"fields": [
  { "field": "c*g", "format": "date_time"}
]

We then, helpfully, interpret the * as a glob pattern. Chaos! In the fake example above we'd match the original c*g field but we also match cat.dog and condor.clog. And, sadly, those are not dates! So we throw back an IllegalArgumentException.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[mattkime]

I guess I'm curious to hear an argument for supporting * in field names. I think it would be possible to support it in kibana but it would be unlikely to be worth the effort as we'd need to distinguish whether the user wants * the character or * the glob pattern.

---

Based on what @timroes has to say here - https://github.com/elastic/sdh-elasticsearch/issues/4800#issuecomment-905427386 - they should just be disallowed.

[nik9000]

I haven't talked to anyone that thinks it's worth supporting them.

…

On Wed, Aug 25, 2021, 20:14 Matthew Kime ***@***.***> wrote: I guess I'm curious to hear an argument for supporting * in field names. I think it would be possible to support it in kibana but it would be unlikely to be worth the effort as we'd need to distinguish whether the user wants * the character or * the glob pattern — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#76874 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABUXIQV3X4VK3QNFTTJIWLT6WBO7ANCNFSM5CWWU27A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[mattkime]

Where does this stand? Seems like we should be ready to make a call on this.

[nik9000]

I'm having similar thoughts. I see if it can talk to the right folks on Monday. I had hoped making the ticket would do it but it think it fell through the cracks.

…

On Sat, Sep 11, 2021, 00:49 Matthew Kime ***@***.***> wrote: Where does this stand? Seems like we should be ready to make a call on this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#76874 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABUXIQZNT73KQ57OV4EK4LUBLNW5ANCNFSM5CWWU27A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[jimczi]

We discussed this offline and we're not convinced that we should make a call now. The main point of the dynamic mapping is to accept all field names. We could reserve some characters and make them invalid but that doesn't solve the problem. Users would still need to deal with these characters somehow.
IMO, this issue reveals two things:

• We don't have any mechanism to escape characters in APIs that accept field names and patterns.
• We shouldn't fail the entire shard request when a single field fails the fields option parser.

None of these requires a breaking change for our users and can be dealt in the general case.