Add support for time_series_dimension flag in _elastic/{index}/_field_caps API method

[kuzaxak]

Opensearch Dashboards calling _field_caps to build an index pattern. To use quickwit natively as an external data source for it dashboards need to determine a timestamp field to render histogram, such fields listed on index pattern settings page. For that it relies on a time_series_dimension flag in <field>.<type>.time_series_dimension.

Link to the doc.

Describe the solution you'd like

Filed chosen as timestamp_field in index should be marked as time_series_dimension in _field_caps output.

Describe alternatives you've considered

I've made a little proxy to query /api/v1/indexes?index_id_patterns= and patch response of _field_caps method.

Additional context

I'm working on integrating quickwit as an Opensearch Dashboards backend to replace existing setup which is using regular Opensearch.

[fulmicoton]

@kuzaxak have you started experimenting with adding opendashboard support by any chance? I'd love to exchange if you have more info on other missing pieces.

[kuzaxak]

@kuzaxak have you started experimenting with adding opendashboard support by any chance? I'd love to exchange if you have more info on other missing pieces.

Yes, I was able to migrate our Opensearch dashboard completely to quickwit backend. But I'm using a proxy to monkey patch some responses of quickwit.

I can share a proxy code, it is a simple python flask app.

[kuzaxak]

@fulmicoton here you can find a proxy server code.

I'm using quickwit as an external source not to deal with auth, dashboards storage, etc.

1. Enable this feature for OSD:

data_source.enabled: true

2. Add proxy as an additional datasource:

3. Add Index pattern using external source:

This way I'm using security features and SAML from OSD and logs storage and querying from Quickwit.

[fulmicoton]

@kuzaxak That's absolutely awesome, thank you for sharing the script, the approach and the screenshot, The hack is really cool!

[kuzaxak]

Sorry for confusion, this particular field is in spec of ES. And due to missing spec of OpenSearch I was thinking that is it required. Apparently it is not implemented in OS and net needed.

Will continue with next methods to make sure that QuickWit is compatible via light proxy (no request \ response modifications needed).