Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sparse_vector field support #168186

Merged

Conversation

carlosdelest
Copy link
Member

@carlosdelest carlosdelest commented Oct 6, 2023

Summary

Add sparse_vector field support (added to Elasticsearch here) for Discovery. Specific field icon is pending this EUI issue:

Field type:
Screenshot 2023-10-06 at 10 37 28

Field filter:
Screenshot 2023-10-06 at 10 37 44

Field in doc table:
Screenshot 2023-10-06 at 10 38 43

It also uses sparse_vector as the field mapping for ELSER based ML pipelines created using Kibana:

Screenshot 2023-10-06 at 10 40 09

Checklist

Delete any items that are not applicable to this PR.

@carlosdelest carlosdelest marked this pull request as ready for review October 6, 2023 12:17
@carlosdelest carlosdelest requested review from a team as code owners October 6, 2023 12:17
Copy link
Member

@kderusso kderusso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one question

<EuiToken
aria-label="sparse_vector"
className="kbnFieldIcon"
iconType="tokenDenseVector"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to use tokenDenseVector or should we create a new token for sparse vector?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already asked for a specific field icon in elastic/eui#7257. I think we can update that when / if ready.

@@ -69,6 +70,7 @@ export const typeToEuiIconMap: Partial<Record<string, EuiTokenProps>> = {
_source: { iconType: 'editorCodeBlock', color: 'gray' },
point: { iconType: 'tokenShape' }, // there is no separate icon for `point` yet
shape: { iconType: 'tokenShape' },
sparse_vector: { iconType: 'tokenDenseVector' },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here, do we want to use tokenDenseVector or should we create a new token for sparse vector?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andreadelrio do we have an icon we can use or add to eui?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Andrea is already pinged in elastic/eui#7257 (comment). 😉 My plan is to open a separate PR when that's ready.

@kderusso kderusso requested a review from a team October 6, 2023 12:24
Copy link
Contributor

@jughosta jughosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Discovery changes LGTM 👍

Copy link
Contributor

@demjened demjened left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@carlosdelest carlosdelest enabled auto-merge (squash) October 6, 2023 14:30
@@ -851,6 +851,23 @@ export const TYPE_DEFINITION: { [key in DataType]: DataTypeDefinition } = {
</p>
),
},
sparse_vector: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @carlosdelest, just a question: why are we adding these changes to the Index management plugin?
Did you mean to add support for the sparse_vector field type in the Mappings editor?

Screenshot 2023-10-06 at 17 52 47

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ElenaStoeva , thanks for pointing that out - I totally missed it.

I've made the changes here.

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying! Tested locally and LGTM.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cloudSecurityPosture 398.9KB 399.1KB +212.0B
dataVisualizer 613.3KB 613.6KB +371.0B
discover 573.9KB 574.5KB +551.0B
enterpriseSearch 2.6MB 2.6MB +44.0B
eventAnnotationListing 197.6KB 197.8KB +212.0B
graph 387.9KB 388.0KB +44.0B
indexManagement 560.6KB 561.1KB +503.0B
lens 1.4MB 1.4MB +507.0B
maps 2.8MB 2.8MB +44.0B
ml 3.5MB 3.5MB +44.0B
presentationUtil 76.3KB 76.4KB +44.0B
securitySolution 13.0MB 13.0MB +44.0B
stackAlerts 202.8KB 202.9KB +44.0B
unifiedDocViewer 57.7KB 57.9KB +212.0B
unifiedSearch 215.3KB 215.3KB +44.0B
total +2.9KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
osquery 51.2KB 51.3KB +44.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index management changes LGTM

@@ -851,6 +851,23 @@ export const TYPE_DEFINITION: { [key in DataType]: DataTypeDefinition } = {
</p>
),
},
sparse_vector: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying! Tested locally and LGTM.

@carlosdelest carlosdelest merged commit e8a9968 into elastic:main Oct 11, 2023
33 checks passed
@kibanamachine kibanamachine added v8.12.0 backport:skip This commit does not require backporting labels Oct 11, 2023
dej611 pushed a commit to dej611/kibana that referenced this pull request Oct 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants