Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Support testing fill mask models that do not use the [MASK] token #159577

Closed
davidkyle opened this issue Jun 13, 2023 · 4 comments · Fixed by #162168
Closed

[ML] Support testing fill mask models that do not use the [MASK] token #159577

davidkyle opened this issue Jun 13, 2023 · 4 comments · Fixed by #162168
Assignees
@jgowdyelastic
Labels
enhancement New value added to drive a business result Feature:3rd Party Models ML 3rd party models :ml v8.10.0

Comments

@davidkyle
Copy link
Member

Describe the feature:
When using the test trained models feature for Fill Mask models the test button is enabled only if the text contains [MASK]. Some tokenisers use a different mask token, e.g. <mask>.

The model configuration should report what the mask token is and use that value to check the text is valid.

Describe a specific use case for the feature:
Using the xlm-roberta-base model.

Screenshot 2023-06-13 at 14 46 36
@davidkyle davidkyle added the :ml label Jun 13, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@jgowdyelastic
Copy link
Member

@davidkyle is there any way to know what the mask token is?
Looking at the model config I can't see anything.

@jgowdyelastic jgowdyelastic changed the title Support testing fill mask models that do not use the [MASK] token [ML] Support testing fill mask models that do not use the [MASK] token Jun 13, 2023
@jgowdyelastic jgowdyelastic self-assigned this Jun 13, 2023
@davidkyle
Copy link
Member Author

is there any way to know what the mask token is?

Intuition! Failing that a back end change is required

@peteharverson peteharverson added Feature:3rd Party Models ML 3rd party models enhancement New value added to drive a business result labels Jun 13, 2023
@maxhniebergall maxhniebergall mentioned this issue Jul 10, 2023
Merged
@jgowdyelastic
Copy link
Member

mask_token is now available in the model config.
elastic/elasticsearch#97453

@jgowdyelastic jgowdyelastic mentioned this issue Jul 18, 2023
Merged
jgowdyelastic added a commit that referenced this issue Jul 20, 2023
@jgowdyelastic
[ML] Using model supplied mask token (#162168)
3524da7 
Fixes #159577

Using the `mask_token` property from the model config for testing the
model.
This is shown in the input placeholder text, in the input validation and
for displaying the results.

<img width="433" alt="image"
src="https://github.com/elastic/kibana/assets/7405507/bc63f9e6-a3d5-402c-a451-8d80b758acbc">
dgieselaar pushed a commit to dgieselaar/kibana that referenced this issue Jul 23, 2023
@jgowdyelastic @dgieselaar
[ML] Using model supplied mask token (elastic#162168)
27c3db3 
Fixes elastic#159577

Using the `mask_token` property from the model config for testing the
model.
This is shown in the input placeholder text, in the input validation and
for displaying the results.

<img width="433" alt="image"
src="https://github.com/elastic/kibana/assets/7405507/bc63f9e6-a3d5-402c-a451-8d80b758acbc">
ThomThomson pushed a commit to ThomThomson/kibana that referenced this issue Aug 1, 2023
@jgowdyelastic
[ML] Using model supplied mask token (elastic#162168)
140d034 
Fixes elastic#159577

Using the `mask_token` property from the model config for testing the
model.
This is shown in the input placeholder text, in the input validation and
for displaying the results.

<img width="433" alt="image"
src="https://github.com/elastic/kibana/assets/7405507/bc63f9e6-a3d5-402c-a451-8d80b758acbc">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgowdyelastic jgowdyelastic

Labels
enhancement New value added to drive a business result Feature:3rd Party Models ML 3rd party models :ml v8.10.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ML] Uses model supplied mask token for testing trained models jgowdyelastic/kibana
4 participants
@davidkyle @peteharverson @elasticmachine @jgowdyelastic