Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add unsigned_long support to data frame analytics #81501

Closed
droberts195 opened this issue Oct 22, 2020 · 2 comments
Closed

[ML] Add unsigned_long support to data frame analytics #81501

droberts195 opened this issue Oct 22, 2020 · 2 comments
Assignees
Labels

Comments

@droberts195
Copy link
Contributor

A new field type unsigned_long was added to Elasticsearch in elastic/elasticsearch#60050 and core Kibana in #81115. This is a new type of numeric field. The necessary changes for the ML Java data frame analytics code to work with unsigned_long were made in elastic/elasticsearch#64066. Given that this type is supported in core Kibana now it could be argued that it is now a bug that it's not supported in the ML UI.

I did some testing with the ML Java changes combined with the following Kibana patch:

diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/common/fields.ts b/x-pack/plugins/ml/public/application/data_frame_analytics/common/fields.ts
index c606cbd1cc1..561f2f12a3f 100644
--- a/x-pack/plugins/ml/public/application/data_frame_analytics/common/fields.ts
+++ b/x-pack/plugins/ml/public/application/data_frame_analytics/common/fields.ts
@@ -33,6 +33,7 @@ export const MAX_COLUMNS = 10;
 export const DEFAULT_REGRESSION_COLUMNS = 8;
 
 export const BASIC_NUMERICAL_TYPES = new Set([
+  ES_FIELD_TYPES.UNSIGNED_LONG,
   ES_FIELD_TYPES.LONG,
   ES_FIELD_TYPES.INTEGER,
   ES_FIELD_TYPES.SHORT,

With those changes the unsigned_long fields showed up as expected when creating a new data frame analytics job, and it was possible to run that job and view the results. However, in the results grid the unsigned_long fields did not show up. So it seems that some other change is also needed. However, hopefully it is not a massive amount of work to add support for unsigned_long to the data frame analytics UI.

@droberts195 droberts195 added :ml Feature:Data Frame Analytics ML data frame analytics features labels Oct 22, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@LeeDr
Copy link
Contributor

LeeDr commented Oct 27, 2020

I'll just add a note and screenshot from the testing I did on 7.10.0. I created an index pattern with 4 docs which have an unsigned long field "my_counter" which shows a value rounded or truncated to the precision Kibana supports, and also has a string representation of the actual value "string_counter". The values of "my_counter" do appear in the grid view;
2020-10-27 12_41_33-Data Frame Analytics - Machine Learning - Elastic

But when I click the Outlier detection selection I get an error.

Request URL: http://localhost:5601/api/ml/data_frame/analytics/_explain
Response: {"statusCode":500,"error":"Internal Server Error","message":"exception"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants