Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Half and scaled floats missing from spark 2 schema #981

Closed
mdelaney opened this issue May 4, 2017 · 0 comments · Fixed by #982
Closed

Half and scaled floats missing from spark 2 schema #981

mdelaney opened this issue May 4, 2017 · 0 comments · Fixed by #982
Labels
bug :Spark v6.0.0-alpha2 v6.0.0

Comments

@mdelaney
Copy link
Contributor

mdelaney commented May 4, 2017

What kind an issue is this?

Bug report.

Issue description

Type mapping for scaled and half floats is missing for spark 2 sql. Issue #822 is the original issue where the schema mapping was added, looks like this file was missed.

Steps to reproduce

If you attempt to work with any index that uses scaled or half floats using spark 2 you can reproduce the issue. Using metricbeat is a good example as it makes extensive use of scaled floats.

Version Info

OS: : Amazon Linux (4.4.35-33.55.amzn1.x86_64)
JVM : openjdk 1.8.0_121
Hadoop/Spark: 2.7.3-amzn-1
ES-Hadoop : 5.2.3 (issue still exists in master)
ES : 5.2.1
mdelaney pushed a commit to tubular/elasticsearch-hadoop that referenced this issue May 4, 2017
add scaled and half float to schemautils (elastic#981)
175f692 
spark-sql-20 was missed when this was first added
@mdelaney mdelaney mentioned this issue May 4, 2017
Merged
1 task
jbaiera pushed a commit that referenced this issue May 9, 2017
@mdelaney @jbaiera
add scaled and half float to schemautils (#981) (#982)
c0c963b 
spark-sql-20 was missed when this was first added
@jbaiera jbaiera added the v6.0.0 label May 9, 2017
jbaiera pushed a commit that referenced this issue Jun 1, 2017
@mdelaney @jbaiera
add scaled and half float to schemautils (#981) (#982)
55a2890 
spark-sql-20 was missed when this was first added
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug :Spark v6.0.0-alpha2 v6.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add scaled and half float to schemautils (#981) tubular/elasticsearch-hadoop
2 participants
@mdelaney @jbaiera