Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use scaled_floats for percentages in ES mapping #2156

Merged
merged 1 commit into from Aug 2, 2016

Conversation

Projects
None yet
2 participants
@tsg
Copy link
Collaborator

commented Aug 2, 2016

Elasticsearch has recently added scaled_float as an option for storing floating
point numbers. The scaled floats are stored internally as longs, which means
they can take advantage of the integer compression in Lucene. See
elastic/elasticsearch#19264 for details.

The PR moves all percentages to scaled floats. In our fields.yml we assume a
default scaling factor of 1000, which should work well for our percentages
(values between 0 and 1). This scaling factor can also be set to a different
value in fields.yml.

field["type"] = "float"

properties[field["name"]] = {
"type": field.get("type")
}

if field["type"] == "scaled_float":
properties[field["name"]]["scaling_factor"] = \
field.get("scaling_factor", 1000)

This comment has been minimized.

Copy link
@ruflin

ruflin Aug 2, 2016

Collaborator

Nice

Use scaled_floats for percentages in ES mapping
Elasticsearch has recently added scaled_float as an option for storing floating
point numbers. The scaled floats are stored internally as longs, which means
they can take advantage of the integer compression in Lucene. See
elastic/elasticsearch#19264 for details.

The PR moves all percentages to scaled floats. In our `fields.yml` we assume a
default scaling factor of 1000, which should work well for our percentages
(values between 0 and 1). This scaling factor can also be set to a different
value in `fields.yml`.

@tsg tsg force-pushed the tsg:use_scaled_float branch from d6778d7 to 6cc5026 Aug 2, 2016

@tsg tsg removed the in progress label Aug 2, 2016

@ruflin

This comment has been minimized.

Copy link
Collaborator

commented Aug 2, 2016

Flaky appveyor test should be fixed in #2156

@ruflin

This comment has been minimized.

Copy link
Collaborator

commented Aug 2, 2016

LGTM

@ruflin ruflin merged commit 4809f35 into elastic:master Aug 2, 2016

3 of 4 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
CLA Commit author is a member of Elasticsearch
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
default Build finished.
Details

@tsg tsg deleted the tsg:use_scaled_float branch Aug 25, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.