Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Research: Make sure sqlAlchemy decimal places are rounded to two places #3856

Closed
jason-upchurch opened this issue Jul 10, 2019 · 1 comment
Closed
Assignees
Labels
Milestone

Comments

@jason-upchurch
Copy link
Contributor

jason-upchurch commented Jul 10, 2019

Summary

Querying the following in Chrome vs. Chrome Incognito mode yield results having different decimal places:

"https://api.open.fec.gov/v1/schedules/schedule_a/by_size/by_candidate/?sort_hide_null=false&page=1&api_key=DEMO_KEY&candidate_id=P80001571&election_full=false&per_page=20&cycle=2020&sort_null_only=false&sort_nulls_last=false"

Need to verify that decimal precision is set to maximum of 2 places in columns and operations. May need to consider rounding in final step for any operations having intermediate calculations to avoid accumulation of error.

Note that querying swagger ui and using the following script shows 2 decimal places:

import pandas as pd
import json
import requests

api_url = "https://api.open.fec.gov/v1/schedules/schedule_a/by_size/by_candidate/? 
    sort_hide_null=false&page=1&api_key=DEMO_KEY&candidate_id=P80001571&election_full=
    false&per_page=20&cycle=2020&sort_null_only=false&sort_nulls_last=false"

df = pd.read_json(api_url, typ=dict)
for r in df.results:
    print(r['total'])

response = requests.get(api_url)
print(json.loads(response.content.decode('utf-8')))

[edited formatting]

@dorothyyeager dorothyyeager added this to the Sprint 9.5 milestone Jul 11, 2019
@patphongs patphongs changed the title Make sure sqlAlchemy decimal places are rounded to two places Research: Make sure sqlAlchemy decimal places are rounded to two places Jul 16, 2019
@patphongs patphongs modified the milestones: Sprint 9.5, Sprint 9.6 Jul 16, 2019
@jason-upchurch jason-upchurch self-assigned this Jul 24, 2019
@jason-upchurch jason-upchurch modified the milestones: Sprint 9.6, Sprint 9.5 Jul 24, 2019
@jason-upchurch
Copy link
Contributor Author

Closing issue as values are represented as expected in several environments. For the case of arithmetic on values representing dollars and cents, the case of results having greater than two decimal places can be explained as (see python documentation):

On most machines today, floats are approximated using a binary fraction with the numerator using the first 53 bits starting with the most significant bit and with the denominator as a power of two. In the case of 1/10, the binary fraction is 3602879701896397 / 2 ** 55 which is close to but not exactly equal to the true value of 1/10.

Many users are not aware of the approximation because of the way values are displayed. Python only prints a decimal approximation to the true decimal value of the binary approximation stored by the machine. On most machines, if Python were to print the true decimal value of the binary approximation stored for 0.1, it would have to display

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants