Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a data validation from downloaded data #7

Open
samuelhei opened this issue Apr 5, 2018 · 1 comment
Open

Create a data validation from downloaded data #7

samuelhei opened this issue Apr 5, 2018 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@samuelhei
Copy link
Contributor

The data downloaded from Binance will be used to test a trading strategy, it is important that:

  1. There is no duplicated data.
  2. There is no missing data.
@samuelhei samuelhei added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers labels Apr 5, 2018
@anson-vandoren
Copy link
Contributor

@samuelhei with the new parallel implementation, duplicated data is already scrubbed before saving. This is in part because Binance sometimes has some missing slots (i.e., when they were down for a maintenance period), and so normal intervals didn't always grab all the data (interval starts didn't line up to the second post-maintenance). So I added some intentional overlap, and then scrubbed out dupes.

For the missing data, there will still be some even in the best case due to above-mentioned maintenance period. It would be possible to generate a quick report showing what intervals are not present in the data set if you think it'd be useful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants