Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All actions (Trades, Asset Movements e.t.c.) are now saved in the DB #508

Merged
merged 48 commits into from Sep 27, 2019

Conversation

@LefterisJP
Copy link
Contributor

LefterisJP commented Sep 26, 2019

Fix #499
Fix #506

LefterisJP added 30 commits Sep 2, 2019
That's because we need to be able to test the DB upgrades from back
then and there is no residual DBs from then to be found.

From here and on for each upgrade an old DB must be used.
Just like with trades and marging positions, now deposits and
withdrawals are first queried from the DB and only iff newer data may
exist online for the time period is the exchange queried.
Iterating a list of records is better than just using a bunch of
functions that are called sequentially since it's:

1. easier to read
2. can be mocked/modified easier in tests to test only upgrades until
a specific target version
- Adding a Location Enum table in the DB and using it for the trades
location
- Changing the exchange enum in python to a more generic Location enum

Important: Still left to add this enum to other DB tables
LefterisJP added 17 commits Sep 22, 2019
As opposed to the previous logic where a simple executemany() was done
and if it failed we bailed out now we first try that but if it fails
we resort to writting entries to the DB one by one
Ethereum transactions are now also saved in the database.

Initially for each address in a time range we first check if the last
DB trade is at the limit of the range. If yes then that means the
range has been queried before succesfully and then we do not query
again.

Else we query etherscan and save the transactions in the DB.

The logic needs more finetuning, and needs to be made similar to the
rest of the DB + API querying logic, but unfortunately etherscan
querying uses block numbers and not timestamps. Converting between
timestamps and block numbers is non trivial.
@LefterisJP LefterisJP force-pushed the LefterisJP:workon_499 branch from 0db4022 to 1712817 Sep 27, 2019
@LefterisJP LefterisJP merged commit 3e3bc7a into rotki:master Sep 27, 2019
1 of 3 checks passed
1 of 3 checks passed
continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
license/cla Contributor License Agreement is signed.
Details
@LefterisJP LefterisJP deleted the LefterisJP:workon_499 branch Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.