-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature - Distributed storage of market data #3540
Comments
Which users will be authorized to write to the decentralized database? |
I would say everyone is allowed to write data. Probably we can have a tier above a certain amount of reputation are the ones that can write data. I'm having second thoughts on this feature. Users that are creating strategies do need historic data for backtesting and they can download it from the exchanges. Thoughts? EDIT |
This is a chat I just had relevant to this issue, for those of you looking into this problem: Kara Lama, [1/10/2022 10:12 AM] Kara Lama, [1/10/2022 10:14 AM] Kara Lama, [1/10/2022 10:14 AM] Kara Lama, [1/10/2022 10:15 AM] Kara Lama, [1/10/2022 10:27 AM] Luis Molina, [1/10/2022 10:38 AM] Luis Molina, [1/10/2022 10:39 AM] Luis Molina, [1/10/2022 10:39 AM] Luis Molina, [1/10/2022 10:40 AM] Luis Molina, [1/10/2022 10:43 AM] Luis Molina, [1/10/2022 10:43 AM] Luis Molina, [1/10/2022 10:44 AM]
Luis Molina, [1/10/2022 10:46 AM] Kara Lama, [1/10/2022 10:47 AM] Luis Molina, [1/10/2022 10:47 AM] Luis Molina, [1/10/2022 10:47 AM] Luis Molina, [1/10/2022 10:48 AM] Luis Molina, [1/10/2022 10:50 AM] Luis Molina, [1/10/2022 10:53 AM] Luis Molina, [1/10/2022 10:54 AM] Luis Molina, [1/10/2022 10:55 AM] Kara Lama, [1/10/2022 10:55 AM] Luis Molina, [1/10/2022 10:57 AM] Luis Molina, [1/10/2022 10:57 AM] Luis Molina, [1/10/2022 10:57 AM] Luis Molina, [1/10/2022 10:57 AM] Luis Molina, [1/10/2022 10:58 AM] Luis Molina, [1/10/2022 10:58 AM] Luis Molina, [1/10/2022 10:58 AM] Luis Molina, [1/10/2022 10:59 AM] Luis Molina, [1/10/2022 10:59 AM] Luis Molina, [1/10/2022 11:00 AM] Luis Molina, [1/10/2022 11:01 AM] Kara Lama, [1/10/2022 11:04 AM] Kara Lama, [1/10/2022 11:10 AM] Kara Lama, [1/10/2022 11:13 AM] Luis Molina, [1/10/2022 11:14 AM] User X running the platform have access to all the User Profiles plugins of all users, and there are explicitly declared the Repositories each user has for storing their hashes. The way I can imagine this to work is to load the hashes from multiple sources, and create a local hash table for all the available data. That local hash table would prioritize users with more SA for the same files. Kara Lama, [1/10/2022 11:19 AM] Kara Lama, [1/10/2022 11:19 AM] Luis Molina, [1/10/2022 11:20 AM] Luis Molina, [1/10/2022 11:20 AM] Kara Lama, [1/10/2022 11:21 AM] Kara Lama, [1/10/2022 11:22 AM] Luis Molina, [1/10/2022 11:24 AM] Luis Molina, [1/10/2022 11:26 AM] Kara Lama, [1/10/2022 11:26 AM] Kara Lama, [1/10/2022 11:27 AM] Luis Molina, [1/10/2022 11:28 AM] Kara Lama, [1/10/2022 11:29 AM] Luis Molina, [1/10/2022 11:38 AM] Luis Molina, [1/10/2022 11:41 AM] Maybe you would like to browse calculated data at the charts, of any exchange and any market, including indicators and so on. But you would like to calculate everything yourself for your live trading. |
There's GUNjs (https://gun.eco/) that presents solutions for some of the issues. As stated before, this is based on some investigation I did last month. I'm not actively working on this now. |
Is your feature request related to a problem? Please describe.
The platform needs a distributed storage solution to have market historical data always available to all users
Describe the solution you'd like
Many exchanges do not provide a very long historic background of market data. This hinders backtest scenarios to test strategies and validate indicators.
Having a distributed storage like IPFS, market data that is downloaded or processed (indicators) by one user can become available to all users.
Describe alternatives you've considered
Currently each installation has it's own set of market data as a set of JSON files. This can potentially be further optimized using a time-series db.
The text was updated successfully, but these errors were encountered: