Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error - Prepare data from Yahoo 从Yahoo下载数据报错 #613

Closed
prevailings opened this issue Sep 21, 2021 · 2 comments
Closed

Error - Prepare data from Yahoo 从Yahoo下载数据报错 #613

prevailings opened this issue Sep 21, 2021 · 2 comments
Labels
question Further information is requested

Comments

@prevailings
Copy link

English: Already prepared data by "qlib/scripts/get_data.py", but it only include until 20200925.
I tried "python scripts/data_collector/yahoo/collector.py update_data_to_bin --qlib_data_1d_dir ~/.qlib/qlib_data/cn_data --trading_date 20200925 --end_date 20210917", but failed as below:
Question: How to prepare data up to today?

中文:已经使用scripts/get_data.py准备数据,但是数据只包含到20200925。使用scripts/data_collector/yahoo/collector.py更新数据,但是报错,如下所示:
问题:如何更新数据到今天?

"""
S:\Python3.7\Lib\site-packages\qlib\qlib-main>python scripts/data_collector/yahoo/collector.py update_data_to_bin --qlib_data_1d_dir ~/.qlib/qlib_data/cn_data --trading_date 20200925 --end_date 20210917
2021-09-21 22:14:30.214 | INFO | collector:get_instrument_list:196 - get HS stock symbols......
2021-09-21 22:14:43.294 | INFO | collector:get_instrument_list:198 - get 4226 symbols.
2021-09-21 22:14:43.423 | INFO | data_collector.base:collector_data:204 - start collector data......
2021-09-21 22:14:43.424 | INFO | data_collector.base:collector_data:209 - getting data: 1
0%| | 0/4226 [00:00<?, ?it/s]2021-09-21 22:15:31.532 | WARNING | collector:get_data_from_remote:141 - 000001.sz-1d-2020-09-25 00:00:00+08:00-2021-09-17 00:00:00+08:00:HTTPSConnectionPool(host='query2.finance.yahoo.com', port=443): Max retries exceeded with url: /v8/finance/chart/000001.sz?period1=1600963200&period2=1631808000&interval=1d&events=div%2Csplit&formatted=false&lang=en-US&region=US&corsDomain=finance.yahoo.com (Caused by ReadTimeoutError("HTTPSConnectionPool(host='query2.finance.yahoo.com', port=443): Read timed out. (read timeout=5)"))
2021-09-21 22:15:31.536 | WARNING | data_collector.utils:wrapper:429 - _get_simple: 1 :get data error: 000001.sz--2020-09-25 00:00:00+08:00--2021-09-17 00:00:00+08:00
"""

@prevailings prevailings added the question Further information is requested label Sep 21, 2021
@zhupr
Copy link
Collaborator

zhupr commented Sep 21, 2021

@prevailings Hi,
The v2 dataset is dated to 2021-06-11:

  • python scripts/get_data.py qlib_data --target_dir cn_data_v2 --version v2

scripts/data_collector/yahoo/collector.py: using Yahoo Finance API, the network is not very stable, you can try to change the network

@prevailings
Copy link
Author

Have downloaded v2 dataset.
Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants