From c539d81223512752d051e855acf25713691adfa5 Mon Sep 17 00:00:00 2001
From: Jactus <dw1920@nyu.edu>
Date: Fri, 25 Sep 2020 11:57:06 +0800
Subject: [PATCH 1/3] Add docs for explaining prepared dataset

---
 README.md                     | 24 +++++++++++++-----------
 docs/start/initialization.rst |  2 +-
 scripts/get_data.py           |  1 +
 3 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/README.md b/README.md
index c9cb709793..04657df9ec 100644
--- a/README.md
+++ b/README.md
@@ -11,16 +11,16 @@ With Qlib, you can easily try your ideas to create better Quant investment strat
 For more details, please refer to our paper ["Qlib: An AI-oriented Quantitative Investment Platform"](https://arxiv.org/abs/2009.11189).
 
 
-- [Framework of Qlib](#Framework-of-Qlib)
-- [Quick Start](#Quick-Start)
-  - [Installation](#Installation)
-  - [Data Preparation](#Data-Preparation)
-  - [Auto Quant Research Workflow with](#Auto-Quant-Research-Workflow)
-  - [Building Customized Quant Research Workflow by Code](#Building-Customized-Quant-Research-Workflow-by-Code)
-- [More About Qlib](#More-About-Qlib)
-- [Offline Mode and Online Mode](#Offline-Mode-and-Online-Mode)
-  - [Performance of Qlib Data Server](#Performance-of-Qlib-Data-Server)
-- [Contributing](#Contributing)
+- [Framework of Qlib](#framework-of-qlib)
+- [Quick Start](#quick-start)
+  - [Installation](#installation)
+  - [Data Preparation](#data-preparation)
+  - [Auto Quant Research Workflow](#auto-quant-research-workflow)
+  - [Building Customized Quant Research Workflow by Code](#building-customized-quant-research-workflow-by-code)
+- [More About Qlib](#more-about-qlib)
+- [Offline Mode and Online Mode](#offline-mode-and-online-mode)
+  - [Performance of Qlib Data Server](#performance-of-qlib-data-server)
+- [Contributing](#contributing)
 
 
 
@@ -77,7 +77,9 @@ Load and prepare data by running the following code:
 
 This dataset is created by public data collected by [crawler scripts](scripts/data_collector/), which have been released in
 the same repository.
-Users could create the same dataset with it.
+Users could create the same dataset with it. 
+
+*Please pay **ATTENTION** that the data is collected from [Yahoo Finance](https://finance.yahoo.com/lookup) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset.*
 
 <!-- 
 - Run the initialization code and get stock data:
diff --git a/docs/start/initialization.rst b/docs/start/initialization.rst
index 5b50e1eba7..863642ed04 100644
--- a/docs/start/initialization.rst
+++ b/docs/start/initialization.rst
@@ -11,7 +11,7 @@ Initialization
 
 Please follow the steps below to initialize ``Qlib``.
 
-- Download and prepare the Data: execute the following command to download stock data.
+- Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to use their own data if they have high-quality dataset.
     .. code-block:: bash
     
         python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data
diff --git a/scripts/get_data.py b/scripts/get_data.py
index 0ab40df1c3..d279536313 100644
--- a/scripts/get_data.py
+++ b/scripts/get_data.py
@@ -34,6 +34,7 @@ def _download_data(self, file_name: str, target_dir: [Path, str]):
             raise requests.exceptions.HTTPError()
 
         chuck_size = 1024
+        logger.warning(f"The data for the example is collected from Yahoo Finance. Please be aware that the quality of the data might not be perfect. (You can refer to the original data source: https://finance.yahoo.com/lookup.)")
         logger.info(f"{file_name} downloading......")
         with tqdm(total=int(resp.headers.get("Content-Length", 0))) as p_bar:
             with target_path.open("wb") as fp:

From 8ec4e9b5e4b31aeaaa8a61bbc47728cf5f6d6adb Mon Sep 17 00:00:00 2001
From: Jactus <dw1920@nyu.edu>
Date: Fri, 25 Sep 2020 12:52:04 +0800
Subject: [PATCH 2/3] Add doc links for customized dataset

---
 README.md                     | 2 +-
 docs/start/initialization.rst | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 04657df9ec..a893e9ac28 100644
--- a/README.md
+++ b/README.md
@@ -79,7 +79,7 @@ This dataset is created by public data collected by [crawler scripts](scripts/da
 the same repository.
 Users could create the same dataset with it. 
 
-*Please pay **ATTENTION** that the data is collected from [Yahoo Finance](https://finance.yahoo.com/lookup) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset.*
+*Please pay **ATTENTION** that the data is collected from [Yahoo Finance](https://finance.yahoo.com/lookup) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset. For more information, users can refer to the [related document](https://qlib.readthedocs.io/en/latest/component/data.html)*
 
 <!-- 
 - Run the initialization code and get stock data:
diff --git a/docs/start/initialization.rst b/docs/start/initialization.rst
index 863642ed04..09b6b22d71 100644
--- a/docs/start/initialization.rst
+++ b/docs/start/initialization.rst
@@ -11,7 +11,7 @@ Initialization
 
 Please follow the steps below to initialize ``Qlib``.
 
-- Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to use their own data if they have high-quality dataset.
+- Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to prepare their own data if they have high-quality datasets. Please refer to `Data  <../component/data.html>` for more information about customized dataset.
     .. code-block:: bash
     
         python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data

From 40b9203df29ce4bfedec7ba18a5e479b30fbb500 Mon Sep 17 00:00:00 2001
From: Jactus <dw1920@nyu.edu>
Date: Fri, 25 Sep 2020 13:14:56 +0800
Subject: [PATCH 3/3] Add accurate links for reference

---
 README.md                     | 2 +-
 docs/start/initialization.rst | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index a893e9ac28..061647a2b4 100644
--- a/README.md
+++ b/README.md
@@ -79,7 +79,7 @@ This dataset is created by public data collected by [crawler scripts](scripts/da
 the same repository.
 Users could create the same dataset with it. 
 
-*Please pay **ATTENTION** that the data is collected from [Yahoo Finance](https://finance.yahoo.com/lookup) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset. For more information, users can refer to the [related document](https://qlib.readthedocs.io/en/latest/component/data.html)*
+*Please pay **ATTENTION** that the data is collected from [Yahoo Finance](https://finance.yahoo.com/lookup) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset. For more information, users can refer to the [related document](https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format)*
 
 <!-- 
 - Run the initialization code and get stock data:
diff --git a/docs/start/initialization.rst b/docs/start/initialization.rst
index 09b6b22d71..24b270d628 100644
--- a/docs/start/initialization.rst
+++ b/docs/start/initialization.rst
@@ -11,7 +11,7 @@ Initialization
 
 Please follow the steps below to initialize ``Qlib``.
 
-- Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to prepare their own data if they have high-quality datasets. Please refer to `Data  <../component/data.html>` for more information about customized dataset.
+- Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to prepare their own data if they have high-quality datasets. Please refer to `Data  <../component/data.html#converting-csv-format-into-qlib-format>` for more information about customized dataset.
     .. code-block:: bash
     
         python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data