From 80f43630cdc26ef4b8f149cfbfe07e826a3afbeb Mon Sep 17 00:00:00 2001 From: Young Date: Mon, 30 Nov 2020 04:21:33 +0000 Subject: [PATCH 1/3] refine docs --- docs/component/backtest.rst | 36 ++++++++++++++++++------------------ docs/component/data.rst | 9 ++++++--- docs/component/model.rst | 8 ++++---- docs/component/recorder.rst | 7 +++---- docs/component/strategy.rst | 23 ++++++++++++----------- docs/index.rst | 4 ++-- docs/introduction/quick.rst | 2 +- docs/start/integration.rst | 4 ++-- 8 files changed, 48 insertions(+), 45 deletions(-) diff --git a/docs/component/backtest.rst b/docs/component/backtest.rst index d36dba3161..88e01e2de4 100644 --- a/docs/component/backtest.rst +++ b/docs/component/backtest.rst @@ -13,7 +13,7 @@ Introduction .. note:: - ``Intraday Trading`` uses ``Order Executor`` to trade and execute orders output by ``Interday Strategy``. ``Order Executor`` is a component in `Qlib Framework <../introduction/introduction.html#framework>`_, which can execute orders. ``Vwap Executor`` and ``Close Executor`` is supported by ``Qlib`` now. In the future, ``Qlib`` will support ``HighFreq Executor`` also. + ``Intraday Trading`` uses ``Order Executor`` to trade and execute orders output by ``Portfolio Strategy``. ``Order Executor`` is a component in `Qlib Framework <../introduction/introduction.html#framework>`_, which can execute orders. ``VWAP Executor`` and ``Close Executor`` is supported by ``Qlib`` now. In the future, ``Qlib`` will support ``HighFreq Executor`` also. @@ -32,34 +32,34 @@ The simple example of the default strategy is as follows. # pred_score is the prediction score report, positions = backtest(pred_score, topk=50, n_drop=0.5, verbose=False, limit_threshold=0.0095) -To know more about backtesting with a specific ``Strategy``, please refer to `Strategy `_. +To know more about backtesting with a specific ``Strategy``, please refer to `Portfolio Strategy `_. -To know more about the prediction score `pred_score` output by ``Interday Model``, please refer to `Interday Model: Model Training & Prediction `_. +To know more about the prediction score `pred_score` output by ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction `_. Prediction Score ----------------- -The `prediction score` is a pandas DataFrame. Its index is and it must +The `prediction score` is a pandas DataFrame. Its index is and it must contains a `score` column. A prediction sample is shown as follows. .. code-block:: python - instrument datetime score - SH600000 2019-01-04 -0.505488 - SZ002531 2019-01-04 -0.320391 - SZ000999 2019-01-04 0.583808 - SZ300569 2019-01-04 0.819628 - SZ001696 2019-01-04 -0.137140 - ... ... - SZ000996 2019-04-30 -1.027618 - SH603127 2019-04-30 0.225677 - SH603126 2019-04-30 0.462443 - SH603133 2019-04-30 -0.302460 - SZ300760 2019-04-30 -0.126383 - -``Interday Model`` module can make predictions, please refer to `Interday Model: Model Training & Prediction `_. + datetime instrument score + 2019-01-04 SH600000 -0.505488 + 2019-01-04 SZ002531 -0.320391 + 2019-01-04 SZ000999 0.583808 + 2019-01-04 SZ300569 0.819628 + 2019-01-04 SZ001696 -0.137140 + ... ... + 2019-04-30 SZ000996 -1.027618 + 2019-04-30 SH603127 0.225677 + 2019-04-30 SH603126 0.462443 + 2019-04-30 SH603133 -0.302460 + 2019-04-30 SZ300760 -0.126383 + +``Forecast Model`` module can make predictions, please refer to `Forecast Model: Model Training & Prediction `_. Backtest Result ------------------ diff --git a/docs/component/data.rst b/docs/component/data.rst index 55d6c72077..ed95c9bf77 100644 --- a/docs/component/data.rst +++ b/docs/component/data.rst @@ -206,7 +206,7 @@ Data Loader QlibDataLoader --------------- -The ``QlibDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from the data source. +The ``QlibDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from the ``Qlib`` data source. Interface ------------ @@ -234,7 +234,7 @@ DataHandlerLP In addition to use ``Data Handler`` in an automatic workflow with ``qrun``, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data (standardization, remove NaN, etc.) and build datasets. -In order to achieve so, ``Qlib`` provides a base class `qlib.data.dataset.DataHandlerLP <../reference/api.html#qlib.data.dataset.handler.DataHandlerLP>`_. The core idea of this class is that: we will have some leanable ``Processors`` which can learn the parameters of data processing. When new data comes in, these `trained` ``Processors`` can then infer on the new data and thus processing real-time data in an efficient way. More information about ``Processors`` will be listed in the next subsection. +In order to achieve so, ``Qlib`` provides a base class `qlib.data.dataset.DataHandlerLP <../reference/api.html#qlib.data.dataset.handler.DataHandlerLP>`_. The core idea of this class is that: we will have some leanable ``Processors`` which can learn the parameters of data processing(e.g., parameters for zscore normalization). When new data comes in, these `trained` ``Processors`` can then process the new data and thus processing real-time data in an efficient way becomes possible. More information about ``Processors`` will be listed in the next subsection. Interface @@ -321,7 +321,10 @@ Dataset The ``Dataset`` module in ``Qlib`` aims to prepare data for model training and inferencing. -The motivation of this module is that we want to maximize the flexibility of of different models to handle data that are suitable for themselves. This module gives the model the rights to process their data in an unique way. For instance, models such as ``GBDT`` may work well on data that contains `nan` or `None` value, while neural networks such as ``MLP`` will break down on such data. +The motivation of this module is that we want to maximize the flexibility of of different models to handle data that are suitable for themselves. This module gives the model the flexibility to process their data in an unique way. For instance, models such as ``GBDT`` may work well on data that contains `nan` or `None` value, while neural networks such as ``MLP`` will break down on such data. + +If user's model need process its data in a different way, user could implement his own ``Dataset`` class. If the model's +data processing is not special, ``DatasetH`` can be used directly. The ``DatasetH`` class is the `dataset` with `Data Handler`. Here is the most important interface of the class: diff --git a/docs/component/model.rst b/docs/component/model.rst index e4aa4ca915..96fb93945f 100644 --- a/docs/component/model.rst +++ b/docs/component/model.rst @@ -1,15 +1,15 @@ .. _model: ============================================ -Interday Model: Model Training & Prediction +Forecast Model: Model Training & Prediction ============================================ Introduction =================== -``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``qrun``, please refer to `Workflow: Workflow Management `_. +``Forecast Model`` is designed to make the `prediction score` about stocks. Users can use the ``Forecast Model`` in an automatic workflow by ``qrun``, please refer to `Workflow: Workflow Management `_. -Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Model`` can be used as an independent module also. +Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Forecast Model`` can be used as an independent module also. Base Class & Interface ====================== @@ -63,7 +63,7 @@ For other interfaces such as `finetune`, please refer to `Model API <../referenc Example ================== -``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are treated as the baselines of ``Interday Model``. The following steps show how to run`` LightGBM`` as an independent module. +``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are treated as the baselines of ``Forecast Model``. The following steps show how to run`` LightGBM`` as an independent module. - Initialize ``Qlib`` with `qlib.init` first, please refer to `Initialization <../start/initialization.html>`_. - Run the following code to get the `prediction score` `pred_score` diff --git a/docs/component/recorder.rst b/docs/component/recorder.rst index 4304dcce50..a7516587c5 100644 --- a/docs/component/recorder.rst +++ b/docs/component/recorder.rst @@ -7,7 +7,7 @@ Qlib Recorder: Experiment Management Introduction =================== -``Qlib`` contains an experiment management system named ``QlibRecorder``, which is designed to help users handle experiment and analysis results in an efficient way. +``Qlib`` contains an experiment management system named ``QlibRecorder``, which is designed to help users handle experiment and analyse results in an efficient way. There are three components of the system: @@ -34,8 +34,7 @@ Here is a general view of the structure of the system: - Recorder 2 - ... - ... - -Currently, the components of this experiment management system are implemented using the machine learning platform: ``MLFlow`` (`link `_). +This experiment management system defines a set of interface and provided a concrete implementation based on the machine learning platform: ``MLFlow`` (`link `_). Qlib Recorder @@ -94,4 +93,4 @@ The ``RecordTemp`` class is a class that enables generate experiment results suc - ``SigAnaRecord``: This class generates the `IC`, `ICIR`, `Rank IC` and `Rank ICIR` of the model. - ``PortAnaRecord``: This class generates the results of `backtest`. The detailed information about `backtest` as well as the available `strategy`, users can refer to `Strategy <../component/strategy.html>`_ and `Backtest <../component/backtest.html>`_. -For more information about the APIs, please refer to `Record Template API <../reference/api.html#module-qlib.workflow.record_temp>`_. \ No newline at end of file +For more information about the APIs, please refer to `Record Template API <../reference/api.html#module-qlib.workflow.record_temp>`_. diff --git a/docs/component/strategy.rst b/docs/component/strategy.rst index 0bdf453fe4..0720dcdade 100644 --- a/docs/component/strategy.rst +++ b/docs/component/strategy.rst @@ -1,18 +1,18 @@ .. _strategy: ======================================== -Interday Strategy: Portfolio Management +Portfolio Strategy: Portfolio Management ======================================== .. currentmodule:: qlib Introduction =================== -``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management `_. +``Portfolio Strategy`` is designed to adopt different portfolio strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Forecast Model``. Users can use the ``Portfolio Strategy`` in an automatic workflow by ``Workflow`` module, please refer to `Workflow: Workflow Management `_. -Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Strategy`` can be used as an independent module also. +Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Portfolio Strategy`` can be used as an independent module also. -``Qlib`` provides several implemented trading strategies. Also, ``Qlib`` supports custom strategy, users can customize strategies according to their own needs. +``Qlib`` provides several implemented portfolio strategies. Also, ``Qlib`` supports custom strategy, users can customize strategies according to their own needs. Base Class & Interface ====================== @@ -26,19 +26,20 @@ Qlib provides a base class ``qlib.contrib.strategy.BaseStrategy``. All strategy Return the proportion of your total value you will use in investment. Dynamically risk_degree will result in Market timing. - `generate_order_list` - Rerturn the order list. + Return the order list. Users can inherit `BaseStrategy` to customize their strategy class. WeightStrategyBase -------------------- -Qlib alse provides a class ``qlib.contrib.strategy.WeightStrategyBase`` that is a subclass of `BaseStrategy`. +Qlib also provides a class ``qlib.contrib.strategy.WeightStrategyBase`` that is a subclass of `BaseStrategy`. `WeightStrategyBase` only focuses on the target positions, and automatically generates an order list based on positions. It provides the `generate_target_weight_position` interface. - `generate_target_weight_position` - - According to the current position and trading date to generate the target position. The cash is not considered. + - According to the current position and trading date to generate the target position. The cash is not considered in + the output weight distribution. - Return the target position. .. note:: @@ -81,7 +82,7 @@ TopkDropoutStrategy Usage & Example ==================== -``Interday Strategy`` can be specified in the ``Intraday Trading(Backtest)``, the example is as follows. +``Portfolio Strategy`` can be specified in the ``Intraday Trading(Backtest)``, the example is as follows. .. code-block:: python @@ -110,12 +111,12 @@ Usage & Example pred_score, strategy=strategy, **BACKTEST_CONFIG ) -Also, the above example has been given in ``examples\train_backtest_analyze.ipynb``. +Also, the above example has been given in ``examples/train_backtest_analyze.ipynb``. -To know more about the `prediction score` `pred_score` output by ``Interday Model``, please refer to `Interday Model: Model Training & Prediction `_. +To know more about the `prediction score` `pred_score` output by ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction `_. To know more about ``Intraday Trading``, please refer to `Intraday Trading: Model&Strategy Testing `_. Reference =================== -To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html#module-qlib.contrib.strategy.strategy>`_. +To know more about ``Portfolio Strategy``, please refer to `Strategy API <../reference/api.html#module-qlib.contrib.strategy.strategy>`_. diff --git a/docs/index.rst b/docs/index.rst index 1e43cf99e6..15a36b4892 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -37,8 +37,8 @@ Document Structure Workflow: Workflow Management Data Layer: Data Framework&Usage - Interday Model: Model Training & Prediction - Interday Strategy: Portfolio Management + Forecast Model: Model Training & Prediction + Strategy: Portfolio Management Intraday Trading: Model&Strategy Testing Qlib Recorder: Experiment Management Analysis: Evaluation & Results Analysis diff --git a/docs/introduction/quick.rst b/docs/introduction/quick.rst index f228ce2af1..1abe3fe762 100644 --- a/docs/introduction/quick.rst +++ b/docs/introduction/quick.rst @@ -91,4 +91,4 @@ Auto Quant Research Workflow Custom Model Integration =============================================== -``Qlib`` provides a batch of models (such as ``lightGBM`` and ``MLP`` models) as examples of ``Interday Model``. In addition to the default model, users can integrate their own custom models into ``Qlib``. If users are interested in the custom model, please refer to `Custom Model Integration <../start/integration.html>`_. +``Qlib`` provides a batch of models (such as ``lightGBM`` and ``MLP`` models) as examples of ``Forecast Model``. In addition to the default model, users can integrate their own custom models into ``Qlib``. If users are interested in the custom model, please refer to `Custom Model Integration <../start/integration.html>`_. diff --git a/docs/start/integration.rst b/docs/start/integration.rst index e36805c012..2f3adb15fd 100644 --- a/docs/start/integration.rst +++ b/docs/start/integration.rst @@ -5,7 +5,7 @@ Custom Model Integration Introduction =================== -``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are examples of ``Interday Model``. In addition to the default models ``Qlib`` provide, users can integrate their own custom models into ``Qlib``. +``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are examples of ``Forecast Model``. In addition to the default models ``Qlib`` provide, users can integrate their own custom models into ``Qlib``. Users can integrate their own custom models according to the following steps. @@ -141,4 +141,4 @@ Also, ``Model`` can also be tested as a single module. An example has been given Reference ===================== -To know more about ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.model.base>`_. +To know more about ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.model.base>`_. From ff9d48fab88d9bf1245ca066e4ac33a8a7db3932 Mon Sep 17 00:00:00 2001 From: Jactus Date: Mon, 30 Nov 2020 18:54:31 +0800 Subject: [PATCH 2/3] Update docs --- docs/component/data.rst | 42 +++++++++++--- docs/component/model.rst | 41 +------------ docs/component/recorder.rst | 2 + docs/component/workflow.rst | 57 +++++++------------ docs/start/integration.rst | 17 +++--- .../ALSTM/workflow_config_alstm.yaml | 5 +- .../CatBoost/workflow_config_catboost.yaml | 5 +- .../benchmarks/GATs/workflow_config_gats.yaml | 5 +- .../benchmarks/GRU/workflow_config_gru.yaml | 5 +- .../benchmarks/LSTM/workflow_config_lstm.yaml | 5 +- .../LightGBM/workflow_config_lightgbm.yaml | 5 +- .../Linear/workflow_config_linear.yaml | 5 +- .../benchmarks/MLP/workflow_config_mlp.yaml | 5 +- .../benchmarks/SFM/workflow_config_sfm.yaml | 5 +- .../benchmarks/TFT/workflow_config_tft.yaml | 10 +++- .../XGBoost/workflow_config_xgboost.yaml | 5 +- qlib/model/base.py | 25 +++++++- qlib/model/trainer.py | 14 ++--- qlib/workflow/__init__.py | 43 +++++++------- qlib/workflow/cli.py | 6 +- qlib/workflow/exp.py | 12 ++-- qlib/workflow/expm.py | 20 +++---- 22 files changed, 180 insertions(+), 159 deletions(-) diff --git a/docs/component/data.rst b/docs/component/data.rst index ed95c9bf77..cb1103e722 100644 --- a/docs/component/data.rst +++ b/docs/component/data.rst @@ -29,7 +29,18 @@ Qlib Format Data ------------------ We've specially designed a data structure to manage financial data, please refer to the `File storage design section in Qlib paper `_ for detailed information. -Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data +Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data. + +``Qlib`` provides two different off-the-shelf dataset, which can be accessed through this `link `_: + +======================== ================= ================ +Dataset US Market China Market +======================== ================= ================ +Alpha360 √ √ + +Alpha158 √ √ +======================== ================= ================ + Qlib Format Dataset -------------------- @@ -45,7 +56,7 @@ In addition to China-Stock data, ``Qlib`` also includes a US-Stock dataset, whic python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/us_data --region us -After running the above command, users can find china-stock and us-stock data in Qlib format in the ``~/.qlib/csv_data/cn_data`` directory and ``~/.qlib/csv_data/us_data`` directory respectively. +After running the above command, users can find china-stock and us-stock data in ``Qlib`` format in the ``~/.qlib/csv_data/cn_data`` directory and ``~/.qlib/csv_data/us_data`` directory respectively. ``Qlib`` also provides the scripts in ``scripts/data_collector`` to help users crawl the latest data on the Internet and convert it to qlib format. @@ -54,8 +65,7 @@ When ``Qlib`` is initialized with this dataset, users could build and evaluate t Converting CSV Format into Qlib Format ------------------------------------------- -``Qlib`` has provided the script ``scripts/dump_bin.py`` to convert data in CSV format into `.bin` files (Qlib format). - +``Qlib`` has provided the script ``scripts/dump_bin.py`` to convert **any** data in CSV format into `.bin` files (``Qlib`` format) as long as they are in the correct format. Users can download the demo china-stock data in CSV format as follows for reference to the CSV format. @@ -130,9 +140,21 @@ After conversion, users can find their Qlib format data in the directory `~/.qli In the convention of `Qlib` data processing, `open, close, high, low, volume, money and factor` will be set to NaN if the stock is suspended. -China-Stock Mode & US-Stock Mode +Multiple Stock Modes -------------------------------- +``Qlib`` now provides two different stock modes for users: China-Stock Mode & US-Stock Mode. Here are some different settings of these two modes: + +============== ================= ================ +Region Trade Unit Limit Threshold +============== ================= ================ +China 100 0.099 + +US 1 None +============== ================= ================ + +The `trade unit` defines the unit number of stocks can be used in a trade, and the `limit threshold` defines the bound set to the percentage of ups and downs of a stock. + - If users use ``Qlib`` in china-stock mode, china-stock data is required. Users can use ``Qlib`` in china-stock mode according to the following steps: - Download china-stock in qlib format, please refer to section `Qlib Format Dataset <#qlib-format-dataset>`_. - Initialize ``Qlib`` in china-stock mode @@ -208,13 +230,19 @@ QlibDataLoader The ``QlibDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from the ``Qlib`` data source. +StaticDataLoader +--------------- + +The ``StaticDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from file or as provided. + + Interface ------------ Here are some interfaces of the ``QlibDataLoader`` class: -.. autoclass:: qlib.data.dataset.loader.QlibDataLoader - :members: load, load_group_df +.. autoclass:: qlib.data.dataset.loader.DataLoader + :members: API ----------- diff --git a/docs/component/model.rst b/docs/component/model.rst index 96fb93945f..d9d59a4d70 100644 --- a/docs/component/model.rst +++ b/docs/component/model.rst @@ -18,45 +18,10 @@ Base Class & Interface The base class provides the following interfaces: -- `__init__(**kwargs)` - - Initialization. - -- `fit(self, dataset, **kwargs)` - - Train model. - - Parameter: - - `dataset`, ``Qlib``'s ``DatasetH`` type. For more information about ``DatasetH``, users can refer to the related document: `Qlib Dataset <../component/data.html#dataset>`_. - The `dataset` is passed into the `model`'s method because there are some unique data preprocessing procedures for each, we want to give each model maximum flexibility to handle the data that is suitable for their own. - The following code example shows how to retrieve `x_train`, `y_train` and `w_train` from the `dataset`: - - .. code-block:: Python - - # get features and labels - df_train, df_valid = dataset.prepare( - ["train", "valid"], col_set=["feature", "label"], data_key=DataHandlerLP.DK_L - ) - x_train, y_train = df_train["feature"], df_train["label"] - x_valid, y_valid = df_valid["feature"], df_valid["label"] - - # get weights - try: - wdf_train, wdf_valid = dataset.prepare(["train", "valid"], col_set=["weight"], data_key=DataHandlerLP.DK_L) - w_train, w_valid = wdf_train["weight"], wdf_valid["weight"] - except KeyError as e: - w_train = pd.DataFrame(np.ones_like(y_train.values), index=y_train.index) - w_valid = pd.DataFrame(np.ones_like(y_valid.values), index=y_valid.index) - -- `predict(self, dataset, **kwargs)` - - Predict test data. - - Parameter: - - `dataset`, ``Qlib``'s ``DatasetH`` type. The usage is similar to the example above. - - Returns: - - Predic results with type: `pandas.Series`. - -- `finetune(self, dataset, **kwargs)` - - Finetune the model. - - Parameter: - - `dataset`, ``Qlib``'s ``DatasetH`` type. The usage is similar to the example above. +.. autoclass:: qlib.model.base.Model + :members: +``Qlib`` also provides a base class `qlib.model.base.ModelFT <../reference/api.html#qlib.model.base.ModelFT>`_, which includes the method for finetuning the model. For other interfaces such as `finetune`, please refer to `Model API <../reference/api.html#module-qlib.model.base>`_. diff --git a/docs/component/recorder.rst b/docs/component/recorder.rst index a7516587c5..baf12448b4 100644 --- a/docs/component/recorder.rst +++ b/docs/component/recorder.rst @@ -72,6 +72,8 @@ The ``Experiment`` class is solely responsible for a single experiment, and it w For other interfaces such as `search_records`, `delete_recorder`, please refer to `Experiment API <../reference/api.html#experiment>`_. +``Qlib`` also provides a default ``Experiment``, which will be created and used under certain situations when users use the APIs such as `log_metrics` or `get_exp`. If the default ``Experiment`` is used, there will be related logged information when running ``Qlib``. Users are able to change the name of the default ``Experiment`` in the config file of ``Qlib`` or during ``Qlib``'s `initialization <../start/initialization.html#parameters>`_, which is set to be '`Experiment`'. + Recorder =================== diff --git a/docs/component/workflow.rst b/docs/component/workflow.rst index c44f1100fd..5b81c7e789 100644 --- a/docs/component/workflow.rst +++ b/docs/component/workflow.rst @@ -11,8 +11,8 @@ Introduction The components in `Qlib Framework <../introduction/introduction.html#framework>`_ are designed in a loosely-coupled way. Users could build their own Quant research workflow with these components like `Example `_. -Besides, ``Qlib`` provides more user-friendly interfaces named ``qrun`` to automatically run the whole workflow defined by configuration. A concrete execution of the whole workflow is called an `experiment`. -With ``qrun``, user can easily run an `experiment`, which includes the following steps: +Besides, ``Qlib`` provides more user-friendly interfaces named ``qrun`` to automatically run the whole workflow defined by configuration. Running the whole workflow is called an `execution`. +With ``qrun``, user can easily start an `execution`, which includes the following steps: - Data - Loading @@ -25,7 +25,7 @@ With ``qrun``, user can easily run an `experiment`, which includes the following - Forecast signal analysis - Backtest -For each `experiment`, ``Qlib`` has a complete system to tracking all the information as well as artifacts generated during training, inference and evaluation phase. For more information about how Qlib handles `experiment`, please refer to the related document: `Recorder: Experiment Management <../component/recorder.html>`_. +For each `execution`, ``Qlib`` has a complete system to tracking all the information as well as artifacts generated during training, inference and evaluation phase. For more information about how ``Qlib`` handles this, please refer to the related document: `Recorder: Experiment Management <../component/recorder.html>`_. Complete Example =================== @@ -35,8 +35,9 @@ Below is a typical config file of ``qrun``. .. code-block:: YAML - provider_uri: "~/.qlib/qlib_data/cn_data" - region: cn + qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config @@ -100,12 +101,16 @@ After saving the config into `configuration.yaml`, users could start the workflo .. code-block:: bash - qrun -c configuration.yaml + qrun configuration.yaml .. note:: `qrun` will be placed in your $PATH directory when installing ``Qlib``. +.. note:: + + The symbol `&` in `yaml` file stands for an anchor of a field, which is useful when another fields include this parameter as part of the value. Taking the configuration file above as an example, users can directly change the value of `market` and `benchmark` without traversing the entire configuration file. + Configuration File =================== @@ -114,17 +119,15 @@ Let's get into details of ``qrun`` in this section. Before using ``qrun``, users need to prepare a configuration file. The following content shows how to prepare each part of the configuration file. -Qlib Data Section +Qlib Init Section -------------------- -At first, the configuration file needs to contain several basic parameters about the data, which will be used for qlib initialization, data handling and backtest. +At first, the configuration file needs to contain several basic parameters which will be used for qlib initialization. .. code-block:: YAML provider_uri: "~/.qlib/qlib_data/cn_data" region: cn - market: &market csi300 - benchmark: &benchmark SH000300 The meaning of each field is as follows: @@ -139,34 +142,14 @@ The meaning of each field is as follows: The value of `region` should be aligned with the data stored in `provider_uri`. -- `market` - Type: str. Index name, the default value is `csi500`. - -- `benchmark` - Type: str, list or pandas.Series. Stock index symbol, the default value is `SH000905`. - - .. note:: - * If `benchmark` is str, it will use the daily change as the 'bench'. +Task Section +-------------------- - * If `benchmark` is list, it will use the daily average change of the stock pool in the list as the 'bench'. - - * If `benchmark` is pandas.Series, whose `index` is trading date and the value T is the change from T-1 to T, it will be directly used as the 'bench'. An example is as following: - - .. code-block:: python - - print(D.features(D.instruments('csi500'), ['$close/Ref($close, 1)-1'])['$close/Ref($close, 1)-1'].head()) - 2017-01-04 0.011693 - 2017-01-05 0.000721 - 2017-01-06 -0.004322 - 2017-01-09 0.006874 - 2017-01-10 -0.003350 -.. note:: - - The symbol `&` in `yaml` file stands for an anchor of a field, which is useful when another fields include this parameter as part of the value. Taking the configuration file above as an example, users can directly change the value of `market` and `benchmark` without traversing the entire configuration file. +The `task` field in the configuration corresponds to a `task`, which contains the parameters of three different subsections: `Model`, `Dataset` and `Record`. Model Section --------------------- +~~~~~~~~~~~~~~~~~~~~ In the `task` field, the `model` section describes the parameters of the model to be used for training and inference. For more information about the base ``Model`` class, please refer to `Qlib Model <../component/model.html>`_. @@ -202,7 +185,7 @@ The meaning of each field is as follows: ``Qlib`` provides a util named: ``init_instance_by_config`` to initialize any class inside ``Qlib`` with the configuration includes the fields: `class`, `module_path` and `kwargs`. Dataset Section --------------------- +~~~~~~~~~~~~~~~~~~~~ The `dataset` field describes the parameters for the ``Dataset`` module in ``Qlib`` as well those for the module ``DataHandler``. For more information about the ``Dataset`` module, please refer to `Qlib Model <../component/data.html#dataset>`_. @@ -237,9 +220,9 @@ Here is the configuration for the ``Dataset`` module which will take care of dat test: [2017-01-01, 2020-08-01] Record Section --------------------- +~~~~~~~~~~~~~~~~~~~~ -The `record` field is about the parameters the ``Record`` module in ``Qlib``. ``Record`` is responsible for generating certain analysis and evaluation results such as `prediction`, `information Coefficient (IC)` and `backtest`. +The `record` field is about the parameters the ``Record`` module in ``Qlib``. ``Record`` is responsible for tracking training process and results such as `information Coefficient (IC)` and `backtest` in a standard format. The following script is the configuration of `backtest` and the `strategy` used in `backtest`: diff --git a/docs/start/integration.rst b/docs/start/integration.rst index 2f3adb15fd..3ecae1090a 100644 --- a/docs/start/integration.rst +++ b/docs/start/integration.rst @@ -19,8 +19,8 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html# - Override the `__init__` method - ``Qlib`` passes the initialized parameters to the \_\_init\_\_ method. - - The parameter must be consistent with the hyperparameters in the configuration file. - - Code Example: In the following example, the hyperparameter filed of the configuration file should contain parameters such as `loss:mse`. + - The hyperparameters of model in the configuration must be consistent with those defined in the `__init__` method. + - Code Example: In the following example, the hyperparameters of model in the configuration file should contain parameters such as `loss:mse`. .. code-block:: Python def __init__(self, loss='mse', **kwargs): @@ -31,9 +31,9 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html# self._model = None - Override the `fit` method - - ``Qlib`` calls the fit method to train the model - - The parameters must include training feature `dataset`. - - The parameters could include some optional parameters with default values, such as `num_boost_round = 1000` for `GBDT`. + - ``Qlib`` calls the fit method to train the model. + - The parameters must include training feature `dataset`, which is designed in the interface. + - The parameters could include some `optional` parameters with default values, such as `num_boost_round = 1000` for `GBDT`. - Code Example: In the following example, `num_boost_round = 1000` is an optional parameter. .. code-block:: Python @@ -69,7 +69,7 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html# ) - Override the `predict` method - - The parameters must include training feature `dataset`, which will be userd to get the test dataset. + - The parameters must include the parameter `dataset`, which will be userd to get the test dataset. - Return the `prediction score`. - Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_ for the parameter types of the fit method. - Code Example: In the following example, users need to use `LightGBM` to predict the label(such as `preds`) of test data `x_test` and return it. @@ -81,8 +81,9 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html# x_test = dataset.prepare("test", col_set="feature", data_key=DataHandlerLP.DK_I) return pd.Series(self.model.predict(x_test.values), index=x_test.index) -- Override the `finetune` method - - The parameters must include training feature `dataset`. +- Override the `finetune` method (Optional) + - This method is optional to the users, and when users one to use this method on their own models, they should inherit the ``ModelFT`` base class, which includes the interface of `finetune`. + - The parameters must include the parameter `dataset`. - Code Example: In the following example, users will use `LightGBM` as the model and finetune it. .. code-block:: Python diff --git a/examples/benchmarks/ALSTM/workflow_config_alstm.yaml b/examples/benchmarks/ALSTM/workflow_config_alstm.yaml index dd57761f36..66367034dd 100644 --- a/examples/benchmarks/ALSTM/workflow_config_alstm.yaml +++ b/examples/benchmarks/ALSTM/workflow_config_alstm.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/CatBoost/workflow_config_catboost.yaml b/examples/benchmarks/CatBoost/workflow_config_catboost.yaml index 9c15dc25be..af556dc871 100644 --- a/examples/benchmarks/CatBoost/workflow_config_catboost.yaml +++ b/examples/benchmarks/CatBoost/workflow_config_catboost.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/GATs/workflow_config_gats.yaml b/examples/benchmarks/GATs/workflow_config_gats.yaml index c38b4b3129..95e0d06d10 100644 --- a/examples/benchmarks/GATs/workflow_config_gats.yaml +++ b/examples/benchmarks/GATs/workflow_config_gats.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/GRU/workflow_config_gru.yaml b/examples/benchmarks/GRU/workflow_config_gru.yaml index bdfcd4e55d..381581a77c 100644 --- a/examples/benchmarks/GRU/workflow_config_gru.yaml +++ b/examples/benchmarks/GRU/workflow_config_gru.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/LSTM/workflow_config_lstm.yaml b/examples/benchmarks/LSTM/workflow_config_lstm.yaml index 6512a0df38..cb3b2a7894 100644 --- a/examples/benchmarks/LSTM/workflow_config_lstm.yaml +++ b/examples/benchmarks/LSTM/workflow_config_lstm.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml b/examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml index 790fc3ae55..76a6347db0 100644 --- a/examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml +++ b/examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/Linear/workflow_config_linear.yaml b/examples/benchmarks/Linear/workflow_config_linear.yaml index 70d3eaf682..ef2fee4c55 100644 --- a/examples/benchmarks/Linear/workflow_config_linear.yaml +++ b/examples/benchmarks/Linear/workflow_config_linear.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/MLP/workflow_config_mlp.yaml b/examples/benchmarks/MLP/workflow_config_mlp.yaml index e01c4eb3a6..f9bfb46e49 100644 --- a/examples/benchmarks/MLP/workflow_config_mlp.yaml +++ b/examples/benchmarks/MLP/workflow_config_mlp.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/SFM/workflow_config_sfm.yaml b/examples/benchmarks/SFM/workflow_config_sfm.yaml index 3fa3f932c9..edf176e62a 100644 --- a/examples/benchmarks/SFM/workflow_config_sfm.yaml +++ b/examples/benchmarks/SFM/workflow_config_sfm.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/examples/benchmarks/TFT/workflow_config_tft.yaml b/examples/benchmarks/TFT/workflow_config_tft.yaml index d8ee14e71f..dba37ab637 100644 --- a/examples/benchmarks/TFT/workflow_config_tft.yaml +++ b/examples/benchmarks/TFT/workflow_config_tft.yaml @@ -1,7 +1,8 @@ sys: rel_path: . -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config @@ -46,6 +47,11 @@ task: - class: SignalRecord module_path: qlib.workflow.record_temp kwargs: {} + - class: SigAnaRecord + module_path: qlib.workflow.record_temp + kwargs: + ana_long_short: False + ann_scaler: 252 - class: PortAnaRecord module_path: qlib.workflow.record_temp kwargs: diff --git a/examples/benchmarks/XGBoost/workflow_config_xgboost.yaml b/examples/benchmarks/XGBoost/workflow_config_xgboost.yaml index 1352c496d0..4caaa6f622 100644 --- a/examples/benchmarks/XGBoost/workflow_config_xgboost.yaml +++ b/examples/benchmarks/XGBoost/workflow_config_xgboost.yaml @@ -1,5 +1,6 @@ -provider_uri: "~/.qlib/qlib_data/cn_data" -region: cn +qlib_init: + provider_uri: "~/.qlib/qlib_data/cn_data" + region: cn market: &market csi300 benchmark: &benchmark SH000300 data_handler_config: &data_handler_config diff --git a/qlib/model/base.py b/qlib/model/base.py index c9bef11526..4a81d5a314 100644 --- a/qlib/model/base.py +++ b/qlib/model/base.py @@ -27,13 +27,32 @@ def fit(self, dataset: Dataset): .. note:: - The the attribute names of learned model should `not` start with '_'. So that the model could be + The attribute names of learned model should `not` start with '_'. So that the model could be dumped to disk. Parameters ---------- dataset : Dataset dataset will generate the processed data from model training. + + The following code example shows how to retrieve `x_train`, `y_train` and `w_train` from the `dataset`: + + .. code-block:: Python + + # get features and labels + df_train, df_valid = dataset.prepare( + ["train", "valid"], col_set=["feature", "label"], data_key=DataHandlerLP.DK_L + ) + x_train, y_train = df_train["feature"], df_train["label"] + x_valid, y_valid = df_valid["feature"], df_valid["label"] + + # get weights + try: + wdf_train, wdf_valid = dataset.prepare(["train", "valid"], col_set=["weight"], data_key=DataHandlerLP.DK_L) + w_train, w_valid = wdf_train["weight"], wdf_valid["weight"] + except KeyError as e: + w_train = pd.DataFrame(np.ones_like(y_train.values), index=y_train.index) + w_valid = pd.DataFrame(np.ones_like(y_valid.values), index=y_valid.index) """ raise NotImplementedError() @@ -45,6 +64,10 @@ def predict(self, dataset: Dataset) -> object: ---------- dataset : Dataset dataset will generate the processed dataset from model training. + + Returns + ------- + Prediction results with certain type such as `pandas.Series`. """ raise NotImplementedError() diff --git a/qlib/model/trainer.py b/qlib/model/trainer.py index 0ef0620218..305cf9ed2d 100644 --- a/qlib/model/trainer.py +++ b/qlib/model/trainer.py @@ -6,29 +6,29 @@ from qlib.workflow.record_temp import SignalRecord -def task_train(config: dict, experiment_name): +def task_train(task_config: dict, experiment_name): """ task based training Parameters ---------- - config : dict - A dict describing the training process + task_config : dict + A dict describes a task setting. """ # model initiaiton - model = init_instance_by_config(config.get("task")["model"]) - dataset = init_instance_by_config(config.get("task")["dataset"]) + model = init_instance_by_config(task_config["model"]) + dataset = init_instance_by_config(task_config["dataset"]) # start exp with R.start(experiment_name=experiment_name): # train model - R.log_params(**flatten_dict(config.get("task"))) + R.log_params(**flatten_dict(task_config)) model.fit(dataset) recorder = R.get_recorder() # generate records: prediction, backtest, and analysis - for record in config.get("task")["record"]: + for record in task_config.get["record"]: if record["class"] == SignalRecord.__name__: srconf = {"model": model, "dataset": dataset, "recorder": recorder} record["kwargs"].update(srconf) diff --git a/qlib/workflow/__init__.py b/qlib/workflow/__init__.py index c0745f6d45..e65bfb03ff 100644 --- a/qlib/workflow/__init__.py +++ b/qlib/workflow/__init__.py @@ -90,7 +90,11 @@ def end_exp(self, recorder_status=Recorder.STATUS_FI): def search_records(self, experiment_ids, **kwargs): """ - Get a pandas DataFrame of records that fit the search criteria. Here is the example code of the method: + Get a pandas DataFrame of records that fit the search criteria. + + The arguments of this function are not set to be rigid, and they will be different with different implementation of + ``ExpManager`` in ``Qlib``. ``Qlib`` now provides an implementation of ``ExpManager`` with mlflow, and here is the + example code of the this method with the ``MLflowExpManager``: .. code-block:: Python @@ -139,7 +143,8 @@ def list_recorders(self, experiment_id=None, experiment_name=None): If user doesn't provide the id or name of the experiment, this method will try to retrieve the default experiment and list all the recorders of the default experiment. If the default experiment doesn't exist, the method will first - create the default experiment, and then create a new recorder under it. + create the default experiment, and then create a new recorder under it. (More information about the default experiment + can be found `here <../component/recorder.html#qlib.workflow.exp.Experiment>`_). Here is the example code: @@ -168,27 +173,27 @@ def get_exp(self, experiment_id=None, experiment_name=None, create: bool = True) - If '`create`' is True: - - If ``R``'s running: + - If `active experiment` exists: - no id or name specified, return the active experiment. - - if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be running. + - if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be active. - - If ``R``'s not running: + - If `active experiment` not exists: - - no id or name specified, create a default experiment, and the experiment is set to be running. + - no id or name specified, create a default experiment, and the experiment is set to be active. - - if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given name or the default experiment, and the experiment is set to be running. + - if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given name or the default experiment, and the experiment is set to be active. - Else If '`create`' is False: - - If ``R``'s running: + - If ``active experiment` exists: - no id or name specified, return the active experiment. - if id or name is specified, return the specified experiment. If no such exp found, raise Error. - - If ``R``'s not running: + - If `active experiment` not exists: - no id or name specified. If the default experiment exists, return it, otherwise, raise Error. @@ -272,13 +277,13 @@ def get_recorder(self, recorder_id=None, recorder_name=None, experiment_name=Non """ Method for retrieving a recorder. - - If ``R``'s running: + - If `active recorder` exists: - no id or name specified, return the active recorder. - if id or name is specified, return the specified recorder. - - If ``R``'s not running: + - If `active recorder` not exists: - no id or name specified, raise Error. @@ -351,8 +356,8 @@ def save_objects(self, local_path=None, artifact_path=None, **kwargs): from a local file/directory, or directly saving objects. User can use valid python's keywords arguments to specify the object to be saved as well as its name (name: value). - - If R's running: it will save the objects through the running recorder. - - If R's not running: the system will create a default experiment, and a new recorder and save objects under it. + - If `active recorder` exists: it will save the objects through the active recorder. + - If `active recorder` not exists: the system will create a default experiment, and a new recorder and save objects under it. .. note:: @@ -384,8 +389,8 @@ def log_params(self, **kwargs): """ Method for logging parameters during an experiment. In addition to using ``R``, one can also log to a specific recorder after getting it with `get_recorder` API. - - If R's running: it will log parameters through the running recorder. - - If R's not running: the system will create a default experiment as well as a new recorder, and log parameters under it. + - If `active recorder` exists: it will log parameters through the active recorder. + - If `active recorder` not exists: the system will create a default experiment as well as a new recorder, and log parameters under it. Here are some use cases: @@ -409,8 +414,8 @@ def log_metrics(self, step=None, **kwargs): """ Method for logging metrics during an experiment. In addition to using ``R``, one can also log to a specific recorder after getting it with `get_recorder` API. - - If R's running: it will log metrics through the running recorder. - - If R's not running: the system will create a default experiment as well as a new recorder, and log metrics under it. + - If `active recorder` exists: it will log metrics through the active recorder. + - If `active recorder` not exists: the system will create a default experiment as well as a new recorder, and log metrics under it. Here are some use cases: @@ -434,8 +439,8 @@ def set_tags(self, **kwargs): """ Method for setting tags for a recorder. In addition to using ``R``, one can also set the tag to a specific recorder after getting it with `get_recorder` API. - - If R's running: it will set tags through the running recorder. - - If R's not running: the system will create a default experiment as well as a new recorder, and set the tags under it. + - If `active recorder` exists: it will set tags through the active recorder. + - If `active recorder` not exists: the system will create a default experiment as well as a new recorder, and set the tags under it. Here are some use cases: diff --git a/qlib/workflow/cli.py b/qlib/workflow/cli.py index 65d9a14b4d..8270d2db79 100644 --- a/qlib/workflow/cli.py +++ b/qlib/workflow/cli.py @@ -49,13 +49,11 @@ def workflow(config_path, experiment_name="workflow", uri_folder="mlruns"): # config the `sys` section sys_config(config, config_path) - provider_uri = config.get("provider_uri") - region = config.get("region") exp_manager = C["exp_manager"] exp_manager["kwargs"]["uri"] = "file:" + str(Path(os.getcwd()).resolve() / uri_folder) - qlib.init(provider_uri=provider_uri, region=region, exp_manager=exp_manager) + qlib.init(**config.get("qlib_init"), exp_manager=exp_manager) - task_train(config, experiment_name=experiment_name) + task_train(config.get("task"), experiment_name=experiment_name) # function to run worklflow by config diff --git a/qlib/workflow/exp.py b/qlib/workflow/exp.py index 09c680e599..a92a9a9eac 100644 --- a/qlib/workflow/exp.py +++ b/qlib/workflow/exp.py @@ -114,24 +114,24 @@ def get_recorder(self, recorder_id=None, recorder_name=None, create: bool = True * If `create` is True: - * If R's running: + * If `active recorder` exists: * no id or name specified, return the active recorder. - * if id or name is specified, return the specified recorder. If no such exp found, create a new recorder with given id or name, and the recorder shoud be running. + * if id or name is specified, return the specified recorder. If no such exp found, create a new recorder with given id or name, and the recorder shoud be active. - * If R's not running: + * If `active recorder` not exists: * no id or name specified, create a new recorder. - * if id or name is specified, return the specified experiment. If no such exp found, create a new recorder with given id or name, and the recorder shoud be running. + * if id or name is specified, return the specified experiment. If no such exp found, create a new recorder with given id or name, and the recorder shoud be active. * Else If `create` is False: - * If R's running: + * If `active recorder` exists: * no id or name specified, return the active recorder. * if id or name is specified, return the specified recorder. If no such exp found, raise Error. - * If R's not running: + * If `active recorder` not exists: * no id or name specified, raise Error. * if id or name is specified, return the specified recorder. If no such exp found, raise Error. diff --git a/qlib/workflow/expm.py b/qlib/workflow/expm.py index cfb0290fca..a50dce7c91 100644 --- a/qlib/workflow/expm.py +++ b/qlib/workflow/expm.py @@ -23,12 +23,12 @@ class ExpManager: def __init__(self, uri, default_exp_name): self.uri = uri self.default_exp_name = default_exp_name - self.active_experiment = None # only one experiment can running each time + self.active_experiment = None # only one experiment can active each time def start_exp(self, experiment_name=None, recorder_name=None, uri=None, **kwargs): """ Start an experiment. This method includes first get_or_create an experiment, and then - set it to be running. + set it to be active. Parameters ---------- @@ -47,7 +47,7 @@ def start_exp(self, experiment_name=None, recorder_name=None, uri=None, **kwargs def end_exp(self, recorder_status: str = Recorder.STATUS_S, **kwargs): """ - End an running experiment. + End an active experiment. Parameters ---------- @@ -90,7 +90,7 @@ def search_records(self, experiment_ids=None, **kwargs): def get_exp(self, experiment_id=None, experiment_name=None, create: bool = True): """ Retrieve an experiment. This method includes getting an active experiment, and get_or_create a specific experiment. - The returned experiment will be running. + The returned experiment will be active. When user specify experiment id and name, the method will try to return the specific experiment. When user does not provide recorder id or name, the method will try to return the current active experiment. @@ -99,24 +99,24 @@ def get_exp(self, experiment_id=None, experiment_name=None, create: bool = True) * If `create` is True: - * If R's running: + * If `active experiment` exists: * no id or name specified, return the active experiment. - * if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be running. + * if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be active. - * If R's not running: + * If `active experiment` not exists: * no id or name specified, create a default experiment. - * if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be running. + * if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name, and the experiment is set to be active. * Else If `create` is False: - * If R's running: + * If `active experiment` exists: * no id or name specified, return the active experiment. * if id or name is specified, return the specified experiment. If no such exp found, raise Error. - * If R's not running: + * If `active experiment` not exists: * no id or name specified. If the default experiment exists, return it, otherwise, raise Error. * if id or name is specified, return the specified experiment. If no such exp found, raise Error. From d0cc8835172c25c0456cdcd48d79bb48794b036a Mon Sep 17 00:00:00 2001 From: Jactus Date: Mon, 30 Nov 2020 19:06:23 +0800 Subject: [PATCH 3/3] Fix trainer --- qlib/model/trainer.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qlib/model/trainer.py b/qlib/model/trainer.py index 305cf9ed2d..8f128d3826 100644 --- a/qlib/model/trainer.py +++ b/qlib/model/trainer.py @@ -28,7 +28,7 @@ def task_train(task_config: dict, experiment_name): recorder = R.get_recorder() # generate records: prediction, backtest, and analysis - for record in task_config.get["record"]: + for record in task_config["record"]: if record["class"] == SignalRecord.__name__: srconf = {"model": model, "dataset": dataset, "recorder": recorder} record["kwargs"].update(srconf)