How to easily use the example models with jupyter note book? workflow_by_code_my.ipynb

[Joshuajxy]

❓ Questions and Help

How to easily use the example models with jupyter note book? such as in workflow_by_code_my.ipynb

I found workflow_by_code_my.ipynb has only one example LGBModel, does it mean we have to modify the notebook file everytime when want to try a new model?

I went through the document but did not find an easy way to debug of visualize the result by using qrun.

Thanks for the help in advance.

[Derek-Wds]

Hi, you only need to modify the model config of the task dictionary in the jupyter notebook. For the parameters, you may want to refer to the benchmarks we provided here or the source code here.

[Joshuajxy]

Thanks a lot, Derek! So the easiest way to visualize the benchmark model result is to modify the model and task with workflow_by_code_my.ipynb, right?

Let me have a try. Thanks for the help again.

[Joshuajxy]

Hi,

I tried to convert yaml into notebook(ALSTM model), but got the following error. could you give some comment on this?

[7232:MainThread](2021-05-09 17:52:18,801) INFO - qlib.ALSTM - [pytorch_alstm_ts.py:62] - ALSTM pytorch version...
[7232:MainThread](2021-05-09 17:52:18,802) INFO - qlib.ALSTM - [pytorch_alstm_ts.py:111] - ALSTM parameters setting:
d_feat : 20
hidden_size : 64
num_layers : 2
dropout : 0.0
n_epochs : 200
lr : 0.001
metric : loss
batch_size : 800
early_stop : 10
optimizer : adam
loss_type : mse
device : cpu
n_jobs : 20
use_GPU : False
seed : None
[7232:MainThread](2021-05-09 17:52:18,809) INFO - qlib.ALSTM - [pytorch_alstm_ts.py:125] - model:
ALSTMModel(
(net): Sequential(
(fc_in): Linear(in_features=20, out_features=64, bias=True)
(act): Tanh()
)
(rnn): GRU(64, 64, num_layers=2, batch_first=True)
(fc_out): Linear(in_features=128, out_features=1, bias=True)
(att_net): Sequential(
(att_fc_in): Linear(in_features=64, out_features=32, bias=True)
(att_dropout): Dropout(p=0.0, inplace=False)
(att_act): Tanh()
(att_fc_out): Linear(in_features=32, out_features=1, bias=False)
(att_softmax): Softmax(dim=1)
)
)
[7232:MainThread](2021-05-09 17:52:18,811) INFO - qlib.ALSTM - [pytorch_alstm_ts.py:126] - model size: 0.0510 MB
[7232:MainThread](2021-05-09 17:55:11,412) INFO - qlib.timer - [log.py:77] - Time cost: 172.595s | Loading data Done
[7232:MainThread](2021-05-09 17:55:12,198) INFO - qlib.timer - [log.py:77] - Time cost: 0.614s | FilterCol Done
[7232:MainThread](2021-05-09 17:55:13,270) INFO - qlib.timer - [log.py:77] - Time cost: 1.070s | RobustZScoreNorm Done
[7232:MainThread](2021-05-09 17:55:13,322) INFO - qlib.timer - [log.py:77] - Time cost: 0.051s | Fillna Done
[7232:MainThread](2021-05-09 17:55:13,436) INFO - qlib.timer - [log.py:77] - Time cost: 0.090s | DropnaLabel Done
[7232:MainThread](2021-05-09 17:55:13,875) INFO - qlib.timer - [log.py:77] - Time cost: 0.437s | CSRankNorm Done
[7232:MainThread](2021-05-09 17:55:13,876) INFO - qlib.timer - [log.py:77] - Time cost: 2.463s | fit & process data Done
[7232:MainThread](2021-05-09 17:55:13,876) INFO - qlib.timer - [log.py:77] - Time cost: 175.060s | Init data Done
[7232:MainThread](2021-05-09 17:55:13,878) INFO - qlib.workflow - [expm.py:254] - No tracking URI is provided. Use the default tracking URI.
[7232:MainThread](2021-05-09 17:55:13,879) INFO - qlib.workflow - [expm.py:290] - <mlflow.tracking.client.MlflowClient object at 0x000001DE62E8B8C8>
[7232:MainThread](2021-05-09 17:55:13,887) INFO - qlib.workflow - [exp.py:242] - Experiment 2 starts running ...
[7232:MainThread](2021-05-09 17:55:14,630) INFO - qlib.workflow - [recorder.py:270] - Recorder e27719a22f0d4524aa8c79a440cc1244 starts running under Experiment 2 ...

AttributeError Traceback (most recent call last)
in
92 with R.start(experiment_name="train_model"):
93 R.log_params(**flatten_dict(task))
---> 94 model.fit(dataset)
95 R.save_objects(trained_model=model)
96 rid = R.get_recorder().id

~\AppData\Local\Continuum\anaconda3\envs\Qlib\lib\site-packages\pyqlib-0.6.3.99-py3.7-win-amd64.egg\qlib\contrib\model\pytorch_alstm_ts.py in fit(self, dataset, evals_result, save_path)
209 dl_valid = dataset.prepare("valid", col_set=["feature", "label"], data_key=DataHandlerLP.DK_L)
210
--> 211 dl_train.config(fillna_type="ffill+bfill") # process nan brought by dataloader
212 dl_valid.config(fillna_type="ffill+bfill") # process nan brought by dataloader
213

~\AppData\Local\Continuum\anaconda3\envs\Qlib\lib\site-packages\pandas-1.2.4-py3.7-win-amd64.egg\pandas\core\generic.py in getattr(self, name)
5463 if self._info_axis._can_hold_identifiers_and_holds_name(name):
5464 return self[name]
-> 5465 return object.getattribute(self, name)
5466
5467 def setattr(self, name: str, value) -> None:

AttributeError: 'DataFrame' object has no attribute 'config'

Following are the python code that was converted from yaml file.

###################################

train model

###################################
data_handler_config = {
"start_time": "2008-01-01",
"end_time": "2020-08-01",
"fit_start_time": "2008-01-01",
"fit_end_time": "2014-12-31",
"instruments": market,
"infer_processors": [{
"class": "FilterCol",
"kwargs": {
"fields_group": "feature",
"col_list": ["RESI5", "WVMA5", "RSQR5", "KLEN", "RSQR10", "CORR5", "CORD5", "CORR10",
"ROC60", "RESI10", "VSTD5", "RSQR60", "CORR60", "WVMA60", "STD5",
"RSQR20", "CORD60", "CORD10", "CORR20", "KLOW"
],
},
},
{
"class": "RobustZScoreNorm",
"kwargs":{
"fields_group": "feature",
"clip_outlier": True,
},
},
{
"class": "Fillna",
"kwargs":{
"fields_group": "feature",
},
},
],
"learn_processors" : [
{
"class" : "DropnaLabel",
},
{
"class" : "CSRankNorm",
"kwargs" : {
"fields_group" : "label",
},
},
],
"label": ["Ref($close, -2) / Ref($close, -1) - 1"],
}

task = {
"model": {
"class": "ALSTM",
"module_path": "qlib.contrib.model.pytorch_alstm_ts",
"kwargs": {
"d_feat": 20,
"hidden_size": 64,
"num_layers": 2,
"dropout": 0.0,
"n_epochs": 200,
"lr": 1e-3,
"early_stop": 10,
"batch_size": 800,
"metric": "loss",
"loss": "mse",
"n_jobs": 20,
"GPU": 0,
"rnn_type": "GRU",
},
},
"dataset": {
"class": "DatasetH",
"module_path": "qlib.data.dataset",
"kwargs": {
"handler": {
"class": "Alpha158",
"module_path": "qlib.contrib.data.handler",
"kwargs": data_handler_config,
},
"segments": {
"train": ("2008-01-01", "2014-12-31"),
"valid": ("2015-01-01", "2016-12-31"),
"test": ("2017-01-01", "2020-08-01"),
},
"step_len": 20,
},
},
}

[Derek-Wds]

Hi @Joshuajxy , it seems you are using the wrong data set class. You should be using the TSDatasetH if running models with ts in name. You can refer to the details in the YAML file of ALSTM here.

[Joshuajxy]

Thanks, Derek. The issue was solved with your suggestion.

But I met another problem as the following error, it looks also related to data set class, but I'm not sure how to fix this problem, could you also share some suggestion?
score IC

pred_label = pd.concat([label_df, pred_df], axis=1, sort=True).reindex(label_df.index)
analysis_position.score_ic_graph(pred_label)

---

TypeError Traceback (most recent call last)
in
----> 1 pred_label = pd.concat([label_df, pred_df], axis=1, sort=True).reindex(label_df.index)
2 analysis_position.score_ic_graph(pred_label)

~\AppData\Local\Continuum\anaconda3\envs\Qlib\lib\site-packages\pandas-1.2.4-py3.7-win-amd64.egg\pandas\core\reshape\concat.py in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
293 verify_integrity=verify_integrity,
294 copy=copy,
--> 295 sort=sort,
296 )
297

~\AppData\Local\Continuum\anaconda3\envs\Qlib\lib\site-packages\pandas-1.2.4-py3.7-win-amd64.egg\pandas\core\reshape\concat.py in init(self, objs, axis, join, keys, levels, names, ignore_index, verify_integrity, copy, sort)
368 "only Series and DataFrame objs are valid"
369 )
--> 370 raise TypeError(msg)
371
372 # consolidate

TypeError: cannot concatenate object of type '<class 'qlib.data.dataset.TSDataSampler'>'; only Series and DataFrame objs are valid

[Derek-Wds]

Hi @Joshuajxy , sorry for any inconvenience that our imperfect example may bring to you. In your example, since you are trying to use the time series dataset of Qlib to retrieve data. The label_df now you have is something similar to pytorch's data sampler, through which you can get data iteratively by their indexes. If you want to get the dataframe of the data, you may want to use this command label_df = dataset.prepare("test", col_set="label").data.

For more details for the TSDataSampler in Qlib, you can refer to the code here.

Hope this can solve your problem!

[Joshuajxy]

Thanks a lot, Derek. The issue was solved with your suggestion.

[LogCreative]

I tried to call label_df = dataset.prepare("test", col_set="label").data and found that it prompts

'TSDataSampler' object has no attribute 'data'.

After referring to the source code, it seems that the usage is now deprecated for TSDataSampler and I add another line to make it work for the current version:

label_df = pd.DataFrame(data=label_df.data_arr[:-1], index=label_df.data_index)

It seems that the last row is set to nan by design. Anyway, it should be ignored when constructing a DataFrame.

qlib/qlib/data/dataset/__init__.py

Line 301 in 7800dd4

# - append last line with full NaN for better performance in `__getitem__`

[you-n-g]

@LogCreative
TSDataSampler is designed for time series.
Its internal data is data_arr in a tabular format. TSDataSampler tries to create time series based on the tabular data.
The last row with nan value is designed for improving performance when creating time series from tabular data.
https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/__init__.py#L241

If you want to get tabular data, the best practice is

1. use step_len=1 https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/__init__.py#L259
2. use DatasetH instead of TSDatasetH
   You can either way of them

Thanks for your attention.