Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update high freq demo #358

Merged
merged 6 commits into from Apr 14, 2021
Merged

update high freq demo #358

merged 6 commits into from Apr 14, 2021

Conversation

javaThonc
Copy link
Contributor

Description

Create a high-frequency demo with the calculation of precision to calculate the right prediction level and a classification version of lightGBM that is designed for the high-frequency data classification method as well as a workflow that can be used to backtest later.

Motivation and Context

Create a high freq demo that can be used to test whether a dataset contains signal for trading

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

@you-n-g
Copy link
Collaborator

you-n-g commented Mar 24, 2021

@javaThonc

  1. We need more documents about this example
  2. It will be better if we could demonstrate the evaluation metrics about high frequency trading algorithms in the documents like https://github.com/microsoft/qlib/tree/main/examples

qlib_init:
provider_uri: "~/.qlib/qlib_data/yahoo_cn_1min"
region: cn
market: &market ['SH605222', 'SZ002796', 'SZ002246', 'SZ000713', 'SZ002820', 'SH601328', 'SZ000668', 'SH603359', 'SZ002144', 'SH600195', 'SH603685', 'SH603386', 'SZ002586', 'SZ000573', 'SZ000605', 'SZ002842', 'SH600068', 'SZ300547', 'SZ000926', 'SZ002036', 'SZ002161', 'SH600715', 'SZ300427', 'SZ002573', 'SZ300142', 'SH605116', 'SZ002951', 'SH600276', 'SZ002437', 'SH603355', 'SZ002893', 'SH600584']
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting for @zhupr to update the instruments list

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated: python get_data.py qlib_data --name qlib_data --target_dir ~/.qlib/qlib_data/cn_data_1min --interval 1min --region cn

qlib/contrib/model/highfreq_gdbt_model.py Outdated Show resolved Hide resolved
qlib/contrib/eva/alpha.py Outdated Show resolved Hide resolved
def calc_prec(
pred: pd.Series, label: pd.Series, date_col="datetime", quantile: float = 0.2, dropna=False, is_alpha=False
) -> Tuple[pd.Series, pd.Series]:
"""calculate the precision
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will excess return/return a more comprehensible name than precision?

qlib/contrib/eva/alpha.py Show resolved Hide resolved
qlib/contrib/eva/alpha.py Show resolved Hide resolved
long precision and short precision in time level
"""
if is_alpha:
label = label - label.mean(level=0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use groupby('datetime')

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

label.mean(level=date_col)

qlib/contrib/eva/alpha.py Show resolved Hide resolved
self.params.update(kwargs)
self.model = None

def _cal_signal_metrics(self, y_test, l_cut, r_cut):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you use calc_prec here?

self.get_path("long_pre.pkl"),
self.get_path("short_pre.pkl"),
]
paths.extend([self.get_path("long_short_r.pkl"), self.get_path("long_avg_r.pkl")])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is extend a better choice than writing it in the list directly here?

@you-n-g
Copy link
Collaborator

you-n-g commented Apr 1, 2021

@javaThonc Please remember to update the evaluation results on high frequency data and send us the profiling results.

Besides, don't forget to update the results of tabnet .

@you-n-g you-n-g merged commit a3a2b5a into microsoft:main Apr 14, 2021
qianyun210603 pushed a commit to qianyun210603/qlib that referenced this pull request Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants