New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update benchmark performance #97
Conversation
examples/benchmarks/README.md
Outdated
| TFT | Alpha158 (with selected 20 features) | 0.0344±0.00 | 0.2071±0.02| 0.0103±0.00 | 0.0632±0.01 | 0.0638±0.00 | 0.5845±0.8| -0.1754±0.02 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why isTFT‘s annual return so stable while it's information ratio so variable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wendi calculated the information ratio's std in a wrong way. It should be 0.08 instead of 0.8. I have just updated it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I calculated the mean of two 5-runs and typed by hand, and missed a "0"...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wendili-cs make sure your rank IC. TFT has high IC & annual return while it's IC is low.
Please double check it though such results ares possible.
@Derek-Wds Are these results generated by repeating 20 times of experiments? |
It's not for now. The running for Alpha360 hasn't finished yet. I will update the results when the running's done. |
Description
Motivation and Context
How Has This Been Tested?
pytest qlib/tests/test_all_pipeline.py
under upper directory ofqlib
.Screenshots of Test Results (if appropriate):
Types of changes