Benchmarking Process

[antoinecarme]

Need to run a benchmarking process to review the current state of PyAF.

In as first time, we will see this as a sanity check (correct some bugs here and there ;).

In a second time, a report is generated with performance figures.

[antoinecarme]

sanity check :

The benchmarks M1, M2, M3, M4 , NN3 and NN5 are now working.

[antoinecarme]

Added a new repository PyAF_Benchmarks to store benchamrk data under :

https://github.com/antoinecarme/PyAF_Benchmarks

[antoinecarme]

Added an internal benchmark (specific to PyAF) based on 4818 stock values of the following indices:

['aex', 'aord', 'bvsp', 'cac40', 'currency', 'dow_jones', 'eurindex', 'euronext', 'eurostoxx50', 'exch', 'ftse100', 'ftse250', 'ftseall', 'gdaxi', 'ibex35', 'ibexnm', 'ipc', 'kospi', 'mc', 'merval', 'mib30', 'mibtel', 'midex', 'my_test', 'nasdaq', 'nasdaqbio', 'ny100', 'nysecomp', 'nyworldlead', 'othindex', 'smi', 'sp500', 'spmib', 'sse', 'ta100', 'tsx', 'usindex']

[antoinecarme]

sanity check :

These benchmarks allow building models on around 20000 time series from different businesses:

M1 : 1001 series
M2 : 29 series
M3 : 3003 series
M4 : 10000 series
NN3 : 111 series
NN5 : 111 series
YahooStocks : 4818

logs are available under https://github.com/antoinecarme/PyAF_Benchmarks

[antoinecarme]

Added a jupyter notebook to generate benchmark reports

https://github.com/antoinecarme/PyAF_Benchmarks/blob/master/reporting/bench-debrief.ipynb

Added first report generation data :

https://github.com/antoinecarme/PyAF_Benchmarks/tree/master/reporting/data