Runtimes

[joaquinvanschoren]

Would it be possible to measure the runtime (in milliseconds) of a run? Ideally both the model training time and prediction time per model in cross-validation, but even the overall runtime of a run would be very interesting to know.

[janvanrijn]

Added this feature. Is in branch issue208. Should author a pull request soon

[janvanrijn]

Todo: check right scale (seconds bs milli seconds)

[janvanrijn]

The runtimes do not make sense to me. The unit test takes 3 minutes. Below some details. According to Python manual, process_time measures (seconds) the process takes. I multiply this by 1000 to receive milliseconds (shown below) The numbers seem too low to me.

/home/vanrijn/projects/pythonvirtual/openml-python2/bin/python3.5 /home/vanrijn/appl/pycharm-community-2016.2.2/helpers/pycharm/utrunner.py /home/vanrijn/projects/openml-python/tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload true
Testing started at 13:06 ...
modelfit start time: 0.830650306
modelfit duration: 4.492052999999996
modelpredict start time: 0.8351803720000001
modelpredict duration: 0.18008400000002478
usercpu_time_millis_training 4.492052999999996
usercpu_time_millis_testing 0.18008400000002478
usercpu_time_millis 4.672137000000021
modelfit start time: 1.011894559
modelfit duration: 0.41847999999999885
modelpredict start time: 1.012372798
modelpredict duration: 0.24624000000006419
usercpu_time_millis_training 0.41847999999999885
usercpu_time_millis_testing 0.24624000000006419
usercpu_time_millis 0.664720000000063
modelfit start time: 1.170907659
modelfit duration: 50.08675400000007
modelpredict start time: 1.22103077
modelpredict duration: 0.45400400000006336
usercpu_time_millis_training 50.08675400000007
usercpu_time_millis_testing 0.45400400000006336
usercpu_time_millis 50.54075800000013
modelfit start time: 1.400338321
modelfit duration: 1817.572009
modelpredict start time: 3.218093266
modelpredict duration: 49.79189100000036
modelfit start time: 3.554305636
modelfit duration: 77.3404740000001
modelpredict start time: 3.6316989790000003
modelpredict duration: 54.0461729999997
usercpu_time_millis_training 1817.572009
usercpu_time_millis_testing 49.79189100000036
usercpu_time_millis 1867.3639000000003
modelfit start time: 3.749932313
modelfit duration: 77.27686700000014
modelpredict start time: 3.827406502
modelpredict duration: 1.0462300000000369
modelfit start time: 4.062806429
modelfit duration: 6.182788000000272
modelpredict start time: 4.069021121
modelpredict duration: 1.0471880000002542
usercpu_time_millis_training 77.27686700000014
usercpu_time_millis_testing 1.0462300000000369
usercpu_time_millis 78.32309700000017

Process finished with exit code 0