Log autotvm/auto_schduler Builder/Runner/session-ctor stack traces #8696
Description
Right now when AutoTVM and auto_scheduler builders and runners encounter exception conditions, the exceptions are caught and stored into result objects, but never printed. This is WAI when the log level is e.g. below logging.INFO, because during autotuning, some schedules are expected to fail to build and autotuning should be robust to that.
However, there are a lot of valid debugging scenarios (e.g. debugging new schedules, debugging targets, debugging changes to autotvm, debugging failed tests) where currently, you first need to modify python/tvm/autotvm or python/tvm/auto_scheduler code to print out the exception.
The task here is twofold, the second part could be deferred:
part 1:
- add logging.debug() calls to autotvm and auto_scheduler to print the exception
- add unit tests to verify these get called
call sites to log exceptions from in autotvm:
- builder
- runner
- establishing the session (kind of the same as the previous item, but a different class of errors can occur here--those errors raised from
session_constructor)
part 2: do this more scalably for autotvm, and provide a way to inspect the autotvm results:
- improve the return value of autotvm: output some kind of class that summarizes e.g. # successful trials, # failed trials, tune log results by trial, exceptions by trial
- determine the appropriate way to serialize this to disk for consumption from
tvmc - add testing for the above
@mikepapadim if you have time for at least part 1, that would be quite helpful!