Add Traceback to the AutoMLSearch Logs - #1840
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1840 +/- ##
========================================
+ Coverage 99.9% 100.0% +0.2%
========================================
Files 255 255
Lines 20662 20644 -18
========================================
+ Hits 20628 20636 +8
+ Misses 34 8 -26
Continue to review full report at Codecov.
|
| automl = AutoMLSearch(X_train=X, y_train=y, problem_type="binary", error_callback=None, train_best_pipeline=False, n_jobs=1) | ||
| automl.search() | ||
| assert msg in caplog.text | ||
| traceback_string1 = "return self._execute_mock_call(*args, **kwargs)" |
There was a problem hiding this comment.
These are messages that exist in the tracebacks only
| traceback_string2 = "return self._mock_call(*args, **kwargs)" | ||
| traceback_string3 = "cv_pipeline.fit(X_train, y_train)" | ||
| for messages in [traceback_string1, traceback_string2, traceback_string3, msg]: | ||
| assert messages not in caplog.text |
There was a problem hiding this comment.
Only silent_error_callback doesn't log the traceback
There was a problem hiding this comment.
It's confusing though cause there is no traceback since nothing errors. I think we should update the test. Could be done in a separate PR but it'd be a small change.
There was a problem hiding this comment.
@bchen1116 @freddyaboulton if an issue is coming out of this, can we also maybe parameterize these tests into one? Ping me if you do it in a separate PR, I'd like to make sure it doesn't get lost in the backlog.
There was a problem hiding this comment.
@freddyaboulton @chukarsten simplified the tests, so I don't think we'll need to file that in a new issue.
freddyaboulton
left a comment
freddyaboulton
left a comment
There was a problem hiding this comment.
@bchen1116 Looks good! Thanks for updating all of our existing tests!
I have a comment on adding some info to the traceback in the event of a PipelineScoreError that I'd like to resolve before merge.
| traceback_string2 = "return self._mock_call(*args, **kwargs)" | ||
| traceback_string3 = "cv_pipeline.fit(X_train, y_train)" | ||
| for messages in [traceback_string1, traceback_string2, traceback_string3, msg]: | ||
| assert messages not in caplog.text |
There was a problem hiding this comment.
It's confusing though cause there is no traceback since nothing errors. I think we should update the test. Could be done in a separate PR but it'd be a small change.
| traceback_string2 = "return self._mock_call(*args, **kwargs)" | ||
| traceback_string3 = "cv_pipeline.fit(X_train, y_train)" | ||
| for messages in [traceback_string1, traceback_string2, traceback_string3, msg]: | ||
| assert messages not in caplog.text |
There was a problem hiding this comment.
@bchen1116 @freddyaboulton if an issue is coming out of this, can we also maybe parameterize these tests into one? Ping me if you do it in a separate PR, I'd like to make sure it doesn't get lost in the backlog.
freddyaboulton
left a comment
freddyaboulton
left a comment
There was a problem hiding this comment.
Looks great @bchen1116 !
3 participants
fix #1778
Screenshot of the
evalml_debug.logfile: