-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Type annotate get_status_dict and note that we can pass Exception or CapturedException which is not subclass #7403
Conversation
I decided to start type annotating there instead of relying on numpy docstrings
@yarikoptic What types are the fields of a status/result dict supposed to be? |
well, to be figured out from typical uses which could be grep'ed up. |
base documentation is https://github.com/datalad/datalad/blob/HEAD/docs/source/design/result_records.rst |
@yarikoptic That documentation states that |
Looking at the hits on "error_message" and because the name is singular❯ git grep -p error_message
datalad/core/distributed/clone.py=def clone_dataset(
datalad/core/distributed/clone.py: #error_message=ce.message,
datalad/core/local/tests/test_results.py=def test_generic_result_renderer():
datalad/core/local/tests/test_results.py: (dict(message='funky', error_message='extra-funky'),
datalad/core/local/tests/test_status.py=def test_untracked_annex_query(path=None):
datalad/core/local/tests/test_status.py: error_message='File unknown to git',
datalad/distributed/drop.py=def _drop_dataset(ds, paths, what, reckless, recursive, recursion_limit, jobs):
datalad/distributed/drop.py: error_messages = r.get('error-messages')
datalad/distributed/drop.py: if error_messages:
datalad/distributed/drop.py: res['error_message'] = '\n'.join(
datalad/distributed/drop.py: m.strip() for m in error_messages
datalad/distributed/tests/test_drop.py=def test_drop_file_content(path=None, outside_path=None):
datalad/distributed/tests/test_drop.py: error_message='File unknown to git',
datalad/distributed/tests/test_drop.py=def test_drop_uninit_annexrepo(origpath=None, path=None):
datalad/distributed/tests/test_drop.py:def test_drop_allkeys_result_contains_annex_error_messages(path=None):
datalad/distributed/tests/test_drop.py: # message(s) to be present in the result record error_message
datalad/distributed/tests/test_drop.py: error_message='git-annex error message here',
datalad/distributed/tests/test_ria_git_remote.py=def _test_bare_git_version_1(host, dspath, store):
datalad/distributed/tests/test_ria_git_remote.py: error_message='** Based on the location log, one.txt\n'
datalad/distributed/tests/test_ria_git_remote.py: error_message='** Based on the location log, subdir/two\n'
datalad/distributed/tests/test_ria_git_remote.py=def _test_bare_git_version_2(host, dspath, store):
datalad/distributed/tests/test_ria_git_remote.py: error_message='** Based on the location log, one.txt\n'
datalad/interface/results.py=def get_status_dict(action=None, ds=None, path=None, type=None, logger=None,
datalad/interface/results.py: error_message=None, **kwargs):
datalad/interface/results.py: if error_message is not None:
datalad/interface/results.py: d['error_message'] = error_message
datalad/interface/results.py: if error_message is None and isinstance(exception, CapturedException):
datalad/interface/results.py: d['error_message'] = exception.message
datalad/interface/results.py=def annexjson2result(d, ds, **kwargs):
datalad/interface/results.py: res['error_message'] = '\n'.join(m.strip() for m in d['error-messages'])
datalad/interface/utils.py=def generic_result_renderer(res):
datalad/interface/utils.py: res['error_message'][0] % res['error_message'][1:]
datalad/interface/utils.py: if isinstance(res['error_message'], tuple) else res[
datalad/interface/utils.py: 'error_message']), ac.RED)
datalad/interface/utils.py: if res.get('error_message', None) and res.get('status', None) != 'ok' else ''))
datalad/runner/exception.py=class CommandError(RuntimeError):
datalad/runner/exception.py: to_str += _format_json_error_messages(
datalad/runner/exception.py:def _format_json_error_messages(recs: list[dict]) -> str:
datalad/runner/tests/test_exception.py=from ..exception import (
datalad/runner/tests/test_exception.py: _format_json_error_messages,
datalad/runner/tests/test_exception.py=def test_format_error_with_duplicates() -> None:
datalad/runner/tests/test_exception.py: result = _format_json_error_messages(json_objects)
datalad/runner/tests/test_exception.py: result = _format_json_error_messages(json_objects + json_objects)
datalad/runner/tests/test_exception.py=def test_format_no_errors() -> None:
datalad/runner/tests/test_exception.py: result = _format_json_error_messages(json_objects)
datalad/runner/tests/test_exception.py: result = _format_json_error_messages(json_objects + json_objects)
datalad/support/annexrepo.py=class AnnexRepo(GitRepo, RepoInterface):
datalad/support/annexrepo.py: rec['error_message'] = '\n'.join(m.strip() for m in j['error-messages'])
docs/source/design/result_records.rst=A string label categorizing the state of an entity. Common values are:
docs/source/design/result_records.rst:``error_message``
it seems that it is always a string and documentation might be wrong/needs fixing. Did you spot where it was set to a |
@yarikoptic There's at least a provision for if datalad/datalad/interface/utils.py Lines 253 to 256 in 9c91ff9
|
I guess it was added to mimic the behavior of |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## maint #7403 +/- ##
==========================================
+ Coverage 88.76% 91.42% +2.65%
==========================================
Files 327 325 -2
Lines 44693 43399 -1294
Branches 5924 5818 -106
==========================================
+ Hits 39671 39676 +5
+ Misses 5007 3707 -1300
- Partials 15 16 +1
☔ View full report in Codecov by Sentry. |
@jwodder is working on more annotations, so merging this would be beneficial since it does help to pick up possible typing snafus. I will merge tomorrow unless other @datalad/developers chime in with objections/concerns. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just commenting on Exception vs CapturedException: I believe the latter was primarily invented for passing to get_result_dict(). A regular exception binds a traceback and the long-lived nature of results caused complex/incomprehensible behavior when interaction with other parts of datalad that bind ressources. I have not been in touch with that, only @yarikoptic and primarily @bpoldrack , I believe. AFAIR it was open files staying open and processes not ending.
Unless there is a good reason to risk such regressions, I'd stay away from it.
I am not sure what you @mih have in mind since this PR AFAIK just adds types annotations -- it is just a matter of current state that we could pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yarikoptic asked me to take a look at this at the devcall, and while I'm not very knowledgeable about type annotations, I did not spot anything conceptually wrong with the PR, so I approve.
thank you @adswa ! neuroimaging CI failed due to some
and since the rest is green -- let's proceed! |
PR released in |
I decided to start type annotating there instead of relying on numpy docstrings
Ref #6884
TODOs