Remove metadata-related code after its move to -deprecated #7014
Conversation
|
I think the dependency on |
Codecov ReportBase: 89.42% // Head: 90.62% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## master #7014 +/- ##
==========================================
+ Coverage 89.42% 90.62% +1.20%
==========================================
Files 358 325 -33
Lines 46553 44001 -2552
Branches 6322 0 -6322
==========================================
- Hits 41628 39876 -1752
+ Misses 4910 4125 -785
+ Partials 15 0 -15
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
This commit adds the extractors that were removed from datalad core in the datalad PR #7014 (datalad/datalad#7014) including their tests to datalad-metalad. This is done to keep them available with only `datalad` and `datalad-metalad` installed.
|
Should we move this out of draft and review it? The extension tests can best be fixed after this PR is released. Besides removing metadata-related code and fixing tests, the following has been done
The following has NOT been tackled yet in this PR:
|
This commit adds the metadata code that is removed from datalad core in the datalad PR #7014 (datalad/datalad#7014). It is tested against the datalad commit ff0f6a71b460075b7aa85a9da8052ab570f04590
This changeset is part of the transition from the 2nd-gen metadata
implementation to then gen3 code provided by the -metalad extension.
This removal has side-effect on other code, not directly related
to metadata. This following list can be considered a TODO for further
maintenance work:
Now unused code:
- `datalad/support/json_py.py:load_xzstream()`
- `datalad/interface/common_opts.py:reporton_opt`
- `datalad/utils.py:all_same()`
- `datalad/utils.py:as_unicode()`
- `datalad/utils.py:unicode_srctypes` made obsolete by removal of `as_unicode()`
Now barely used (technical debt?):
- `nosave_opt`: Residual use in `addurls()` and `download_url()`
- `datalad/support/path.py:split_ext()`: Residual use in a single spot in `addurls()`
- `datalad/utils.py:path_startswith()`: 2 pieces of code still use this outside the tests
- `datalad/utils.py:get_suggestions_msg()`: CLI parser and `addurls()` make the total of 2 calls made to this helper
- `datalad/utils.py:shortened_repr()`: one call in `get()` and one in `foreach_dataset()`
Issues to file:
- [ ] Investigate whether the dependency on `simplejson` is still needed.
- [ ] `datalad/interface/utils.py:discover_dataset_trace_to_targets()`
is now an internal helper of `save()`
Closes datalad#7012
This commit removes tests that are related to the metadata code that was contained in datalad-core. These tests fail after the removal of the metadata code.
This commit removes parts of the test `test_duecredit_dataset`, that tried to patch `Dataset.metadata`. The patching fails now because the metadata code has been removed.
This commit removes `simplejson` dependency in favor of the built-in `json`-module. Encoding is now a property of the streams and not the encoder.
This commit removes the `reporton_opt` parameter definition because with the removal of metalad from datalad core it is not used anymore.
This commit adds a `readme.txt` to `docs/source/_extras`, to ensure that the directory is added to the git-repo
The current implementation cannot rely on the presence of metalad-commands, since the extension might not be installed. This requires some code that is conditionally executed, based on the availability of certain metalad-commands.
This commit brings back the ability to clean old metadata indices into core.
christian-monch
force-pushed
the
mnt-metadata-mv
branch
from
ed372fd
to
d2acc02
Compare
|
appveyor mac fail: |
I am not able to reproduce it locally on |
|
Ok, so still not ready. Apparently the move to deprecated is still incomplete. datalad-deprecated is trying to import annotate_path stuff from core, that has moved to deprecated. |
bpoldrack
force-pushed
the
mnt-metadata-mv
branch
from
42d1465
to
2c884c1
Compare
bpoldrack
force-pushed
the
mnt-metadata-mv
branch
from
2c884c1
to
d2acc02
Compare
|
Code Climate has analyzed commit b61c0e4 and detected 4 issues on this pull request. Here's the issue category breakdown:
View more on Code Climate. |
|
Remaining failures are currently happening everywhere and are clearly unrelated. Looking into this. However, extension tests now pass. I'm gonna go ahead and merge this finally. |
|
PR released in |
Follow up to datalad#7014 which removed all metadata handling. Without this change we get ❯ make -C docs html make: Entering directory '/home/yoh/proj/datalad/datalad-maint/docs' DATALAD_SPHINX_RUN=1 sphinx-build -b html -d build/doctrees -W source build/html Running Sphinx v6.1.3 running build_manpage running build_cfginfo usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help error: invalid command 'build_schema' ... proceeding normally without crashing so we missed that
This changeset is part of the transition from the 2nd-gen metadata implementation to then gen3 code provided by the -metalad extension.
This removal has side-effect on other code, not directly related to metadata. This following list can be considered a TODO for further maintenance work:
Now unused code:
datalad/support/json_py.py:load_xzstream():load_xzstreamanddump_xzstreamremoved in 8263128datalad/interface/common_opts.py:reporton_opt: removed in 1c26072datalad/utils.py:all_same(): removed in 5e50038datalad/utils.py:as_unicode(): removed in 13514badatalad/utils.py:unicode_srctypesmade obsolete by removal ofas_unicode(): removed in 13514ba (same as above)Now barely used (technical debt?):
nosave_opt: Residual use inaddurls()anddownload_url()datalad/support/path.py:split_ext(): Residual use in a single spot inaddurls()<- Dissolvedatalad.support.pathand fully replace with pathlib #4056datalad/utils.py:path_startswith(): 2 pieces of code still use this outside the testsdatalad/utils.py:get_suggestions_msg(): CLI parser andaddurls()make the total of 2 calls made to this helperdatalad/utils.py:shortened_repr(): one call inget()and one inforeach_dataset()Issues to file:
simplejsonis still needed,simplejsoncan be removed by PR Removesimplejsonin favor of usingjson#7035datalad/interface/utils.py:discover_dataset_trace_to_targets()is now an internal helper ofsave()SEARCH_INDEX_DOTGITDIRRelated TODOs:
_datalad_build_supportand removeBuildSchemaand related code -- maybe that was only in -core?Closes #7012