Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unconditionally store image path in POSIX notation in configuration #243

Merged
merged 10 commits into from
Oct 12, 2023

Conversation

mih
Copy link
Member

@mih mih commented Oct 10, 2023

[This PR sits on #242, only the last commit is unique and relevant here]

This establishes a fixed format for the configuration. Previously,
the path was stored in platform conventions without any indication
of the nature of that platform. This turned platform detection into
guesswork. Configuration created on windows was not functional on other platforms.

Some of that guesswork is now implemented in a path normalization
helper (_normalize_image_path()). It is executed on-read, in order
to keep older dataset configuration functional for the cases

  • we can tell the platform conventions, because the image is placed
    under .datalad\\environments\\ (the default)
  • we can locate a matching element on the file system

Notably, this guesswork does not cover the case of an image that
was configured on windows, and is hosted in a currently uninstalled
subdataset. This seems rare enough of a case to not deserve the implied complexity of supporting it in a backward compatible way.

Closes #224

TODO:

  • run this on windows and confirm that it works (stores POSIX, and operates on platform paths)
  • adjust tests to simulate an old setup with windows paths

The missing lines in the coverage are

image

I am not planning on adding a test for this case (a path that is verified to be in windows convention by finding a matching filesystem item). If this is unacceptable, I am OK with removing support for this entirely.

This is a plain code move from `containers_list` into a standalone function.
Added value is

- reusability
- documentation, incl. type annotation

Also adds rendering of the utility module's documentation.

This is a first step towards resolving datalad#238
This is a common use case in commands other than `containers_list`.
Albeit not strictly necessary (all configuration could be read
and processed first, and sub-selection could happen in user code),
it is a low-complexity change. Moreover, we envision more configuration
post-processing to happen in the future
(see datalad#238). This
would change the cost assessment of loading everything upfront.
This changeset maintains the exact behavior of the command, and
merely adds documentation and replaces custom configuration
access with the new `get_container_configuration()` helper.

The documentation may communicate suboptimal command semantics,
but these were given before, and are only written down now.

Also see datalad#240
for additional aspects re a future refactoring.

Configuration writes now use the ``scope=branch`` approach. It was
introduced with datalad v0.16 and requires no dependency adjustments.
...for the purpose of a standardization of container configuration reads
on `get_container_configuration()`.
This is done by turning of the result rendering. This revealed that
the command implementations cause uncondtional result rendering
datalad#241
This is the last change for establishing `get_container_configuration()`
as the single code piece for reading container documentation.

This opens the door for implementing on-read normalization of
configuration items, such as the image path.

Closes datalad#238
@codeclimate
Copy link

codeclimate bot commented Oct 10, 2023

Code Climate has analyzed commit cd49d66 and detected 0 issues on this pull request.

View more on Code Climate.

Possible a little less complex to understand.
@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (c505e52) 94.69% compared to head (cd49d66) 94.70%.
Report is 10 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #243      +/-   ##
==========================================
+ Coverage   94.69%   94.70%   +0.01%     
==========================================
  Files          25       25              
  Lines        1093     1133      +40     
==========================================
+ Hits         1035     1073      +38     
- Misses         58       60       +2     
Files Coverage Δ
datalad_container/containers_add.py 89.80% <100.00%> (+0.13%) ⬆️
datalad_container/containers_list.py 95.74% <100.00%> (-0.81%) ⬇️
datalad_container/containers_remove.py 97.43% <100.00%> (-0.07%) ⬇️
datalad_container/containers_run.py 84.84% <ø> (ø)
datalad_container/tests/test_containers.py 100.00% <100.00%> (ø)
datalad_container/tests/test_run.py 100.00% <100.00%> (ø)
datalad_container/utils.py 92.68% <94.11%> (+6.96%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This establishes a fixed format for the configuration. Previously,
the path was stored in platform conventions without any indication
of the nature of that platform. This turned platform detection into
guesswork.

Some of that guesswork is now implemented in a path normalization
helper (`_normalize_image_path()`). it is executed on-read, in order
to keep older dataset configuration functional for the cases

- we can tell the platform conventions, because the image is placed
  under `.datalad\\environments\\` (the default)
- we can locate a matching element on the file system

Notably, this guesswork does not cover the case of an image that
was configured on windows, and is hosted in a currently uninstalled
subdataset.

Closes datalad#224
Copy link
Member

@adswa adswa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks sensible to me

@yarikoptic yarikoptic added windows patch Increment the patch version when merged labels Oct 12, 2023
Copy link
Member

@yarikoptic yarikoptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conditionally on that non-r""-ed use of \\ in strings. IIRC it was literally double \\ in the file.

The faulty path delimiter is a `\\`, not just a single backlash.

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
@mih
Copy link
Member Author

mih commented Oct 12, 2023

Thanks for the reviews! Very helpful!

@mih mih merged commit cce7edb into datalad:master Oct 12, 2023
8 checks passed
@mih mih deleted the path-normalize branch October 12, 2023 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
patch Increment the patch version when merged windows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

containers-add produces platform-specific configuration
3 participants