Describe a DataLad GitHub Action

[thewtex]

📝 Documentation

• Describe specifications for a DataLad GitHub Action re: DataLad GitHub Action #6929. These may find their way to the GitHub Action README.

[ci skip]
[appveyor skip]

[codecov]

Codecov Report

Merging #6931 (5bf357a) into master (0a18814) will decrease coverage by 1.77%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6931      +/-   ##
==========================================
- Coverage   90.02%   88.24%   -1.78%     
==========================================
  Files         354      354              
  Lines       46273    46352      +79     
  Branches     6595        0    -6595     
==========================================
- Hits        41655    40904     -751     
- Misses       4602     5448     +846     
+ Partials       16        0      -16     

Impacted Files	Coverage Δ
datalad/cmdline/__init__.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/interface/run_procedure.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/cmdline/helpers.py	0.00% <0.00%> (-78.27%)	⬇️
datalad/support/nda_.py	0.00% <0.00%> (-54.55%)	⬇️
datalad/tests/utils.py	21.60% <0.00%> (-32.47%)	⬇️
datalad/__main__.py	39.65% <0.00%> (-29.32%)	⬇️
datalad/distribution/uninstall.py	63.26% <0.00%> (-28.58%)	⬇️
datalad/__init__.py	78.00% <0.00%> (-20.00%)	⬇️
datalad/ui/utils.py	50.00% <0.00%> (-19.45%)	⬇️
datalad/api.py	75.60% <0.00%> (-14.64%)	⬇️
... and 86 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[yarikoptic]

are you sure that getting all the data should be default? E.g. in this demo use case it would be 14GB! (will CI have enough time to fetch it all? ;-))

[thewtex]

My use case involves a dedicated small unit testing dataset, but, yes I can see how result in a large download :-).

[yarikoptic]

both install and get have --jobs options. We might want to add here as jobs and get_jobs.

Also, for the use-case of "get all data", I think we might better just add also get_data option to invoke install with --get-data and avoid individual get_paths. Also get_paths on itself might be "tricky" to ask for all data -- i.e. . would not be recursive.

[thewtex]

I made a pass at this -- please let me know if I did not follow correctly and more changes are needed!

[vsoch]

Question (and going to work on this tomorrow) - is there possibly another functionality aside "get"? E.g., for actions I like to design them as small modules, so you could do like:

    - name: Install Datalad
      uses: datalad/datalad-action/install@main

# This is the "get" idea
    - name: Download Dataset
      uses: datalad/datalad-action/download@main

    - name: Something else?
      uses: datalad/datalad-action/??@main

Minimally if we can install / get, those should be separate (and the get can use the install) so that's my proposed modification to the above!

[vsoch]

Just FYI (as far as I know) you cannot have nested fields like this.

[vsoch]

@yarikoptic can you make me a member of the org or create datalad-action and make me maintainer? I have the install start ready and I'd like to push and test and then work on get.

[vsoch]

Could someone show me an example of the python and/or bash you want for the (core) of the get action? I see both as examples in the discussion so I'm not sure which to use - I was going with Python, and it took me 10 minutes to find (what I think) is the install function in core/local/run.py but I don't see any mention of recursion... so not sure how that comes in. Thanks!

[yarikoptic]

sorry @vsoch - I am not entirely sure what example you would like? but in general anything "datalad COMMAND [OPTIONS] ARGS" roughly corresponds to

import datalad.api as dl
dl.COMMAND(*args, **OPTIONS)

and imho bash scripts are good too, just make them set -eu on the top to ensure them not struggling through the pile of errors

In general -- as long as there is action which simplifies installation of datalad, and exposes it to run of subsequent steps/actions -- I do not think there is much need for really modular reflection of various datalad commands in the action -- datalad COMMANDS could be just coded in across platforms. I think the main goal of having datalad action is to condense and simplify for simplistic use case - get datasets with some data, and for that git-annex and datalad need to be installed and few installations targets given.

[vsoch]

@yarikoptic I already figured it out.

[yarikoptic]

@TheWeX, there is https://github.com/datalad/datalad-action now from @vsoch . Could you have a look at it and give it a shot?

[yarikoptic]

Let's adjust status (proposed suggestion), take out of draft, and get it merged - we have an implementation by @vsoch so any further discussion etc should happen in its issues.

[codeclimate]

Analysis results are not available for those commits

View more on Code Climate.

[mih]

I am not sure why the doc build CI did not run here, but this is invalid RST, and broke it:

Warning, treated as error:
167
/home/runner/work/datalad/datalad/docs/source/design/github_actions.rst:62:Title underline too short.

[yarikoptic-gitmate]

PR released in 0.18.0