Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: wtf -S to specify which sections to query/display (by default -- all) #3440

Merged
merged 6 commits into from May 22, 2019

Conversation

@yarikoptic
Copy link
Member

@yarikoptic yarikoptic commented May 20, 2019

-S dependencies is probably one of the most useful, but unfortunately it is not including concise versioning of datalad and datalad extensions, which are all heavy sections...

Copy link
Contributor

@kyleam kyleam left a comment

Nice. My only comment is that after this restructuring infos is now set to an empty dict away from the code that populates it. From a readability standpoint, I'd find it nicer if infos was initiated just above where the callables are invoked or at the same time as a dict comprehension. (res would have to be moved down too.)

… all)

-S dependencies   is probably one of the most useful, but unfortunately it is not
including concise versioning of datalad and datalad extensions, which are all heavy
sections...
@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented May 20, 2019

thanks @kyleam , did that with a few other minor tune ups (eventually I will learn how to use git range-diff ;)) (I expected git range-diff 'HEAD^..HEAD@{2}' 'HEAD^..HEAD@{0}' to be it but it wasn't ;))

@codecov
Copy link

@codecov codecov bot commented May 20, 2019

Codecov Report

Merging #3440 into 0.11.x will decrease coverage by 56.64%.
The diff coverage is 10.44%.

Impacted file tree graph

@@             Coverage Diff             @@
##           0.11.x    #3440       +/-   ##
===========================================
- Coverage   91.05%   34.41%   -56.65%     
===========================================
  Files         255      254        -1     
  Lines       33450    33466       +16     
===========================================
- Hits        30459    11516    -18943     
- Misses       2991    21950    +18959
Impacted Files Coverage Δ
datalad/plugin/tests/test_plugins.py 0% <0%> (-89.16%) ⬇️
datalad/plugin/wtf.py 19.73% <16.27%> (-62%) ⬇️
datalad/customremotes/tests/__init__.py 0% <0%> (-100%) ⬇️
datalad/tests/test_protocols.py 0% <0%> (-100%) ⬇️
datalad/support/tests/test_stats.py 0% <0%> (-100%) ⬇️
datalad/tests/test_constraints.py 0% <0%> (-100%) ⬇️
datalad/support/tests/test_ansi_colors.py 0% <0%> (-100%) ⬇️
datalad/distribution/tests/test_subdataset.py 0% <0%> (-100%) ⬇️
datalad/interface/tests/test_annotate_paths.py 0% <0%> (-100%) ⬇️
datalad/cmdline/tests/__init__.py 0% <0%> (-100%) ⬇️
... and 204 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 16ac107...af7dc19. Read the comment docs.

@kyleam
Copy link
Contributor

@kyleam kyleam commented May 20, 2019

eventually I will learn how to use git range-diff ;)) (I expected git range-diff 'HEAD^..HEAD@{2}' 'HEAD^..HEAD@{0}' to be it but it wasn't ;))

:]

Something like the above could work, but of course it's completely dependent on the state of your HEAD reflog and that could easily not line up. If you did it pre-push (and the root was the same for both), it could be as simple as git range-diff yarikoptic/enh-wtf-S...enh-wtf-S (that's three dots).

kyleam
kyleam approved these changes May 20, 2019
Copy link
Contributor

@kyleam kyleam left a comment

Thanks for the updates.

@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented May 21, 2019

grr... couldn't stop. Please re-review again. Now my primary use case will be

datalad wtf -S datalad -S extensions -S dependencies -D html_details -c

to just then paste e.g. here

DataLad 0.11.4.dev107 WTF (datalad, extensions, dependencies)

WTF

datalad

  • full_version: 0.11.4.dev107-g03da-dirty
  • version: 0.11.4.dev107

extensions

dependencies

  • tqdm: 4.28.1
  • git: git
  • wrapt: 1.11.1
  • boto: 2.44.0
  • keyrings.alt: 3.1
  • six: 1.12.0
  • keyring: 16.0.0
  • cmd:git: 2.20.1
  • iso8601: 0.1.11
  • mutagen: 1.40.0
  • exifread: 2.1.2
  • gitdb: 2.0.3
  • msgpack: 0.5.6
  • cmd:annex: 7.20190322+git133-g59922f1f4-1~ndall+1
  • requests: 2.21.0
  • cmd:system-ssh: 7.9p1
  • cmd:system-git: 2.21.0.593.g511ec345e18
  • humanize: 0.5.1
  • appdirs: 1.4.3
  • scrapy: 1.5.1

unless tests fail -- I am done with it. Feel free to tune up to your liking or drop last commits ;-)

@kyleam
Copy link
Contributor

@kyleam kyleam commented May 21, 2019

As I said on the call, I'm in favor of the action="append" change. I didn't look closely at the updates, but adding an option for the details-formatted output is fine with me.

args=("-D", "--decor"),
constraints=EnsureChoice('html_details') | EnsureNone(),
doc="""decoration around the rendering to facilitate embedding into
issues etc, e.g. use 'html_details_md' for posting collapsable entry
Copy link
Contributor

@kyleam kyleam May 21, 2019

'html_details_md' doesn't match the above value.

Copy link
Member Author

@yarikoptic yarikoptic May 21, 2019

thanks!

metavar="SECTION",
constraints=EnsureChoice(*sorted(SECTION_CALLABLES)) | EnsureNone(),
doc="""section to include. If not set, all sections. This option
can be given more than once to include multiple sections."""),
Copy link
Contributor

@kyleam kyleam May 21, 2019

This would benefit from [CMD] and [PY] markers.

Copy link
Member Author

@yarikoptic yarikoptic May 22, 2019

did for [CMD using similar phrasing as found in others


if clipboard:
external_versions.check(
'pyperclip', msg="It is needed to be able to use clipboard")
import pyperclip
report = _render_report(res)
pyperclip.copy(report)
pyperclip.copy(assure_bytes(report))
Copy link
Contributor

@kyleam kyleam May 21, 2019

Our tastes regarding collapsing unrelated changes into commits differ, so I try not to remark upon it too much, but this really should get its own commit!

Copy link
Member Author

@yarikoptic yarikoptic May 22, 2019

did that too ;-)

yarikoptic added 2 commits May 21, 2019
…b issue without clutter

Example use

   datalad wtf -S datalad -S extensions -S dependencies -D html_details -c

to be ready to just paste into the issue as is to get a nice

<details><summary>DataLad 0.11.4.dev107 WTF (datalad, extensions, dependencies)</summary>

  - full_version: 0.11.4.dev107-g03da-dirty
  - version: 0.11.4.dev107
  - tqdm: 4.28.1
  - git: git
  - wrapt: 1.11.1
  - boto: 2.44.0
  - keyrings.alt: 3.1
  - six: 1.12.0
  - keyring: 16.0.0
  - cmd:git: 2.20.1
  - iso8601: 0.1.11
  - mutagen: 1.40.0
  - exifread: 2.1.2
  - gitdb: 2.0.3
  - msgpack: 0.5.6
  - cmd:annex: 7.20190322+git133-g59922f1f4-1~ndall+1
  - requests: 2.21.0
  - cmd:system-ssh: 7.9p1
  - cmd:system-git: 2.21.0.593.g511ec345e18
  - humanize: 0.5.1
  - appdirs: 1.4.3
  - scrapy: 1.5.1
</details>
kyleam
kyleam approved these changes May 22, 2019
@kyleam kyleam merged commit af7dc19 into datalad:0.11.x May 22, 2019
2 of 4 checks passed
kyleam added a commit that referenced this issue May 22, 2019
Teach wtf the --section/-S option for limiting the output to specific
sections and the --decor/-D option, which currently knows how to
format the output as GitHub's `<details>` section.
@yarikoptic yarikoptic deleted the enh-wtf-S branch May 23, 2019
yarikoptic added a commit that referenced this issue May 28, 2019
0.11.5 (May 23, 2019) -- stability is not overrated

Should be faster and less buggy, with a few enhancements.

 Fixes

- [create-sibling][]  ([#3318][])
  - Siblings are no longer configured with a post-update hook unless a
    web interface is requested with `--ui`.
  - `git submodule update --init` is no longer called from the
    post-update hook.
  - If `--inherit` is given for a dataset without a superdataset, a
    warning is now given instead of raising an error.
- The internal command runner failed on Python 2 when its `env`
  argument had unicode values.  ([#3332][])
- The safeguard that prevents creating a dataset in a subdirectory
  that already contains tracked files for another repository failed on
  Git versions before 2.14.  For older Git versions, we now warn the
  caller that the safeguard is not active.  ([#3347][])
- A regression introduced in v0.11.1 prevented [save][] from committing
  changes under a subdirectory when the subdirectory was specified as
  a path argument.  ([#3106][])
- A workaround introduced in v0.11.1 made it possible for [save][] to
  do a partial commit with an annex file that has gone below the
  `annex.largefiles` threshold.  The logic of this workaround was
  faulty, leading to files being displayed as typechanged in the index
  following the commit.  ([#3365][])
- The resolve_path() helper confused paths that had a semicolon for
  SSH RIs.  ([#3425][])
- The detection of SSH RIs has been improved.  ([#3425][])

 Enhancements and new features

- The internal command runner was too aggressive in its decision to
  sleep.  ([#3322][])
- The "INFO" label in log messages now retains the default text color
  for the terminal rather than using white, which only worked well for
  terminals with dark backgrounds.  ([#3334][])
- A short flag `-R` is now available for the `--recursion-limit` flag,
  a flag shared by several subcommands.  ([#3340][])
- The authentication logic for [create-sibling-github][] has been
  revamped and now supports 2FA.  ([#3180][])
- New configuration option `datalad.ui.progressbar` can be used to
  configure the default backend for progress reporting ("none", for
  example, results in no progress bars being shown).  ([#3396][])
- A new progress backend, available by setting datalad.ui.progressbar
  to "log", replaces progress bars with a log message upon completion
  of an action.  ([#3396][])
- DataLad learned to consult the [NO_COLOR][] environment variable and
  the new `datalad.ui.color` configuration option when deciding to
  color output.  The default value, "auto", retains the current
  behavior of coloring output if attached to a TTY ([#3407][]).
- [clean][] now removes annex transfer directories, which is useful
  for cleaning up failed downloads. ([#3374][])
- [clone][] no longer refuses to clone into a local path that looks
  like a URL, making its behavior consistent with `git clone`.
  ([#3425][])
- [wtf][]
  - Learned to fall back to the `dist` package if `platform.dist`,
    which has been removed in the yet-to-be-release Python 3.8, does
    not exist.  ([#3439][])
  - Gained a `--section` option for limiting the output to specific
    sections and a `--decor` option, which currently knows how to
    format the output as GitHub's `<details>` section.  ([#3440][])

* tag '0.11.5': (96 commits)
  [DATALAD RUNCMD] make update-changelog
  Version boost and finalize CHANGELOG.md record
  ENH: new Makefile rule linkissues-changelog to link issues, which now will also be prerequisite for update-changelog
  CHANGELOG.md: Add entries for recently merged PRs
  ENH: require "distro" for python >= 3.8
  ENH: compat with python 3.8 which removed .dist -- try distro
  CLN: wtf: Remove unused (and duplicated) import
  DOC: wtf: Avoid double period in -S's description
  ENH: -D|--decor html_details -- to make it ready for pasting to github issue without clutter
  BF: assure bytes while giving to pyperclip upon its demand (on Py2)
  RF: move always present path + type "section" into "location" section, retain order of sections from cmdline
  RF: switch from nargs="*" to action=append for wtf -S
  ENH: wtf -S to specify which sections to query/display (by default -- all)
  MNT: Avoid invalid escape sequences in strings
  BF: export_to_figshare: Don't test identity of string literal
  BF(TST): do not assume user naiveness - treat any url-like looking path as a path
  BF: Check for /, \ or # in the username@hostname part while detecting SSHRI
  CHANGELOG.md: Add entry for gh-3374
  BF: revert back (remove) check for path being PathRI
  BF: list annex-transfer also in cmdline opt choice for "what"
  ...
yarikoptic added a commit that referenced this issue May 28, 2019
0.11.5 (May 23, 2019) -- stability is not overrated

Should be faster and less buggy, with a few enhancements.

 Fixes

- [create-sibling][]  ([#3318][])
  - Siblings are no longer configured with a post-update hook unless a
    web interface is requested with `--ui`.
  - `git submodule update --init` is no longer called from the
    post-update hook.
  - If `--inherit` is given for a dataset without a superdataset, a
    warning is now given instead of raising an error.
- The internal command runner failed on Python 2 when its `env`
  argument had unicode values.  ([#3332][])
- The safeguard that prevents creating a dataset in a subdirectory
  that already contains tracked files for another repository failed on
  Git versions before 2.14.  For older Git versions, we now warn the
  caller that the safeguard is not active.  ([#3347][])
- A regression introduced in v0.11.1 prevented [save][] from committing
  changes under a subdirectory when the subdirectory was specified as
  a path argument.  ([#3106][])
- A workaround introduced in v0.11.1 made it possible for [save][] to
  do a partial commit with an annex file that has gone below the
  `annex.largefiles` threshold.  The logic of this workaround was
  faulty, leading to files being displayed as typechanged in the index
  following the commit.  ([#3365][])
- The resolve_path() helper confused paths that had a semicolon for
  SSH RIs.  ([#3425][])
- The detection of SSH RIs has been improved.  ([#3425][])

 Enhancements and new features

- The internal command runner was too aggressive in its decision to
  sleep.  ([#3322][])
- The "INFO" label in log messages now retains the default text color
  for the terminal rather than using white, which only worked well for
  terminals with dark backgrounds.  ([#3334][])
- A short flag `-R` is now available for the `--recursion-limit` flag,
  a flag shared by several subcommands.  ([#3340][])
- The authentication logic for [create-sibling-github][] has been
  revamped and now supports 2FA.  ([#3180][])
- New configuration option `datalad.ui.progressbar` can be used to
  configure the default backend for progress reporting ("none", for
  example, results in no progress bars being shown).  ([#3396][])
- A new progress backend, available by setting datalad.ui.progressbar
  to "log", replaces progress bars with a log message upon completion
  of an action.  ([#3396][])
- DataLad learned to consult the [NO_COLOR][] environment variable and
  the new `datalad.ui.color` configuration option when deciding to
  color output.  The default value, "auto", retains the current
  behavior of coloring output if attached to a TTY ([#3407][]).
- [clean][] now removes annex transfer directories, which is useful
  for cleaning up failed downloads. ([#3374][])
- [clone][] no longer refuses to clone into a local path that looks
  like a URL, making its behavior consistent with `git clone`.
  ([#3425][])
- [wtf][]
  - Learned to fall back to the `dist` package if `platform.dist`,
    which has been removed in the yet-to-be-release Python 3.8, does
    not exist.  ([#3439][])
  - Gained a `--section` option for limiting the output to specific
    sections and a `--decor` option, which currently knows how to
    format the output as GitHub's `<details>` section.  ([#3440][])

* tag '0.11.5':
  BF: make test for url download more reliable in cases where connection fails
  RF: remove stale commented out duecredit in setup.py.  It has now its own section
@yarikoptic yarikoptic added this to the 0.11.x milestone Jul 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants