dvc/dagascii: Use pager instead of AsciiCanvas._do_draw

[ghost]

Uses Stdlib's pydoc to
draw the output in the interactive mode while doing e.g.
dvc pipeline show ...

Fixes #2807

• ❗ Have you followed the guidelines in the Contributing to DVC list?

• 📖 Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.

• ❌ Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addresses. Please review them carefully and fix those that actually improve code or fix bugs.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Related MR:
iterative/dvc.org#831

[ghost]

[casperdcl]

Was just looking through pydoc source and found this wonderful comment https://github.com/python/cpython/blob/a5ed2fe0eedefa1649aa93ee74a0bafc8e628a10/Lib/pydoc.py#L1479

pipes completely broken in Windows

[shcheklein]

@xliiv is it a draft/WIP still or ready to review?

[efiop]

Also, could you please elaborate on how pydoc's pager is different from git's pager?

[ghost]

@xliiv is it a draft/WIP still or ready to review?

It was ready for review. Now it needs some work

1. show a wide DAG
2. remove unused code
3. describe difference between pydoc's pager & git's pager

Also we can temporary make it opt-in via some kind of switch/config in case it doesn't work like it should?

[ghost]

@efiop i used a narrow window instead, is it ok? i thought that would be quicker.
If you know how to make text wide quickly, please share :)

[casperdcl]

@efiop pydoc.pager could wind up doing a number of things - including os.system('more ...'). I don't think there'll be any guarantees about how scrolling (either sideways or horizontally) works.

[ghost]

Btw isn't this a bit redundant

    if not hasattr(sys.stdin, "isatty"):
        return plainpager
    if not hasattr(sys.stdout, "isatty"):
        return plainpager
    if not sys.stdin.isatty() or not sys.stdout.isatty():
        return plainpager

from
https://github.com/python/cpython/blob/a5ed2fe0eedefa1649aa93ee74a0bafc8e628a10/Lib/pydoc.py#L1471-L1476

[casperdcl]

@xliiv I saw that but I think there are many more things wrong with that whole file. I've discussed it with a few people and we all agree it's our new favourite look-at-this-and-feel-better-about-your-badly-written-code example.

[casperdcl]

@efiop I'm tempted to say don't rely on pydoc and perhaps find another alternative to asciimatics if it isn't fit for purpose anymore.

[ghost]

@efiop I'm tempted to say don't rely on pydoc

@casperdcl could you argue that, i'm just curious of your opinion

[casperdcl]

erm... this is what pydoc does.

if not isatty:
  return plain text
elif MANPAGER/PAGER in env:
  if win/dumb/emacs:
    return plain text to pager
  else:
    return unicode text to pager
elif dumb/emacs:
  return plain text
elif win:
  return plain text to more
elif less exists:
  return unicode text to less
else:
  try: really hard to use more
  except: return ttypager # what?

P.S. ttypager is https://github.com/python/cpython/blob/0aca3a3a1e68b4ca2d334ab5255dfc267719096e/Lib/pydoc.py#L1546-L1588 a.k.a undocumented logike moste insane. I mean, I could probably explain everything it's trying to do but, erm, I don't want to. Really don't want to.

[efiop]

@casperdcl Great points! Have you seen the git source linked in the original issue? E.g. https://github.com/git/git/blob/v2.24.0/pager.c#L48 What are your thoughts about that? At least pydoc is built-in and maintained by someone else. Also seems like a pretty core feature that we could kinda rely on working on most pythons.

[ghost]

😍 thanks a lot!

[efiop]

Btw, @xliiv, would this work on windows in cmd/powershell/gitbash? Esp with horizontal scrolling.

[efiop]

https://github.com/iterative/dvc/pull/2815/files#r349441964

@xliiv You could add a few stages to the pipeline 🙂 E.g.

DEPS=                                                                           
for i in {1..10}; do                                                            
        echo $i > $i                                                            
        dvc add $i                                                              
        DEPS="$DEPS -d $i"                                                      
done                                                                            
                                                                                
dvc run $DEPS -f run.dvc echo "run"                                             
                                                                                
dvc pipeline show --ascii run.dvc

Or, you could draw something wide directly and feed it to pydoc directly and see how that shows up 🙂 But making the terminal smaller also does totally count 🙂

[casperdcl]

@efiop I meant to say we can rely on pydoc working but not on how it works (UX). Basically a less vs more experience.

[efiop]

@casperdcl Could you elaborate please? more is a legacy thingy, but it still supports horizontal scrolling with more -S. Seems pretty suitable.

[efiop]

@casperdcl Or you mean that the fact that pydoc's pager is all over the place, depending on which tool it chooses to use, is a UX concern?

[ghost]

@efiop thanks for the snippet.

Btw, @xliiv, would this work on windows in cmd/powershell/gitbash? Esp with horizontal scrolling.

I'll spin up a Windows VM and i'll test it.

Todos

• use the snippet to test a wide dag
• test it on Win Cmd
• test it on Win Powershell
• test it on Win Gitbash
• resolve code comments

[efiop]

@casperdcl Regarding the UX. Tried both more and less and they seem to be able to navigate horizontaly and vertically pretty normally, as long as you are running a sane terminal. E.g. arrows work fine, so that is nice :) Still feels better than our own ncurses implementation 🙂

[ghost]

Regarding the UX. Tried both more and less and they seem to be able to navigate horizontaly and vertically pretty normally, as long as you are running a sane terminal.

@efiop Are you sure? I tested this approach (and standalone more) today on Windows (cmd, Power shell) and looks like more is useless. It doesn't support arrows. Can scroll only forward, etc.

See also
https://unix.stackexchange.com/questions/81129/what-are-the-differences-between-most-more-and-less

So, for now, we have to find a different solution IMO. I'll think of a new approach and i'll put the solution here when find something.

[casperdcl]

Though if a submitter rebases to remove/squash/drop extraneous commits then it always looks nicer and easier to track, merge, fix conflicts, and debug.

[casperdcl]

Btw @efiop (realted to another discussion somewhere, cc @shcheklein) I don't think it's a problem for maintainers to also do some amount of rebasing before merging. I'm against outright squash-merges in most cases though.

[efiop]

@casperdcl We are not doing squashes anymore. The current consensus is to format nicely if you can, but we don't require any specific formatting from outside contributors. And in both cases everything gets simply merged, without squashes.

[efiop]

@xliiv Also note at the tests failed:

___________________ test_less_pager_returned_when_less_found ___________________
[gw3] linux -- Python 3.7.1 /home/travis/virtualenv/python3.7.1/bin/python
    def test_less_pager_returned_when_less_found():
        with mock.patch.object(os, "system") as m:
            m.return_value = 0
            pager = dagascii.find_pager()
    
>       assert pager.cmd == dagascii.DEFAULT_PAGER_FORMATTED
E       AttributeError: 'function' object has no attribute 'cmd'
tests/unit/test_dagascii.py:13: AttributeError
_________________ test_tempfilepager_returned_when_var_defined _________________
[gw3] linux -- Python 3.7.1 /home/travis/virtualenv/python3.7.1/bin/python
    def test_tempfilepager_returned_when_var_defined():
        os.environ[DVC_PAGER] = dagascii.DEFAULT_PAGER
        pager = dagascii.find_pager()
    
>       assert pager.cmd == dagascii.DEFAULT_PAGER
E       AttributeError: 'function' object has no attribute 'cmd'
tests/unit/test_dagascii.py:28: AttributeError

[ghost]

once build is successful and i have approval, i'll squash commits

[efiop]

@xliiv FYI: tests failed.

[efiop]

Thanks! 🙂

[efiop]

once build is successful and i have approval, i'll squash commits

Oops, sorry about that, merged as-is.

[ghost]

Thanks! slightly_smiling_face

Glad could do it :)

Oops, sorry about that, merged as-is.

everything is ok 👍