Python's pip and pipenv completions #4448

cprieto · 2017-10-04T06:37:07Z

pip and pipenv completions

Both pip and pipenv has its own completion support for fish, so we only call their native completion support if the command is installed in the path, nothing fancy but this avoids having to manually install the completion.

* We call native pip completion for fish if pip is installed

* We call pipenv native completion if pipenv is installed

mqudsi · 2017-10-04T16:14:52Z

share/completions/pipenv.fish

@@ -0,0 +1,3 @@
+if test (command -v pipenv)
+    eval (pipenv --completion)


Are there any guarantees that --completion will always be fish-compatible?

It is guarantee to always return completion for the shell pointed by the SHELL environment variable. This is the way is documented in the official documentation and I checked the source code (of Pipenv) as well.

It's certainly producing fish-compatible output intended to be eval'd.

$ pipenv --completion complete --command pipenv --arguments "(env _PIPENV_COMPLETE=complete-fish COMMANDLINE=(commandline -cp) pipenv)" -f

Yeah, I checked the source code for pipenv and click-completion before submitting the patch. Pretty good code if you ask me.

The only concern will be if you set SHELL as something that is not fish, but I think in that case not only this completion will fail.

floam · 2017-10-05T03:15:19Z

share/completions/pip.fish

@@ -0,0 +1,3 @@
+if test (command -v pip)


This can just be if command -sq pip

floam · 2017-10-05T03:15:49Z

share/completions/pipenv.fish

@@ -0,0 +1,3 @@
+if test (command -v pipenv)


This can just be if command -sq pipenv

* Changed usage of `test (command -v)` for just `command -sq`

floam · 2017-10-05T03:31:21Z

On my system, I notice there is only pip2 and pip2.7 - no pip. How do we want to handle this? @faho? A symlink? Git handles them fine.

floam · 2017-10-05T03:32:24Z

That won't just work, of course, the function name would be wrong. I guess the one file could have multiple functions in it.

cprieto · 2017-10-05T03:39:41Z

@floam I actually thought about it (in my system I have both pip2 and pip3 and depending of the virtualenv used it changes and create an alias for the correct pip pointed to your Python version). In fact, the first version I did had the functions pip2 and pip3 for that case but I thought it will be too much. If the scenario where both pip exists and are not aliased by the virtualenv it will make sense, though I have no idea how common that scenario would be (in my Mac and Ubuntu both look to do the same pip2/3 aliasing to pip).

should I add the same conditions for both?

* In some systems pip is not aliased and we have pip2 and pip3 * In those cases, we just load the completions for those commands

floam · 2017-10-05T04:01:40Z

You'll need to add a pip2.fish and a pip3.fish - either symlinked or just copies with a function each.

@floam

* As requested by @floam

@floam

* Add pip completion * We call native pip completion for fish if pip is installed * Add pipenv completion * We call pipenv native completion if pipenv is installed * Applied changes as requested by @floam * Changed usage of `test (command -v)` for just `command -sq` * Add completions for pip2/3 * In some systems pip is not aliased and we have pip2 and pip3 * In those cases, we just load the completions for those commands * Separate pip2/3 completions in their own file as requested by @floam

floam · 2017-10-05T04:16:15Z

Merged, thanks!

floam · 2017-10-05T04:21:43Z

Hm, I do notice something keeping pip2 completions from working here, but this ought to be addressed upstream:

$ pip2 completion --fish

# pip fish completion start
function __fish_complete_pip
    set -lx COMP_WORDS (commandline -o) ""
    set -lx COMP_CWORD (math (contains -i -- (commandline -t) $COMP_WORDS)-1)
    set -lx PIP_AUTO_COMPLETE 1
    string split \  -- (eval $COMP_WORDS[1])
end
complete -fa "(__fish_complete_pip)" -c pip
# pip fish completion end

Running pip2 completion --fish outputs a complete command for pip, not pip2.

floam · 2017-10-05T04:21:52Z

Any chance you could report this there?

cprieto · 2017-10-05T04:23:11Z

Sure, I will open the ticket at pip, I was afraid something like this could happen. Let me see what can be done.

cprieto · 2017-10-05T04:25:38Z

I think I have a simple quick way to solve this and submit a patch to pip, let me work on that this arvo taking advantage I don't have much to do.

floam · 2017-10-05T04:30:17Z

It's also rather unfortunate that pip is quite slow to output the code, causing a noticeable lag on hitting tab:

$ pip2 completion --fish > /dev/null
$ echo $CMD_DURATION
636

... but there's not a lot we here can do about that.

floam · 2017-10-05T04:35:45Z

Hey, there's already an open PR over there for the command name thing:

pypa/pip#4755

cprieto · 2017-10-05T04:38:19Z

Yeah I saw the patch before leaving (right now on a tram in my way home). Should we wait and see?

floam · 2017-10-05T04:40:37Z

Yep.

Work around bug pypa/pip#4755 Don't expect all users to be running a version of pip2/3 that includes the fix (once it's upstreamed). Will continue to work if/when pip2/3 emit the correct output. pip is already very slow at printing the completions (see #4448) so the `sed` call overhead is neglible.

mqudsi · 2017-10-05T11:08:54Z

Fixed in b11ca2c and 3f9273c

floam · 2017-10-05T11:10:46Z

How will that continue to work when they fix it, @mqudsi? Won't it cause -c pip2 -> -c pip22?

@floam

Thanks @floam. See #4448 (cherry picked from commit 78fe534)

mqudsi · 2017-10-05T11:29:01Z

Thanks. Corrected and pushed.

floam · 2017-10-05T11:33:07Z

Ok, it still doesn't work. I assumed that was it. Did you test it? I don't think the regex is correct for PCRE. Might want to check the documentation they have. Sorry for saying it looked good.

I'm getting complete -fa "(__fish_complete_pip)" -c pip.

mqudsi · 2017-10-05T11:35:46Z

Ugh, I tested it.. only my test case emitted a plain pip in the first place :/ Sorry.

mqudsi · 2017-10-05T11:39:09Z

It's the string replace that's the issue. I have to put both the pattern and the completion after --. I had originally tested it with sed -r and then I botched the test with string replace.

floam · 2017-10-05T11:40:11Z

pip2 completion --fish | string replace -r -- "\b-c\s+pip\b" "-c pip2" also prints output with -c pip here.

mqudsi · 2017-10-05T11:55:59Z

Word boundaries don't match words starting with -, which is... annoying.

mqudsi@ZBook ~/r/fish-shell> echo "hello -c pip something" | string replace -r -- ' -c\s+pip\b' ' -c pip3'
hello -c pip3 something

Fixed and pushed (yet again).

floam · 2017-10-05T12:08:29Z

Looks like --disable-pip-version-check speeds it up spitting out code a little bit, around 100ms. Still unpleasantly slow, though.

cprieto · 2017-10-05T12:10:01Z

I will take a look at that with the python profiler tomorrow. Thanks @floam

faho · 2017-10-06T09:19:09Z

Soo... if the output of pip completion --fish looks like this:

# pip fish completion start
function __fish_complete_pip
    set -lx COMP_WORDS (commandline -o) ""
    set -lx COMP_CWORD (math (contains -i -- (commandline -t) $COMP_WORDS)-1)
    set -lx PIP_AUTO_COMPLETE 1
    string split \  -- (eval $COMP_WORDS[1])
end
complete -fa "(__fish_complete_pip)" -c pip

Then why are we bothering with the source thing at all? Why not just include that function directly? (Also, I'd replace the eval $COMP_WORDS[1] with a direct call to pip or pip2 or pip3)

floam · 2017-10-06T09:22:34Z

I'm not sure there's any guarantee the interface will remain stable. There are open PRs there that add new features and change the function IIRC. Since they're providing the feature officially through pip completion it seems wise to me to just let them handle it.

faho · 2017-10-06T09:25:38Z

That current interface is a direct translation to the bash completion system, so it kinda has to remain stable. In case they add any fish-specific features, we can remove this again, or conditionalize it on the version.

floam · 2017-10-06T09:27:21Z

Hm, pip takes like 500ms just to tell you what version it is. And it seems way more future-proof just to let it do its thing.

faho · 2017-10-06T09:30:25Z

I'm sorry, I don't follow? By using the function directly, we can remove that first call to pip, so we're 500ms faster.

Or did I not have enough coffee this morning?

floam · 2017-10-06T09:34:05Z

Right, I just mean that a version check would render useless the optimization, and mean we have to do the work to adjust down the road if anything changed.

floam · 2017-10-06T09:34:52Z

If you're OK with having to have to potentially track pip though, yeah, your way would make the first completion faster today.

faho · 2017-10-06T09:40:17Z

Ah, okay.

Would it be possible to get the pip folks to just ship a completion file instead?

That would seem like the best of both worlds - skip the source step and track whatever pip is doing.

floam · 2017-10-06T09:43:12Z

I guess it'd need to have some placeholder in there to be replaced by fish with the actual command name being used, since the same file could apply to pip, pip2, pip2.7.

floam · 2017-10-06T09:44:08Z

Or multiple files. But sure, couldn't hurt to ask.

floam · 2017-10-06T09:45:54Z

Now, the way python modules are installed, it might be an exercise actually locating a file installed into its share directory or wherever.

edit: nevermind, obviously we'd want them to have it install that into a system directory on install, just like the symlinks in /usr/local/bin.

faho · 2017-10-06T09:49:50Z

Note: pip's documentation currently says to use

pip completion --fish > ~/.config/fish/completions/pip.fish

for fish. So I'd wager that, in practice, tracking of upstream changes doesn't happen anyway.

Now, I'd love to tell you how often the completion script changed but it changed files at the end of august and I can't get github to track that.

floam · 2017-10-06T09:51:09Z

OK, I'm sold if you want to do it.

cprieto · 2017-10-06T09:54:29Z

What’s the suggestion again? Using a placeholder for the command name? I’m happy to do it

faho · 2017-10-06T09:55:30Z

@cprieto: The suggestion is to embed the output of pip completion --fish directly into the completion script, instead of running pip completion --fish | source to get it.

cprieto · 2017-10-06T09:55:56Z

I am happy removing the pipX.fish files from the code and wait until everything looks fine in the pip side

faho · 2017-10-06T09:55:59Z

Plus replacing the eval $COMP_WORDS[1] with pipX.

cprieto · 2017-10-06T10:02:56Z

Ok I’m doing it, no problem

cprieto added 2 commits October 4, 2017 17:33

Add pip completion

1a2a2e6

* We call native pip completion for fish if pip is installed

Add pipenv completion

e0a2279

* We call pipenv native completion if pipenv is installed

mqudsi reviewed Oct 4, 2017

View reviewed changes

floam requested changes Oct 5, 2017

View reviewed changes

Applied changes as requested by @floam

78ed58b

* Changed usage of `test (command -v)` for just `command -sq`

floam added this to the fish 2.7.0 milestone Oct 5, 2017

floam added the completions label Oct 5, 2017

floam self-assigned this Oct 5, 2017

Add completions for pip2/3

86830ac

* In some systems pip is not aliased and we have pip2 and pip3 * In those cases, we just load the completions for those commands

Separate pip2/3 completions in their own file

0fb8053

* As requested by @floam

floam merged commit 1770e47 into fish-shell:master Oct 5, 2017

floam mentioned this pull request Oct 5, 2017

Use correct executable name for completion command pypa/pip#4755

Merged

mqudsi added a commit that referenced this pull request Oct 5, 2017

Fix copy-and-paste pip3 -> pip2 typo

de45232

Thanks @floam. See #4448 (cherry picked from commit 78fe534)

nesl247 mentioned this pull request Jan 11, 2018

Fish 2.7.1 includes completions sentriz/fish-pipenv#12

Closed

faho mentioned this pull request Jun 10, 2018

pip hangs when using fish-shell #5040

Closed

github-actions bot locked as resolved and limited conversation to collaborators Apr 17, 2020

		@@ -0,0 +1,3 @@
		if test (command -v pipenv)
		eval (pipenv --completion)

Python's pip and pipenv completions #4448

Python's pip and pipenv completions #4448

Conversation

cprieto commented Oct 4, 2017

pip and pipenv completions

mqudsi Oct 4, 2017

Choose a reason for hiding this comment

cprieto Oct 5, 2017

Choose a reason for hiding this comment

floam Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

cprieto Oct 5, 2017

Choose a reason for hiding this comment

floam Oct 5, 2017

Choose a reason for hiding this comment

floam Oct 5, 2017

Choose a reason for hiding this comment

floam commented Oct 5, 2017 • edited Loading

floam commented Oct 5, 2017

cprieto commented Oct 5, 2017

floam commented Oct 5, 2017

floam commented Oct 5, 2017

floam commented Oct 5, 2017 • edited Loading

floam commented Oct 5, 2017

cprieto commented Oct 5, 2017

cprieto commented Oct 5, 2017

floam commented Oct 5, 2017 • edited Loading

floam commented Oct 5, 2017

cprieto commented Oct 5, 2017

floam commented Oct 5, 2017

mqudsi commented Oct 5, 2017

floam commented Oct 5, 2017

mqudsi commented Oct 5, 2017

floam commented Oct 5, 2017 • edited Loading

mqudsi commented Oct 5, 2017

mqudsi commented Oct 5, 2017

floam commented Oct 5, 2017 • edited Loading

mqudsi commented Oct 5, 2017 • edited Loading

floam commented Oct 5, 2017 • edited Loading

cprieto commented Oct 5, 2017

faho commented Oct 6, 2017

floam commented Oct 6, 2017 • edited Loading

faho commented Oct 6, 2017

floam commented Oct 6, 2017 • edited Loading

faho commented Oct 6, 2017

floam commented Oct 6, 2017

floam commented Oct 6, 2017 • edited Loading

faho commented Oct 6, 2017

floam commented Oct 6, 2017

floam commented Oct 6, 2017 • edited Loading

floam commented Oct 6, 2017 • edited Loading

faho commented Oct 6, 2017

floam commented Oct 6, 2017

cprieto commented Oct 6, 2017

faho commented Oct 6, 2017

cprieto commented Oct 6, 2017

faho commented Oct 6, 2017

cprieto commented Oct 6, 2017

floam Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

mqudsi commented Oct 5, 2017 •

edited

Loading

floam commented Oct 5, 2017 •

edited

Loading

floam commented Oct 6, 2017 •

edited

Loading

floam commented Oct 6, 2017 •

edited

Loading

floam commented Oct 6, 2017 •

edited

Loading

floam commented Oct 6, 2017 •

edited

Loading

floam commented Oct 6, 2017 •

edited

Loading