Make REPL tab completion more extensible

[ericphanson]

I think it would be really nice to be able to tab-complete AWSS3.jl's S3Paths. I took a look at the REPL code, and it seems like one would need to modify REPL.REPLCompletions.completions for this to be possible; we can't e.g. add a dispatch or such. Here are two options I came up with, but perhaps there are different/better ones too.

Does either of these sound OK? Are there better ways?

Option 1: String macros customize their tab completion?

One option I think would be to allow string macros to customize how they tab-complete. In

julia/stdlib/REPL/src/REPLCompletions.jl

Lines 797 to 798 in f879d20

	startpos = nextind(partial, reverseind(partial, m.offset))
	r = startpos:pos

we take the part of the string starting after the quote; if we take the part before, we get the name of the string macro. E.g. FilePathsBase defines a string macro @p_str, and we can parse out the p by adding

if m.match == "\"" # string macro?
    before_start_pos = prevind(partial, reverseind(partial, m.offset))
    str_macro_name = partial[1:before_start_pos]
    if !isempty(str_macro_name)
        @show str_macro_name
    end
end

However, I'm not sure how to actually get the macro without eval (if we didn't mind eval, then eval(Symbol("@", str_macro_name, "_str")) gives me var"@p_str" which we can dispatch on with f(::typeof(var"@p_str")) = ...). It seems bad to eval user code when trying to tab-complete it. However, if we could get the macro as an Julia object, then we could call some function ok, ret = tab_completion(macro_object, ...) and use dispatch, so that string-macro-authors could provide dispatches for their macros (probably with tab_completion having a fallback Any that uses the current path completion logic).

Option 2: Add some hooks

So then the second option I thought of would be to just have some hooks, by adding something like

for hook in TAB_COMPLETION_HOOKS
    ok, ret = hook(inc_tag, string, pos)
    ok && return ret
end

after string-path completion (I put it here:

julia/stdlib/REPL/src/REPLCompletions.jl

Line 812 in f879d20

). Hooks could decide they don't apply and return false for ok. In this case, one would need to make sure different hooks don't clash. Adding that code for hooks, and writing a hook,

using AWSS3
using REPL: REPLCompletions
using .REPLCompletions: Completion, PathCompletion

push!(REPLCompletions.TAB_COMPLETION_HOOKS,
      function (inc_tag, string, pos)
          partial = string[1:pos]
          m = match(r"[\t\n\r\"`><=*?|]| (?!\\)", reverse(partial))
          startpos = nextind(partial, reverseind(partial, m.offset))
          r = startpos:pos
          path = replace(string[r], r"\\ " => " ")
          p = tryparse(S3Path, path)
          p === nothing && return false, ()
          dir, prefix = splitdir(p)
          isdir(dir) || return false, ()
          match_list = Completion[PathCompletion(joinpath(dir, pa))
                                  for pa in readdir(dir) if startswith(pa, prefix)]
          ok = !isempty(match_list)
          return ok, (match_list, r, ok)
      end)

# some patches...
Base.basename(fp::S3Path) = isempty(fp.segments) ? "" : fp.segments[end] # FilePathsBase's version errors for bucket with no key
Base.splitdir(fp::S3Path) = isdir(fp) ? (fp, "") : (dirname(fp), basename(fp)) # FilePathsBase's version doesn't seem to behave correctly for directories

I get semi-working tab-completion! As suggested completions, it prints the entire bucket + key instead of just the part being completed, presumably because r is not the correct second-argument to return, however it does complete correctly (e.g. if there's only one option, it fills it in).

[vtjnash]

Hooks sounds great to me, as you described. Though the set of completions should probably be the union of all results, and not the first result.