zsh completion fails

[gedge]

_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]

ack v3.8.0 (standalone version) on macOS (latest)

[petdance]

Is this something you're actually using? What did you run to cause the problem? Do you know how to fix it?

[lorenmh]

I'm not the issue creator, but I came here to post the same issue. Google pointed me to this homebrew issue where a homebrew maintainer mentioned it was likely coming from this file:
https://github.com/zsh-users/zsh/blob/d051857e03d474bf7a122148f323bd90d8c16b36/Completion/Unix/Command/_ack#L22C1-L27C3

I downgraded to 3.7.0 and tab-completion appears to work with zsh. (appears not to work with 3.7.0 as well)

edit: seems it's coming from oh-my-zsh in my case

[petdance]

Can someone please give specifics on what happens to make this error happen. Is this something that happens when you start zsh and it tries to scrape filetypes to set up file completions?

[lorenmh]

tldr; I believe this issue has to do with an oh-my-zsh update, I think you can probably close the issue, apologies for wasting your time.

Issue manifested itself today for me, when I tried to do tab completion with ack:

# in zsh
% ack <tab>

results in:

% ack
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]

so, a bunch of garbage is spit out into the terminal and tab completion doesnt work. First assumption was that it was be ack because of the recent update. Downgrading to 3.7.0 still has the issue. Removing oh-my-zsh from my .zshrc fixes the issue for me. I checked out my oh-my-zsh directory and sure enough it magically was updated today for me. Am assuming others maybe have auto-updates enabled for oh-my-zsh and are hitting this too.

[petdance]

Interesting. zsh doesn't figure out the tab completion until the first time you do ack<tab>?

Seems to me the most robust way to have zsh get the available arguments for ack is to get them from ack itself. For example, ripgrep will generate zsh completions.

$ rg --generate=complete-zsh
#compdef rg

##
# zsh completion function for ripgrep
#
# Run ci/test-complete after building to ensure that the options supported by
# this function stay in synch with the `rg` binary.
#
# For convenience, a completion reference guide is included at the bottom of
# this file.
#
# Originally based on code from the zsh-users project — see copyright notice
# below.

_rg() {
  local curcontext=$curcontext no='!' descr ret=1
  local -a context line state state_descr args tmp suf
  local -A opt_args
... 647 more lines ...

I'd be glad to work with someone to figure something out to do this. I'm no expert in how shell completions work myself, but it seems like it should be fairly simple with someone who does. See also #332

[lorenmh]

I was wrong about this issue having to do with oh-my-zsh auto-updating, I think there was a change in ack 3.8.0 which somehow is causing this issue. There's some (I believe?) autocompletion information which is cached by zsh/oh-my-zsh, which is why I was still seeing the issue when I downgraded to 3.7.0. Sorry I can't be more help because I don't fully understand how these completions work, but the following is a repro of the issue for me. To remedy the issue on my device, I need to downgrade to 3.7.0 and also remove the cached ~/.oh-my-zsh/cache/ack-types:

repro:

% sudo curl https://beyondgrep.com/ack-v3.7.0 > /usr/local/bin/ack && sudo chmod 0755 /usr/local/bin/ack
% ack --version
ack v3.7.0 (standalone version)
Running under Perl v5.34.1 at /usr/bin/perl

Copyright 2005-2023 Andy Lester.

This program is free software.  You may modify or distribute it
under the terms of the Artistic License v2.0.
% rm ~/.oh-my-zsh/cache/ack-types
% source ~/.oh-my-zsh/oh-my-zsh.sh
% ack # I pressed tab here, no completion because empty, populates completion cache
# produces this cached file
% cat ~/.oh-my-zsh/cache/ack-types
_ack_raw_types=( ${(Q)"${(z)$(<<\EO:_ack_raw_types
'Usage: ack [OPTION]... PATTERN [FILES OR DIRECTORIES]' 'The following is the list of filetypes supported by ack.  You can specify a' 'filetype to include with -t TYPE or --type=TYPE.  You can exclude a' 'filetype with -T TYPE or --type=noTYPE.' 'Note that some files may appear in multiple types.  For example, a file' 'called Rakefile is both Ruby (--type=ruby) and Rakefile (--type=rakefile).' '    actionscript .as .mxml' '    ada          .ada .adb .ads' '    asm          .asm .s' '    asp          .asp' '    aspx         .master .ascx .asmx .aspx .svc' '    batch        .bat .cmd' '    bazel        .bzl; .bazelrc; BUILD; WORKSPACE' '    cc           .c .h .xs' '    cfmx         .cfc .cfm .cfml' '    clojure      .clj .cljs .edn .cljc' '    cmake        CMakeLists.txt; .cmake' '    coffeescript .coffee' '    cpp          .cpp .cc .cxx .m .hpp .hh .h .hxx' '    crystal      .cr .ecr' '    csharp       .cs' '    css          .css' '    dart         .dart' '    delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj' '    elisp        .el' '    elixir       .ex .exs' '    elm          .elm' '    erlang       .erl .hrl' '    fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp' '    go           .go' '    groovy       .groovy .gtmpl .gpp .grunit .gradle' '    gsp          .gsp' '    haskell      .hs .lhs' '    hh           .h' '    hpp          .hpp .hh .h .hxx' '    html         .htm .html .xhtml' '    jade         .jade' '    java         .java .properties' '    js           .js' '    json         .json' '    jsp          .jsp .jspx .jspf .jhtm .jhtml' '    kotlin       .kt .kts' '    less         .less' '    lisp         .lisp .lsp' '    lua          .lua' '    make         .mk; .mak; makefile; Makefile; Makefile.Debug; Makefile.Release; GNUmakefile' '    markdown     .md .markdown' '    matlab       .m' '    objc         .m .h' '    objcpp       .mm .h' '    ocaml        .ml .mli .mll .mly' '    perl         .pl .pm .pod .t .psgi' '    perltest     .t' '    php          .php .phpt .php3 .php4 .php5 .phtml' '    plone        .pt .cpt .metadata .cpy .py' '    pod          .pod' '    powershell   .ps1 .psm1' '    purescript   .purs' '    python       .py' '    rake         Rakefile' '    rr           .R .Rmd' '    rst          .rst' '    ruby         .rb .rhtml .rjs .rxml .erb .rake .spec; Rakefile' '    rust         .rs' '    sass         .sass .scss' '    scala        .scala .sbt' '    scheme       .scm .ss' '    shell        .sh .bash .csh .tcsh .ksh .zsh .fish' '    smalltalk    .st' '    smarty       .tpl' '    sql          .sql .ctl' '    stylus       .styl' '    svg          .svg' '    swift        .swift' '    tcl          .tcl .itcl .itk' '    tex          .tex .cls .sty' '    toml         .toml' '    ts           .ts .tsx' '    ttml         .tt .tt2 .ttml' '    vb           .bas .cls .frm .ctl .vb .resx' '    verilog      .v .vh .sv' '    vhdl         .vhd .vhdl' '    vim          .vim' '    xml          .xml .dtd .xsd .xsl .xslt .ent .wsdl' '    yaml         .yaml .yml'
EO:_ack_raw_types
)}"} )
% rm ~/.oh-my-zsh/cache/ack-types. # clear out the cached file and update ack to 3.8.0
% sudo curl https://beyondgrep.com/ack-v3.8.0 > /usr/local/bin/ack && sudo chmod 0755 /usr/local/bin/ack
% ack --version
ack v3.8.0 (standalone version)
Running under Perl v5.34.1 at /usr/bin/perl

Copyright 2005-2024 Andy Lester.

This program is free software.  You may modify or distribute it
under the terms of the Artistic License v2.0.
% ack # I pressed tab here, garbage output to terminal
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]
_arguments:comparguments:327: invalid option definition: --[pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)]
% cat ~/.oh-my-zsh/cache/ack-types 
_ack_raw_types=( ${(Q)"${(z)$(<<\EO:_ack_raw_types
'Usage: ack [OPTION]... PATTERN [FILES OR DIRECTORIES]' 'The following is the list of filetypes supported by ack.  You can specify a' 'filetype to include with -t TYPE or --type=TYPE.  You can exclude a' 'filetype with -T TYPE or --type=noTYPE.' 'Note that some files may appear in multiple types.  For example, a file' 'called Rakefile is both Ruby (--type=ruby) and Rakefile (--type=rakefile).' '    actionscript .as .mxml' '    ada          .ada .adb .ads' '    asm          .asm .s' '    asp          .asp' '    aspx         .master .ascx .asmx .aspx .svc' '    batch        .bat .cmd' '    bazel        .bzl; .bazelrc; BUILD; WORKSPACE' '    cc           .c .h .xs' '    cfmx         .cfc .cfm .cfml' '    clojure      .clj .cljs .edn .cljc' '    cmake        CMakeLists.txt; .cmake' '    coffeescript .coffee' '    cpp          .cpp .cc .cxx .m .hpp .hh .h .hxx' '    crystal      .cr .ecr' '    csharp       .cs' '    css          .css' '    dart         .dart' '    delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj' '    elisp        .el' '    elixir       .ex .exs' '    elm          .elm' '    erlang       .erl .hrl' '    fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp' '    go           .go' '    groovy       .groovy .gtmpl .gpp .grunit .gradle' '    gsp          .gsp' '    haskell      .hs .lhs' '    hh           .h' '    hpp          .hpp .hh .h .hxx' '    html         .htm .html .xhtml' '    jade         .jade' '    java         .java .properties' '    js           .js' '    json         .json' '    jsp          .jsp .jspx .jspf .jhtm .jhtml' '    kotlin       .kt .kts' '    less         .less' '    lisp         .lisp .lsp' '    lua          .lua' '    make         .mk; .mak; makefile; Makefile; Makefile.Debug; Makefile.Release; GNUmakefile' '    markdown     .md .markdown' '    matlab       .m' '    objc         .m .h' '    objcpp       .mm .h' '    ocaml        .ml .mli .mll .mly' '    perl         .pl .pm .pod .t .psgi' '    perltest     .t' '    php          .php .phpt .php3 .php4 .php5 .phtml' '    plone        .pt .cpt .metadata .cpy .py' '    pod          .pod' '    powershell   .ps1 .psm1' '    purescript   .purs' '    pytest       Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)' '    python       .py' '    rake         Rakefile' '    rr           .R .Rmd' '    rst          .rst' '    ruby         .rb .rhtml .rjs .rxml .erb .rake .spec; Rakefile' '    rust         .rs' '    sass         .sass .scss' '    scala        .scala .sbt' '    scheme       .scm .ss' '    shell        .sh .bash .csh .tcsh .ksh .zsh .fish' '    smalltalk    .st' '    smarty       .tpl' '    sql          .sql .ctl' '    stylus       .styl' '    svg          .svg' '    swift        .swift' '    tcl          .tcl .itcl .itk' '    terraform    .tf .tfvars' '    tex          .tex .cls .sty' '    toml         .toml' '    ts           .ts .tsx' '    ttml         .tt .tt2 .ttml' '    vb           .bas .cls .frm .ctl .vb .resx' '    verilog      .v .vh .sv' '    vhdl         .vhd .vhdl' '    vim          .vim' '    xml          .xml .dtd .xsd .xsl .xslt .ent .wsdl' '    yaml         .yaml .yml'
EO:_ack_raw_types
)}"} )

Note that with 3.8.0, however this file is generated, there's Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$)' ' in the ~/.oh-my-zsh/cache/ack-types file now, whereas with 3.7.0 it was not there. I'm not sure if this issue is really coming from ack or elsewhere.

[n1vux]

* `arguments:comparguments` is a known error message from ZSH. * ack v3.8.0 did add pytest as a new file type, so yes, 3.8.0 new feature triggers this error in ZSH. ``` $ git diff v3.7.0 -- lib/App/Ack/ConfigDefault.pm ... +# Pytest +# https://pytest.org/ +--type-add=pytest:match:_test[.]py$ +--type-add=pytest:match:^test_.*[.]py$ ... ``` * The message from ZSH echos the output from `ack --help-types` , so ZSH (or oh-my-zsh) has meta-config to query `ack --help-types` (and possibly --version to know when to re-cache?) ``` $ ./ack-standalone --help-types | ack pytest pytest Filename matches (?^i:_test[.]py$); Filename matches (?^i:^test_.*[.]py$) ``` It looks suspiciously like ZSH failed to quote one or more metacharacters that are special to ZSH as well as to ack, perl, etc when the option definition string extracted from ack --help-types includes an embedded regex, and so zsh generates an **invalid option definition** which it then detects and rejects. pytest is not the first default-config entry that results in `matches ${re}` in the `--help-types` output, but it is the first to have **two** such on one line. `$ ./ack-standalone --help-types | ack matches` This suggests ZSH checks for an embedded RE and patches that, but doesn't do-while any un-quoted REs remain? This all means it is ZSH (or oh-my-zsh)'s bug, not previously triggered by ack v.3.7.0 , triggered by ack v.3.8.0 . Thus this should be reported to ZSH &/or oh-my-zsh as their bug to fix. *Workarounds* for Loren, Geraint, and any other ZSH users to try - * delete one of the two pytest lines in lib/App/Ack/ConfigDefault.pm (or equivalent section of ack-standalone) * combine the two into a single `--type-add=pytest:match:(?x:\A test _ .* | _test )[.]py$` * delete the whole pytest paragraph in that file if you don't test python so don't need that 3.8.0 upgrade * revert to ack 3.70 * if ZSH is detecting version number ≥ prior seen for re-caching, may need to force re-caching on each of the above. (Forgetting ack would be sufficient, if that granularity is available.) If we want to patch a v3.8.1 with the second workaround, that might avoid triggering the bug in ZSH; one of the singles already uses (?:) and | so it should work. (OTOH doing so would take pressure off ZSH|omz to clean their own nest and would require them to specifically use v3.8.0 for bug reproduction.) Andy's offer to collaborate with a ZSH volunteer to implement a `--generate=complete-zsh` that gives the --help --man --help-types output distilled into ZSH format is very generous, and might be an safer way for ZSH to ingest ack options, but the existing bug should be fixed first as it will just crop up later for some other command's options later otherwise.

[petdance]

I think there was a change in ack 3.8.0 which somehow is causing this issue.

What's causing the issue is that the zsh script was written

1. without my having any knowledge of it
2. with the assumption that it was valid to scrape the output of ack --help-types, which was intended for humans
3. with the assumption that the output of --help-types would never change
4. fragile enough that the unexpected change blew up the entire script.

This is not an ack bug.

If we want to patch a v3.8.1 with the second workaround

I do not want to change the output of ack --help-types to accommodate a script doing what it should not be doing.

[gedge]

Apologies for the poor error report, and thanks for the responses - glad to hear I'm not alone.

For the record, I'm not using "oh-my-zsh".

Further, I assumed that the zsh-completion was done 'in house', for some reason. My mistake, I will seek out the author of the latter. Thanks for the in-depth look, @n1vux.

[gedge]

a workaround is to change the two offending lines to (seems to be the square brackets causing the issue):

--type-add=pytest:match:_test\.py$
--type-add=pytest:match:^test_.*\.py$

[gedge]

for my brew installed ack, the diff is:

--- /usr/local/Cellar/ack/3.8.0/bin/ack.orig    2024-12-21 08:41:43
+++ /usr/local/Cellar/ack/3.8.0/bin/ack 2024-12-31 09:24:38
@@ -3771,8 +3771,8 @@

 # Pytest
 # https://pytest.org/
---type-add=pytest:match:_test[.]py$
---type-add=pytest:match:^test_.*[.]py$
+--type-add=pytest:match:_test\.py$
+--type-add=pytest:match:^test_.*\.py$

 # R
 # https://www.r-project.org/

[petdance]

Using \. instead of [.] could be an OK short-term solution. I specifically prefer using [.] rather than escaping the metacharacter, because escaping characters is always fraught with peril. I'll look into this more.

[n1vux]

On Tue, Dec 31, 2024, 03:55 Geraint Edwards ***@***.***> wrote: Apologies for the poor error report, and thanks for the responses - glad to hear I'm not alone.

NOT a poor report. Zsh error message was misleading with ack: prefix instead of zsh: prefix, so unless you were used to debugging zsh completions (which should be rare problems this decade), it wouldmtake research to disambiguate. Reporting it here was useful as others such as Loren see it here.

For the record, I'm not using "oh-my-zsh".

Ok, that's useful disambiguation!

Further, I assumed that the zsh-completion was done 'in house', for some reason.

If it *seems* simple and the app hasn't already built a ZSH specific exporter (e.g. because app author is personally a ZSH user), quite reasonable. Alas it's brittle if they emit it to a formatted configuration file w/o rigorous escape-quoting. (And exporting it some other ways becomes a security hole so ...) My mistake, I will seek out the author of the latter. Thanks for the

in-depth look, @n1vux <https://github.com/n1vux>.

Entirely Welcome. We're glad to have feedback from users with diverse choices of OS, Shell, as we can't really test on every combination.

[n1vux]

On Tue, Dec 31, 2024, 15:09 Andy Lester ***@***.***> wrote: Using \. instead of [.] could be an OK short-term solution. I specifically prefer using [.] rather than escaping the metacharacter, because escaping characters is always fraught with peril.

Agreed completely; equivalent in every way except for the shoe-gun opportunities. AND BONUS, it's more readable! But totally acceptable as a workaround. (I'm guessing this [.]ness means my single line suggestion did not work?)

…

I'll look into this more.