Add a way to toggle debug mode in Fish shell

[edouard-lopez]

Based from How to toggle debug mode in Fish shell?

Bash legacy

In bash I do set -x and set +x enable/disable the debug mode in a limited scope.

I'm aware of how to debug fish script?, but I would like to toggle debug mode in a more precise way, e.g. inside a function.

Question

What is the equivalent of set -x/set -x in Fish shell?

Would you mind implementing such future?

[faho]

As always, we probably shouldn't mindlessly copy the thing bash does. In this case we actually can't use set -x since that's already taken for exporting a variable.

But first, let's look at what bash actually does. I've never used it much, so this is kinda new to me and I might miss a bunch of subtleties.

> set -x
> echo $test
+ echo

> test=foo
> echo $test
+ echo foo
foo
> false
+ false
>

What immediately seems lacking here is that neither return status nor the exact commandlines before expansion are printed, which might be quite useful.

It also seems like bash has a "BASH_XTRACEFD" variable that can be used to send this debug output somewhere else, which seems quite useful but takes a file descriptor instead of a file name.

Now to what we already have. There is the "-d" or "--debug-level" option, which is lacking in a few ways:

• It can only be specified at startup, so you need to start a new shell. If you have defined a function interactively, you'd need to redo that just to debug it
• It prints a lot information that is useful when debugging fish itself (like "Job is constructed" and "Skipping fork" or "trying to match mapping") but completely superfluous when debugging a fish script
• It prints information related to every piece of fish code that is run, including stuff run by events - the prompt and such, which completely drowns out the stuff actually run by the user

Now, I know @krader1961 wanted to do some reorganizing regarding our debug levels - I can't see much of a difference between debug level 2 and 3 - but I still don't think that'd solve it.

So, how might this look? Let's assume you turn on debugging somehow (via an environment variable?), and then you run somescript $somevar.

It would output something like

Running `somescript someval` (from commandline `somescript $somevar`)
Status: 0
Running `and someotherscript`
Status: 1
Skipping `and yetanotherscript`

I.e. the "from commandline" is only output if there has been an expansion.

[krader1961]

This has been discussed in issue #805 but I'm not going to close this as a duplicate because that issue is mostly concerned with other matters. However, see this comment.

Note that this can also be enabled via set -o xtrace in bash and this facility also exists in zsh and ksh. It's a feature I wish fish had. However, I'd prefer that if we do implement this we do so in a better fashion than those shells. For example, the debug output should include the file name, line number, and exit status of the command. Possibly also the name of the function if the statement is in a function block. As @faho says we definitely do not want to blindly mimic bash.

There's a developing consensus on some related issues to augment begin;...; end to accept options which would control the behavior of the enclosed block of statements. So we might recognize begin -x and begin --xtrace. TBD is whether enabling this feature has lexical or dynamic scoping. If the former should there be way to optionally enable dynamic scoping? In other words, consider the following contrived example:

function b
    echo inside b
end

begin -x
    echo about to call b
    b
    echo returned from b
end

With lexical scoping only the statements in the begin block are traced. With dynamic scoping the statements inside function b are also traced. I've seen situations where you want only the former and others where the latter is useful. For simplicity I'm inclined to only implement dynamic scoping of the feature. By tagging the output with information about where the statement originated the there is little point to limiting tracing to the current lexical scope.

[faho]

So we might recognize begin -x and begin --xtrace.

Personally, I'd prefer begin --debug or begin --trace - no short option and simpler terms. This isn't something you'd type constantly, so clarity beats terseness.

Also, I like my bikesheds blue.

However, I can see this route not being entirely optimal - if you used it to interactively debug a certain function, you'd do begin --debug; somefunction; end, which seems a tiny bit awkward.

Also, I can't see any case where you'd always want tracing on, because it, unlike the set -e equivalent, always prints something.

However, that slight awkwardness might be offset by having consistency with the other options.

By tagging the output with information about where the statement originated the there is little point to limiting tracing to the current lexical scope.

Agreed.

[edouard-lopez]

With dynamic scoping the statements inside function b are also traced.

Fine with me.

Personally, I'd prefer begin --debug or begin --trace - no short option and simpler terms. This isn't something you'd type constantly, so clarity beats terseness.

Better to be explicit

[floam]

> begin --debug; somefunction; end

that's no fun - but it'd be easy to offer a wrapper script. trace somefunction seems pleasant.

[floam]

Except I guess I have a trace in $PATH.

[frederickjh]

Just a comment that in bash you can get the line numbers while debugging by adding the following:

#!/bin/bash
# For testing. First line outputs line numbers. 
# Second line says to output what is going on in script
PS4=':${LINENO}+'
#set -x

I add this first thing, to the top of all my bash scripts

[BarbzYHOOL]

Just discovered set -x (and set -e) in bash, it's awesome, would love it in fish

[frederickjh]

However, I can see this route not being entirely optimal - if you used it to interactively debug a certain function, you'd do begin --debug; somefunction; end, which seems a tiny bit awkward.

Maybe a syntax something like:

--debugon;
<code to debug>
--debugoff

would make more sense and also would be explicit.

---

Also a comment regarding my previous comment on line numbers . The PS4 is the debugging trace line prefix in bash. One of bash's prompts. Maybe a syntax for adding line numbers to the debug if that is not the default would be to add `--debuglinenumberson` or something shorter at the topp of the script.

The bash documentations says:

PS1 The value of this parameter is expanded (see PROMPTING below)
and used as the primary prompt string. The default value is
\s-\v\$ ''. PS2 The value of this parameter is expanded as with PS1 and used as the secondary prompt string. The default is > ''.
PS3 The value of this parameter is used as the prompt for the select
command (see SHELL GRAMMAR above).
PS4 The value of this parameter is expanded as with PS1 and the
value is printed before each command bash displays during an
execution trace. The first character of PS4 is replicated mul‐
tiple times, as necessary, to indicate multiple levels of indi‐
rection. The default is ``+ ''.

[ridiculousfish]

I'm eager to implement this. A few questions we need first:

Interface

Should the interface be a block-scope argument (begin --trace; ... ; end), a variable (set fish_trace 1), or a new command (fish_trace --on)?

I'm leaning towards a variable. The reason is that a variable gets a lot of flexibility for free via the scoping:

1. You can debug startup scripts through an environment variable:

    env fish_trace=1 fish ...
   

2. You can debug hangs in completions, prompts, etc by setting this as a global variable:

    set -g fish_trace 1
   

3. You can trace the "top level" of a script or drill down into functions, by setting it as a local vs global variable.

The downside is that it's somewhat harder to use options with a variable. For example if we wanted to control the tracing output we would need to somehow encode that in the value of the variable (fish_trace=ALL or whatever).

When to print

bash prints commands after expansion but before execution. Example:

zzz=99999
sleep $zzz

will print:

+ sleep 99999

The advantage of printing before running is that long-running commands get printed. The disadvantage is that you can't see the exit status, which would be really useful in debugging. I'd like to find a way to trace both the exit status and the command itself.

What to print

bash and zsh expand the $PS4 variable like a prompt and prints that. zsh has substitutions like %N which are flexible but opaque (this is zsh after all).

My sense is that rather than providing a DSL here, we should just have different trace levels:

set fish_trace 1 # prints basic information, e.g. no file or line numbers
set fish_trace 2 # prints more detailed information including file and line numbers

and so on.

[cben]

bash prints commands after expansion but before execution.

FWIW bash also has a set -v option, printing commands before expansion.
In my experience it's much less useful than -x — essentially you only learn where you are in the script. In very dense code it's sometimes useful to enable both set -vx but that's largely redundant.

The advantage of printing before running is that long-running commands get printed. The disadvantage is that you can't see the exit status, which would be really useful in debugging. I'd like to find a way to trace both the exit status and the command itself.

Printing after execution is much worse if command outputs anything.
The only logical order (leaving aside formatting) is (1) starting CMD (2) output of CMD (3) exit status.

I think if you print only non-zero exit statuses, it's fine to print them on a separate line after command exists.

bash and zsh expand the $PS4 variable like a prompt and prints that.

There are 2 neat aspects to this:

1. First, you can redefine PS4 to print details timestamp,$LINENO, $FUNCNAME etc.
   https://stackoverflow.com/questions/14639125/logging-bash-scripts-with-verbose-and-xtrace-set-vx-and-customizing-ps4
   bash also has DEBUG trap that can run arbitrary code. (see answer on above stackoverflow)
   Considering fish's approach to a prompt is running a function, if we want to copy this flexibility, we should run a function too. e.g. function trace --on-proccess-start.
   But I must say that's too low-level knob, in 95% of cases I just want "turn default tracing on".

2. It indicates sub-shell nesting (by replicating the first char of PS4, which is pretty arbitrary).
   echo $(echo 1) 2 logs:

   ++ echo 1
   + echo 1 2
   

   I long imagined this works across sub-processes — a script launching another script would trace that too by passing down a modified PS4 — but there is no such thing apparently. It's only for internal command expansion?

   Anyway, even indicating internal nesting can very useful when reading the trace.
   IMHO should increment for command expansion, when entering a function, and maybe for control structures like for and begin..end?

[cben]

TLDR for above comment: even a simple fish_trace=1 variable would be great ❤️! Can fine-tune output and add higher levels later.
And better invest in useful defaults than in full configurability (this is fish after all 😉)

[cben]

• One tweak I'd love in interactive mode, is suppress tracing while printing the prompt! bash happily traces the prompt too, making interactive set -x nearly unusable...
  (To debug the prompt function itself, one could set -l fish_trace=1 inside the function, or just execute fish_prompt directly.)

[tobia]

Thank you for adding this feature. It's very useful in a variety of situations.

The downside is that it's somewhat harder to use options with a variable.

[Edit by @faho: Because this keeps being confused: These are not implemented]

For a debugging tool such as fish_trace I think it's OK to encode an increase in detail as higher numbers, and orthogonal options as successive array values. For example,

set fish_trace 1    # print the expanded commandline before execution
set fish_trace 2    # prefix the commandline with file and line number

set fish_trace 1 1  # print the commandline before execution, 
                    #   and print its exit status after execution
set fish_trace 1 2  # as above, but print the exit status of all subprocesses (?)

set fish_trace 0 1  # only print the exit status of commandlines

And so on.