fix handling of odd strings by test builtin

[krader1961]

The test builtin currently has unexpected behavior with respect to
expressions such as '' -eq 0. That currently evaluates to true with a
return status of zero. This change addresses that oddity while also
ensuring that other unusual strings (e.g., numbers with leading and
trailing whitespace) are handled consistently.

I based this on PR #3579 so it includes commits not already merged to the master branch. The only commit that is relevant for this PR is the final one: 210296b.

[krader1961]

@ridiculousfish: I think the only part of this that needs a quick look from you is my use of parser_t::expand_argument_list() to tokenize the test expressions being tested in src/fish_tests.cpp. The original code to tokenize the statements did not correctly handle quoted strings and that function seemed like the most straightforward solution.

[krader1961]

Also, this fixes issue #3346. I'll update the commit comment to mention that when I merge it.

[floam]

I haven't tried this PR yet - there are conflicts.

Does this let builtin test "" -lt 1 and command test "" -lt 1 behave similarly, emitting an error message? Fish was wrong here, test should require numeric arguments for this comparison.

[krader1961]

Does this let builtin test "" -lt 1 and command test "" -lt 1 behave similarly, ...

Sort of. Fish now fails the test but does not emit a diagnostic. Note that even without this change doing something nonsensical like test x -lt 1 does not emit a diagnostic. So my change is the minimum needed to provide sane behavior consistent with command test without introducing a new error message. But I'm inclined to think we should take this opportunity to emit a diagnostic.

[floam]

But I'm inclined to think we should take this opportunity to emit a diagnostic.

👍

Though, beware all - a number of our scripts end up relying on the no-error fish test quirk and it's a bit less convenient when comparing values that might be in a variable that might exist - see fish_vi_cursor.fish for examples of a bunch of stuff that won't work with a system test - the easiest way to "fix" it without adding an additional test is to prepend a 0 before using a var as a sole argument, like is done in __fish_config_interactive.

e.g.

- test "$VTE_VERSION" -lt 4000
+ test "0$VTE_VERSION" -lt 4000

[floam]

(Or, you could just redirect stderr to /dev/null, silencing all errors.)

[krader1961]

Ugh! Adding a diagnostic causes test failures because the unit tests either assume an empty string is equivalent to "0" or don't suppress errors. I'm still going to add the error message because it's the right thing to do, and fix the unit tests. However, this does make me worry that there are a not insignificant number of scripts that depend on the current, silent, behavior.

[floam]

However, this does make me worry that there are a not insignificant number of scripts that depend on the current, silent, behavior.

And unfortunately test errors are notoriously cryptic and hard to track down in fish IMO.

What would probably be a nice thing to do given potential pain from new errors in random scripts people may suffer after a change like this, would be to improve our error output to make identifying where a test failure occurred easier - namely some indication of what the hell the arguments were.

I had something along the lines of this happening locally at one point, but can't find the commit.

$ [ $foobar -gt 1 ]
[: Non-numeric argument at index 0: ‘’
[: given args: ‘’, ‘-gt‘, ‘1‘, ‘]‘

Such improvements should be a handled in different issue.

[krader1961]

And unfortunately test errors are notoriously cryptic and hard to track down in fish IMO.

See issue #2860 that I opened eight months ago to pointing out that all diagnostic messages need to provide more context. There are too many issues of that type that need to be a high priority to resolve. Not sure how to get attention focused on those issues.

[floam]

I think in this case it'd be wiser to let an error() bubble up onto the wcstring_list_t errors list to be handled uniformly, rather than call debug here.

[krader1961]

Okay, that's probably the right thing to do but I need to think about it. The problem with that approach is it loses context if we ever fix isssue #2860. I need to better understand how the errors list is utilized.

[krader1961]

Ahh, I had a brain cramp. Yes, we should definitely append that error to the streams.err stream so it can be redirected. Thanks for catching that, @floam. As part of fixing issue #2860 we'll need to add a generic mechanism for decorating such messages with useful context.

[krader1961]

J.H.C. Switching to appending to the streams.err stream does not allow redirecting such errors to /dev/null. It's still the right thing to do but clearly doesn't allow such errors to be silently discarded.

[floam]

I didn't mean appending the string directly to streams.err, but rather use the test_parser::error() or add_error or any other means to get it onto the errors list on test_parser. It may be necessary to check the state of errnoinstead after using parse_number (because you can't return anything but true or false here, or access errors) and catch that in some caller above (binary_primary_evaluate?), or perhaps check for a sane number at some other stage in evaluation.

[krader1961]

Closed with commit 2f33c24 and commit 54a76bb.