-
Notifications
You must be signed in to change notification settings - Fork 10.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[lit] Drop "Script:", make -v and -a imply -vv
This patch and D156954 were discussed in <https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>. **Motivation**: -a shows output from all tests, and -v shows output from just failed tests. Without this patch, that output from each test includes a section called "Script:", which includes all shell commands that lit has computed from RUN directives and will attempt to run for that test. The effect of -vv (which also implies -v if neither -a or -v is specified) is to extend that output with shell commands as they are executing so you can easily see which one failed. For example, when using lit's internal shell and -vv: ``` Script: -- : 'RUN: at line 1'; echo hello world : 'RUN: at line 2'; 3c40 hello world : 'RUN: at line 3'; echo hello world -- Exit Code: 127 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "echo" "hello" "world" hello world $ ":" "RUN: at line 2" $ "3c40" "hello" "world" '3c40': command not found error: command failed with exit status: 127 -- ``` Notice that all shell commands that actually execute appear in the output twice, once for "Script:" and once for -vv. Especially for tests with many RUN directives, the result is noisy. When searching through the output for a particular shell command, it is easy to get lost and mistake shell commands under "Script:" for shell commands that actually executed. **Change**: With this patch, a test's output changes in two ways. First, the "Script:" section is never shown. Second, omitting -vv no longer disables printing of shell commands as they execute. That is, -a and -v imply -vv, and so -vv is deprecated as it is just an alias for -v. **Secondary motivation**: We are also working to introduce a PYTHON directive, which can appear between RUN directives. How should PYTHON directives be represented in the "Script:" section, which has previously been just a shell script? We could probably think of something, but adding info about PYTHON directive execution in the -vv trace seems more straight-forward and more useful. (This patch also removes a confusing point in the -vv documentation: at least when using bash as an external shell, -vv echoes commands to the shell's stderr not stdout.) Reviewed By: awarzynski, Endill, ldionne, MaskRay Differential Revision: https://reviews.llvm.org/D154984
- Loading branch information
1 parent
77e965e
commit 09b6e45
Showing
11 changed files
with
138 additions
and
139 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,92 +1,148 @@ | ||
# CHECK: -- Testing:{{.*}} | ||
# CHECK-NEXT: PASS: shtest-if-else :: test.txt (1 of 1) | ||
# CHECK-NEXT: Script: | ||
# CHECK-NEXT: -- | ||
# CHECK: -- Testing:{{.*}} | ||
# CHECK-NEXT: PASS: shtest-if-else :: test.txt (1 of 1) | ||
# CHECK-NEXT: Exit Code: 0 | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: Command Output (stdout): | ||
# CHECK-NEXT: -- | ||
|
||
# RUN: %if feature %{ echo "test-1" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo "test-1" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-1 | ||
# CHECK-EMPTY: | ||
|
||
# If %else is not present it is treated like %else %{%}. Empty commands | ||
# are ignored. | ||
# | ||
# RUN: %if nofeature %{ echo "fail" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]' | ||
# CHECK-NOT: fail | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK-NOT: fail | ||
|
||
# RUN: %if nofeature %{ echo "fail" %} %else %{ echo "test-2" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo "test-2" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-2 | ||
# CHECK-EMPTY: | ||
|
||
# Spaces inside curly braces are not ignored | ||
# | ||
# RUN: echo test-%if feature %{ 3 %} %else %{ fail %}-test | ||
# RUN: echo test-%if feature %{ 4 4 %} %else %{ fail %}-test | ||
# RUN: echo test-%if nofeature %{ fail %} %else %{ 5 5 %}-test | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo test- 3 -test | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo test- 4 4 -test | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo test- 5 5 -test | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-3]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test- 3 -test | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-6]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test- 4 4 -test | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-9]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test- 5 5 -test | ||
# CHECK-EMPTY: | ||
|
||
# Escape line breaks for multi-line expressions | ||
# | ||
# RUN: %if feature \ | ||
# RUN: %{ echo \ | ||
# RUN: "test-5" \ | ||
# RUN: %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-4]]'; echo "test-5" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-4]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-5 | ||
# CHECK-EMPTY: | ||
|
||
# RUN: %if nofeature \ | ||
# RUN: %{ echo "fail" %} \ | ||
# RUN: %else \ | ||
# RUN: %{ echo "test-6" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-4]]'; echo "test-6" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-4]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-6 | ||
# CHECK-EMPTY: | ||
|
||
# RUN: echo "test%if feature %{%} %else %{%}-7" | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo "test-7" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-7 | ||
# CHECK-EMPTY: | ||
|
||
# Escape %if. Without %if..%else context '%{' and '%}' are treated | ||
# literally. | ||
# | ||
# RUN: echo %%if feature %{ echo "test-8" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo %if feature %{ echo "test-8" %} | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: %if feature %{ echo test-8 %} | ||
# CHECK-EMPTY: | ||
|
||
# Nested expressions are supported: | ||
# | ||
# RUN: echo %if feature %{ %if feature %{ %if nofeature %{"fail"%} %else %{"test-9"%} %} %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo "test-9" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-9 | ||
# CHECK-EMPTY: | ||
|
||
# Binary expression evaluation and regex match can be used as | ||
# conditions. | ||
# | ||
# RUN: echo %if feature && !nofeature %{ "test-10" %} | ||
# RUN: echo %if feature && nofeature %{ "fail" %} %else %{ "test-11" %} | ||
# RUN: echo %if {{fea.+}} %{ "test-12" %} %else %{ "fail" %} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo "test-10" | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo "test-11" | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo "test-12" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-3]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-10 | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-6]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-11 | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-9]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: test-12 | ||
# CHECK-EMPTY: | ||
|
||
# Spaces between %if and %else are ignored. If there is no %else - | ||
# space after %if %{...%} is not ignored. | ||
# | ||
# RUN: echo XX %if feature %{YY%} ZZ | ||
# RUN: echo AA %if feature %{BB%} %else %{CC%} DD | ||
# RUN: echo AA %if nofeature %{BB%} %else %{CC%} DD | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo XX YY ZZ | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo AA BB DD | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-3]]'; echo AA CC DD | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-3]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: XX YY ZZ | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-6]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: AA BB DD | ||
# CHECK-EMPTY: | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-9]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: AA CC DD | ||
# CHECK-EMPTY: | ||
|
||
# '{' and '}' can be used without escaping | ||
# | ||
# RUN: %if feature %{echo {}%} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo {} | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: {} | ||
# CHECK-EMPTY: | ||
|
||
# Spaces are not required | ||
# | ||
# RUN: echo %if feature%{"ok"%}%else%{"fail"%} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo "ok" | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: ok | ||
# CHECK-EMPTY: | ||
|
||
# Substitutions with braces are handled correctly | ||
# | ||
# RUN: echo %{sub} %if feature%{test-%{sub}%}%else%{"fail"%} | ||
# CHECK-NEXT: {{^.*'RUN}}: at line [[#@LINE-1]]'; echo ok test-ok | ||
|
||
# CHECK-NEXT: -- | ||
# CHECK-NEXT: Exit Code: 0 | ||
# CHECK-NEXT: {{^.*"RUN}}: at line [[#@LINE-1]]" | ||
# CHECK: # command output: | ||
# CHECK-NEXT: ok test-ok | ||
# CHECK-EMPTY: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.