test suite: error-pattern annotations are not checked in UI tests
#52531
Labels
A-testsuite
Area: The testsuite used to check the correctness of rustc
Comments
petrochenkov
added
the
A-testsuite
davidtwco
added a commit
to davidtwco/rust
that referenced
this issue
Aug 10, 2018
Previously, even if no expected errors were supplied, if a test execution failed then supplied error patterns would not be checked. This commit modifies the conditional that determines whether error patterns or expected errors are checked to remedy this. Further, this commit modifies the error pattern checking logic so that each pattern is checked against all lines of the string. This is required for UI tests as the stderr is in JSON format - all on one line - so in the previous implementation when the first pattern was found on the first line (which was actually the entire error) then no other patterns would be found on subsequent lines (as there weren't any).
bors
added a commit
that referenced
this issue
Aug 10, 2018
Move `compile-fail` tests to `ui` Fixes #46841, #52531, #44844. r? @nikomatsakis
davidtwco
added a commit
to davidtwco/rust
that referenced
this issue
Aug 13, 2018
Previously, even if no expected errors were supplied, if a test execution failed then supplied error patterns would not be checked. This commit modifies the conditional that determines whether error patterns or expected errors are checked to remedy this. Further, this commit modifies the error pattern checking logic so that each pattern is checked against all lines of the string. This is required for UI tests as the stderr is in JSON format - all on one line - so in the previous implementation when the first pattern was found on the first line (which was actually the entire error) then no other patterns would be found on subsequent lines (as there weren't any).
bors
added a commit
that referenced
this issue
Aug 13, 2018
Move `compile-fail` tests to `ui` Fixes #46841, #52531, #44844. r? @nikomatsakis
bors
added a commit
that referenced
this issue
Aug 14, 2018
Move `compile-fail` tests to `ui` Fixes #46841, #52531, #44844. r? @nikomatsakis
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sometimes
//~ ERRORattached to a line in the local file cannot be used for testing an error message, because the message is attached to a "remote" span from some other file.In this case
error-patternannotations can be used in compile-fail tests to check that some diagnostics present in the output regardless of span.The problem is that such annotations are completely ignored in UI tests and don't check anything, so every such annotation is a potential bug going unnoticed.
error-patternshould either work in UI tests, or some #52175 -like mechanism should be used to remap remote spans into local spans and then//~ ERRORcould be used for everything.The text was updated successfully, but these errors were encountered: