diff --git a/clang/docs/InternalsManual.rst b/clang/docs/InternalsManual.rst index b7d88d3d67d0a..f8e3da5f97368 100644 --- a/clang/docs/InternalsManual.rst +++ b/clang/docs/InternalsManual.rst @@ -3309,6 +3309,173 @@ are similar. as syntax highlighting, cross-referencing, and so on. The ``c-index-test`` helper program can be used to test these features. +Testing +------- +All functional changes to Clang should come with test coverage demonstrating +the change in behavior. + +Verifying Diagnostics +^^^^^^^^^^^^^^^^^^^^^ +Clang ``-cc1`` supports the ``-verify`` command line option as a way to +validate diagnostic behavior. This option will use special comments within the +test file to verify that expected diagnostics appear in the correct source +locations. If all of the expected diagnostics match the actual output of Clang, +then the invocation will return normally. If there are discrepancies between +the expected and actual output, Clang will emit detailed information about +which expected diagnostics were not seen or which unexpected diagnostics were +seen, etc. A complete example is: + +.. code-block: c++ + + // RUN: %clang_cc1 -verify %s + int A = B; // expected-error {{use of undeclared identifier 'B'}} + +If the test is run and the expected error is emitted on the expected line, the +diagnostic verifier will pass. However, if the expected error does not appear +or appears in a different location than expected, or if additional diagnostics +appear, the diagnostic verifier will fail and emit information as to why. + +The ``-verify`` command optionally accepts a comma-delimited list of one or +more verification prefixes that can be used to craft those special comments. +Each prefix must start with a letter and contain only alphanumeric characters, +hyphens, and underscores. ``-verify`` by itself is equivalent to +``-verify=expected``, meaning that special comments will start with +``expected``. Using different prefixes makes it easier to have separate +``RUN:`` lines in the same test file which result in differing diagnostic +behavior. For example: + +.. code-block:: c++ + + // RUN: %clang_cc1 -verify=foo,bar %s + + int A = B; // foo-error {{use of undeclared identifier 'B'}} + int C = D; // bar-error {{use of undeclared identifier 'D'}} + int E = F; // expected-error {{use of undeclared identifier 'F'}} + +The verifier will recognize ``foo-error`` and ``bar-error`` as special comments +but will not recognize ``expected-error`` as one because the ``-verify`` line +does not contain that as a prefix. Thus, this test would fail verification +because an unexpected diagnostic would appear on the declaration of ``E``. + +Multiple occurrences accumulate prefixes. For example, +``-verify -verify=foo,bar -verify=baz`` is equivalent to +``-verify=expected,foo,bar,baz``. + +Specifying Diagnostics +^^^^^^^^^^^^^^^^^^^^^^ +Indicating that a line expects an error or a warning is simple. Put a comment +on the line that has the diagnostic, use +``expected-{error,warning,remark,note}`` to tag if it's an expected error, +warning, remark, or note (respectively), and place the expected text between +``{{`` and ``}}`` markers. The full text doesn't have to be included, only +enough to ensure that the correct diagnostic was emitted. (Note: full text +should be included in test cases unless there is a compelling reason to use +truncated text instead.) + +Here's an example of the most commonly used way to specify expected +diagnostics: + +.. code-block: c++ + + int A = B; // expected-error {{use of undeclared identifier 'B'}} + +You can place as many diagnostics on one line as you wish. To make the code +more readable, you can use slash-newline to separate out the diagnostics. + +Alternatively, it is possible to specify the line on which the diagnostic +should appear by appending ``@`` to ``expected-``, for example: + +.. code-block: c++ + + #warning some text + // expected-warning@10 {{some text}} + +The line number may be absolute (as above), or relative to the current line by +prefixing the number with either ``+`` or ``-``. + +If the diagnostic is generated in a separate file, for example in a shared +header file, it may be beneficial to be able to declare the file in which the +diagnostic will appear, rather than placing the ``expected-*`` directive in the +actual file itself. This can be done using the following syntax: + +.. code-block: c++ + + // expected-error@path/include.h:15 {{error message}} + +The path can be absolute or relative and the same search paths will be used as +for ``#include`` directives. The line number in an external file may be +substituted with ``*`` meaning that any line number will match (useful where +the included file is, for example, a system header where the actual line number +may change and is not critical). + +As an alternative to specifying a fixed line number, the location of a +diagnostic can instead be indicated by a marker of the form ``#``. +Markers are specified by including them in a comment, and then referenced by +appending the marker to the diagnostic with ``@#``, as with: + +.. code-block: c++ + + #warning some text // #1 + // ... other code ... + // expected-warning@#1 {{some text}} + +The name of a marker used in a directive must be unique within the compilation. + +The simple syntax above allows each specification to match exactly one +diagnostic. You can use the extended syntax to customize this. The extended +syntax is ``expected- {{diag text}}``, where ```` is one of +``error``, ``warning``, ``remark``, or ``note``, and ```` is a positive +integer. This allows the diagnostic to appear as many times as specified. For +example: + +.. code-block: c++ + + void f(); // expected-note 2 {{previous declaration is here}} + +Where the diagnostic is expected to occur a minimum number of times, this can +be specified by appending a ``+`` to the number. For example: + +.. code-block: c++ + + void f(); // expected-note 0+ {{previous declaration is here}} + void g(); // expected-note 1+ {{previous declaration is here}} + +In the first example, the diagnostic becomes optional, i.e. it will be +swallowed if it occurs, but will not generate an error if it does not occur. In +the second example, the diagnostic must occur at least once. As a short-hand, +"one or more" can be specified simply by ``+``. For example: + +.. code-block: c++ + + void g(); // expected-note + {{previous declaration is here}} + +A range can also be specified by ``-``. For example: + +.. code-block: c++ + + void f(); // expected-note 0-1 {{previous declaration is here}} + +In this example, the diagnostic may appear only once, if at all. + +Regex matching mode may be selected by appending ``-re`` to the diagnostic type +and including regexes wrapped in double curly braces in the directive, such as: + +.. code-block: c++ + + expected-error-re {{format specifies type 'wchar_t **' (aka '{{.+}}')}} + +Examples matching error: "variable has incomplete type 'struct s'" + +.. code-block: c++ + + // expected-error {{variable has incomplete type 'struct s'}} + // expected-error {{variable has incomplete type}} + + // expected-error-re {{variable has type 'struct {{.}}'}} + // expected-error-re {{variable has type 'struct {{.*}}'}} + // expected-error-re {{variable has type 'struct {{(.*)}}'}} + // expected-error-re {{variable has type 'struct{{[[:space:]](.*)}}'}} + Feature Test Macros =================== Clang implements several ways to test whether a feature is supported or not. diff --git a/clang/include/clang/Frontend/VerifyDiagnosticConsumer.h b/clang/include/clang/Frontend/VerifyDiagnosticConsumer.h index 500a7e11ab9ac..ddfae2666c4c3 100644 --- a/clang/include/clang/Frontend/VerifyDiagnosticConsumer.h +++ b/clang/include/clang/Frontend/VerifyDiagnosticConsumer.h @@ -32,156 +32,8 @@ class TextDiagnosticBuffer; /// VerifyDiagnosticConsumer - Create a diagnostic client which will use /// markers in the input source to check that all the emitted diagnostics match -/// those expected. -/// -/// INVOKING THE DIAGNOSTIC CHECKER: -/// -/// VerifyDiagnosticConsumer is typically invoked via the "-verify" option to -/// "clang -cc1". "-verify" is equivalent to "-verify=expected", so all -/// diagnostics are typically specified with the prefix "expected". For -/// example: -/// -/// \code -/// int A = B; // expected-error {{use of undeclared identifier 'B'}} -/// \endcode -/// -/// Custom prefixes can be specified as a comma-separated sequence. Each -/// prefix must start with a letter and contain only alphanumeric characters, -/// hyphens, and underscores. For example, given just "-verify=foo,bar", -/// the above diagnostic would be ignored, but the following diagnostics would -/// be recognized: -/// -/// \code -/// int A = B; // foo-error {{use of undeclared identifier 'B'}} -/// int C = D; // bar-error {{use of undeclared identifier 'D'}} -/// \endcode -/// -/// Multiple occurrences accumulate prefixes. For example, -/// "-verify -verify=foo,bar -verify=baz" is equivalent to -/// "-verify=expected,foo,bar,baz". -/// -/// SPECIFYING DIAGNOSTICS: -/// -/// Indicating that a line expects an error or a warning is simple. Put a -/// comment on the line that has the diagnostic, use: -/// -/// \code -/// expected-{error,warning,remark,note} -/// \endcode -/// -/// to tag if it's an expected error, remark or warning, and place the expected -/// text between {{ and }} markers. The full text doesn't have to be included, -/// only enough to ensure that the correct diagnostic was emitted. -/// -/// Here's an example: -/// -/// \code -/// int A = B; // expected-error {{use of undeclared identifier 'B'}} -/// \endcode -/// -/// You can place as many diagnostics on one line as you wish. To make the code -/// more readable, you can use slash-newline to separate out the diagnostics. -/// -/// Alternatively, it is possible to specify the line on which the diagnostic -/// should appear by appending "@" to "expected-", for example: -/// -/// \code -/// #warning some text -/// // expected-warning@10 {{some text}} -/// \endcode -/// -/// The line number may be absolute (as above), or relative to the current -/// line by prefixing the number with either '+' or '-'. -/// -/// If the diagnostic is generated in a separate file, for example in a shared -/// header file, it may be beneficial to be able to declare the file in which -/// the diagnostic will appear, rather than placing the expected-* directive in -/// the actual file itself. This can be done using the following syntax: -/// -/// \code -/// // expected-error@path/include.h:15 {{error message}} -/// \endcode -/// -/// The path can be absolute or relative and the same search paths will be used -/// as for #include directives. The line number in an external file may be -/// substituted with '*' meaning that any line number will match (useful where -/// the included file is, for example, a system header where the actual line -/// number may change and is not critical). -/// -/// As an alternative to specifying a fixed line number, the location of a -/// diagnostic can instead be indicated by a marker of the form "#". -/// Markers are specified by including them in a comment, and then referenced -/// by appending the marker to the diagnostic with "@#": -/// -/// \code -/// #warning some text // #1 -/// // expected-warning@#1 {{some text}} -/// \endcode -/// -/// The name of a marker used in a directive must be unique within the -/// compilation. -/// -/// The simple syntax above allows each specification to match exactly one -/// error. You can use the extended syntax to customize this. The extended -/// syntax is "expected- {{diag text}}", where \ is one of -/// "error", "warning" or "note", and \ is a positive integer. This allows -/// the diagnostic to appear as many times as specified. Example: -/// -/// \code -/// void f(); // expected-note 2 {{previous declaration is here}} -/// \endcode -/// -/// Where the diagnostic is expected to occur a minimum number of times, this -/// can be specified by appending a '+' to the number. Example: -/// -/// \code -/// void f(); // expected-note 0+ {{previous declaration is here}} -/// void g(); // expected-note 1+ {{previous declaration is here}} -/// \endcode -/// -/// In the first example, the diagnostic becomes optional, i.e. it will be -/// swallowed if it occurs, but will not generate an error if it does not -/// occur. In the second example, the diagnostic must occur at least once. -/// As a short-hand, "one or more" can be specified simply by '+'. Example: -/// -/// \code -/// void g(); // expected-note + {{previous declaration is here}} -/// \endcode -/// -/// A range can also be specified by "-". Example: -/// -/// \code -/// void f(); // expected-note 0-1 {{previous declaration is here}} -/// \endcode -/// -/// In this example, the diagnostic may appear only once, if at all. -/// -/// Regex matching mode may be selected by appending '-re' to type and -/// including regexes wrapped in double curly braces in the directive, such as: -/// -/// \code -/// expected-error-re {{format specifies type 'wchar_t **' (aka '{{.+}}')}} -/// \endcode -/// -/// Examples matching error: "variable has incomplete type 'struct s'" -/// -/// \code -/// // expected-error {{variable has incomplete type 'struct s'}} -/// // expected-error {{variable has incomplete type}} -/// -/// // expected-error-re {{variable has type 'struct {{.}}'}} -/// // expected-error-re {{variable has type 'struct {{.*}}'}} -/// // expected-error-re {{variable has type 'struct {{(.*)}}'}} -/// // expected-error-re {{variable has type 'struct{{[[:space:]](.*)}}'}} -/// \endcode -/// -/// VerifyDiagnosticConsumer expects at least one expected-* directive to -/// be found inside the source code. If no diagnostics are expected the -/// following directive can be used to indicate this: -/// -/// \code -/// // expected-no-diagnostics -/// \endcode +/// those expected. See clang/docs/InternalsManual.rst for details about how to +/// write tests to verify diagnostics. /// class VerifyDiagnosticConsumer: public DiagnosticConsumer, public CommentHandler {