From 8160ef6e8127f117cba6df2d5c67d8daeb7e02e4 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 13:46:51 +0100 Subject: [PATCH 1/7] update annotation docs to reference signatures --- docs/codeql/ql-language-reference/annotations.rst | 9 ++++++--- docs/codeql/ql-language-reference/signatures.rst | 12 ++++++++++++ 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/codeql/ql-language-reference/annotations.rst b/docs/codeql/ql-language-reference/annotations.rst index cffbbf73c71a..a177f71d5007 100644 --- a/docs/codeql/ql-language-reference/annotations.rst +++ b/docs/codeql/ql-language-reference/annotations.rst @@ -126,7 +126,7 @@ body must also be annotated with ``cached``, otherwise a compiler error is repor ``deprecated`` ============== -**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases| +**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| The ``deprecated`` annotation is applied to names that are outdated and scheduled for removal in a future release of QL. @@ -235,7 +235,7 @@ warning. ``private`` =========== -**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases| +**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| The ``private`` annotation is used to prevent names from being exported. @@ -461,7 +461,7 @@ For more information, see ":ref:`monotonic-aggregates`." Binding sets ============ -**Available for**: |classes|, |characteristic predicates|, |member predicates|, |non-member predicates| +**Available for**: |classes|, |characteristic predicates|, |member predicates|, |non-member predicates|, |predicate signatures|, |type signatures| ``bindingset[...]`` ------------------- @@ -491,3 +491,6 @@ The ``bindingset`` annotation takes a comma-separated list of variables. .. |type-aliases| replace:: :ref:`type aliases ` .. |algebraic datatypes| replace:: :ref:`algebraic datatypes ` .. |expressions| replace:: :ref:`expressions ` +.. |signatures| replace:: :ref:`signatures ` +.. |predicate signatures| replace:: :ref:`predicate signatures ` +.. |type signatures| replace:: :ref:`type signatures ` diff --git a/docs/codeql/ql-language-reference/signatures.rst b/docs/codeql/ql-language-reference/signatures.rst index e80c54c47e41..f0fb8c03d7f5 100644 --- a/docs/codeql/ql-language-reference/signatures.rst +++ b/docs/codeql/ql-language-reference/signatures.rst @@ -10,6 +10,10 @@ Signatures Parameterized modules use signatures as a type system for their parameters. There are three categories of signatures: **predicate signatures**, **type signatures**, and **module signatures**. +.. index:: predicate signature + +.. _predicate-signatures: + Predicate signatures ==================== @@ -36,6 +40,10 @@ For example: signature int operator(int lhs, int rhs); +.. index:: type signature + +.. _type-signatures: + Type signatures =============== @@ -66,6 +74,10 @@ For example: string toString(); } +.. index:: module signature + +.. _module-signatures: + Module signatures ================= From 7893768cb27bd6acdc95c962eac7059e33dd8604 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 13:48:46 +0100 Subject: [PATCH 2/7] update annotation docs to reference type unions --- docs/codeql/ql-language-reference/annotations.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/codeql/ql-language-reference/annotations.rst b/docs/codeql/ql-language-reference/annotations.rst index a177f71d5007..7bcc85356895 100644 --- a/docs/codeql/ql-language-reference/annotations.rst +++ b/docs/codeql/ql-language-reference/annotations.rst @@ -103,7 +103,7 @@ own body, or they must inherit from another class that overrides ``isSource``: ``cached`` ========== -**Available for**: |classes|, |algebraic datatypes|, |characteristic predicates|, |member predicates|, |non-member predicates|, |modules| +**Available for**: |classes|, |algebraic datatypes|, |type unions|, |characteristic predicates|, |member predicates|, |non-member predicates|, |modules| The ``cached`` annotation indicates that an entity should be evaluated in its entirety and stored in the evaluation cache. All later references to this entity will use the @@ -126,7 +126,7 @@ body must also be annotated with ``cached``, otherwise a compiler error is repor ``deprecated`` ============== -**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| +**Available for**: |classes|, |algebraic datatypes|, |type unions|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| The ``deprecated`` annotation is applied to names that are outdated and scheduled for removal in a future release of QL. @@ -235,7 +235,7 @@ warning. ``private`` =========== -**Available for**: |classes|, |algebraic datatypes|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| +**Available for**: |classes|, |algebraic datatypes|, |type unions|, |member predicates|, |non-member predicates|, |imports|, |fields|, |modules|, |aliases|, |signatures| The ``private`` annotation is used to prevent names from being exported. @@ -490,6 +490,7 @@ The ``bindingset`` annotation takes a comma-separated list of variables. .. |aliases| replace:: :ref:`aliases ` .. |type-aliases| replace:: :ref:`type aliases ` .. |algebraic datatypes| replace:: :ref:`algebraic datatypes ` +.. |type unions| replace:: :ref:`type unions ` .. |expressions| replace:: :ref:`expressions ` .. |signatures| replace:: :ref:`signatures ` .. |predicate signatures| replace:: :ref:`predicate signatures ` From bd3bcf981a8d75d54ad08e32b044bcc9c8ae92c7 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 13:33:48 +0100 Subject: [PATCH 3/7] language reference section on 'additional' annotation --- docs/codeql/ql-language-reference/annotations.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/codeql/ql-language-reference/annotations.rst b/docs/codeql/ql-language-reference/annotations.rst index 7bcc85356895..1983a57f1010 100644 --- a/docs/codeql/ql-language-reference/annotations.rst +++ b/docs/codeql/ql-language-reference/annotations.rst @@ -97,6 +97,20 @@ own body, or they must inherit from another class that overrides ``isSource``: // doesn't need to override `isSource`, because it inherits it from ConfigA } +.. index:: additional +.. _additional: + +``additional`` +============== + +**Available for**: |classes|, |algebraic datatypes|, |type unions|, |non-member predicates|, |modules|, |aliases|, |signatures| + +The ``additional`` annotation can be used on declarations directly inside of modules that implement |module signatures|. +All declarations in such modules that are not required by a module signature must be annotated with ``additional``. + +Omitting ``additional`` on such declarations, or using the annotation in any other context, will result in a compiler error. +Other than that, the annotation has no effect. + .. index:: cached .. _cached: @@ -495,3 +509,4 @@ The ``bindingset`` annotation takes a comma-separated list of variables. .. |signatures| replace:: :ref:`signatures ` .. |predicate signatures| replace:: :ref:`predicate signatures ` .. |type signatures| replace:: :ref:`type signatures ` +.. |module signatures| replace:: :ref:`module signatures ` From 341a1191a382c158e2df1b7f5f5f41d9601f8350 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 15:38:14 +0100 Subject: [PATCH 4/7] language reference section on 'extensible' annotation --- docs/codeql/ql-language-reference/annotations.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/codeql/ql-language-reference/annotations.rst b/docs/codeql/ql-language-reference/annotations.rst index 1983a57f1010..a9d5642e73da 100644 --- a/docs/codeql/ql-language-reference/annotations.rst +++ b/docs/codeql/ql-language-reference/annotations.rst @@ -165,6 +165,16 @@ For example, the name ``DataFlowNode`` is deprecated and has the following QLDoc This QLDoc comment appears when you use the name ``DataFlowNode`` in a QL editor. +.. index:: extensible +.. _extensible: + +``extensible`` +============== + +**Available for**: |non-member predicates| + +The ``extensible`` annotation is used to mark predicates that are populated at evaluation time through data extensions. + .. index:: external .. _external: From 3159b299f75fd621a7f49f5fcadb1979a091a8aa Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 15:44:25 +0100 Subject: [PATCH 5/7] member predicates cannot be 'external' --- docs/codeql/ql-language-reference/ql-language-specification.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/codeql/ql-language-reference/ql-language-specification.rst b/docs/codeql/ql-language-reference/ql-language-specification.rst index f834949c3cde..f384c680d1d5 100644 --- a/docs/codeql/ql-language-reference/ql-language-specification.rst +++ b/docs/codeql/ql-language-reference/ql-language-specification.rst @@ -1053,7 +1053,7 @@ A member predicate ``p`` with enclosing class ``C`` *shadows* a member predicate Member predicates have one or more *root definitions*. If a member predicate overrides no other member predicate, then it is its own root definition. Otherwise, its root definitions are those of any member predicate that it overrides. -A valid member predicate must have a body unless it is abstract or external, in which case it must not have a body. +A valid member predicate must have a body unless it is abstract, in which case it must not have a body. A valid member predicate must override another member predicate if it is annotated override. From f0b39099e3441c48c255227e4da50d3d125512e7 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Wed, 1 Oct 2025 15:43:09 +0100 Subject: [PATCH 6/7] discuss 'extensible' whenever the spec mentions 'external' --- .../ql-language-reference/ql-language-specification.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/codeql/ql-language-reference/ql-language-specification.rst b/docs/codeql/ql-language-reference/ql-language-specification.rst index f384c680d1d5..1d84cc31c739 100644 --- a/docs/codeql/ql-language-reference/ql-language-specification.rst +++ b/docs/codeql/ql-language-reference/ql-language-specification.rst @@ -36,7 +36,7 @@ Architecture A *QL program* consists of a query module defined in a QL file and a number of library modules defined in QLL files that it imports (see "`Import directives <#import-directives>`__"). The module in the QL file includes one or more queries (see "`Queries <#queries>`__"). A module may also include *import directives* (see "`Import directives <#import-directives>`__"), non-member predicates (see "`Non-member predicates <#non-member-predicates>`__"), class definitions (see "`Classes <#classes>`__"), and module definitions (see "`Modules <#modules>`__"). -QL programs are interpreted in the context of a *database* and a *library path* . The database provides a number of definitions: database types (see "`Types <#types>`__"), entities (see "`Values <#values>`__"), built-in predicates (see "`Built-ins <#built-ins>`__"), and the *database content* of built-in predicates and external predicates (see "`Evaluation <#evaluation>`__"). The library path is a sequence of file-system directories that hold QLL files. +QL programs are interpreted in the context of a *database* and a *library path* . The database provides a number of definitions: database types (see "`Types <#types>`__"), entities (see "`Values <#values>`__"), built-in predicates (see "`Built-ins <#built-ins>`__"), and the *database content* of built-in predicates, external predicates, and extensible predicates (see "`Evaluation <#evaluation>`__"). The library path is a sequence of file-system directories that hold QLL files. A QL program can be *evaluated* (see "`Evaluation <#evaluation>`__") to produce a set of tuples of values (see "`Values <#values>`__"). @@ -935,6 +935,7 @@ When a predicate is a top-level clause in a module, it is called a non-member pr A valid non-member predicate can be annotated with ``additional``, ``cached``, ``deprecated``, ``extensible``, ``external``, ``transient``, ``private``, and ``query``. Note, the ``transient`` annotation can only be applied if the non-member predicate is also annotated with ``external``. +Note, the annotations ``extensible`` and ``external`` cannot both be used on the same non-member predicate. The head of the predicate gives a name, an optional *result type*, and a sequence of variables declarations that are *arguments*: @@ -952,7 +953,7 @@ The body of a predicate is of one of three forms: In the first form, with just a semicolon, the predicate is said to not have a body. In the second form, the body of the predicate is the given formula (see "`Formulas <#formulas>`__"). In the third form, the body is a higher-order relation. -A valid non-member predicate must have a body, either a formula or a higher-order relation, unless it is external, in which case it must not have a body. +A valid non-member predicate must have a body, either a formula or a higher-order relation, unless it is external or extensible, in which case it must not have a body. The typing environment for the body of the formula, if present, maps the variables in the head of the predicate to their associated types. If the predicate has a result type, then the typing environment also maps ``result`` to the result type. @@ -2180,7 +2181,7 @@ If a QL program has no valid stratification, then the program itself is not vali Layer evaluation ~~~~~~~~~~~~~~~~ -The store is first initialized with the *database content* of all built-in predicates and external predicates. The database content of a predicate is a set of ordered tuples that are included in the database. +The store is first initialized with the *database content* of all built-in predicates, external predicates, and extensible predicates. The database content of a predicate is a set of ordered tuples that are included in the database. Each layer of the stratification is *populated* in order. To populate a layer, each predicate in the layer is repeatedly populated until the store stops changing. The way that a predicate is populated is as follows: From a2d31be1523f00cfe468987f890c1371529bf6e1 Mon Sep 17 00:00:00 2001 From: Philip Ginsbach Date: Thu, 2 Oct 2025 09:02:20 +0100 Subject: [PATCH 7/7] improve the wording based on PR review feedback --- docs/codeql/ql-language-reference/annotations.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/codeql/ql-language-reference/annotations.rst b/docs/codeql/ql-language-reference/annotations.rst index a9d5642e73da..b792e807c931 100644 --- a/docs/codeql/ql-language-reference/annotations.rst +++ b/docs/codeql/ql-language-reference/annotations.rst @@ -105,8 +105,8 @@ own body, or they must inherit from another class that overrides ``isSource``: **Available for**: |classes|, |algebraic datatypes|, |type unions|, |non-member predicates|, |modules|, |aliases|, |signatures| -The ``additional`` annotation can be used on declarations directly inside of modules that implement |module signatures|. -All declarations in such modules that are not required by a module signature must be annotated with ``additional``. +The ``additional`` annotation can be used on declarations in explicit modules. +All declarations that are not required by a module signature in modules that implement |module signatures| must be annotated with ``additional``. Omitting ``additional`` on such declarations, or using the annotation in any other context, will result in a compiler error. Other than that, the annotation has no effect.