Merge pull request github#3138 from jf205/recent-changes

docs: fix links in Python articles (rc/1.23)

Author: jf205

docs/language/learn-ql/cpp/dataflow.rst

@@ -45,7 +45,7 @@ or using the predicates ``exprNode`` and ``parameterNode``:
     */
    ParameterNode parameterNode(Parameter p) { ... }
 
-The predicate ``localFlowStep(Node nodeFrom, Node, nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
+The predicate ``localFlowStep(Node nodeFrom, Node nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
 
 For example, finding flow from a parameter ``source`` to an expression ``sink`` in zero or more local steps can be achieved as follows:
 

docs/language/learn-ql/java/dataflow.rst

@@ -45,7 +45,7 @@ or using the predicates ``exprNode`` and ``parameterNode``:
     */
    ParameterNode parameterNode(Parameter p) { ... }
 
-The predicate ``localFlowStep(Node nodeFrom, Node, nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
+The predicate ``localFlowStep(Node nodeFrom, Node nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. The predicate can be applied recursively (using the ``+`` and ``*`` operators), or through the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
 
 For example, finding flow from a parameter ``source`` to an expression ``sink`` in zero or more local steps can be achieved as follows:
 

docs/language/learn-ql/python/introduce-libraries-python.rst

@@ -158,7 +158,7 @@ Summary
 
 The most commonly used standard classes in the syntactic part of the library are organized as follows:
 
-``Module``, ``Class``, ``Function``, ``Stmt``, and ``Expr`` - they are all subclasses of `AstNode <https://help.semmle.com/qldoc/python/semmle/python/AST.qll/type.AST$AstNode.html>`__.
+``Module``, ``Class``, ``Function``, ``Stmt``, and ``Expr`` - they are all subclasses of `AstNode <https://help.semmle.com/qldoc/python/semmle/python/AstExtended.qll/type.AstExtended$AstNode.html>`__.
 
 Abstract syntax tree
 ''''''''''''''''''''
@@ -323,8 +323,8 @@ The CodeQL library for Python also supplies classes to specify taint-tracking an
 Summary
 ~~~~~~~
 
-- `TaintKind <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintKind.html>`__
-- `Configuration <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintTracking$Configuration.html>`__
+- `TaintKind <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/type.TaintTracking$TaintKind.html>`__
+- `Configuration <https://help.semmle.com/qldoc/python/semmle/python/dataflow/Configuration.qll/type.Configuration$TaintTracking$Configuration.html>`__
 
 These classes are explained in more detail in :doc:`Tutorial: Taint tracking and data flow analysis in Python <taint-tracking>`.
 

docs/language/learn-ql/python/taint-tracking.rst

@@ -4,7 +4,7 @@ Taint tracking and data flow analysis in Python
 Overview
 --------
 
-Taint tracking is used to analyze how potentially insecure, or 'tainted' data flows throughout a program at runtime. 
+Taint tracking is used to analyze how potentially insecure, or 'tainted' data flows throughout a program at runtime.
 You can use taint tracking to find out whether user-controlled input can be used in a malicious way,
 whether dangerous arguments are passed to vulnerable functions, and whether confidential or sensitive data can leak.
 You can also use it to track invalid, insecure, or untrusted data in other analyses.
@@ -13,36 +13,36 @@ Taint tracking differs from basic data flow in that it considers non-value-prese
 For example, in the assignment ``dir = path + "/"``, if ``path`` is tainted then ``dir`` is also tainted,
 even though there is no data flow from ``path`` to ``path + "/"``.
 
-Separate CodeQL libraries have been written to handle 'normal' data flow and taint tracking in :doc:`C/C++ <../cpp/dataflow>`, :doc:`C# <../csharp/dataflow>`, :doc:`Java <../java/dataflow>`, and :doc:`JavaScript <../javascript/dataflow>`. You can access the appropriate classes and predicates that reason about these different modes of data flow by importing the appropriate library in your query. 
-In Python analysis, we can use the same taint tracking library to model both 'normal' data flow and taint flow, but we are still able make the distinction between steps that preserve value and those that don't by defining additional data flow properties. 
+Separate CodeQL libraries have been written to handle 'normal' data flow and taint tracking in :doc:`C/C++ <../cpp/dataflow>`, :doc:`C# <../csharp/dataflow>`, :doc:`Java <../java/dataflow>`, and :doc:`JavaScript <../javascript/dataflow>`. You can access the appropriate classes and predicates that reason about these different modes of data flow by importing the appropriate library in your query.
+In Python analysis, we can use the same taint tracking library to model both 'normal' data flow and taint flow, but we are still able make the distinction between steps that preserve value and those that don't by defining additional data flow properties.
 
 For further information on data flow and taint tracking with CodeQL, see :doc:`Introduction to data flow <../intro-to-data-flow>`.
 
 Fundamentals of taint tracking and data flow analysis
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The taint tracking library is in the `TaintTracking <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/module.TaintTracking.html>`__ module.
+The taint tracking library is in the `TaintTracking <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/module.TaintTracking.html>`__ module.
 Any taint tracking or data flow analysis query has three explicit components, one of which is optional, and an implicit component.
 The explicit components are:
 
-1. One or more ``sources`` of potentially insecure or unsafe data, represented by the `TaintTracking::Source <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintSource.html>`__ class.
-2. One or more ``sinks``, to where the data or taint may flow, represented by the `TaintTracking::Sink <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintSink.html>`__ class.
-3. Zero or more ``sanitizers``, represented by the `Sanitizer <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$Sanitizer.html>`__ class.
+1. One or more ``sources`` of potentially insecure or unsafe data, represented by the `TaintTracking::Source <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/type.TaintTracking$TaintSource.html>`__ class.
+2. One or more ``sinks``, to where the data or taint may flow, represented by the `TaintTracking::Sink <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/type.TaintTracking$TaintSink.html>`__ class.
+3. Zero or more ``sanitizers``, represented by the `Sanitizer <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/type.TaintTracking$Sanitizer.html>`__ class.
 
 A taint tracking or data flow query gives results when there is the flow of data from a source to a sink, which is not blocked by a sanitizer.
 
-These three components are bound together using a `TaintTracking::Configuration <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintTracking$Configuration.html>`__.
+These three components are bound together using a `TaintTracking::Configuration <https://help.semmle.com/qldoc/python/semmle/python/dataflow/Configuration.qll/type.Configuration$TaintTracking$Configuration.html>`__.
 The purpose of the configuration is to specify exactly which sources and sinks are relevant to the specific query.
 
-The final, implicit component is the "kind" of taint, represented by the `TaintKind <https://help.semmle.com/qldoc/python/semmle/python/security/TaintTracking.qll/type.TaintTracking$TaintKind.html>`__ class.
+The final, implicit component is the "kind" of taint, represented by the `TaintKind <https://help.semmle.com/qldoc/python/semmle/python/dataflow/TaintTracking.qll/type.TaintTracking$TaintKind.html>`__ class.
 The kind of taint determines which non-value-preserving steps are possible, in addition to value-preserving steps that are built into the analysis.
-In the above example ``dir = path + "/"``, taint flows from ``path`` to ``dir`` if the taint represents a string, but not if the taint is ``None``. 
+In the above example ``dir = path + "/"``, taint flows from ``path`` to ``dir`` if the taint represents a string, but not if the taint is ``None``.
 
 Limitations
 ~~~~~~~~~~~
 
 Although taint tracking is a powerful technique, it is worth noting that it depends on the underlying data flow graphs.
-Creating a data flow graph that is both accurate and covers a large enough part of a program is a challenge, 
+Creating a data flow graph that is both accurate and covers a large enough part of a program is a challenge,
 especially for a dynamic language like Python. The call graph might be incomplete, the reachability of code is an approximation,
 and certain constructs, like ``eval``, are just too dynamic to analyze.
 
@@ -61,18 +61,18 @@ A simple taint tracking query has the basic form:
      */
 
     import semmle.python.security.TaintTracking
-    
+
     class MyConfiguration extends TaintTracking::Configuration {
-    
+
         MyConfiguration() { this = "My example configuration" }
-    
+
         override predicate isSource(TaintTracking::Source src) { ... }
-    
+
         override predicate isSink(TaintTracking::Sink sink) { ... }
-        
+
         /* optionally */
         override predicate isExtension(Extension extension) { ... }
-    
+
     }
 
     from MyConfiguration config, TaintTracking::Source src, TaintTracking::Sink sink
@@ -107,17 +107,17 @@ The sink is defined by using a custom ``TaintTracking::Sink`` class.
         }
 
     }
-    
+
     class HttpToUnsafeConfiguration extends TaintTracking::Configuration {
-    
-        HttpToUnsafeConfiguration() { 
+
+        HttpToUnsafeConfiguration() {
             this = "Example config finding flow from http request to 'unsafe' function"
         }
-        
+
         override predicate isSource(TaintTracking::Source src) { src instanceof HttpRequestTaintSource }
-    
+
         override predicate isSink(TaintTracking::Sink sink) { sink instanceof UnsafeSink }
-        
+
     }
 
     from HttpToUnsafeConfiguration config, TaintTracking::Source src, TaintTracking::Sink sink
@@ -183,17 +183,17 @@ Thus, our example query becomes:
         }
 
     }
-    
+
     class HttpToUnsafeConfiguration extends TaintTracking::Configuration {
-    
-        HttpToUnsafeConfiguration() { 
+
+        HttpToUnsafeConfiguration() {
             this = "Example config finding flow from http request to 'unsafe' function"
         }
-        
+
         override predicate isSource(TaintTracking::Source src) { src instanceof HttpRequestTaintSource }
-    
+
         override predicate isSink(TaintTracking::Sink sink) { sink instanceof UnsafeSink }
-        
+
     }
 
     from HttpToUnsafeConfiguration config, TaintedPathSource src, TaintedPathSink sink
@@ -205,7 +205,7 @@ Thus, our example query becomes:
 Custom taint kinds and flows
 ----------------------------
 
-In the above examples, we have assumed the existence of a suitable ``TaintKind``, 
+In the above examples, we have assumed the existence of a suitable ``TaintKind``,
 but sometimes it is necessary to model the flow of other objects, such as database connections, or ``None``.
 
 The ``TaintTracking::Source`` and ``TaintTracking::Sink`` classes have predicates that determine which kind of taint the source and sink model, respectively.

docs/language/ql-spec/language.rst

@@ -1075,10 +1075,16 @@ Apart from the presence or absence of the rank variable, all other reduced forms
 
 -  If the formula is omitted, then it is taken to be ``any()``.
 -  If there are no aggregation expressions, then either: 
-   + The aggregation id is ``count`` or ``strictcount`` and the expression is taken to be ``1``. 
-   + There must be precisely one variable declaration, and the aggregation expression is taken to be a reference to that variable.
+
+   - The aggregation id is ``count`` or ``strictcount`` and the expression is taken to be ``1``. 
+   - There must be precisely one variable declaration, and the aggregation expression is taken to be a reference to that variable.
+
 -  If the aggregation id is ``concat`` or ``strictconcat`` and it has a single expression then the second expression is taken to be ``""``.
--  If the ``monotonicAggregates`` language pragma is not enabled, or the original formula and variable declarations are both omitted, then the aggregate is transformed as follows: - For each aggregation expression ``expr_i``, a fresh variable ``v_i`` is declared with the same type as the expression in addition to the original variable declarations. - The new range is the conjunction of the original range and a term ``v_i = expr_i`` for each aggregation expression ``expr_i``. - Each original aggregation expression ``expr_i`` is replaced by a new aggregation expression ``v_i``.
+-  If the ``monotonicAggregates`` language pragma is not enabled, or the original formula and variable declarations are both omitted, then the aggregate is transformed as follows: 
+
+   - For each aggregation expression ``expr_i``, a fresh variable ``v_i`` is declared with the same type as the expression in addition to the original variable declarations. 
+   - The new range is the conjunction of the original range and a term ``v_i = expr_i`` for each aggregation expression ``expr_i``.
+   - Each original aggregation expression ``expr_i`` is replaced by a new aggregation expression ``v_i``.
 
 The variables in the variable declarations list must not occur in the typing environment.