Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations.

Patch by Ivan Levkivskyi.

Author: 1st1

Doc/glossary.rst

@@ -964,6 +964,18 @@ Glossary
       ``'\r'``.  See :pep:`278` and :pep:`3116`, as well as
       :func:`bytes.splitlines` for an additional use.
 
+   variable annotation
+      A type metadata value associated with a module global variable or
+      a class attribute. Its syntax is explained in section :ref:`annassign`.
+      Annotations are stored in the :attr:`__annotations__` special
+      attribute of a class or module object and can be accessed using
+      :func:`typing.get_type_hints`.
+
+      Python itself does not assign any particular meaning to variable
+      annotations. They are intended to be interpreted by third-party libraries
+      or type checking tools. See :pep:`526`, :pep:`484` which describe
+      some of their potential uses.
+
    virtual environment
       A cooperatively isolated runtime environment that allows Python users
       and applications to install and upgrade Python distribution packages

Doc/library/dis.rst

@@ -607,6 +607,12 @@ iterations of the loop.
 
    .. versionadded:: 3.3
 
+.. opcode:: SETUP_ANNOTATIONS
+
+   Checks whether ``__annotations__`` is defined in ``locals()``, if not it is
+   set up to an empty ``dict``. This opcode is only emmitted if a class
+   or module body contains :term:`variable annotations <variable annotation>`
+   statically.
 
 .. opcode:: IMPORT_STAR
 
@@ -890,6 +896,11 @@ All of the following opcodes use their arguments.
    Deletes local ``co_varnames[var_num]``.
 
 
+.. opcode:: STORE_ANNOTATION (namei)
+
+   Stores TOS as ``locals()['__annotations__'][co_names[namei]] = TOS``.
+
+
 .. opcode:: LOAD_CLOSURE (i)
 
    Pushes a reference to the cell contained in slot *i* of the cell and free

Doc/reference/datamodel.rst

@@ -686,33 +686,36 @@ Modules
    Attribute assignment updates the module's namespace dictionary, e.g.,
    ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``.
 
-   .. index:: single: __dict__ (module attribute)
-
-   Special read-only attribute: :attr:`~object.__dict__` is the module's namespace as a
-   dictionary object.
-
-   .. impl-detail::
-
-      Because of the way CPython clears module dictionaries, the module
-      dictionary will be cleared when the module falls out of scope even if the
-      dictionary still has live references.  To avoid this, copy the dictionary
-      or keep the module around while using its dictionary directly.
-
    .. index::
       single: __name__ (module attribute)
       single: __doc__ (module attribute)
       single: __file__ (module attribute)
+      single: __annotations__ (module attribute)
       pair: module; namespace
 
    Predefined (writable) attributes: :attr:`__name__` is the module's name;
    :attr:`__doc__` is the module's documentation string, or ``None`` if
-   unavailable; :attr:`__file__` is the pathname of the file from which the
+   unavailable; :attr:`__annotations__` (optional) is a dictionary containing
+   :term:`variable annotations <variable annotation>` collected during module
+   body execution; :attr:`__file__` is the pathname of the file from which the
    module was loaded, if it was loaded from a file. The :attr:`__file__`
    attribute may be missing for certain types of modules, such as C modules
    that are statically linked into the interpreter; for extension modules
    loaded dynamically from a shared library, it is the pathname of the shared
    library file.
 
+   .. index:: single: __dict__ (module attribute)
+
+   Special read-only attribute: :attr:`~object.__dict__` is the module's
+   namespace as a dictionary object.
+
+   .. impl-detail::
+
+      Because of the way CPython clears module dictionaries, the module
+      dictionary will be cleared when the module falls out of scope even if the
+      dictionary still has live references.  To avoid this, copy the dictionary
+      or keep the module around while using its dictionary directly.
+
 Custom classes
    Custom class types are typically created by class definitions (see section
    :ref:`class`).  A class has a namespace implemented by a dictionary object.
@@ -761,13 +764,17 @@ Custom classes
       single: __dict__ (class attribute)
       single: __bases__ (class attribute)
       single: __doc__ (class attribute)
+      single: __annotations__ (class attribute)
 
    Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is
    the module name in which the class was defined; :attr:`~object.__dict__` is the
    dictionary containing the class's namespace; :attr:`~class.__bases__` is a
    tuple (possibly empty or a singleton) containing the base classes, in the
    order of their occurrence in the base class list; :attr:`__doc__` is the
-   class's documentation string, or None if undefined.
+   class's documentation string, or None if undefined;
+   :attr:`__annotations__` (optional) is a dictionary containing
+   :term:`variable annotations <variable annotation>` collected during
+   class body execution.
 
 Class instances
    .. index::

Doc/reference/simple_stmts.rst

@@ -16,6 +16,7 @@ simple statements is:
               : | `assert_stmt`
               : | `assignment_stmt`
               : | `augmented_assignment_stmt`
+              : | `annotated_assignment_stmt`
               : | `pass_stmt`
               : | `del_stmt`
               : | `return_stmt`
@@ -312,6 +313,49 @@ For targets which are attribute references, the same :ref:`caveat about class
 and instance attributes <attr-target-note>` applies as for regular assignments.
 
 
+.. _annassign:
+
+Annotated assignment statements
+-------------------------------
+
+.. index::
+   pair: annotated; assignment
+   single: statement; assignment, annotated
+
+Annotation assignment is the combination, in a single statement,
+of a variable or attribute annotation and an optional assignment statement:
+
+.. productionlist::
+   annotated_assignment_stmt: `augtarget` ":" `expression` ["=" `expression`]
+
+The difference from normal :ref:`assignment` is that only single target and
+only single right hand side value is allowed.
+
+For simple names as assignment targets, if in class or module scope,
+the annotations are evaluated and stored in a special class or module
+attribute :attr:`__annotations__`
+that is a dictionary mapping from variable names to evaluated annotations.
+This attribute is writable and is automatically created at the start
+of class or module body execution, if annotations are found statically.
+
+For expressions as assignment targets, the annotations are evaluated if
+in class or module scope, but not stored.
+
+If a name is annotated in a function scope, then this name is local for
+that scope. Annotations are never evaluated and stored in function scopes.
+
+If the right hand side is present, an annotated
+assignment performs the actual assignment before evaluating annotations
+(where applicable). If the right hand side is not present for an expression
+target, then the interpreter evaluates the target except for the last
+:meth:`__setitem__` or :meth:`__setattr__` call.
+
+.. seealso::
+
+   :pep:`526` - Variable and attribute annotation syntax
+   :pep:`484` - Type hints
+
+
 .. _assert:
 
 The :keyword:`assert` statement

Grammar/Grammar

@@ -45,12 +45,13 @@ stmt: simple_stmt | compound_stmt
 simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
 small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt |
              import_stmt | global_stmt | nonlocal_stmt | assert_stmt)
-expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) |
+expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) |
                      ('=' (yield_expr|testlist_star_expr))*)
+annassign: ':' test ['=' test]
 testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [',']
 augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' |
             '<<=' | '>>=' | '**=' | '//=')
-# For normal assignments, additional restrictions enforced by the interpreter
+# For normal and annotated assignments, additional restrictions enforced by the interpreter
 del_stmt: 'del' exprlist
 pass_stmt: 'pass'
 flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt

Include/Python-ast.h

@@ -65,11 +65,12 @@ struct _mod {
 
 enum _stmt_kind {FunctionDef_kind=1, AsyncFunctionDef_kind=2, ClassDef_kind=3,
                   Return_kind=4, Delete_kind=5, Assign_kind=6,
-                  AugAssign_kind=7, For_kind=8, AsyncFor_kind=9, While_kind=10,
-                  If_kind=11, With_kind=12, AsyncWith_kind=13, Raise_kind=14,
-                  Try_kind=15, Assert_kind=16, Import_kind=17,
-                  ImportFrom_kind=18, Global_kind=19, Nonlocal_kind=20,
-                  Expr_kind=21, Pass_kind=22, Break_kind=23, Continue_kind=24};
+                  AugAssign_kind=7, AnnAssign_kind=8, For_kind=9,
+                  AsyncFor_kind=10, While_kind=11, If_kind=12, With_kind=13,
+                  AsyncWith_kind=14, Raise_kind=15, Try_kind=16,
+                  Assert_kind=17, Import_kind=18, ImportFrom_kind=19,
+                  Global_kind=20, Nonlocal_kind=21, Expr_kind=22, Pass_kind=23,
+                  Break_kind=24, Continue_kind=25};
 struct _stmt {
     enum _stmt_kind kind;
     union {
@@ -116,6 +117,13 @@ struct _stmt {
             expr_ty value;
         } AugAssign;
 
+        struct {
+            expr_ty target;
+            expr_ty annotation;
+            expr_ty value;
+            int simple;
+        } AnnAssign;
+        
         struct {
             expr_ty target;
             expr_ty iter;
@@ -461,6 +469,9 @@ stmt_ty _Py_Assign(asdl_seq * targets, expr_ty value, int lineno, int
 #define AugAssign(a0, a1, a2, a3, a4, a5) _Py_AugAssign(a0, a1, a2, a3, a4, a5)
 stmt_ty _Py_AugAssign(expr_ty target, operator_ty op, expr_ty value, int
                       lineno, int col_offset, PyArena *arena);
+#define AnnAssign(a0, a1, a2, a3, a4, a5, a6) _Py_AnnAssign(a0, a1, a2, a3, a4, a5, a6)
+stmt_ty _Py_AnnAssign(expr_ty target, expr_ty annotation, expr_ty value, int
+                      simple, int lineno, int col_offset, PyArena *arena);
 #define For(a0, a1, a2, a3, a4, a5, a6) _Py_For(a0, a1, a2, a3, a4, a5, a6)
 stmt_ty _Py_For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq *
                 orelse, int lineno, int col_offset, PyArena *arena);

Include/graminit.h

@@ -17,71 +17,72 @@
 #define simple_stmt 270
 #define small_stmt 271
 #define expr_stmt 272
-#define testlist_star_expr 273
-#define augassign 274
-#define del_stmt 275
-#define pass_stmt 276
-#define flow_stmt 277
-#define break_stmt 278
-#define continue_stmt 279
-#define return_stmt 280
-#define yield_stmt 281
-#define raise_stmt 282
-#define import_stmt 283
-#define import_name 284
-#define import_from 285
-#define import_as_name 286
-#define dotted_as_name 287
-#define import_as_names 288
-#define dotted_as_names 289
-#define dotted_name 290
-#define global_stmt 291
-#define nonlocal_stmt 292
-#define assert_stmt 293
-#define compound_stmt 294
-#define async_stmt 295
-#define if_stmt 296
-#define while_stmt 297
-#define for_stmt 298
-#define try_stmt 299
-#define with_stmt 300
-#define with_item 301
-#define except_clause 302
-#define suite 303
-#define test 304
-#define test_nocond 305
-#define lambdef 306
-#define lambdef_nocond 307
-#define or_test 308
-#define and_test 309
-#define not_test 310
-#define comparison 311
-#define comp_op 312
-#define star_expr 313
-#define expr 314
-#define xor_expr 315
-#define and_expr 316
-#define shift_expr 317
-#define arith_expr 318
-#define term 319
-#define factor 320
-#define power 321
-#define atom_expr 322
-#define atom 323
-#define testlist_comp 324
-#define trailer 325
-#define subscriptlist 326
-#define subscript 327
-#define sliceop 328
-#define exprlist 329
-#define testlist 330
-#define dictorsetmaker 331
-#define classdef 332
-#define arglist 333
-#define argument 334
-#define comp_iter 335
-#define comp_for 336
-#define comp_if 337
-#define encoding_decl 338
-#define yield_expr 339
-#define yield_arg 340
+#define annassign 273
+#define testlist_star_expr 274
+#define augassign 275
+#define del_stmt 276
+#define pass_stmt 277
+#define flow_stmt 278
+#define break_stmt 279
+#define continue_stmt 280
+#define return_stmt 281
+#define yield_stmt 282
+#define raise_stmt 283
+#define import_stmt 284
+#define import_name 285
+#define import_from 286
+#define import_as_name 287
+#define dotted_as_name 288
+#define import_as_names 289
+#define dotted_as_names 290
+#define dotted_name 291
+#define global_stmt 292
+#define nonlocal_stmt 293
+#define assert_stmt 294
+#define compound_stmt 295
+#define async_stmt 296
+#define if_stmt 297
+#define while_stmt 298
+#define for_stmt 299
+#define try_stmt 300
+#define with_stmt 301
+#define with_item 302
+#define except_clause 303
+#define suite 304
+#define test 305
+#define test_nocond 306
+#define lambdef 307
+#define lambdef_nocond 308
+#define or_test 309
+#define and_test 310
+#define not_test 311
+#define comparison 312
+#define comp_op 313
+#define star_expr 314
+#define expr 315
+#define xor_expr 316
+#define and_expr 317
+#define shift_expr 318
+#define arith_expr 319
+#define term 320
+#define factor 321
+#define power 322
+#define atom_expr 323
+#define atom 324
+#define testlist_comp 325
+#define trailer 326
+#define subscriptlist 327
+#define subscript 328
+#define sliceop 329
+#define exprlist 330
+#define testlist 331
+#define dictorsetmaker 332
+#define classdef 333
+#define arglist 334
+#define argument 335
+#define comp_iter 336
+#define comp_for 337
+#define comp_if 338
+#define encoding_decl 339
+#define yield_expr 340
+#define yield_arg 341

Include/opcode.h

@@ -60,6 +60,7 @@ extern "C" {
 #define WITH_CLEANUP_FINISH      82
 #define RETURN_VALUE             83
 #define IMPORT_STAR              84
+#define SETUP_ANNOTATIONS        85
 #define YIELD_VALUE              86
 #define POP_BLOCK                87
 #define END_FINALLY              88
@@ -98,6 +99,7 @@ extern "C" {
 #define LOAD_FAST               124
 #define STORE_FAST              125
 #define DELETE_FAST             126
+#define STORE_ANNOTATION        127
 #define RAISE_VARARGS           130
 #define CALL_FUNCTION           131
 #define MAKE_FUNCTION           132

Include/symtable.h

@@ -91,6 +91,7 @@ PyAPI_FUNC(void) PySymtable_Free(struct symtable *);
 #define DEF_FREE 2<<4          /* name used but not defined in nested block */
 #define DEF_FREE_CLASS 2<<5    /* free variable from class's method */
 #define DEF_IMPORT 2<<6        /* assignment occurred via import */
+#define DEF_ANNOT 2<<7         /* this name is annotated */
 
 #define DEF_BOUND (DEF_LOCAL | DEF_PARAM | DEF_IMPORT)