Merge pull request #1251 from google/google_sync

Google sync

README.md

@@ -102,7 +102,7 @@ Platform support:
 * Windows is currently not supported unless you use [WSL][wsl].
 
 <sub>\*
-On Alpine Linux, installing may fail due to issues with upstream
+On Alpine Linux, installation may fail due to issues with upstream
 dependencies.  See the details of [this issue][scikit-build-issue] for a
 possible fix.
 <br />
@@ -166,7 +166,7 @@ Common options:
   Defaults to the version that pytype is running under.
 * `-o, --output`: The directory into which all pytype output goes, including
   generated .pyi files. Defaults to `.pytype`.
-* `-d, --disable`. Comma or space separated list of error names to ignore.
+* `-d, --disable`. Comma or space-separated list of error names to ignore.
   Detailed explanations of pytype's error names are in
   [this doc][error-classes]. Defaults to empty.
 
@@ -231,7 +231,7 @@ pythonpath =
     .:
     ~/repo2
 
-# Comma or space separated list of error names to ignore.
+# Comma or space-separated list of error names to ignore.
 disable =
     attribute-error
 ```
@@ -257,7 +257,7 @@ Python file.
 * `pytype-single`, a debugging tool for pytype developers, which analyzes a
 single Python file assuming that .pyi files have already been generated for all
 of its dependencies.
-* `pyxref`, a cross references generator.
+* `pyxref`, a cross-references generator.
 
 ## 2022 Roadmap
 

docs/developers/config.md

@@ -17,7 +17,7 @@ freshness: { owner: 'mdemello' reviewed: '2021-11-29' }
       * [Adding a new option](#adding-a-new-option)
       * [Config files](#config-files)
 
-<!-- Added by: rechen, at: 2022-02-03T17:05-08:00 -->
+<!-- Added by: rechen, at: 2022-07-11T17:31-07:00 -->
 
 <!--te-->
 
@@ -151,41 +151,40 @@ postprocessing step. This invokes the `config.Postprocessor` class, which
 copies options from the raw `input_options` to a final `output_options`. The
 `Postprocessor` class does several things:
 
-1. Define `_store_*()` methods, corresponding to some of the options. If
-   `Postprocessor._store_foo()` exists, it will be called with `options.foo` as
-   an argument; i.e.
-
-   ```
-   if hasattr(postprocessor, '_store_foo'):
-     output_options.foo = postprocessor._store_foo(input_options.foo)
-   else:
-     output_options.foo = input_options.foo
-   ```
-
-2. Arrange the options into a dependency graph, so that some options can use the
-   *postprocessed* values of other options in their own postprocessing step. For
-   example, `options.module_name` is postprocessed via
-
-   ```
-   @uses(["input", "pythonpath"])
-   def _store_module_name(self, module_name):
-     if module_name is None:
-       module_name = module_utils.get_module_name(
-           self.output_options.input, self.output_options.pythonpath)
-       self.output_options.module_name = module_name
-   ```
-
-   where `self.output_options.pythonpath` is used to construct
-   `self.output_options.module_name`. The postprocessor uses the
-   `@uses["pythonpath"]` decorator to make sure that `_store_pythonpath()` is
-   run before `_store_module_name`, so that `output_options.pythonpath` has the
-   correct value when we read it.
-
-3. Populate some options that do not correspond to inputs. For example
-   `_store_python_version` sets both `output_options.python_version` and
-   `output_options.python_exe`. The latter is derived from the python version
-   and cached in `options.python_exe`, but it can not be set indepedently.
-
+1.  Define `_store_*()` methods, corresponding to some of the options. If
+    `Postprocessor._store_foo()` exists, it will be called with `options.foo` as
+    an argument; i.e.
+
+    ```
+    if hasattr(postprocessor, '_store_foo'):
+      output_options.foo = postprocessor._store_foo(input_options.foo)
+    else:
+      output_options.foo = input_options.foo
+    ```
+
+2.  Arrange the options into a dependency graph, so that some options can use
+    the *postprocessed* values of other options in their own postprocessing
+    step. For example, `options.module_name` is postprocessed via
+
+    ```
+    @uses(["input", "pythonpath"])
+    def _store_module_name(self, module_name):
+      if module_name is None:
+        module_name = module_utils.get_module_name(
+            self.output_options.input, self.output_options.pythonpath)
+        self.output_options.module_name = module_name
+    ```
+
+    where `self.output_options.pythonpath` is used to construct
+    `self.output_options.module_name`. The postprocessor uses the
+    `@uses["pythonpath"]` decorator to make sure that `_store_pythonpath()` is
+    run before `_store_module_name`, so that `output_options.pythonpath` has the
+    correct value when we read it.
+
+3.  Populate some options that do not correspond to inputs. For example
+    `_store_python_version` sets both `output_options.python_version` and
+    `output_options.python_exe`. The latter is derived from the python version
+    and cached in `options.python_exe`, but it can not be set independently.
 
 ## Adding a new option
 

docs/developers/typegraph.md

@@ -22,7 +22,7 @@ freshness: { owner: 'tsudol' reviewed: '2020-11-20' }
          * [A More Complex Example](#a-more-complex-example)
          * [Shortcircuiting and the solver cache](#shortcircuiting-and-the-solver-cache)
 
-<!-- Added by: rechen, at: 2022-02-03T17:05-08:00 -->
+<!-- Added by: rechen, at: 2022-07-11T17:31-07:00 -->
 
 <!--te-->
 
@@ -225,7 +225,7 @@ When a `PyObject*` value is added to a Binding, it's transformed into a
 `BindingData` pointer with a cleanup function that decrements the reference
 count. This approach ensures that each `PyObject*` has a correct reference
 count, that no data objects will be deleted before the Binding is cleaned up,
-and that deleting a Binding (or a whole Program) will correctly decrememnt the
+and that deleting a Binding (or a whole Program) will correctly decrement the
 reference count. This avoids potential memory leaks caused by `PyObject`s not
 being cleaned up.
 

docs/index.md

@@ -99,7 +99,7 @@ Platform support:
 * Windows is currently not supported unless you use [WSL][wsl].
 
 <sub>\*
-On Alpine Linux, installing may fail due to issues with upstream
+On Alpine Linux, installation may fail due to issues with upstream
 dependencies.  See the details of [this issue][scikit-build-issue] for a
 possible fix.
 <br />
@@ -163,7 +163,7 @@ Common options:
   Defaults to the version that pytype is running under.
 * `-o, --output`: The directory into which all pytype output goes, including
   generated .pyi files. Defaults to `.pytype`.
-* `-d, --disable`. Comma or space separated list of error names to ignore.
+* `-d, --disable`. Comma or space-separated list of error names to ignore.
   Detailed explanations of pytype's error names are in
   [this doc][error-classes]. Defaults to empty.
 
@@ -228,7 +228,7 @@ pythonpath =
     .:
     ~/repo2
 
-# Comma or space separated list of error names to ignore.
+# Comma or space-separated list of error names to ignore.
 disable =
     attribute-error
 ```
@@ -254,7 +254,7 @@ Python file.
 * `pytype-single`, a debugging tool for pytype developers, which analyzes a
 single Python file assuming that .pyi files have already been generated for all
 of its dependencies.
-* `pyxref`, a cross references generator.
+* `pyxref`, a cross-references generator.
 
 ## 2022 Roadmap
 

pytype/attribute.py

@@ -274,19 +274,25 @@ def _get_attribute(self, node, obj, cls, name, valself):
             node, obj, name, valself, skip=())
       else:
         node, attr = self._get_member(node, obj, name, valself)
-    if attr is None and obj.maybe_missing_members:
-      # The VM hit maximum depth while initializing this instance, so it may
-      # have attributes that we don't know about. These attributes take
-      # precedence over class attributes and __getattr__, so we set `attr` to
-      # Any immediately.
-      attr = self.ctx.new_unsolvable(node)
+    # If the VM hit maximum depth while initializing this instance, it may have
+    # attributes that we don't know about.
+    is_unknown_instance_attribute = attr is None and obj.maybe_missing_members
     if attr is None and cls:
       # Check for the attribute on the class.
       node, attr = self.get_attribute(node, cls, name, valself)
-      if attr is None:
+      if attr:
+        # If the attribute is a method, then we allow it to take precedence over
+        # the possible unknown instance attribute, since otherwise method lookup
+        # on classes with _HAS_DYNAMIC_ATTRIBUTES would always return Any.
+        if (is_unknown_instance_attribute and
+            any(isinstance(v, abstract.FUNCTION_TYPES) for v in attr.data)):
+          is_unknown_instance_attribute = False
+      elif not is_unknown_instance_attribute:
         # Fall back to __getattr__ if the attribute doesn't otherwise exist.
         node, attr = self._get_attribute_computed(
             node, cls, name, valself, compute_function="__getattr__")
+    if is_unknown_instance_attribute:
+      attr = self.ctx.new_unsolvable(node)
     if attr is not None:
       attr = self._filter_var(node, attr)
     return node, attr

pytype/convert.py

@@ -2,6 +2,7 @@
 
 import logging
 import types
+from typing import Any, Dict
 
 from pytype import blocks
 from pytype import datatypes
@@ -65,7 +66,7 @@ def __init__(self, ctx):
     super().__init__(ctx)
     ctx.convert = self  # to make constant_to_value calls below work
 
-    self._convert_cache = {}
+    self._convert_cache: Dict[Any, Any] = {}
     self._resolved_late_types = {}  # performance cache
 
     # Initialize primitive_classes to empty to allow constant_to_value to run.
@@ -303,7 +304,7 @@ def get_maybe_abstract_instance(self, data):
         return self.primitive_class_instances[data_type]
     return data
 
-  def _create_new_unknown_value(self, action):
+  def _create_new_unknown_value(self, action) -> abstract.Unknown:
     if not action or not self.ctx.vm.frame:
       return abstract.Unknown(self.ctx)
     # We allow only one Unknown at each point in the program, regardless of

pytype/tests/test_methods2.py

@@ -115,6 +115,16 @@ def f(x: Union[Foo, Bar]):
           return x()
     """)
 
+  def test_lookup_on_dynamic_class(self):
+    self.Check("""
+      class Foo:
+        _HAS_DYNAMIC_ATTRIBUTES = True
+        def f(self) -> str:
+          return ''
+        def g(self):
+          assert_type(self.f(), str)
+    """)
+
 
 class TestMethodsPy3(test_base.BaseTest):
   """Test python3-specific method features."""