Skip to content
This repository has been archived by the owner on Apr 14, 2022. It is now read-only.

Add persistence tests for 60+ system modules #1429

Merged
merged 216 commits into from Aug 15, 2019

Conversation

MikhailArkhipov
Copy link

@MikhailArkhipov MikhailArkhipov commented Aug 13, 2019

Closes #1417
Closes #1370

  • Adds persistence of TypeVar declarations
  • Fixes issue when types imported via * overwrite local declarations (happens in socket.py)
  • Handles self-deriving class such as class A(Generic[T], A[str]) which happens to exist in typeshed stubs
  • Allows overwrite of method with function (happens in random stubs)
  • Improves documentation formatting in TypeVar
  • Add caching level setting
  • Add AbstractSet to typing specialization

@MikhailArkhipov MikhailArkhipov changed the title Add persistence tests for 50 system modules Add persistence tests for 60+ system modules Aug 13, 2019
@MikhailArkhipov MikhailArkhipov merged commit f267b80 into microsoft:db Aug 15, 2019
@MikhailArkhipov MikhailArkhipov deleted the dbsystest branch August 22, 2019 21:46
MikhailArkhipov pushed a commit that referenced this pull request Sep 16, 2019
* Persistent analysis, part I (#1224)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Merge master into database branch (#1312)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (#1227)

* Fix addBrackets setting (#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (#1246)

* Trying to augment argument set with Evaluator and Expression context (#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (#1285)

* Multiple analysis fixes (#1297)

Multiple analysis fixes.
- Fixes #1151: Reliable way to determine final analysis of a module
- Fixes #1201: NRE in SymbolCollector.AddProperty
- Fixes #1228: NRE in TryFindMissingDependencies
- Fixes #1294: Handle valid case for reloading AST
- Fixes #1295: FindReferences doesn't work on package init members
- Fixes #1296: DependencyResolver graph misses intermediate imports
- Part of the fix for #1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback

* Treat persistent module as regular specialized (#1334)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code

* Store location of module members in the database  (#1339)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype

* Multiple fixes and additional tests for persistence (#1372)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (#1405)

* Remove all references to dropped modules and force GC on reload (#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Add persistence tests for 60+ system modules (#1429)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines

* Test update (we no longer reassign classes)

* Fix typing tests

* Update baseline

* Fix missing keys issue (#1451)

* Typevar fix

* Persistence of generics and named tuples (#1459)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback

* - Fix #1455: AF: Library module of type Stub has been analyzed already! (#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast

* Update baselines since __spec__ member was added

* Undo some of changes to match master.

* Fix null ref

* Resolve conflicts

* Fix null ref on bases due to new AnyStr behavior

* Handling dependencies in module persistence (#1471)

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Separate creation and population for better handling of forward references.

* Add dependency provider

* Merge issues

* Better class creation

* Partial

* Dependency providers

* Add IsPersistent flag

* Make special global scope

* Move dependency fetch earlier

* Replace AST walk by interface

* Make analysis sequence closer to master

* Undo some

* Undo debug

* Show dependencies in log

* null check

* Update tests

* Make sure factory looks in correct scope
Enable assertions in tests

* Separate stub dependencies

* Don't try and merge from empty stub

* Match generic parameters by name

* Use qualified names in generics
Minor tests cleanup

* Back to names

* Update tests

* usings

* Fix dependencies for stub

* Null check

* PR feedback

* PR feedback + fix merge issue in generics

* Test fixes + PR feedback

* Baseline updates

* PR feedback

* Restore original order

* Order

* Merge issues

* Make cancellation closer to original

* Pass cancellation to module walker

* Simplify

* Upgrade to 3.0

* Undo change + .NET 3

* Get back to 2.0

* Update components

* Display caching level
Round time to reasonble fractions.

* Fix null ref due to conflicting changes

* Null ref fix

* Simplify

* Pass AST whish is required during analysis when module does not have its own yet.

* tensorflow test

* Fix from import model
Add tensorflow test

* Undo debug

* Improve goto def in import and fix class base processing (#1529)

* Update signing project to 3.0

* Upgrade to .NET 3.0 (#1521)

* Upgrade to 3.0

* Get back to 2.0

* Add global.json for .NET SDK

* Ensure FileInfo is not relative in DirectoryInfoProxy (#1519)

* Ensure FileInfo is not relative in DirectoryInfoProxy

* Add tests

* Hadle import

* Update dependencies (#1526)

* Handle from import

* Add test

* Fix bad find and replace in AssertionsUtilities (#1527)

* Merge issues

* Add test (fails)

* using

* Fix merge of classes

* Fix regression in bases
Split merging into separate class

* Fix null refs

* Make common CanUpdateAnalysis check

* Usings

* Test fix

* Remove CompletionItemKind.None

* Add info to asserts

* PR feedback

* Undo test change, enable class reassignment
Fix stub merge for TypeVar

* Prevent merge of unrelated types

* more tests

* Fix variable update

* Null check

* Baselines

* Move analysis complete event

* More stable enumeration

* Improve restoring imports in tests

* Make extension for tests

* Add URI null check

* Fix completion typo

* Add assert on unresolved modules

* Add more details to assert

* Using

* To 3.7

* Implement check for big/little endian

* Sync

* Synchronization
Fix import model restore

* Add hard assert

* Fix exception

* Fix typo

* Add lock
MikhailArkhipov pushed a commit that referenced this pull request Sep 16, 2019
* Persistent analysis, part I (#1224)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Merge master into database branch (#1312)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (#1227)

* Fix addBrackets setting (#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (#1246)

* Trying to augment argument set with Evaluator and Expression context (#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (#1285)

* Multiple analysis fixes (#1297)

Multiple analysis fixes.
- Fixes #1151: Reliable way to determine final analysis of a module
- Fixes #1201: NRE in SymbolCollector.AddProperty
- Fixes #1228: NRE in TryFindMissingDependencies
- Fixes #1294: Handle valid case for reloading AST
- Fixes #1295: FindReferences doesn't work on package init members
- Fixes #1296: DependencyResolver graph misses intermediate imports
- Part of the fix for #1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback

* Treat persistent module as regular specialized (#1334)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code

* Store location of module members in the database  (#1339)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype

* Multiple fixes and additional tests for persistence (#1372)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (#1405)

* Remove all references to dropped modules and force GC on reload (#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Add persistence tests for 60+ system modules (#1429)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines

* Test update (we no longer reassign classes)

* Fix typing tests

* Update baseline

* Fix missing keys issue (#1451)

* Typevar fix

* Persistence of generics and named tuples (#1459)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback

* - Fix #1455: AF: Library module of type Stub has been analyzed already! (#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast

* Update baselines since __spec__ member was added

* Undo some of changes to match master.

* Fix null ref

* Resolve conflicts

* Fix null ref on bases due to new AnyStr behavior

* Handling dependencies in module persistence (#1471)

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Separate creation and population for better handling of forward references.

* Add dependency provider

* Merge issues

* Better class creation

* Partial

* Dependency providers

* Add IsPersistent flag

* Make special global scope

* Move dependency fetch earlier

* Replace AST walk by interface

* Make analysis sequence closer to master

* Undo some

* Undo debug

* Show dependencies in log

* null check

* Update tests

* Make sure factory looks in correct scope
Enable assertions in tests

* Separate stub dependencies

* Don't try and merge from empty stub

* Match generic parameters by name

* Use qualified names in generics
Minor tests cleanup

* Back to names

* Update tests

* usings

* Fix dependencies for stub

* Null check

* PR feedback

* PR feedback + fix merge issue in generics

* Test fixes + PR feedback

* Baseline updates

* PR feedback

* Restore original order

* Order

* Merge issues

* Make cancellation closer to original

* Pass cancellation to module walker

* Simplify

* Upgrade to 3.0

* Undo change + .NET 3

* Get back to 2.0

* Update components

* Display caching level
Round time to reasonble fractions.

* Fix null ref due to conflicting changes

* Null ref fix

* Simplify

* Pass AST whish is required during analysis when module does not have its own yet.

* tensorflow test

* Fix from import model
Add tensorflow test

* Undo debug

* Improve goto def in import and fix class base processing (#1529)

* Update signing project to 3.0

* Upgrade to .NET 3.0 (#1521)

* Upgrade to 3.0

* Get back to 2.0

* Add global.json for .NET SDK

* Ensure FileInfo is not relative in DirectoryInfoProxy (#1519)

* Ensure FileInfo is not relative in DirectoryInfoProxy

* Add tests

* Hadle import

* Update dependencies (#1526)

* Handle from import

* Add test

* Fix bad find and replace in AssertionsUtilities (#1527)

* Merge issues

* Add test (fails)

* using

* Fix merge of classes

* Fix regression in bases
Split merging into separate class

* Fix null refs

* Make common CanUpdateAnalysis check

* Usings

* Test fix

* Remove CompletionItemKind.None

* Add info to asserts

* PR feedback

* Undo test change, enable class reassignment
Fix stub merge for TypeVar

* Prevent merge of unrelated types

* more tests

* Fix variable update

* Null check

* Baselines

* Move analysis complete event

* More stable enumeration

* Improve restoring imports in tests

* Make extension for tests

* Add URI null check

* Fix completion typo

* Add assert on unresolved modules

* Add more details to assert

* Using

* To 3.7

* Implement check for big/little endian

* Sync

* Synchronization
Fix import model restore

* Add hard assert

* Fix exception

* Fix typo

* Add lock

* Adding support for egg and zip files (#1477)

* Preliminary zip file

* Refactoring and adding .egg support

* refactoring

* More refactoring

* removing irrelevant classes

* Preliminary working in memory

* More things to make it work

* Getting rid of comment

* Refactoring

* More refactoring

* more changes

* refactoring

* Removing unneeded changes

* adding back

* PR feedback

* Test fix
jakebailey pushed a commit to jakebailey/python-language-server that referenced this pull request Nov 1, 2019
* Persistent analysis, part I (microsoft#1224)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Merge master into database branch (microsoft#1312)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (microsoft#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (microsoft#1227)

* Fix addBrackets setting (microsoft#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (microsoft#1246)

* Trying to augment argument set with Evaluator and Expression context (microsoft#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (microsoft#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (microsoft#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (microsoft#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (microsoft#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (microsoft#1285)

* Multiple analysis fixes (microsoft#1297)

Multiple analysis fixes.
- Fixes microsoft#1151: Reliable way to determine final analysis of a module
- Fixes microsoft#1201: NRE in SymbolCollector.AddProperty
- Fixes microsoft#1228: NRE in TryFindMissingDependencies
- Fixes microsoft#1294: Handle valid case for reloading AST
- Fixes microsoft#1295: FindReferences doesn't work on package init members
- Fixes microsoft#1296: DependencyResolver graph misses intermediate imports
- Part of the fix for microsoft#1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (microsoft#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (microsoft#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (microsoft#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback

* Treat persistent module as regular specialized (microsoft#1334)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code

* Store location of module members in the database  (microsoft#1339)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype

* Multiple fixes and additional tests for persistence (microsoft#1372)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (microsoft#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (microsoft#1405)

* Remove all references to dropped modules and force GC on reload (microsoft#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Add persistence tests for 60+ system modules (microsoft#1429)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines

* Test update (we no longer reassign classes)

* Fix typing tests

* Update baseline

* Fix missing keys issue (microsoft#1451)

* Typevar fix

* Persistence of generics and named tuples (microsoft#1459)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback

* - Fix microsoft#1455: AF: Library module of type Stub has been analyzed already! (microsoft#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast

* Update baselines since __spec__ member was added

* Undo some of changes to match master.

* Fix null ref

* Resolve conflicts

* Fix null ref on bases due to new AnyStr behavior

* Handling dependencies in module persistence (microsoft#1471)

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Separate creation and population for better handling of forward references.

* Add dependency provider

* Merge issues

* Better class creation

* Partial

* Dependency providers

* Add IsPersistent flag

* Make special global scope

* Move dependency fetch earlier

* Replace AST walk by interface

* Make analysis sequence closer to master

* Undo some

* Undo debug

* Show dependencies in log

* null check

* Update tests

* Make sure factory looks in correct scope
Enable assertions in tests

* Separate stub dependencies

* Don't try and merge from empty stub

* Match generic parameters by name

* Use qualified names in generics
Minor tests cleanup

* Back to names

* Update tests

* usings

* Fix dependencies for stub

* Null check

* PR feedback

* PR feedback + fix merge issue in generics

* Test fixes + PR feedback

* Baseline updates

* PR feedback

* Restore original order

* Order

* Merge issues

* Make cancellation closer to original

* Pass cancellation to module walker

* Simplify

* Upgrade to 3.0

* Undo change + .NET 3

* Get back to 2.0

* Update components

* Display caching level
Round time to reasonble fractions.

* Fix null ref due to conflicting changes

* Null ref fix

* Simplify

* Pass AST whish is required during analysis when module does not have its own yet.

* tensorflow test

* Fix from import model
Add tensorflow test

* Undo debug

* Improve goto def in import and fix class base processing (microsoft#1529)

* Update signing project to 3.0

* Upgrade to .NET 3.0 (microsoft#1521)

* Upgrade to 3.0

* Get back to 2.0

* Add global.json for .NET SDK

* Ensure FileInfo is not relative in DirectoryInfoProxy (microsoft#1519)

* Ensure FileInfo is not relative in DirectoryInfoProxy

* Add tests

* Hadle import

* Update dependencies (microsoft#1526)

* Handle from import

* Add test

* Fix bad find and replace in AssertionsUtilities (microsoft#1527)

* Merge issues

* Add test (fails)

* using

* Fix merge of classes

* Fix regression in bases
Split merging into separate class

* Fix null refs

* Make common CanUpdateAnalysis check

* Usings

* Test fix

* Remove CompletionItemKind.None

* Add info to asserts

* PR feedback

* Undo test change, enable class reassignment
Fix stub merge for TypeVar

* Prevent merge of unrelated types

* more tests

* Fix variable update

* Null check

* Baselines

* Move analysis complete event

* More stable enumeration

* Improve restoring imports in tests

* Make extension for tests

* Add URI null check

* Fix completion typo

* Add assert on unresolved modules

* Add more details to assert

* Using

* To 3.7

* Implement check for big/little endian

* Sync

* Synchronization
Fix import model restore

* Add hard assert

* Fix exception

* Fix typo

* Add lock
jakebailey pushed a commit to jakebailey/python-language-server that referenced this pull request Nov 1, 2019
* Persistent analysis, part I (microsoft#1224)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Merge master into database branch (microsoft#1312)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (microsoft#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (microsoft#1227)

* Fix addBrackets setting (microsoft#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (microsoft#1246)

* Trying to augment argument set with Evaluator and Expression context (microsoft#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (microsoft#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (microsoft#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (microsoft#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (microsoft#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (microsoft#1285)

* Multiple analysis fixes (microsoft#1297)

Multiple analysis fixes.
- Fixes microsoft#1151: Reliable way to determine final analysis of a module
- Fixes microsoft#1201: NRE in SymbolCollector.AddProperty
- Fixes microsoft#1228: NRE in TryFindMissingDependencies
- Fixes microsoft#1294: Handle valid case for reloading AST
- Fixes microsoft#1295: FindReferences doesn't work on package init members
- Fixes microsoft#1296: DependencyResolver graph misses intermediate imports
- Part of the fix for microsoft#1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (microsoft#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (microsoft#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (microsoft#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback

* Treat persistent module as regular specialized (microsoft#1334)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code

* Store location of module members in the database  (microsoft#1339)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype

* Multiple fixes and additional tests for persistence (microsoft#1372)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (microsoft#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (microsoft#1405)

* Remove all references to dropped modules and force GC on reload (microsoft#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Add persistence tests for 60+ system modules (microsoft#1429)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines

* Test update (we no longer reassign classes)

* Fix typing tests

* Update baseline

* Fix missing keys issue (microsoft#1451)

* Typevar fix

* Persistence of generics and named tuples (microsoft#1459)

* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback

* - Fix microsoft#1455: AF: Library module of type Stub has been analyzed already! (microsoft#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast

* Update baselines since __spec__ member was added

* Undo some of changes to match master.

* Fix null ref

* Resolve conflicts

* Fix null ref on bases due to new AnyStr behavior

* Handling dependencies in module persistence (microsoft#1471)

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Separate creation and population for better handling of forward references.

* Add dependency provider

* Merge issues

* Better class creation

* Partial

* Dependency providers

* Add IsPersistent flag

* Make special global scope

* Move dependency fetch earlier

* Replace AST walk by interface

* Make analysis sequence closer to master

* Undo some

* Undo debug

* Show dependencies in log

* null check

* Update tests

* Make sure factory looks in correct scope
Enable assertions in tests

* Separate stub dependencies

* Don't try and merge from empty stub

* Match generic parameters by name

* Use qualified names in generics
Minor tests cleanup

* Back to names

* Update tests

* usings

* Fix dependencies for stub

* Null check

* PR feedback

* PR feedback + fix merge issue in generics

* Test fixes + PR feedback

* Baseline updates

* PR feedback

* Restore original order

* Order

* Merge issues

* Make cancellation closer to original

* Pass cancellation to module walker

* Simplify

* Upgrade to 3.0

* Undo change + .NET 3

* Get back to 2.0

* Update components

* Display caching level
Round time to reasonble fractions.

* Fix null ref due to conflicting changes

* Null ref fix

* Simplify

* Pass AST whish is required during analysis when module does not have its own yet.

* tensorflow test

* Fix from import model
Add tensorflow test

* Undo debug

* Improve goto def in import and fix class base processing (microsoft#1529)

* Update signing project to 3.0

* Upgrade to .NET 3.0 (microsoft#1521)

* Upgrade to 3.0

* Get back to 2.0

* Add global.json for .NET SDK

* Ensure FileInfo is not relative in DirectoryInfoProxy (microsoft#1519)

* Ensure FileInfo is not relative in DirectoryInfoProxy

* Add tests

* Hadle import

* Update dependencies (microsoft#1526)

* Handle from import

* Add test

* Fix bad find and replace in AssertionsUtilities (microsoft#1527)

* Merge issues

* Add test (fails)

* using

* Fix merge of classes

* Fix regression in bases
Split merging into separate class

* Fix null refs

* Make common CanUpdateAnalysis check

* Usings

* Test fix

* Remove CompletionItemKind.None

* Add info to asserts

* PR feedback

* Undo test change, enable class reassignment
Fix stub merge for TypeVar

* Prevent merge of unrelated types

* more tests

* Fix variable update

* Null check

* Baselines

* Move analysis complete event

* More stable enumeration

* Improve restoring imports in tests

* Make extension for tests

* Add URI null check

* Fix completion typo

* Add assert on unresolved modules

* Add more details to assert

* Using

* To 3.7

* Implement check for big/little endian

* Sync

* Synchronization
Fix import model restore

* Add hard assert

* Fix exception

* Fix typo

* Add lock

* Adding support for egg and zip files (microsoft#1477)

* Preliminary zip file

* Refactoring and adding .egg support

* refactoring

* More refactoring

* removing irrelevant classes

* Preliminary working in memory

* More things to make it work

* Getting rid of comment

* Refactoring

* More refactoring

* more changes

* refactoring

* Removing unneeded changes

* adding back

* PR feedback

* Test fix
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant