Pyright Fails to Resolve RDKit Imports and Attributes Due to C++ Extensions

[ELO2026X]

Title: Pyright Fails to Resolve RDKit Imports and Attributes Due to C++ Extensions

Description

I am encountering persistent issues with Pyright when working with RDKit, a cheminformatics library that relies heavily on C++ extensions. Pyright incorrectly flags valid RDKit imports and attributes as unresolved or unknown, leading to false positives in my IDE. This behavior confuses users and the IDE into thinking there are unresolved bugs, even though the code works correctly at runtime.

Key Issues

1. Missing Imports:

   • Problem: Pyright cannot resolve imports like "rdkit.Chem.Draw".

   • Cause: RDKit's C++ extensions are not fully understood by Pyright's static type checker. Pyright is designed to analyze Python code and struggles with libraries that rely on compiled C++ components.

   • Impact: This results in false positives where Pyright incorrectly flags valid imports as unresolved, leading to confusion and unnecessary debugging efforts.

2. Attribute Access Issues:

   • Problem: Pyright reports that certain attributes (e.g., "EmbedMolecule", "MolWt", "NumHDonors", etc.) are unknown or not recognized in RDKit modules like rdkit.Chem.AllChem, rdkit.Chem.Descriptors, and rdkit.Chem.Lipinski.

   • Cause: These attributes are part of RDKit's C++ extensions, and Pyright cannot infer their existence or types during static analysis.

   • Impact: Developers see warnings or errors in their IDE, even though the code works correctly at runtime. This creates unnecessary noise and reduces trust in Pyright's diagnostics.

3. Type Mismatch Issues:

   • Problem: Pyright reports type mismatches, such as passing a list[str] to a parameter expecting a different type (e.g., Axes | None).

   • Cause: This is unrelated to RDKit and is a general Pyright type-checking issue. It indicates a mismatch between the expected and provided types in the code.

   • Impact: This can lead to runtime errors if the types are not compatible, but the warnings are often misleading or incorrect.

---

Error Log Examples

Here are specific instances of Pyright warnings and errors:

1. Missing Imports:

   • "Import 'rdkit.Chem.Draw' could not be resolved" (Lines 3-4).

2. Attribute Access Issues:

   • "EmbedMolecule" is not a known attribute of module "rdkit.Chem.AllChem" (Line 51).

   • "MolWt" is not a known attribute of module "rdkit.Chem.Descriptors" (Line 64).

   • "NumHDonors" is not a known attribute of module "rdkit.Chem.Lipinski" (Line 66).

3. Type Mismatches:

   • "Argument of type 'list[str]' cannot be assigned to parameter 'columns' of type 'Axes | None'" (Lines 75, 109).

---

Impact on Users

These issues persist and confuse users and the IDE software into thinking unresolved bugs exist. The false positives:

• Create unnecessary noise in the IDE.

• Reduce trust in Pyright's diagnostics.

• Increase debugging time as users attempt to resolve non-existent issues.

• Hinder the adoption of Pyright for projects that rely on libraries with C++ extensions, such as RDKit.

---

Recommendations for the Pyright Team

1. Improve C++ Extension Support:

   • Enhance Pyright's ability to analyze and resolve imports and attributes from libraries that rely on C++ extensions, such as RDKit.

   • Consider integrating with tools like mypy or pybind11 to better handle C++-based Python libraries.

2. Provide Custom Configuration Options:

   • Allow users to specify custom rules or ignore specific modules/attributes in pyrightconfig.json to reduce false positives.

   • Example:

     {
     
       "ignoreModules": ["rdkit.Chem.Draw"],
     
       "ignoreAttributes": ["rdkit.Chem.AllChem.EmbedMolecule"]
     
     }
     

3. Generate Stubs for C++ Libraries:

   • Provide a mechanism to automatically generate or include type stubs for popular libraries like RDKit to improve type inference.

   • Consider collaborating with library maintainers to distribute type stubs alongside the libraries.

4. Document Workarounds:

   • Add documentation on how to handle libraries with C++ extensions, including suppressing warnings and adding type annotations.

   • Provide examples of how to write custom type stubs for such libraries.

5. Improve Type Inference for Dynamic Attributes:

   • Enhance Pyright's ability to infer types for dynamically generated attributes in C++-based libraries.

---

Temporary Workarounds

1. Suppress Warnings:

   • Use a pyrightconfig.json file to suppress specific warnings:

     {
     
       "reportMissingImports": false,
     
       "reportAttributeAccessIssue": false
     
     }
     

2. Verify RDKit Installation:

   • Ensure RDKit is installed correctly using conda:

     conda install -c conda-forge rdkit
     

3. Add Type Annotations:

   • Use type annotations or stubs for RDKit modules to help Pyright understand the types:

     from rdkit import Chem
     
     from typing import Any
     
     def embed_molecule(mol: Chem.Mol) -> Any:
     
         return Chem.AllChem.EmbedMolecule(mol)

4. Fix Type Mismatches:

   • Ensure the correct types are passed to functions to avoid type-checking errors.

---

Conclusion

The issues stem from Pyright's inability to fully analyze RDKit's C++ extensions, leading to false positives for missing imports and attribute access. While temporary workarounds exist, improving Pyright's support for C++-based libraries would greatly enhance the developer experience. I hope the Pyright team can address these limitations in future updates.

---

Additional Context

• IDE: TRAE IDE

• Pyright Extension:

  • Name: Pyright

  • Version: v1.1.396

  • Publisher: ms-pyright

  • Repository: Pyright GitHub Repository

  • VS Code Marketplace Link: Pyright Extension

RDKit Version: RDKit version 2023.09.1

Python Version: Python 3.12.3

problems log:

[{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportMissingImports",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportMissingImports"
}
},
"severity": 8,
"message": "Import "rdkit.Chem.Draw" could not be resolved",
"source": "Pyright",
"startLineNumber": 3,
"startColumn": 6,
"endLineNumber": 3,
"endColumn": 21,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportMissingImports",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportMissingImports"
}
},
"severity": 8,
"message": "Import "rdkit.Chem.Draw" could not be resolved",
"source": "Pyright",
"startLineNumber": 4,
"startColumn": 6,
"endLineNumber": 4,
"endColumn": 21,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""EmbedMolecule" is not a known attribute of module "rdkit.Chem.AllChem"",
"source": "Pyright",
"startLineNumber": 51,
"startColumn": 13,
"endLineNumber": 51,
"endColumn": 26,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""MMFFOptimizeMolecule" is not a known attribute of module "rdkit.Chem.AllChem"",
"source": "Pyright",
"startLineNumber": 52,
"startColumn": 13,
"endLineNumber": 52,
"endColumn": 33,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""MolWt" is not a known attribute of module "rdkit.Chem.Descriptors"",
"source": "Pyright",
"startLineNumber": 64,
"startColumn": 40,
"endLineNumber": 64,
"endColumn": 45,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""MolLogP" is not a known attribute of module "rdkit.Chem.Descriptors"",
"source": "Pyright",
"startLineNumber": 65,
"startColumn": 28,
"endLineNumber": 65,
"endColumn": 35,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""NumHDonors" is not a known attribute of module "rdkit.Chem.Lipinski"",
"source": "Pyright",
"startLineNumber": 66,
"startColumn": 41,
"endLineNumber": 66,
"endColumn": 51,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""NumHAcceptors" is not a known attribute of module "rdkit.Chem.Lipinski"",
"source": "Pyright",
"startLineNumber": 67,
"startColumn": 44,
"endLineNumber": 67,
"endColumn": 57,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""TPSA" is not a known attribute of module "rdkit.Chem.Descriptors"",
"source": "Pyright",
"startLineNumber": 68,
"startColumn": 54,
"endLineNumber": 68,
"endColumn": 58,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""NumRotatableBonds" is not a known attribute of module "rdkit.Chem.Descriptors"",
"source": "Pyright",
"startLineNumber": 69,
"startColumn": 39,
"endLineNumber": 69,
"endColumn": 56,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportAttributeAccessIssue",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportAttributeAccessIssue"
}
},
"severity": 8,
"message": ""RingCount" is not a known attribute of module "rdkit.Chem.Descriptors"",
"source": "Pyright",
"startLineNumber": 71,
"startColumn": 34,
"endLineNumber": 71,
"endColumn": 43,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportArgumentType",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportArgumentType"
}
},
"severity": 8,
"message": "Argument of type "list[str]" cannot be assigned to parameter "columns" of type "Axes | None" in function "init"\n  Type "list[str]" is not assignable to type "Axes | None"\n    "list[str]" is not assignable to "ExtensionArray"\n    "list[str]" is not assignable to "ndarray[Unknown, Unknown]"\n    "list[str]" is not assignable to "Index"\n    "list[str]" is not assignable to "Series"\n    "list[str]" is incompatible with protocol "SequenceNotStr[Unknown]"\n      "index" is an incompatible type\n        Type "(value: str, start: SupportsIndex = 0, stop: SupportsIndex = sys.maxsize, /) -> int" is not assignable to type "(value: Any, /, start: int = 0, stop: int = ...) -> int"\n ...",
"source": "Pyright",
"startLineNumber": 75,
"startColumn": 53,
"endLineNumber": 75,
"endColumn": 74,
"extensionID": "ms-pyright.pyright"
},{
"resource": "/c:/Users/emili/Downloads/Chemical_Compound_Visualization/app.py",
"owner": "python1",
"code": {
"value": "reportArgumentType",
"target": {
"$mid": 1,
"path": "/microsoft/pyright/blob/main/docs/configuration.md",
"scheme": "https",
"authority": "github.com",
"fragment": "reportArgumentType"
}
},
"severity": 8,
"message": "Argument of type "list[str]" cannot be assigned to parameter "columns" of type "Axes | None" in function "init"\n  Type "list[str]" is not assignable to type "Axes | None"\n    "list[str]" is not assignable to "ExtensionArray"\n    "list[str]" is not assignable to "ndarray[Unknown, Unknown]"\n    "list[str]" is not assignable to "Index"\n    "list[str]" is not assignable to "Series"\n    "list[str]" is incompatible with protocol "SequenceNotStr[Unknown]"\n      "index" is an incompatible type\n        Type "(value: str, start: SupportsIndex = 0, stop: SupportsIndex = sys.maxsize, /) -> int" is not assignable to type "(value: Any, /, start: int = 0, stop: int = ...) -> int"\n ...",
"source": "Pyright",
"startLineNumber": 109,
"startColumn": 49,
"endLineNumber": 109,
"endColumn": 76,
"extensionID": "ms-pyright.pyright"
}]

[erictraut]

Static type checkers like pyright and mypy rely on type information provided in either source (".py") or stub (".pyi") files. If the code is written in a language other than Python, you'll need to provide stub files that describe the interface. For more details, refer to this documentation. Stub files were introduced over ten years ago as part of PEP 484 as a way to deal with this use case.

If you would like to tell pyright to simply ignore all of the symbols in a package, you can create a top-level stub file in your local project that simply contains the following contents.

# typings/package_name/__init__.pyi
def __getattr__(name: str) -> Any: ...

This mechanism has been standardized across type checkers, so it will work with pyright, mypy, etc.