Support Pandas IO operation Return Types

[ShivnarenSrinivasan]

Bug Report

Incorrect type inference for annotated pandas functions

Example Code

import sqlite3
import pandas as pd

def load_excel() -> pd.DataFrame:
    df = pd.read_excel('example.xlsx')
    return df


def load_sql() -> pd.DataFrame:
    con = sqlite3.Connection('a.db')
    df = pd.read_sql('', con)
    return df

Expected Behavior

Return type should be DataFrame (not Any), as that is the source code annotation for read_excel and read_sql.

Actual Behavior

Terminal output:

Your Environment

• Mypy version used: 0.961
• Mypy command-line flags: None
• Mypy configuration options: file attached
• Python version used: 3.10
• Operating system and version: Windows 10 21H2

[JelleZijlstra]

Do you have stubs installed for pandas? As I recall, pandas is not a py.typed library (see PEP-561), so mypy will not use its inline types.

[ShivnarenSrinivasan]

Thanks, @JelleZijlstra—I do have pandas-stubs installed:

However, I see that the stubs are installed within the package itself--i.e, the .pyi files are added alongside the source files.
Not like the usual types-xxx install, where the packages are installed in a separate directory.

Anything different I need to do to get it to work?

[hauntsaninja]

No, it's just a shortcoming of the version of the stubs you're using. If you look at:
https://github.com/VirtusLab/pandas-stubs/blob/5b38cae915aa2aedc39df511e58545b0e9c9715d/third_party/3/pandas/io/excel/_base.pyi#L13
you'll see that it's genuinely returning Any.

It looks like there's a better version of the stubs available at https://github.com/pandas-dev/pandas-stubs (and is meant to replace the existing ones), but that hasn't been released yet.

[ShivnarenSrinivasan]

No, it's just a shortcoming of the version of the stubs you're using. If you look at: https://github.com/VirtusLab/pandas-stubs/blob/5b38cae915aa2aedc39df511e58545b0e9c9715d/third_party/3/pandas/io/excel/_base.pyi#L13 you'll see that it's genuinely returning Any.

It looks like there's a better version of the stubs available at https://github.com/pandas-dev/pandas-stubs (and is meant to replace the existing ones), but that hasn't been released yet.

Thanks--I tried, to clone the repo and pip install it that way (i.e v1.42), but same result, unfortunately.

Guess I'll wait for the official release.

[hauntsaninja]

Here, I opened pandas-dev/pandas-stubs#33 to fix read_excel. Similar changes need to be made for read_sql to work. I guess those stubs aren't quite ready for prime time :-)