Can't use mypy because of missing `py.typed` file in the distribution

[ycrouin]

Describe the bug
This SageMaker sdk package doesn't ship with a py.typed file in the distribution.
Because of this, when we install this package, mypy can't use the existing type annotations.
It is quite straightforward to make a package PEP-561 compatible. Mypy documents it here: https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages

To reproduce

pip install sagemaker
touch sad_mypy.py
pip install mypy
mypy sad_mypy.py

Expected behavior
Mypy should be able to use the type annotations from the package.

Screenshots or logs

[YHallouard]

I faced the same problem and no stubs are provided :)

[eugeneyarovoi]

Hi, wanted to bump this issue, because I think it could be a pretty easy fix. The codebase is already type-annotated, so if you just add the py.typed file to the distribution (this is a blank marker file), the library will be able to be typechecked by mypy right away.

Of course, maybe if you were to turn this on, you would want to run a full check of the codebase to ensure that mypy agrees with all your types, but I don't think this is strictly required. There may be a few places where existing type annotations are actually incorrect and need to be fixed (you might see people reporting it then, which isn't necessarily a bad thing!)

There are a few places in the codebase where mypy is a little picky: some functions accept e.g. List[Union[Step, StepCollection]], into which users might pass a List[Step] which mypy sees to be incompatible, because List is invariant. (This is actually technically correct - you could append a StepCollection to a List[Union[Step, StepCollection]], but not to a List[Step]). In some cases like that, ideally the type annotations would be changed to Sequence[Union[Step, StepCollection]] to fix those typechecker issues (Sequence is a read-only abstract list type, so it is covariant).

But even without fixing some of these issues, I tend to think it would do more good than harm to allow typecheckers to check user calls against the types declared by this package. The user can # ignore the few warnings that are incorrect.

I myself added a script to my repo to just always place the py.typed file into the installation location of the sagemaker package as a precommit hook. That can be a solution as well for users who want this functionality, but it is not a neat solution.

@ycrouin @YHallouard You probably already know this, but there are mypy settings that can be used to suppress the warnings that happen because such-and-such module doesn't have typehints. Sure, you will get things that are of type Any in those places, but at least you'll be able to run mypy! Ignoring a particular module's types for import purposes can be configured in your mypy config file, see the mypy docs.

[Ce11an]

Has anyone attempted to write their own stubs for SageMaker? Looking for a solution until this issue is resolved. Thanks 🙏🏼

[eugeneyarovoi]

@Ce11an I'm not from the Sagemaker team, but your best solution will probably be either:

1. Place py.typed into your sagemaker package installation as a precommit hook; or
2. Configure your mypy config file to ignore that this module isn't typed. All types coming from the module will be Any.

[YHallouard]

We chose the option 2 for our project but this is not very cool.

May we could create a type package, but it would be easier to be maintained by aws sagemaker team. Mypy provide an automated way to create py.typed

[eugeneyarovoi]

py.typed is just a blank file if I recall correctly.

[Ce11an]

We are also using option 2. The pre-commit solution is a good idea, but I am looking for something cleaner. I am going to try using stubgen in the meantime. This seems like an easy issue to solve for the SDK...

[fulcus]

Hi team, any update on this?

[ParagEkbote]

This issue was resolved in #5027. It should be closed to prevent duplicate PRs.