Use explicit optional instead of implicit optional in whole project

[MiguelX413]

Checklist for PRs

• Added .. versionadded:: version, .. versionchanged:: version or .. deprecated:: version to the docstrings for user facing changes (for methods/class descriptions, arguments and attributes)
• Created new or adapted existing unit tests
• Documented code changes according to the CSI standard
• Added myself alphabetically to AUTHORS.rst (optional)
• Added new classes & modules to the docs and all suitable __all__ s

If the PR contains API changes (otherwise, you can delete this passage)

• New classes:

  • Added self._id_attrs and corresponding documentation
  • __init__ accepts api_kwargs as kw-only

• Added new shortcuts:

  • In Chat & User for all methods that accept chat/user_id
  • In Message for all methods that accept chat_id and message_id
  • For new Message shortcuts: Added quote argument if methods accepts reply_to_message_id
  • In CallbackQuery for all methods that accept either chat_id and message_id or inline_message_id

• If relevant:

  • Added new constants at telegram.constants and shortcuts to them as class variables
  • Link new and existing constants in docstrings instead of hard coded number and strings
  • Add new message types to Message.effective_attachment
  • Added new handlers for new update types
    • Add the handlers to the warning loop in the ConversationHandler
  • Added new filters for new message (sub)types
  • Added or updated documentation for the changed class(es) and/or method(s)
  • Added the new method(s) to _extbot.py
  • Added or updated bot_methods.rst
  • Updated the Bot API version number in all places: README.rst and README_RAW.rst (including the badge), as well as telegram.constants.BOT_API_VERSION_INFO
  • Added logic for arbitrary callback data in tg.ext.Bot for new methods that either accept a reply_markup in some form or have a return type that is/contains telegram.Message

This was causing mypy to have problems verifying the integrity of my own codebase which depended on this project.

[Bibo-Joshi]

Hi. Thanks for the PR. Does the current implicit typing have any negative effect on your usage of the python-telegram-bot library? If it does not, then I would like to reject this PR for the reasons discussed in #3367.

Edit:
Ah, I didnt see your comment at the very end. Can you give an example of how exactly the current status affects the type checking of your code base?

[MiguelX413]

Hi. Thanks for the PR. Does the current implicit typing have any negative effect on your usage of the python-telegram-bot library? If it does not, then I would like to reject this PR for the reasons discussed in #3367.

Edit: Ah, I didnt see your comment at the very end. Can you give an example of how exactly the current status affects the type checking of your code base?

@Bibo-Joshi Mypy says the following about https://github.com/MiguelX413/IgTgBot/blob/4586c01fb89283eada0b2c1567e2ee67a02d1bd2/formatted_text.py#L35-L81 only without this PR:

formatted_text.py:53: error: Argument "url" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]
formatted_text.py:54: error: Argument "user" to "MessageEntity" has incompatible type "Optional[User]"; expected "User"  [arg-type]
formatted_text.py:55: error: Argument "language" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]
formatted_text.py:56: error: Argument "custom_emoji_id" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]
formatted_text.py:75: error: Argument "url" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]
formatted_text.py:76: error: Argument "user" to "MessageEntity" has incompatible type "Optional[User]"; expected "User"  [arg-type]
formatted_text.py:77: error: Argument "language" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]
formatted_text.py:78: error: Argument "custom_emoji_id" to "MessageEntity" has incompatible type "Optional[str]"; expected "str"  [arg-type]

[Bibo-Joshi]

I see. Since there is a user impact then, we should indeed completely drop the implicit optionals in the whole library. For this, the mypy config file needs to be updated and all the function/method signatures need to be updated. I would guess that some regex-replacing + mypy checking will make this a rather straightforward task. @MiguelX413 would you maybe like to extend your PR?

[MiguelX413]

I see. Since there is a user impact then, we should indeed completely drop the implicit optionals in the whole library. For this, the mypy config file needs to be updated and all the function/method signatures need to be updated. I would guess that some regex-replacing + mypy checking will make this a rather straightforward task. @MiguelX413 would you maybe like to extend your PR?

Sure! @Bibo-Joshi

[Bibo-Joshi]

Thanks for the updates! A number of tests fail because the type hints of shortcut messages don't match the type hint in the Bot class (e.g. Message.reply_text doesn't match Bot.send_message). IISC that's due to some classes being importing only if TYPE_CHECKING and the tests can't resolve the forward references. If you need help with addressing this, please do reach out.

On a side note: please avoid force-pushing (at least once you got review comments), since that makes it hard to keep track of what one has already reviewed :)

[MiguelX413]

Thanks for the updates! A number of tests fail because the type hints of shortcut messages don't match the type hint in the Bot class (e.g. Message.reply_text doesn't match Bot.send_message). IISC that's due to some classes being importing only if TYPE_CHECKING and the tests can't resolve the forward references. If you need help with addressing this, please do reach out.

On a side note: please avoid force-pushing (at least once you got review comments), since that makes it hard to keep track of what one has already reviewed :)

Noted!

[MiguelX413]

Thanks for the updates! A number of tests fail because the type hints of shortcut messages don't match the type hint in the Bot class (e.g. Message.reply_text doesn't match Bot.send_message). IISC that's due to some classes being importing only if TYPE_CHECKING and the tests can't resolve the forward references. If you need help with addressing this, please do reach out.

On a side note: please avoid force-pushing (at least once you got review comments), since that makes it hard to keep track of what one has already reviewed :)

I'm not really sure where to start with the testing system tbh @Bibo-Joshi

[lemontree210]

I'm not really sure where to start with the testing system tbh @Bibo-Joshi

I tried to fix the testing issues. Also, I merged master with some conflict resolution (and fixed implicit optionals in new args of existing methods), but some new methods that were added to our master and were merged without conflicts contain the implicit optionals and you might want to run your script again :)

If the tests still fail after that, we'll look into this again.

[lemontree210]

I merged master with some conflict resolution (and fixed implicit optionals in new args of existing methods), but some new methods that were added to our master and were merged without conflicts contain the implicit optionals

UPD: I just did some bulk replacing in my IDE, mypy is happy now.

[Bibo-Joshi]

LGTM 👍 @harshil21 do you have anything to add?

[harshil21]

@Bibo-Joshi nope LGTM ✅

[Bibo-Joshi]

Thank you very much for your contribution @MiguelX413 !