click.prompt typing clarifications / improvements

[AndreasBackx]

This improves the typing of click.prompt. However there is still an opening question / issue...

Brief summary

The current code makes the following change to the public API of click.prompt:

 def prompt(
     text: str,
-    default: t.Any | None = None,
+    default: V | None = None,
     hide_input: bool = False,
     confirmation_prompt: bool | str = False,
-    type: ParamType[t.Any] | t.Any | None = None,
-    value_proc: t.Callable[[str], t.Any] | None = None,
+    type: ParamType[V] | V | None = None,
+    value_proc: t.Callable[[str], V] | None = None,
     prompt_suffix: str = ": ",
     show_default: bool | str = True,
     err: bool = False,
     show_choices: bool = True,
-) -> t.Any:
+) -> V:

This changes the behaviour in that you can no longer pass types that aren't the same type as the type you expect at the end. For example:

click.prompt("Iteration amount?", default="100", type=int)

Would no longer work under the current implementation because "100" is not of the type int. In the old implementation this worked because default values still were passed as int("100") before being returned. The remaining question is then what behaviour we should go for.

You could say, "why" not keep the current implementation? Well we could, but it's not actually typed correctly and actually allows behaviour we don't account for. That behaviour is that value_proc: t.Callable[[str], V] | None = None accepts only str to convert to the target type. So technically you shouldn't be allowed to pass Any to it. However.... ParamType.__call__, which makes ParamType also a callable, is used to convert the default to the target type as well and that accepts Any, not str. So there's not really a strict contract anywhere.

To make the decision on this PR easier I propose the following options:

1. Restrict default to same type (current implementation)

• Accept only the same type as the type passed in type in default.
• DO NOT pass default to value_proc or the constructor of type.

 def prompt(
     text: str,
-    default: t.Any | None = None,
+    default: V | None = None,
     hide_input: bool = False,
     confirmation_prompt: bool | str = False,
-    type: ParamType[t.Any] | t.Any | None = None,
-    value_proc: t.Callable[[str], t.Any] | None = None,
+    type: ParamType[V] | V | None = None,
+    value_proc: t.Callable[[str], V] | None = None,
     prompt_suffix: str = ": ",
     show_default: bool | str = True,
     err: bool = False,
     show_choices: bool = True,
-) -> t.Any:
+) -> V:

2. Public API typing change (or correction) with unlikely users

• Accept the same type as the type passed in type in default and also str.
• If default type is different from the type of type, pass it to value_proc or the constructor of type to convert it.
  • Make ParamType.__call__ also onyl accept str, not Any.

 def prompt(
     text: str,
-    default: t.Any | None = None,
+    default: V | str | None = None,
     hide_input: bool = False,
     confirmation_prompt: bool | str = False,
-    type: ParamType[t.Any] | t.Any | None = None,
-    value_proc: t.Callable[[str], t.Any] | None = None,
+    type: ParamType[V] | V | None = None,
+    value_proc: t.Callable[[str], V] | None = None,
     prompt_suffix: str = ": ",
     show_default: bool | str = True,
     err: bool = False,
     show_choices: bool = True,
-) -> t.Any:
+) -> V:

    @t.overload
    def __call__(
        self,
        value: None,
        param: Parameter | None = None,
        ctx: Context | None = None,
    ) -> None: ...

    @t.overload
    def __call__(
        self,
-       value: t.Any,
+       value: str,
        param: Parameter | None = None,
        ctx: Context | None = None,
    ) -> ParamTypeValue: ...

    def __call__(
        self,
-       value: t.Any,
+       value: str,
        param: Parameter | None = None,
        ctx: Context | None = None,
    ) -> ParamTypeValue | None:
        if value is not None:
            return self.convert(value, param, ctx)
        return None

    def convert(
        self,
-       value: t.Any,
+       value: str,
		param: Parameter | None,
		ctx: Context | None,
    ) -> ParamTypeValue:

3. Change value_proc to accept Any

• Accept Any in default, keeping the same accepted type as before.
• If default type is different from the type of type, pass it to value_proc or the constructor of type to convert it.

 def prompt(
     text: str,
-    default: t.Any | None = None,
+    default: V | t.Any | None = None,
     hide_input: bool = False,
     confirmation_prompt: bool | str = False,
-    type: ParamType[t.Any] | t.Any | None = None,
-    value_proc: t.Callable[[str], t.Any] | None = None,
+    type: ParamType[V] | V | None = None,
+    value_proc: t.Callable[[t.Any], V] | None = None,
     prompt_suffix: str = ": ",
     show_default: bool | str = True,
     err: bool = False,
     show_choices: bool = True,
-) -> t.Any:
+) -> V:

My preference?

Even though I've implemented the first option, I think the second option makes the most sense. I don't think there's a usecase for accepting beyond str for parsing.

[kdeldycke]

The question isn't about the ideal type, it's what the other developers already rely on. We're in Hyrum's Law territory here: Click is too entranched in the ecosystem that we have users depending on behaviors we never strictly specified. Let's pin the current runtime behavior with tests before tightening anything.

Does option 3 breaks the actual unittests? Can you come up with tests that encode the actually behaviour we allow but we don't account for?

[AndreasBackx]

@kdeldycke I've made changes to the typing, though it has lead to the introduction of a new generic type being added to ParamType, one that identifies that input value type. This should be fine and should make typing work throughout.

However, to make it easier for people and not require custom ParamType instances (and the rest of our codebase atm) to use it yet, I've defaulted it to t.Any. That default however requires Python 3.13, or for us to start importing typing_extensions for 3.12<=, which is currently not being done as it's only used for typing. Is it okay to add typing_extensions here as an import?

[kdeldycke]

@kdeldycke I've made changes to the typing, though it has lead to the introduction of a new generic type being added to ParamType, one that identifies that input value type. This should be fine and should make typing work throughout.

Yeah that looks good! And indeed: it passes on Python 3.13 without issues.

Is it okay to add typing_extensions here as an import?

If it would be my project I would say yes. But not sure what's our policy for Click. So thanks for asking! @davidism, @Rowlando13, @ThiefMaster, any opinion on adding typing_extensions dependency?

In the meantime @AndreasBackx add that dependency to this PR so you can demonstrate that it make your changes pass on all Pythons supported by Click. Could also help other maintainers form an opinion by reading the actual impact of your changes.