Add #[pyo3(from_python_with = ...)] attribute

[davidhewitt]

I've been thinking for a while it would be interesting to have an attribute which can be used to customize the from-python conversion in our proc macros. This need has come up in Gitter as well as #884.

The attribute will go on #[pyfunction] arguments or #[derive(FromPyObject)] fields. It takes a value which is a path to a function fn (&PyAny) -> Result<T, E>, where T is the annotated type and E: Into<PyErr>. For examples see the detailed design below.

This issue is invite users to comment on the design and for anyone interested to volunteer to implement it (before I eventually get around to it).

In detail, the two applications I see for this attribute are:

In #[pyfunction] arguments

Consider the following pyfunction:

#[pyfunction]
fn foo(datetime: chrono::DateTime) { ... }

Currently chrono::DateTime does not implement FromPyObject, so this function won't compile. An existing workaround is to create a newtype MyDateTime(chrono::DateTime) and implement FromPyObject for that, but it's a lot of boilerplate.

The attribute would allow the user to leverage a single function without adding extra types:

fn extract_datetime(any: &PyAny) -> PyResult<chrono::DateTime> { ... }

#[pyfunction]
fn foo(
    #[pyo3(from_python_with = extract_datetime)]
    datetime: chrono::DateTime
) { ... }

On #[derive(FromPyObject)] fields

Similarly, this applies in a #[derive(FromPyObject)] struct:

#[derive(FromPyObject)]
#[pyo3(transparent)]
struct Foo {
    #[pyo3(from_python_with = extract_datetime)]
    datetime: chrono::DateTime
}

The cool thing here is that this lets us add #[derive(FromPyObject)] to a struct which has a field which itself doesn't implement FromPyObject.

Bikeshedding

• should it be from_py_with instead of from_python_with?
• should the function be allowed to take more specific argument types than &PyAny? E.g. in the examples above it could be helpful to take &PyDateTime. But it's also trivial to convert PyAny with .downcast().

[davidhewitt]

cc @gilescope

[daniil-konovalenko]

Hi, I would like to try to implement this, if it's okay.
I haven't touched procedural macros yet though so I think it may take a while

[gilescope]

Go for it! (quote! macro is usually pretty helpful with proc macros)

…

On Tue, 12 Jan 2021 at 11:03, Daniil Konovalenko ***@***.***> wrote: Hi, I would like to try to implement this, if it's okay. I haven't touched procedural macros yet though so I think it may take a while — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1239 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGEJCCP4K73EY4DOD2FGELSZQUBNANCNFSM4SRUEOGQ> .

[davidhewitt]

Please feel welcome to take it on! I'm happy to answer any questions you have.