Using #[serde_as] with custom (de-)serialize functions

[Lehona]

I have implemented custom (de-)serialize functions for a remote type, e.g. Py<T> (doesn't really matter what exactly it is). With serde, I can simply use these custom functions via either #[serde(with = "custom_serde")] (where custom_serde is a module containing the (de-)serialize function) or #[serde(serialize_with = "custom_serde::serialize")] and #[serde(deserialize_with = "custom_serde::deserialize")].

However, I also have structs that contain e.g. Vec<Py<Inner>>, which cannot be (de-)serialized with this approach. To my understanding, this is exactly the conundrum that serde_with is supposed to solve. From my research, however, I only found ways to specify a type to serialize with, i.e., I would need to implement the (De-)Serialize traits for Py<T> directly rather than forward to a custom implementation via annotation.

Are free-standing (de-)serializers not yet supported or am I missing something? In any case I think the documentation could use some work here - #[serde_as] is meant to replace #[serde(with = "...")] yet doesn't seem to support its basic features such as declaring a module which contains (de-)serialize functions.

I don't think I can implement (De-)Serialize for Py<T> due to the orphan rule and the workaround that serde supports would rely on the internal representation of Py<T>, which I don't care for and thus would rather not depend on. Please feel free to correct me if there's an easy way I'm missing.

[jonasbb]

Hi @Lehona,

I agree that documentation can be improved by a lot. So far this crate doesn't have many users, so I don't know where the hurdles are which need documenting. I don't know if the usage can ever be quite as simple as writing a module with a serialize and a deserialize function, since serde_as works by building a specific type "tree" and uses that to guide the de-/serialization. I see how I can document the interaction better though.

Coming back to your question though. I think there is no problem in supporting a foreign type, only a bit of boilerplate. I just focus on the SerializeAs trait for now, but the same applies to DeserializeAs. The trait works in reverse, which allows you to use it with any type.

You write an implementation like this for LocalType. LocalType can be a unit-like type in your crate.

impl SerializeAs<RemoteType> for LocalType {/*...*/}

Then you can use it like:

#[serde_as]
#[derive(...)]
struct Data {
	#[serde_as(as = "Vec<LocalType>")]
	vec: Vec<RemoteType>,
}

In your case RemoteType could be Py<T> or Py<Inner>. You can also have multiple implementations, for example SerializeAs<Py<Inner>> and SerializeAs<Py<Other>>, as long as they are not generic.

The SerializeAs implementation now only needs to call the functions for the existing module which you use in #[serde(with = "...")].
For example here is the implementation for DisplayFromStr:

serde_with/src/ser/impls.rs

Lines 88 to 98 in 2651ea1

	impl<T> SerializeAs<T> for DisplayFromStr
	where
	T: Display,
	{
	fn serialize_as<S>(source: &T, serializer: S) -> Result<S::Ok, S::Error>
	where
	S: Serializer,
	{
	crate::rust::display_fromstr::serialize(source, serializer)
	}
	}

I hope this helps you. Feel free to keep asking questions if you have trouble implementing it :)

[Lehona]

Implementing SerializeAs was straightforward after your comment. I encountered some problems with DeserializeAs because I relied on the DeserializeOwned trait from Serde, which is no longer tied to a lifetime. At some point the lifetime errors just vanished, though, and I can now (de-)serialize

#[pyclass]
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct Container {
    #[pyo3(get, set)]
    #[serde_as(as = "Vec<LocalPy>")]
    pub inner: Vec<Py<Inner>>
}

Thank you very much! I suggest explicitly mentioning in your docs that serde_with will only work with types and that any module or function-based (de-)serialization will require an implementation of the *As-Traits on e.g. a unit struct.
I think I might've figured it out myself in that case, but I kept searching for a way to directly use my functions/module, to mirror serde's with.

[jonasbb]

Glad to hear that it works :) I leave this open for now and try to document how to use serde_as with remote types / with existing with-like modules.