New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helper library of useful de/serialize_with functions #553

Open
dtolnay opened this Issue Sep 23, 2016 · 25 comments

Comments

6 participants
@dtolnay
Member

dtolnay commented Sep 23, 2016

For example the ones from #550 and serde-rs/serde-rs.github.io#22. These don't need to be in serde itself but if there are ones that people ask for over and over, we can stick those in a helper crate and provide a better out-of-the-box experience compared to "implement your own serialize_with" or "paste this code into your project."

@dtolnay dtolnay added the discussion label Sep 23, 2016

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Sep 28, 2016

Member

Another one: include struct name when serializing struct #554.

Member

dtolnay commented Sep 28, 2016

Another one: include struct name when serializing struct #554.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Oct 13, 2016

Member

Another deserialize_with: a generic comma-separated list of FromStr values #581.

Member

dtolnay commented Oct 13, 2016

Another deserialize_with: a generic comma-separated list of FromStr values #581.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Dec 31, 2016

Member

#661 - as_base64 for AsRef<[u8]> and from_base64 for From<&[u8]>

Member

dtolnay commented Dec 31, 2016

#661 - as_base64 for AsRef<[u8]> and from_base64 for From<&[u8]>

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Apr 26, 2017

Member

Add a way for deserializing numbers from strings containing numbers (which is a feature that serde 0.9 had implicitly)

Member

oli-obk commented Apr 26, 2017

Add a way for deserializing numbers from strings containing numbers (which is a feature that serde 0.9 had implicitly)

@albel727

This comment has been minimized.

Show comment
Hide comment
@albel727

albel727 Apr 26, 2017

I suggest providing helper functions that would work for anything implementing TryFrom or FromStr traits.

albel727 commented Apr 26, 2017

I suggest providing helper functions that would work for anything implementing TryFrom or FromStr traits.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay May 7, 2017

Member

Deserializing map and set types while treating duplicate entries as error, and treating duplicate entries as first-one-wins (rather than the current default of last-one-wins). #916

Member

dtolnay commented May 7, 2017

Deserializing map and set types while treating duplicate entries as error, and treating duplicate entries as first-one-wins (rather than the current default of last-one-wins). #916

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg May 7, 2017

Deserializing map and set types while treating duplicate entries as error, and treating duplicate entries as first-one-wins (rather than the current default of last-one-wins). #916

How can you treat duplicate entries as both errors and first-one-wins? 🤔

alexreg commented May 7, 2017

Deserializing map and set types while treating duplicate entries as error, and treating duplicate entries as first-one-wins (rather than the current default of last-one-wins). #916

How can you treat duplicate entries as both errors and first-one-wins? 🤔

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay May 7, 2017

Member

Two functions.

Member

dtolnay commented May 7, 2017

Two functions.

@clarcharr

This comment has been minimized.

Show comment
Hide comment
@clarcharr

clarcharr May 7, 2017

Contributor

It'd be nice if we could have the same level of configuration available for structs, where instead of erroring on duplicates, we could customise them as first-one-wins or last-one-wins.

Perhaps it might be nice to offer merging them together as well if they're of a map-like or list-like type.

Contributor

clarcharr commented May 7, 2017

It'd be nice if we could have the same level of configuration available for structs, where instead of erroring on duplicates, we could customise them as first-one-wins or last-one-wins.

Perhaps it might be nice to offer merging them together as well if they're of a map-like or list-like type.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay May 7, 2017

Member

Have you ever wanted that in a project? Can you explain the use case in more detail?

Member

dtolnay commented May 7, 2017

Have you ever wanted that in a project? Can you explain the use case in more detail?

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Jun 1, 2017

Member

Serialize using Display and deserialize using FromStr, as in serde-rs/json#329.

Member

dtolnay commented Jun 1, 2017

Serialize using Display and deserialize using FromStr, as in serde-rs/json#329.

@spease

This comment has been minimized.

Show comment
Hide comment
@spease

spease Sep 2, 2017

I'm not sure if it would make sense to use Display for serialization, since Display is "for user-facing output" and so may drop data or not be interchangeable with FromStr.

std::string::ToString might be more appropriate for serialization, but I don't know if this is how it's intended to be used either.

spease commented Sep 2, 2017

I'm not sure if it would make sense to use Display for serialization, since Display is "for user-facing output" and so may drop data or not be interchangeable with FromStr.

std::string::ToString might be more appropriate for serialization, but I don't know if this is how it's intended to be used either.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Sep 22, 2017

Member

#994 -- represent a field as nested JSON string.

{
    "sms":"{\"Source\":4477665544,\"Destination\":1231231}",
    "uuid":"69e123f4-4ced-4f8f-9853-df20ebc3937b"
}
Member

dtolnay commented Sep 22, 2017

#994 -- represent a field as nested JSON string.

{
    "sms":"{\"Source\":4477665544,\"Destination\":1231231}",
    "uuid":"69e123f4-4ced-4f8f-9853-df20ebc3937b"
}
@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Nov 5, 2017

Member

#1042 -- a helper for the "double option" pattern for distinguishing null vs absent value.

Member

dtolnay commented Nov 5, 2017

#1042 -- a helper for the "double option" pattern for distinguishing null vs absent value.

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Feb 6, 2018

Member

Stringification of a field (type is String, deserializes from numbers and strings, or even more?).

Example implementation: https://play.rust-lang.org/?gist=f0e00482fa5bb52c7c9e3db213d271db&version=stable

Member

oli-obk commented Feb 6, 2018

Stringification of a field (type is String, deserializes from numbers and strings, or even more?).

Example implementation: https://play.rust-lang.org/?gist=f0e00482fa5bb52c7c9e3db213d271db&version=stable

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Feb 22, 2018

Member

Note that there now is https://github.com/vityafx/serde-aux which has a few of these functions.

Member

oli-obk commented Feb 22, 2018

Note that there now is https://github.com/vityafx/serde-aux which has a few of these functions.

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 22, 2018

Oh, I was going to start a library of my own along these lines, but it seems someone beat me to it. Maybe we can build on that...

alexreg commented Feb 22, 2018

Oh, I was going to start a library of my own along these lines, but it seems someone beat me to it. Maybe we can build on that...

@spease

This comment has been minimized.

Show comment
Hide comment
@spease

spease Feb 26, 2018

Yeah, I was thinking of doing one myself if I got around to it, but I'd rather see there be one library. It's a pain to remember too many library names (and versions if you aren't using cargo-edit)

spease commented Feb 26, 2018

Yeah, I was thinking of doing one myself if I got around to it, but I'd rather see there be one library. It's a pain to remember too many library names (and versions if you aren't using cargo-edit)

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 26, 2018

@spease Probably it will officially come under serde control/ownership once it matures, but we could just submit PRs to this serde-aux project for now. (serde-helpers or serde-util is a better name in my mind.)

alexreg commented Feb 26, 2018

@spease Probably it will officially come under serde control/ownership once it matures, but we could just submit PRs to this serde-aux project for now. (serde-helpers or serde-util is a better name in my mind.)

@spease

This comment has been minimized.

Show comment
Hide comment
@spease

spease Feb 27, 2018

Here's another misc function. The use case for this is to get enum variant names where I needed to do #[serde(rename_all)] to match an API's case, and serialize them with a pipe (|) in between them.

use serde::ser::{self, Serialize, Serializer, SerializeStructVariant, SerializeTupleVariant, Impossible};
use std;

// Many thanks to dtolnay
pub fn variant_name<T: Serialize>(t: &T) -> &'static str {
    #[derive(Debug)]
    struct NotEnum;
    type Result<T> = std::result::Result<T, NotEnum>;
    impl std::error::Error for NotEnum {
        fn description(&self) -> &str { "not struct" }
    }
    impl std::fmt::Display for NotEnum {
        fn fmt(&self, _f: &mut std::fmt::Formatter) -> std::fmt::Result { unimplemented!() }
    }
    impl ser::Error for NotEnum {
        fn custom<T: std::fmt::Display>(_msg: T) -> Self { NotEnum }
    }

    struct VariantName;
    impl Serializer for VariantName {
        type Ok = &'static str;
        type Error = NotEnum;
        type SerializeSeq = Impossible<Self::Ok, Self::Error>;
        type SerializeTuple = Impossible<Self::Ok, Self::Error>;
        type SerializeTupleStruct = Impossible<Self::Ok, Self::Error>;
        type SerializeTupleVariant = Enum;
        type SerializeMap = Impossible<Self::Ok, Self::Error>;
        type SerializeStruct = Impossible<Self::Ok, Self::Error>;
        type SerializeStructVariant = Enum;
        fn serialize_bool(self, _v: bool) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i8(self, _v: i8) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i16(self, _v: i16) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i32(self, _v: i32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i64(self, _v: i64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u8(self, _v: u8) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u16(self, _v: u16) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u32(self, _v: u32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u64(self, _v: u64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_f32(self, _v: f32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_f64(self, _v: f64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_char(self, _v: char) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_str(self, _v: &str) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_bytes(self, _v: &[u8]) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_none(self) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_some<T: ?Sized + Serialize>(self, _value: &T) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit(self) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit_struct(self, _name: &'static str) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str) -> Result<Self::Ok> { Ok(variant) }
        fn serialize_newtype_struct<T: ?Sized + Serialize>(self, _name: &'static str, _value: &T) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_newtype_variant<T: ?Sized + Serialize>(self, _name: &'static str, _variant_index: u32, variant: &'static str, _value: &T) -> Result<Self::Ok> { Ok(variant) }
        fn serialize_seq(self, _len: Option<usize>) -> Result<Self::SerializeSeq> { Err(NotEnum) }
        fn serialize_tuple(self, _len: usize) -> Result<Self::SerializeTuple> { Err(NotEnum) }
        fn serialize_tuple_struct(self, _name: &'static str, _len: usize) -> Result<Self::SerializeTupleStruct> { Err(NotEnum) }
        fn serialize_tuple_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str, _len: usize) -> Result<Self::SerializeTupleVariant> { Ok(Enum(variant)) }
        fn serialize_map(self, _len: Option<usize>) -> Result<Self::SerializeMap> { Err(NotEnum) }
        fn serialize_struct(self, _name: &'static str, _len: usize) -> Result<Self::SerializeStruct> { Err(NotEnum) }
        fn serialize_struct_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str, _len: usize) -> Result<Self::SerializeStructVariant> { Ok(Enum(variant)) }
    }

    struct Enum(&'static str);
    impl SerializeStructVariant for Enum {
        type Ok = &'static str;
        type Error = NotEnum;
        fn serialize_field<T: ?Sized + Serialize>(&mut self, _key: &'static str, _value: &T) -> Result<()> { Ok(()) }
        fn end(self) -> Result<Self::Ok> {
            Ok(self.0)
        }
    }
    impl SerializeTupleVariant for Enum {
        type Ok = &'static str;
        type Error = NotEnum;
        fn serialize_field<T: ?Sized + Serialize>(&mut self, _value: &T) -> Result<()> { Ok(()) }
        fn end(self) -> Result<Self::Ok> {
            Ok(self.0)
        }
    }

    t.serialize(VariantName).unwrap()
}

spease commented Feb 27, 2018

Here's another misc function. The use case for this is to get enum variant names where I needed to do #[serde(rename_all)] to match an API's case, and serialize them with a pipe (|) in between them.

use serde::ser::{self, Serialize, Serializer, SerializeStructVariant, SerializeTupleVariant, Impossible};
use std;

// Many thanks to dtolnay
pub fn variant_name<T: Serialize>(t: &T) -> &'static str {
    #[derive(Debug)]
    struct NotEnum;
    type Result<T> = std::result::Result<T, NotEnum>;
    impl std::error::Error for NotEnum {
        fn description(&self) -> &str { "not struct" }
    }
    impl std::fmt::Display for NotEnum {
        fn fmt(&self, _f: &mut std::fmt::Formatter) -> std::fmt::Result { unimplemented!() }
    }
    impl ser::Error for NotEnum {
        fn custom<T: std::fmt::Display>(_msg: T) -> Self { NotEnum }
    }

    struct VariantName;
    impl Serializer for VariantName {
        type Ok = &'static str;
        type Error = NotEnum;
        type SerializeSeq = Impossible<Self::Ok, Self::Error>;
        type SerializeTuple = Impossible<Self::Ok, Self::Error>;
        type SerializeTupleStruct = Impossible<Self::Ok, Self::Error>;
        type SerializeTupleVariant = Enum;
        type SerializeMap = Impossible<Self::Ok, Self::Error>;
        type SerializeStruct = Impossible<Self::Ok, Self::Error>;
        type SerializeStructVariant = Enum;
        fn serialize_bool(self, _v: bool) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i8(self, _v: i8) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i16(self, _v: i16) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i32(self, _v: i32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_i64(self, _v: i64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u8(self, _v: u8) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u16(self, _v: u16) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u32(self, _v: u32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_u64(self, _v: u64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_f32(self, _v: f32) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_f64(self, _v: f64) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_char(self, _v: char) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_str(self, _v: &str) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_bytes(self, _v: &[u8]) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_none(self) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_some<T: ?Sized + Serialize>(self, _value: &T) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit(self) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit_struct(self, _name: &'static str) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_unit_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str) -> Result<Self::Ok> { Ok(variant) }
        fn serialize_newtype_struct<T: ?Sized + Serialize>(self, _name: &'static str, _value: &T) -> Result<Self::Ok> { Err(NotEnum) }
        fn serialize_newtype_variant<T: ?Sized + Serialize>(self, _name: &'static str, _variant_index: u32, variant: &'static str, _value: &T) -> Result<Self::Ok> { Ok(variant) }
        fn serialize_seq(self, _len: Option<usize>) -> Result<Self::SerializeSeq> { Err(NotEnum) }
        fn serialize_tuple(self, _len: usize) -> Result<Self::SerializeTuple> { Err(NotEnum) }
        fn serialize_tuple_struct(self, _name: &'static str, _len: usize) -> Result<Self::SerializeTupleStruct> { Err(NotEnum) }
        fn serialize_tuple_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str, _len: usize) -> Result<Self::SerializeTupleVariant> { Ok(Enum(variant)) }
        fn serialize_map(self, _len: Option<usize>) -> Result<Self::SerializeMap> { Err(NotEnum) }
        fn serialize_struct(self, _name: &'static str, _len: usize) -> Result<Self::SerializeStruct> { Err(NotEnum) }
        fn serialize_struct_variant(self, _name: &'static str, _variant_index: u32, variant: &'static str, _len: usize) -> Result<Self::SerializeStructVariant> { Ok(Enum(variant)) }
    }

    struct Enum(&'static str);
    impl SerializeStructVariant for Enum {
        type Ok = &'static str;
        type Error = NotEnum;
        fn serialize_field<T: ?Sized + Serialize>(&mut self, _key: &'static str, _value: &T) -> Result<()> { Ok(()) }
        fn end(self) -> Result<Self::Ok> {
            Ok(self.0)
        }
    }
    impl SerializeTupleVariant for Enum {
        type Ok = &'static str;
        type Error = NotEnum;
        fn serialize_field<T: ?Sized + Serialize>(&mut self, _value: &T) -> Result<()> { Ok(()) }
        fn end(self) -> Result<Self::Ok> {
            Ok(self.0)
        }
    }

    t.serialize(VariantName).unwrap()
}
@clarcharr

This comment has been minimized.

Show comment
Hide comment
@clarcharr

clarcharr Feb 28, 2018

Contributor

One thing that I'd suggest now that people are actually providing crates for this is to put the functions in a workspace similar to that for the unic crate: while one master serde-aux crate contains everything, smaller serde-aux-* crates contain self-contained bits of functionality.

Ideally, each sub-crate would be a piece that someone would use in one use case, so that people don't just import one giant crate from which most of the stuff in there is unused.

Contributor

clarcharr commented Feb 28, 2018

One thing that I'd suggest now that people are actually providing crates for this is to put the functions in a workspace similar to that for the unic crate: while one master serde-aux crate contains everything, smaller serde-aux-* crates contain self-contained bits of functionality.

Ideally, each sub-crate would be a piece that someone would use in one use case, so that people don't just import one giant crate from which most of the stuff in there is unused.

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 28, 2018

I'm not sure if there's any obvious logical grouping though...

alexreg commented Feb 28, 2018

I'm not sure if there's any obvious logical grouping though...

@spease

This comment has been minimized.

Show comment
Hide comment
@spease

spease Feb 28, 2018

Yeah, and it risks creating a fragmented ecosystem. It already feels like I end up adding the same ~10 crates every time I start a new rust crate. I don't want to have to look up which aux crate something belongs to, or have to search multiple crates for a helper function.

spease commented Feb 28, 2018

Yeah, and it risks creating a fragmented ecosystem. It already feels like I end up adding the same ~10 crates every time I start a new rust crate. I don't want to have to look up which aux crate something belongs to, or have to search multiple crates for a helper function.

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 28, 2018

@spease Right. Let's stick to a serde-aux / serde-util crate, for these reasons, I think.

alexreg commented Feb 28, 2018

@spease Right. Let's stick to a serde-aux / serde-util crate, for these reasons, I think.

@spease

This comment has been minimized.

Show comment
Hide comment
@spease

spease Apr 27, 2018

Came here to note that I find myself dealing with #908 for multiple classes, and that should probably go in serde-aux. Looks like it's already been noted.

spease commented Apr 27, 2018

Came here to note that I find myself dealing with #908 for multiple classes, and that should probably go in serde-aux. Looks like it's already been noted.

jonasbb added a commit to jonasbb/serde_with that referenced this issue Jul 10, 2018

Add deserialization helpers which error on duplicate keys/values for …
…maps/sets

Inspired by a comment on serde's helper library issue:
serde-rs/serde#553 (comment)

This behaviour can be used to detect error in the serialized data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment