Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Flattening #1179

Merged
merged 51 commits into from
Mar 22, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
5a91ac5
Initial work on supporting structs as map with unknown field collection
mitsuhiko Mar 14, 2018
b4dbae2
Added support for serialization of structs as maps
mitsuhiko Mar 14, 2018
39413c8
Implement deserializer for map mode and collection fields
mitsuhiko Mar 14, 2018
1bd2c61
Explicitly pass value requirements for the capture path
mitsuhiko Mar 14, 2018
77b07f3
Added tests for unknown_fields_into
mitsuhiko Mar 14, 2018
07d0734
Make clippy happy
mitsuhiko Mar 14, 2018
583c0d8
Make proc-macro2/nightly happy
mitsuhiko Mar 14, 2018
299cd2d
Replace unknown_fields_into with serde(flatten)
jan-auer Mar 15, 2018
571bb8c
Derive serialization for serde(flatten)
jan-auer Mar 15, 2018
5ae06bb
Store flatten flag in container attributes
jan-auer Mar 15, 2018
6627540
Added support basic deserialization in derive
jan-auer Mar 15, 2018
5457394
Fixed various issues with combinding flatten and deny_unknown_fields
jan-auer Mar 15, 2018
9e8cda4
Added basic not fully working FlatMapSerializer
mitsuhiko Mar 15, 2018
b692923
Non working changes to the flatten serializer
mitsuhiko Mar 15, 2018
112dfd7
Correctly suppress the end() call for flattened serialization
mitsuhiko Mar 15, 2018
ebf80ac
Implement deserialization support for flatten
mitsuhiko Mar 16, 2018
b4ef7ac
Updated tests for flatten
mitsuhiko Mar 16, 2018
d1833c5
Added support for basic enums in flatten
mitsuhiko Mar 16, 2018
a8c8c20
Added support for struct variant enum serialization
mitsuhiko Mar 16, 2018
b8602a7
Added test for tag/content enum flattening
mitsuhiko Mar 16, 2018
49e302d
Improved error message for flattening on unsupported types
mitsuhiko Mar 16, 2018
352fe7b
Removed an unused field that was left over from a merge conflict
mitsuhiko Mar 16, 2018
ca41e16
Added some missing conditionals for feature compilation
mitsuhiko Mar 16, 2018
bfdcbae
Fixed an unused import error
mitsuhiko Mar 16, 2018
2f57cec
format! -> format_args! for an error message
mitsuhiko Mar 16, 2018
ffcde25
Fixed some clippy warnings
mitsuhiko Mar 16, 2018
ebc61ba
Added newtype struct support for flattening
mitsuhiko Mar 18, 2018
f1af2dc
Added support for newtype variant serialization
mitsuhiko Mar 18, 2018
61b167b
Attempted support for in_place deserialization for structs as map
mitsuhiko Mar 18, 2018
d44f129
Do not emit an in-place deserialization path for struct as map
mitsuhiko Mar 18, 2018
58d52e7
Remove #[serde(repr = "map")]
mitsuhiko Mar 18, 2018
ad40f97
Switch to using Content keys internally for flattening to later suppo…
mitsuhiko Mar 18, 2018
205f606
Various clippy fixes
mitsuhiko Mar 18, 2018
c5a3128
Added a more complex flattening test
mitsuhiko Mar 18, 2018
7cf1846
Use more consistent error messages for bad flattening
mitsuhiko Mar 18, 2018
f02dbf3
Added non string key support for flattening
mitsuhiko Mar 18, 2018
7c596c7
Remove unnecessary as_str
mitsuhiko Mar 20, 2018
6e324e8
Some refactoring to use a bit less unwrap()
mitsuhiko Mar 20, 2018
abeea89
Fully qualify some calls in generated code and fix a bad comment
mitsuhiko Mar 20, 2018
8637dda
Refactored a test
mitsuhiko Mar 20, 2018
5b884b5
Added some missing UFCs
mitsuhiko Mar 20, 2018
50c636a
Remove now dead as_map detection (can be cattrs.has_flatten)
mitsuhiko Mar 20, 2018
695c3ee
Do not imply flatten from skip_serialize
mitsuhiko Mar 20, 2018
e4ef087
Added support for borrowing when flattening
mitsuhiko Mar 20, 2018
1d92569
Added explanatory comment about fetching data from buffered content
mitsuhiko Mar 20, 2018
96393bf
Added checks for flatten attribute
mitsuhiko Mar 20, 2018
bb2ecb3
Added compilefail tests for flatten conflicts
mitsuhiko Mar 20, 2018
99614c7
Added flatten on enum compile fail test
mitsuhiko Mar 20, 2018
27f935f
Correctly serialize newtype variants for flatten
mitsuhiko Mar 20, 2018
0fde3c2
Fix a warning caused by no-default-features
mitsuhiko Mar 20, 2018
3d647f4
Fixed a compilefail test for flatten on enums
mitsuhiko Mar 20, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 184 additions & 9 deletions serde/src/private/de.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ use lib::*;
use de::{Deserialize, DeserializeSeed, Deserializer, Error, IntoDeserializer, Visitor};

#[cfg(any(feature = "std", feature = "alloc"))]
use de::Unexpected;
use de::{Unexpected, MapAccess};

#[cfg(any(feature = "std", feature = "alloc"))]
pub use self::content::{Content, ContentDeserializer, ContentRefDeserializer,
InternallyTaggedUnitVisitor, TagContentOtherField,
TagContentOtherFieldVisitor, TagOrContentField, TagOrContentFieldVisitor,
TaggedContentVisitor, UntaggedUnitVisitor};
TaggedContentVisitor, UntaggedUnitVisitor, EnumDeserializer};

/// If the missing field is of type `Option<T>` then treat is as `None`,
/// otherwise it is an error.
Expand Down Expand Up @@ -269,6 +269,14 @@ mod content {
}

impl<'de> Content<'de> {
pub fn as_str(&self) -> Option<&str> {
match *self {
Content::Str(x) => Some(x),
Content::String(ref x) => Some(x),
_ => None,
}
}

fn unexpected(&self) -> Unexpected {
match *self {
Content::Bool(b) => Unexpected::Bool(b),
Expand Down Expand Up @@ -1118,11 +1126,7 @@ mod content {
}
};

visitor.visit_enum(EnumDeserializer {
variant: variant,
value: value,
err: PhantomData,
})
visitor.visit_enum(EnumDeserializer::new(variant, value))
}

fn deserialize_unit_struct<V>(
Expand Down Expand Up @@ -1170,7 +1174,7 @@ mod content {
}
}

struct EnumDeserializer<'de, E>
pub struct EnumDeserializer<'de, E>
where
E: de::Error,
{
Expand All @@ -1179,6 +1183,18 @@ mod content {
err: PhantomData<E>,
}

impl<'de, E> EnumDeserializer<'de, E>
where E: de::Error
{
pub fn new(variant: Content<'de>, value: Option<Content<'de>>) -> EnumDeserializer<'de, E> {
EnumDeserializer {
variant: variant,
value: value,
err: PhantomData,
}
}
}

impl<'de, E> de::EnumAccess<'de> for EnumDeserializer<'de, E>
where
E: de::Error,
Expand All @@ -1199,7 +1215,7 @@ mod content {
}
}

struct VariantDeserializer<'de, E>
pub struct VariantDeserializer<'de, E>
where
E: de::Error,
{
Expand Down Expand Up @@ -2062,3 +2078,162 @@ where
T::deserialize_in_place(deserializer, self.0)
}
}

#[cfg(any(feature = "std", feature = "alloc"))]
pub struct FlatMapDeserializer<'a, 'de: 'a, E>(
pub &'a mut Vec<Option<(Content<'de>, Content<'de>)>>,
pub PhantomData<E>
);

#[cfg(any(feature = "std", feature = "alloc"))]
impl<'a, 'de, E> Deserializer<'de> for FlatMapDeserializer<'a, 'de, E>
where E: Error
{
type Error = E;

fn deserialize_any<V>(self, _: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
Err(Error::custom("can only flatten structs and maps"))
}

fn deserialize_enum<V>(
self,
name: &'static str,
variants: &'static [&'static str],
visitor: V
) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
for item in self.0.iter_mut() {
// items in the vector are nulled out when used. So we can only use
// an item if it's still filled in and if the field is one we care
// about.
let use_item = match *item {
None => false,
Some((ref c, _)) => c.as_str().map_or(false, |x| variants.contains(&x))
};

if use_item {
let (key, value) = item.take().unwrap();
return visitor.visit_enum(EnumDeserializer::new(
key,
Some(value)
));
}
}

Err(Error::custom(format_args!(
"no variant of enum {} not found in flattened data",
name
)))
}

fn deserialize_map<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
visitor.visit_map(FlatMapAccess::new(self.0.iter_mut(), None))
}

fn deserialize_struct<V>(
self,
_: &'static str,
fields: &'static [&'static str],
visitor: V
) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
visitor.visit_map(FlatMapAccess::new(self.0.iter_mut(), Some(fields)))
}

fn deserialize_newtype_struct<V>(
self,
_name: &str,
visitor: V,
) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
visitor.visit_newtype_struct(self)
}

forward_to_deserialize_any! {
bool i8 i16 i32 i64 u8 u16 u32 u64 f32 f64 char str string bytes
byte_buf option unit unit_struct seq tuple tuple_struct identifier
ignored_any
}
}

#[cfg(any(feature = "std", feature = "alloc"))]
pub struct FlatMapAccess<'a, 'de: 'a, E> {
iter: slice::IterMut<'a, Option<(Content<'de>, Content<'de>)>>,
pending_content: Option<Content<'de>>,
fields: Option<&'static [&'static str]>,
_marker: PhantomData<E>,
}

#[cfg(any(feature = "std", feature = "alloc"))]
impl<'a, 'de, E> FlatMapAccess<'a, 'de, E> {
fn new(
iter: slice::IterMut<'a, Option<(Content<'de>, Content<'de>)>>,
fields: Option<&'static [&'static str]>
) -> FlatMapAccess<'a, 'de, E> {
FlatMapAccess {
iter: iter,
pending_content: None,
fields: fields,
_marker: PhantomData,
}
}
}

#[cfg(any(feature = "std", feature = "alloc"))]
impl<'a, 'de, E> MapAccess<'de> for FlatMapAccess<'a, 'de, E>
where E: Error
{
type Error = E;

fn next_key_seed<T>(&mut self, seed: T) -> Result<Option<T::Value>, Self::Error>
where
T: DeserializeSeed<'de>,
{
while let Some(item) = self.iter.next() {
// items in the vector are nulled out when used. So we can only use
// an item if it's still filled in and if the field is one we care
// about. In case we do not know which fields we want, we take them all.
let use_item = match *item {
None => false,
Some((ref c, _)) => {
c.as_str().map_or(self.fields.is_none(), |key| {
match self.fields {
None => true,
Some(fields) if fields.contains(&key) => true,
_ => false
}
})
}
};

if use_item {
let (key, content) = item.take().unwrap();
self.pending_content = Some(content);
return seed.deserialize(ContentDeserializer::new(key)).map(Some);
}
}
Ok(None)
}

fn next_value_seed<T>(&mut self, seed: T) -> Result<T::Value, Self::Error>
where
T: DeserializeSeed<'de>,
{
match self.pending_content.take() {
Some(value) => seed.deserialize(ContentDeserializer::new(value)),
None => Err(Error::custom("value is missing")),
}
}
}
Loading