deserialize_with not behaving as expected with internally tagged enums

[d-e-s-o]

Please have a look at the following example:

use serde::Deserialize;
use serde::Deserializer;
use serde_json::from_str as from_json;


#[derive(Clone, Debug, Deserialize, PartialEq)]
#[serde(tag = "tag")]
enum Foo {
  #[serde(rename = "variant1")]
  Variant1(Variant1),
}

fn optional_bar_from_str<'de, D>(deserializer: D) -> Result<Option<Bar>, D::Error>
where
  D: Deserializer<'de>,
{
  match Option::<String>::deserialize(deserializer)? {
    Some(s) => match s.as_ref() {
        "bar1" => Ok(Some(Bar::Bar1)),
        "bar2" => Ok(Some(Bar::Bar2)),
        _ => unimplemented!(),
    },
    None => Ok(None),
  }
}

#[derive(Clone, Debug, Deserialize, PartialEq)]
enum Bar {
    Bar1,
    Bar2,
}

#[derive(Clone, Debug, Deserialize, PartialEq)]
struct Variant1 {
    #[serde(deserialize_with = "optional_bar_from_str")]
    optional_bar: Option<Bar>,
}


fn main() {
  let string = r#"{"tag":"variant1"}"#;
  let _foo = from_json::<Foo>(string).unwrap();
  println!("deserialization successful")
}

(playground)

This code fails with

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `optional_bar`", line: 0, column: 0)', src/libcore/result.rs:1165:5

and it is not clear to me why that would be the case. Or rather, it seems incorrect that it does.

If I apply the following patch:

--- foo.rs
+++ foo.rs
@@ -32,7 +32,6 @@

 #[derive(Clone, Debug, Deserialize, PartialEq)]
 struct Variant1 {
-    #[serde(deserialize_with = "optional_bar_from_str")]
     optional_bar: Option<Bar>,
 }

then deserialization succeeds. If I am reading correctly between the lines then we may end up treating the fact that there is a deserialize_with attribute as: the corresponding field has to exist. That seems bogus.

(if my hypothesis is correct, I suppose as a work around I could create a newtype wrapping Option<Bar> and implement Deserialize on it and that may solve the issue, but I haven't tried that yet [and I am not particularly eager to work around the issue this way because I simply can't have newtypes like this everywhere])

[dpc]

I think I've hit the same problem, and it was surprising. I've just addedd #[serde(default)] and I think that solves the problem for me in this particular case.

[dtolnay]

Using deserialize_with does turn off the Option-specific special case that deals with the field being absent, as you found. Adding serde(default) is the no-longer-Option-specific way to get back that behavior.