Code generation randomly produces code which does not compile

[norru]

Code generation sometimes fails with corrupted code producing type errors. A workaround is to slightly change the code and retry (adding or removing blank lines works). I don't know if deterministic (so once it builds, it always builds correctly) a or random yet (so any build can randomly fail).

I'm using serde_codegen version 0.8.11

UPDATE: it does look deterministic and it appears to have something to do with the optional trailing commas. I'm using rustfmt so the comma is added automatically (and I am not keen on changing this policy):

    #[serde(skip_serializing_if="Option::is_none")]
    pub sequences: Option<Vec<Sequence>>, // removing this comma fixes it
}
// also adding an extra blank line fixes it

[...]/out/reference/template_types.rs:239:82: 239:95 error: mismatched types [E0308]
[...]/out/reference/template_types.rs:239                                                _serde::de::Error::invalid_length("SpeedOfTime"));

[...]/out/reference/template_types.rs:239:82: 239:95 help: run `rustc --explain E0308` to see a detailed explanation
[...]/out/reference/template_types.rs:239:82: 239:95 note: expected type `usize`
[...]/out/reference/template_types.rs:239:82: 239:95 note:    found type `&'static str`

                        let __field4 =
                            match try!(visitor . visit :: < Option < Int > > (
                                        )) {


                                Some(value) => {
                                    value
                                }
                                None => {
                                    try!(visitor . end (  ));
                                    return Err(

                                               _serde::de::Error::invalid_length(4usize));
                                }
                            };
                        let __field5 =
                            match try!(visitor . visit :: < Option < Int > > (
                                        )) {


                                Some(value) => {
                                    value
                                }
                                None => {
                                    try!(visitor . end (  ));
                                    return Err(

                                               _serde::de::Error::invalid_length("SpeedOfTime"));
                                                                                ^^^^^^^^^^^^^^^^ <- error in line 239
                                }
                            };
                        let __field6 =
                            match try!(visitor . visit :: < Option < Int > > (
                                        )) {
                                Some(value) =>
                                {
                                    value
                                }
                                None => {
                                    try!(visitor . end (  ));
                                    return Err(_serde::de::Error::invalid_length(6usize));
                                }
                            };

this is the snippet from the .rs.in . "SpeedOfTime" is a field of another struct?

#[derive(Default,Serialize,Deserialize)]
pub struct Event {
    #[serde(rename="Name")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub name: Option<String>,

    #[serde(rename="Format")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub format: Option<String>,

    #[serde(rename="EventDescription")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub event_description: Option<String>,

    #[serde(rename="Colour")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub colour: Option<String>,

    #[serde(rename="Period")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub period: Option<Int>,

    #[serde(rename="Length")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub length: Option<Int>,

    #[serde(rename="Repeat")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub repeat: Option<Int>,

    #[serde(rename="Start")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub start: Option<Int>,

    #[serde(rename="StartOffsetSeconds")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub start_offset_seconds: Option<Int>,

    #[serde(rename="DoubleXP")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub double_xp: Option<Bool>,

    #[serde(rename="Sequences")]
    #[serde(skip_serializing_if="Option::is_none")]
    pub sequences: Option<Vec<Sequence>>,
}

Cargo.toml:

[package]
name = "event_list_editor_cli"
version = "0.1.0"
authors = ["Nicola Orrù <nicola_orru@scee.net>"]
build = "build.rs"

[build-dependencies]
syntex = "*"
serde_codegen = "*"

[dependencies]
num = "*"
getopts = "*"
csv = "*"
log = "*"
log4rs = "*"
rand = "*"
chrono="*"
bitflags="*"
custom_derive="*"
enum_derive="*"
enum_primitive="*"
serde="*"
serde_json="*"

build.rs

extern crate syntex;
extern crate serde_codegen;

use std::env;
use std::path::Path;
use std::fs;

fn main() {
    let out_dir = env::var_os("OUT_DIR").unwrap();

    let src = Path::new("src/reference/template_types.rs.in");
    let dst = Path::new(&out_dir).join("reference/template_types.rs");

    fs::create_dir_all(dst.parent().unwrap()).unwrap_or(());

    serde_codegen::expand(&src, &dst).unwrap();
}

[dtolnay]

Thanks for the detailed report. This is the weirdest Serde-related bug I have ever seen!

The corrupted code is coming from this line where index_in_seq is definitely a usize. There is no way it should even have access to fields from a different struct.

To help us reproduce this, would it be possible for you to share the entire non-working "event_list_editor_cli" crate? You can email it to me at dtolnay@gmail.com if you don't want it publically on GitHub.

Also please include the output of rustc --version --verbose and your operating system and version.

[norru]

Thanks for replying! I can't send you the whole app as it is forbidden by
whatever NDA agreement I'm under, even posting snippets is frowned upon.
I'll see what I can do, perhaps I could try and isolate the problem.

On 6 October 2016 at 17:50, David Tolnay notifications@github.com wrote:

Thanks for the detailed report. This is the weirdest Serde-related bug I
have ever seen!

The corrupted code is coming from this line
https://github.com/serde-rs/serde/blob/v0.8.11/serde_codegen/src/de.rs#L342
where index_in_seq is definitely a usize. There is no way it should even
have access to fields from a different struct.

To help us reproduce this, would it be possible for you to share the
entire non-working "event_list_editor_cli" crate? You can email it to me at
dtolnay@gmail.com if you don't want it publically on GitHub.

Also please include the output of rustc --version --verbose and your
operating system and version.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#574 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA2Ry-N9g-DSo4LQbP2Irl5RdQ7FiVIoks5qxSbwgaJpZM4KPwEV
.

Cheers,
Nico

[dtolnay]

Closing because I don't think we can make progress on this without being able to reproduce it. Hopefully either nobody else ever hits this, or the next person can work with us to reproduce and fix.

[norru]

I am not allowed to send you the whole project and I'll need to strip it down to something that I can share first. That is not currently my top priority, unfortunately. I have not forgotten about this.

[dtolnay]

Thanks! I'll reopen when you get to it.

[dtolnay]

Looks like #590 has a more promising (open source) way to reproduce the same bug.

[norru]

I did manage to repro and stripped out all "confidential" code.

[norru]

Trying to attach source code...

On 18 October 2016 at 01:32, David Tolnay notifications@github.com wrote:

Looks like #590 #590 has a more
promising (open source) way to reproduce the same bug.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#574 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA2Ry0UXRjUlBliYBOaRnA7OxrfMGnnFks5q1BOcgaJpZM4KPwEV
.

Cheers,
Nico

[norru]

Didn't work :/

[dtolnay]

The other ticket reproducing this issue was fixed in Serde 0.8.14. Can you try with serde_codegen >= 0.8.14 and see whether your issue has been fixed as well?

[norru]

Trying now with codegen 0.8.16

[norru]

Cool, it seems to be fixed!