Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Borrowed Cow<'_, [u8]> fails if raw bytes not valid utf8 #231

Closed
daboross opened this issue Mar 12, 2018 · 4 comments
Closed

Borrowed Cow<'_, [u8]> fails if raw bytes not valid utf8 #231

daboross opened this issue Mar 12, 2018 · 4 comments

Comments

@daboross
Copy link

@daboross daboross commented Mar 12, 2018

I thought I was going crazy for a while, so I might still be, but this seems to be a bug in bincode or serde?

This code:

extern crate bincode;
extern crate serde;
#[macro_use]
extern crate serde_derive;

use std::borrow::Cow;

#[derive(Debug, Serialize, Deserialize)]
pub struct X<'a> {
    #[serde(borrow)]
    pub field: Cow<'a, [u8]>,
}


fn main() {
    let original = X { field: [255, 255, 255, 255][..].into() };

    let ser = bincode::serialize(&original).expect("expected serialization to succeed");
    let _: X = bincode::deserialize(&ser).expect("expected same encoded bytes to decode");

}

fails with:

thread 'main' panicked at 'expected same encoded bytes to decode: InvalidUtf8Encoding(Utf8Error { valid_up_to: 0, error_len: Some(1) })', libcore/result.rs:945:5

I'm not serializing or deserializing any strings, so I am assuming this is a bug. If it's not, could there be another way to fix this?

It does work correctly (accepting invalid UTF8 bytes) if I use either Vec<u8> or &'a [u8].

Thanks!

@daboross daboross changed the title Borrowed byte arrays fail if not valid utf8 Borrowed Cow<'_, [u8]> fails if raw bytes not valid utf8 Mar 12, 2018
@TyOverby
Copy link
Collaborator

@TyOverby TyOverby commented Mar 12, 2018

This is super weird. I can't find any code in bincode or serde that would do this.

CC @dtolnay

@daboross
Copy link
Author

@daboross daboross commented Mar 12, 2018

Could be https://github.com/serde-rs/serde/blob/master/serde/src/private/de.rs#L191?

I'll try changing that and see if it fixes anything.

@dtolnay
Copy link
Collaborator

@dtolnay dtolnay commented Mar 12, 2018

Yes I had just found the same thing. That does fix it. Want to send a PR?

@daboross
Copy link
Author

@daboross daboross commented Mar 12, 2018

Sure, will do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.