Documentation: PG doesn't accept null bytes in UTF8

[anp]

It would be good to document the fact that diesel doesn't sanitize 0x00 from Rust's String type, which causes a runtime error when using the postgres backend. Also would be good to mention that it's problematic from an API perspective, as it would add non-trivial overhead to String serialization, along with low-overhead strategies for mitigating.

Maybe this could be a part of either a document on various backend idiosyncrasies, or possibly a document covering the various Rust type -> SQL type mappings that diesel supports.

[hiimtaylorjones]

I'm down to write some docs about this.

@dikaiosune - could you maybe provide some common examples where this might occur?

I really like the suggestion of documenting what Rust.to.SQL mappings are currently supported by Diesel. I think it'll bring down the amount of Issues of "Why doesn't rust_thing convert to SQL?" in the future.

[anp]

@hiimtaylorjones It came up for me when I was inserting responses from GitHub's API into the database. Still not sure why the heck GitHub was returning null bytes, but it is valid UTF-8, even though (apparently) Postgres doesn't accept null bytes.

I've been thinking about writing something similar, but my plate's a little full at the moment.

[sgrif]

Also to clarify, this only affects PG not the text type in general. SQLite handles nul bytes fine.

[hiimtaylorjones]

@dikaiosune - Sweet. I'll see what I can dig up there.

@sgrif - noted.

[hiimtaylorjones]

Hey everyone,

This totally fell of my radar. For that, I apologize. Though this conversation is about a year old, I decided to still take a shot at the issue.

While, the need for SQL mapping types still exists, I figured that I'd at least write up something to cover the ground in the meantime. That commit can be found here. Please let me know if I'm completely right, wrong, or somewhere in between!

As far as the proposed mapping documentation is concerned, where exactly would be the best place to insert that documentation?

[sgrif]

This is now documented on the Text type. It may be worth documenting elsewhere, but this issue seems resolved for the time being.