New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mail/Drafts] 4-byte emoji in a composed message breaks draft saving #449

Closed
jstanden opened this Issue Sep 27, 2017 · 8 comments

Comments

Projects
2 participants
@jstanden
Owner

jstanden commented Sep 27, 2017

No description provided.

@jstanden jstanden added the bug label Sep 27, 2017

@jstanden jstanden modified the milestones: 8.1.3, 8.1.4 Sep 27, 2017

@jstanden jstanden modified the milestones: 8.1.4, 8.2 Oct 5, 2017

@jstanden

This comment has been minimized.

Owner

jstanden commented Oct 5, 2017

This relies on #59 + #444

@jstanden jstanden modified the milestones: 8.2, 8.2.1 Oct 30, 2017

@jstanden jstanden modified the milestones: 8.2.1, 8.2.2 Nov 6, 2017

@jstanden jstanden modified the milestones: 8.2.2, 8.2.3 Nov 13, 2017

@jstanden jstanden modified the milestones: 8.2.3, 8.2.4, 8.2.5 Nov 21, 2017

@jstanden jstanden modified the milestones: 8.2.5, 8.2.6 Dec 5, 2017

@jstanden jstanden removed this from the 8.2.6 milestone Dec 20, 2017

@jstanden jstanden added this to the 9.0.6 milestone Oct 1, 2018

@jstanden jstanden self-assigned this Oct 1, 2018

@jstanden jstanden added this to Sprint in 9.0.9 Oct 4, 2018

@jstanden jstanden moved this from Sprint to In Development in 9.0.9 Oct 4, 2018

@jstanden jstanden modified the milestones: 9.0.6, 9.0.7 Oct 8, 2018

@jstanden

This comment has been minimized.

Owner

jstanden commented Oct 9, 2018

This should now be fixed in 9.0.7.

@jstanden jstanden closed this Oct 9, 2018

9.0.9 automation moved this from In Development to Completed! Oct 9, 2018

@tullyhansen

This comment has been minimized.

tullyhansen commented Oct 9, 2018

You're my hero! 😍 Now waiting on 9.0.7 to drop in Cerb Cloud…

@jstanden

This comment has been minimized.

Owner

jstanden commented Oct 9, 2018

Hey @tullyhansen! I just applied the fix directly to your instance ahead of the update. See if that works properly now.

@tullyhansen

This comment has been minimized.

tullyhansen commented Oct 9, 2018

It took an effort of will to reload the page with emojis in the middle of a draft (hard-won experience) but… yessssssssss! Thanks so much!

(Now for emoji in sticky notes… have I filed for that already? 😝)

@jstanden

This comment has been minimized.

Owner

jstanden commented Oct 9, 2018

I'll check and file that if it isn't already. :)

@tullyhansen

This comment has been minimized.

tullyhansen commented Oct 9, 2018

#59, I think? And #645 will hopefully get caught in the crossfire.

@jstanden

This comment has been minimized.

Owner

jstanden commented Oct 9, 2018

For sure. At the moment, full utf8mb4 support requires changing the charset/collation of the entire schema in MySQL. We'd need to set that on the MySQL connection to be able to save 4-byte content into any field.

The stopgap fix for drafts mirrors what we're doing with message content -- we just treat it as binary content and don't enforce any charset.

I'm all for using utf8mb4 as a default, but we also need to make sure people aren't using the MyISAM storage engine anymore, since it has a much lower key length limit when using text-based fields in compound keys (1000 bytes vs ~3000 for InnoDB).

That doesn't affect most of our schema, but it does come up. There are also field limits like varchar(255) which effectively store fewer characters than expected with 4-byte characters. That can affect ticket subjects, validation, and the like.

If we had started the project today, we wouldn't have even made the charset configurable, and we would have just used utf8mb4 everywhere. It's a shame MySQL's utf8 default has always been a misleading 3-bytes.

We're getting closer to being able to do that.

jstanden added a commit that referenced this issue Oct 23, 2018

* [Mail/Drafts] Fixed an issue with drafts not being saved if they co…
…ntained Emoji characters (or any other 4-byte characters).

Fixes #449
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment