MSC2265: Proposal for mandating case folding when processing e-mail address localparts

[babolivier]

Rendered

[turt2live]

seems sane to me

[KitsuneRal]

I wonder how much of complication is to mandate lower-case processing (such as lookup and hashing) but case-preserve storing addresses.

[turt2live]

we ultimately can't tell implementations how to store their data (if they want to spend the extra time converting things to uppercase they can), but the requirement for lookups being lowercase is a fairly strong argument imo

[anoadragon453]

Seems like general consensus overall.

@mscbot fcp merge

[mscbot]

Team member @anoadragon453 has proposed to merge this. The next step is review by the rest of the tagged people:

• @KitsuneRal
• @anoadragon453
• @ara4n
• @dbkr
• @erikjohnston
• @richvdh
• @turt2live
• @uhoreg

No concerns currently listed.

Once at least 75% of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[richvdh]

So some questions on this of the sort that arise whenever case mapping comes up:

Are you sure that lower-casing, as opposed to casefolding, is the right thing to do? Examples of the difference:

• ß (german lower-case long 's', upper-case equivalent 'SS') case-folds to 'ss', so that 'hans.voß' matches 'HANS.VOSS'. (On the other hand: it's not entirely obvious that they should be treated the same)
• ς (greek lower-case sigma, when used at the end of the word) case-folds to 'σ' (regular lower-case sigma), so that 'ΣΊΣΥΦΟΣ' matches 'σίσυφος'.

Relatedly: should we consider unicode normalisation, so that (for example) 'ê' (U+00EA, e with circumflex) is treated the same as 'ê' (U+0065 U+0302, e followed by circumflex combining character)?

Neither of the above solve the 'French problem' where (traditionally) accents are omitted on upper-case characters, so 'COTE' should be equivalent to 'côté'...

[babolivier]

Relatedly: should we consider unicode normalisation, so that (for example) 'ê' (U+00EA, e with circumflex) is treated the same as 'ê' (U+0065 U+0302, e followed by circumflex combining character)?

I guess it depends on whether common email providers treat both characters as the same. I'll do some investigation around that.

Neither of the above solve the 'French problem' where (traditionally) accents are omitted on upper-case characters, so 'COTE' should be equivalent to 'côté'...

Should it, though, keeping in mind we're only looking at email addresses here? I just checked on both Gmail and Hotmail and neither of them consider bréndan.abolivier@... as being the same as brendan.abolivier@..., and I'm not aware of any provider that does.

Otherwise, yes, casefold is probably the way to go, I'll update the proposal to reflect that.

[richvdh]

tbh I wasn't aware that gmail let you use non-ascii localparts at all. Certainly being guided by the behaviour of common providers seems like a sensible idea. (also: sorry for not starting a thread.)

[babolivier]

tbh I wasn't aware that gmail let you use non-ascii localparts at all

Neither was I. For full context, I've tried Thunderbird (+OVH), Roundcube (+OVH), Hotmail and Gmail and only this last one accepted a non-ascii localpart in the recipient's address.

[mscbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[mscbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

[turt2live]

Merged 🎉