Add IDN (non-ASCII domain names) support

When i use special characters in e-mails which is valid these days you get an error:

Address in mailbox given [Tëst@intermeshdev.nl] does not comply with RFC 2822, 3.6.2.

Original creation date: 2009-05-22T11:56:20Z
Original reporter: Maciej Lisiewski
Original ticket: http://swiftmailer.lighthouseapp.com/projects/21527/tickets/93

Ignore this one. Utf-8 is not valid but people seem to be trying to use this :)

Original creation date: 2009-05-22T12:23:00Z
Original reporter: Merijn

It should be possible to have UTF-8 characters in the e-mail. Domains with UTF-8 like ość.pl exist and swift can't mail to them!

Original creation date: 2009-12-17T15:12:42Z
Original reporter: Merijn

UTF-8 domains violate RFC 2822, but since they now exist we should certainly slacken to regular expression which checks for the domain portion of the email.

e: chris@w3style.co.uk
t (en): http://twitter.com/d11wtq
t (it): http://twitter.com/cosadici

Il giorno 18/dic/2009, alle ore 02.12, Lighthouse ha scritto:

Original creation date: 2009-12-17T23:14:03Z
Original reporter: Chris Corbyn

Do you have a patch for this?

Original creation date: 2009-12-18T08:11:46Z
Original reporter: Merijn

At the moment there is no "true" UTF-8 in domains - IDNs [ http://en.wikipedia.org/wiki/Internationalized_domain_name ] are still ASCII only, but with UTF-8 characters encoded in Punycode [ http://en.wikipedia.org/wiki/Punycode ]. I will submit my UTF-8->Punycode converter once I find it (give me a couple of hours) and I'll leave it up to you when you want to use it (pre-validation, or post-validation - the lated needing changing the regexp to accommodate for UTF-8 in domain name). I am not entirely sure whenever I made one that does Punycode->UTF-8 as I wrote mine to take care of no IDN support in PHPMailer.

Original creation date: 2009-12-21T06:49:57Z
Original reporter: Maciej Lisiewski

Update: will add code before the end of the month - I have found it, it works, but also it's totally unreadable, commentless and with variable names such as $_abc2 - It has been a quick and dirty fix and once I make it readable I'll post it here.

Original creation date: 2009-12-23T04:24:17Z
Original reporter: Maciej Lisiewski

As promised I include my cleaned up converter function. Function accepts prevalidated UTF-8 domain names (not emails!) - extra explode will be required. There shouldn't be much of a performance impact if this function is called for each email - for ASCII only domain names it doesn't do much.

Original creation date: 2010-01-02T18:36:05Z
Original reporter: Maciej Lisiewski

Correct file attached

Original creation date: 2010-01-02T18:38:32Z
Original reporter: Maciej Lisiewski

Thanks! I'll try it. But shouldn't this function be built into the Swift library?

Original creation date: 2010-01-04T07:50:02Z
Original reporter: Merijn

Yes, I think it should.

Original creation date: 2010-01-04T08:36:35Z
Original reporter: Maciej Lisiewski

Oh, and since I haven't added any licence info in the file: feel free to include any part of my code in swift mailer

Original creation date: 2010-01-19T21:15:14Z
Original reporter: Maciej Lisiewski

We'll check if we can rely on the http://www.php.net/manual/en/function.idn-to-ascii.php function too :)

I'll check how to integrate this in the Address Headers

Original creation date: 2011-03-12T22:54:04Z
Original reporter: xdecock

idn_to_ascii is nice, but is PHP >= 5.3.0 only and requires 2 PECL packages to be installed installed: PECL intl >= 1.0.2 and PECL idn >= 0.1.

Original creation date: 2011-03-13T10:31:11Z
Original reporter: Maciej Lisiewski

Hi,

I just added your code in my branch, however, for now, it encodes every domain, without the STD3-ASCII Rules, i'll check if it's possible to add this validation, but for this, i need to understand what's the problem exactly.

Can you confirm we have permission to embed it as GPL inside Swift?

Original creation date: 2011-03-22T21:33:39Z
Original reporter: xdecock

I think all domain names will have to pass through this:
To encode only non-ascii domain names you'd have to be able to determine if the domain name contains any non-ascii characters - check if each and every character in domain name is <= 0x7F. This is the very first step of the encoding - all the characters that are actually <= 0x7F are just copied - additional check before string if passed to be encoded will just duplicate this step. The only real optimization I can see here is getting rid of ord() I have in my code.
I doubt there is any additional check necessary - even with ord it will encode google.com 100 000 times in 1.1s on old xeon I use for testing. Encoding 6 letter IDN domain with 3 non-ascii characters is only 0.3s slower on 100 000 encodes.

As for adding it to Swift feel free to include it - I am fine with GPL/LGPL/MIT/BSD or any compatible licence(s).

Original creation date: 2011-03-23T00:50:26Z
Original reporter: Maciej Lisiewski