Core validation does not support new TLD domains

[hissy]

Concrete\Core\Utility\Service\Validation\Strings::email() allows max 7 characters to TLD, but it does not support new longer TLDs.

eg: .international (13 characters)
http://www.newtldlist.com/

"How long can a TLD possibly be?"
http://stackoverflow.com/questions/9238640/how-long-can-a-tld-possibly-be

"Each node has a label, which is zero to 63 octets in length."
http://tools.ietf.org/html/rfc1034

In addition, there are (maybe unfortunately) multibyte TLDs.

.みんな
.公司
.닷컴

Wow... "A first step toward more global email"
http://googleblog.blogspot.jp/2014/08/a-first-step-toward-more-global-email.html

[hissy]

Related forum post: http://www.concrete5.org/community/forums/usage/tld-issues-adding-users/

[mlocati]

IMHO we should use a better approach to verify email addresses... What about is_email?

[aembler]

@mlocati That looks like a nice script. We'd want to wrap that functionality inside our existing email() function but it looks like a much better solution to validating email addresses. There also might be a nice object-oriented approach to this that we could get through composer and wrap inside our existing API.

[mlocati]

@hissy Do you know if is_email is ok for multibyte TLDs?

[aembler]

What about something like this? https://packagist.org/packages/egulias/email-validator

Basically is_email + OO approach

[joe-meyer]

So... just toyed around with the egulias package. It seems to work ok, until I turn on the DNS check stuff. As soon as I flip that on (and put it in strict mode since that's the only way to check it actually uses it), I end up getting results saying there is no DNS record for the international domains.

If I ping the Bücher.ch example, it actually comes back and resolves to: xn--bcher-kva.ch. From what I can see his domain parser isn't doing any sort of magic to try and convert the domain to something like that so the checkdnsrr() function is failing.

Here's an output of some of my quick testing against the my branch https://github.com/ExchangeCore/concrete5-5.7.0/compare/fixes-652?expand=1. Let me know if you think this would be worthwhile to pull or if we need to find something else.

With DNS Validation

test@exchangecore.com : true
test@Bücher.ch : false
test@ü.ch : false
whoknowsif!works@exchangecore.com : true
thisisnotanemail : false
thisIsAlsoNotAnEmail@ : false
thisIs@Email : false
thisIsAValid@pa : false
test@католик : false
something@武＠メール.グーグル : false
somethingInvalid@みんなみんなみんな : false

Without DNS Validation

test@exchangecore.com : true
test@Bücher.ch : true
test@ü.ch : true
whoknowsif!works@exchangecore.com : true
thisisnotanemail : false
thisIsAlsoNotAnEmail@ : false
thisIs@Email : true
thisIsAValid@pa : true
test@католик : true
something@武＠メール.グーグル : true
somethingInvalid@みんなみんなみんな : true

[joe-meyer]

Ya looks like Yii2 ran into a similar/same problem: yiisoft/yii2#312.

Also another oddity I just noticed with this package is that @pa is a valid tld with an mx record and it's still kicking back false for the check. Not that to many people are going to use a tld for their email (maybe?)

[aembler]

I would think we wouldn't enable DNS validation or mx record checking unless the developer really explicitly wanted that. Too risky and prone to breakage

Sent from my iPhone

On Aug 13, 2014, at 7:58 AM, Joe Meyer notifications@github.com wrote:

So... just toyed around with the egulias package. It seems to work ok, until I turn on the DNS check stuff. As soon as I flip that on (and put it in strict mode since that's the only way to check it actually uses it), I end up getting results saying there is no DNS record for the international domains.

If I ping the Bücher.ch example, it actually comes back and resolves to: xn--bcher-kva.ch. From what I can see his domain parser isn't doing any sort of magic to try and convert the domain to something like that so the checkdnsrr() function is failing.

Here's an output of some of my quick testing against the my branch https://github.com/ExchangeCore/concrete5-5.7.0/compare/fixes-652?expand=1. Let me know if you think this would be worthwhile to pull or if we need to find something else.

With DNS Validation

test@exchangecore.com : true
test@Bücher.ch : false
test@ü.ch : false
whoknowsif!works@exchangecore.com : true
thisisnotanemail : false
thisIsAlsoNotAnEmail@ : false
thisIs@Email : false
thisIsAValid@pa : false
test@католик : false
something@武＠メール.グーグル : false
somethingInvalid@みんなみんなみんな : false

Without DNS Validation

test@exchangecore.com : true
test@Bücher.ch : true
test@ü.ch : true
whoknowsif!works@exchangecore.com : true
thisisnotanemail : false
thisIsAlsoNotAnEmail@ : false
thisIs@Email : true
thisIsAValid@pa : true
test@католик : true
something@武＠メール.グーグル : true
somethingInvalid@みんなみんなみんな : true

—
Reply to this email directly or view it on GitHub.

[joe-meyer]

@aembler it's not enabled by default, but the option was there in the past, and while it seems to work for non international domains and non top level domains, I wasn't sure if that was "good enough". If you think it is I can put up a pull request and we can just wait until a) someone complains b) the package gets updated and fixes the issue.

At this point we certainly don't lose any functionality by implementing, but the rules for a "valid" email become a little more lenient than what was previously there since it allows for top level domains (I.E. no . character in the domain part of the email address).

[hissy]

@mlocati it seems is_email() do not support multibyte TLDs

[joe-meyer]

Looks like PR #715 got merged. Any thoughts feedback there? Hopefully it helps some with the TLD support, though international support seemed to be a bit lacking, and without relying on the PHP intl extension, i'm not sure how to even start about a PR to fix these issues. Anyway, went as far as I can probably go with this without feedback or guidance. If anyone wants to provide those I'm happy to dig further.

[hissy]

I tested and it works fine with new TLDs (including multibyte domains).

[aembler]

Nice! That is good news.

[aembler]

Thanks everybody.