New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BasicURLNormalizer to optionally convert IDN host names to ASCII/Punycode #522

Merged
merged 1 commit into from Dec 21, 2017

Conversation

Projects
None yet
2 participants
@sebastian-nagel
Collaborator

sebastian-nagel commented Dec 21, 2017

  • conversion of IDN host names to their ASCII/Punycode equivalent is triggered if param "hostIDNtoASCII" is true
  • avoids duplicate page content
  • get clean ASCII keys if sharding/routing is done by host or domain

@jnioche jnioche added this to the 1.8 milestone Dec 21, 2017

@jnioche jnioche merged commit a9e4cb1 into DigitalPebble:master Dec 21, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jnioche

This comment has been minimized.

Member

jnioche commented Dec 21, 2017

Great stuff, thanks @sebastian-nagel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment