"Filter segments for malicious characters", really ?

[bitbucket-import]

When requesting a URI containing URI encoded parenthesis (I mean %28 and %29), they reach the controller maimed and unusable. Their strlen() reads wrong and they wont be matched as parenthesis by regex.

The reason for this is in system/core/URI.php at lines 238-241. Why would the parenthesis be replaced by shitty HTML entities that damage the string ?

// Convert programatic characters to entities
$bad    = array('$',        '(',        ')',        '%28',      '%29');
$good   = array('$',    '(',    ')',    '(',    ')');
return str_replace($bad, $good, $str);

Unless there's a good reason for it, I think this replacement should be removed altogether. You don't want to know how many hours it took me to pinpoint this issue.

With love,

Antoine Gersant

[narfbg]

This really depends on wether #388 is considered a bug or not.

[narfbg]

... or not - ignore my last comment.

[iamjochem]

+1 for scrapping the "conversion programatic characters" - it is pointless, has the potential to waste a lot of time 'debugging' and it is incorrect. the code essentially performs a limited "html entitizing" ... html entitizing is something you do to output not input!

[jim-parry]

Hasn't this been address by #388? Can this be closed?

[narfbg]

#388 has nothing to do with this ... other issues related to this one have been fixed, but the suggestion here is to remove this filter altogether.

[Razican]

I think enough people are having issues with it and that it does not have a clear advantage, so in my opinion it should be removed.

[jim-parry]

I agree that the substitution should be removed. RFC 3986 says that the dollar sign and parentheses are safe characters and do not need encoding. They are also flagged as "reserved" characters, which can be encoded and interpreted by an application, but that appears to be subsequent to any use as a URI.

[ostrolucky]

I'm voting for removing it.

[ivantcholakov]

+1 for "conversion programatic characters" removal. The three presented justifications are good enough.

---

Edit: Justification 4: Such characters within a segment may be needed as a result of sloppy slug generation. If you use url_title() for this purpose, the segment would be clean, and then the "programatic characters" simply may not be enabled using the setting $config['permitted_uri_chars'].

url_title() may be reworked to transliterate from non-Latin languages, but this is another story.

[Razican]

I vote for it too.

[narfbg]

Well, the public opinion seems to be unanimous.

@pfote @benedmunds @druu @lonnieezell Any objections?

[benedmunds]

No objection.

[lonnieezell]

I can't think of a reason it's really needed, but haven't scoured the code about this either.

No objection.