New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strings with NUL bytes are silently truncated in bound parameters #420
Comments
|
I'll look into that. However you may know that postgres doesn't accept null bytes in strings, so the right thing to do would be to raise an exception, right? |
|
Yes, raising an exception would be awesome :) Thanks! On Wed, Mar 30, 2016 at 2:59 PM, Daniele Varrazzo notifications@github.com
|
|
I've fixed this issue here: |
|
Amazing, thanks so much! |
After migrating a Wagtail-based site from MySQL to Postgres, we
noticed that malicious requests to the site that included percent-
encoded Unicode NULLs (`%00`) raised a `ValueError` exception that we
hadn't seen when using MySQL: `A string literal cannot contain NUL
(0x00) characters.` This appears to relate to `psycopg2`'s decision to
raise an exception in these situations, as discussed here:
psycopg/psycopg2#420
While newer versions of Django appear to provide some field validation
that addresses these characters, it doesn't look like Wagtail's
redirect middleware is making use of those validators, and so it seemed
reasonable to clean these characters in the context of 'normalizing'
the paths before looking for corresponding redirects -- especially
since a quick investigation on the internet suggests that U+0000 in
URLs can be used as a means of attack, and also since RFC 3986 says:
Note, however, that the "%00" percent-encoding (NUL) may require
special handling and should be rejected if the application is not
expecting to receive raw data within a component.
After migrating a Wagtail-based site from MySQL to Postgres, we
noticed that malicious requests to the site that included percent-
encoded Unicode NULLs (`%00`) raised a `ValueError` exception that we
hadn't seen when using MySQL: `A string literal cannot contain NUL
(0x00) characters.` This appears to relate to `psycopg2`'s decision to
raise an exception in these situations, as discussed here:
psycopg/psycopg2#420
While newer versions of Django appear to provide some field validation
that addresses these characters, it doesn't look like Wagtail's
redirect middleware is making use of those validators, and so it seemed
reasonable to clean these characters in the context of 'normalizing'
the paths before looking for corresponding redirects -- especially
since a quick investigation on the internet suggests that U+0000 in
URLs can be used as a means of attack, and also since RFC 3986 says:
Note, however, that the "%00" percent-encoding (NUL) may require
special handling and should be rejected if the application is not
expecting to receive raw data within a component.
Minimal repro:
This prints out
[('ben.s.kuhn@gmail.com',)], which is neither what I tried to insert nor what I tried to query for, which came as a surprise to me! I would have expected some kind of exception to be thrown. This has the potential to cause security flaws if a developer assumes that the string being queried/inserted has passed validation because the query worked.The text was updated successfully, but these errors were encountered: