New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Unicode/UTF-8 support to UrlEncode and UrlDecode #175
Comments
@parkone Thanks for raising this issue. This is a known issue and has I'm researching approaches of resolving this that maintain Mac support (unfortunately, |
Is this issue fixed by #205? |
@bdr99 it should be, but I'll add a test case. |
I have submitted another PR #213 to enhance UrlEncode further and to provide similar enhancements and fixes to UrlDecode also. This does NOT AFAIK provide full Unicode support - but it does handle ASCII characters above 128. |
Found this reference implementation for Java: https://www.w3.org/International/URLUTF8Encoder.java
// relevant lines
} else if (ch <= 0x007f) { // other ASCII
sbuf.append(hex[ch]);
} else if (ch <= 0x07FF) { // non-ASCII <= 0x7FF
sbuf.append(hex[0xc0 | (ch >> 6)]);
sbuf.append(hex[0x80 | (ch & 0x3F)]);
} else { // 0x7FF < ch <= 0xFFFF
sbuf.append(hex[0xe0 | (ch >> 12)]);
sbuf.append(hex[0x80 | ((ch >> 6) & 0x3F)]);
sbuf.append(hex[0x80 | (ch & 0x3F)]);
} |
I don't know if you'll find this useful for the UTF8 decode, but I've been trying to decode URLs myself and after hours of searching I stumbled upon this: http://stackoverflow.com/questions/23810324/vba-convert-string-to-unicode Also, perhaps this bit here helps you convert text to an UTF8 byte array, perhaps if you combine it with the second part of parkone's answer you can achieve the encoding without the need of ADODB: http://www.di-mgt.com.au/howto-convert-vba-unicode-to-utf8.html |
I still have issues with non-ASCII characters with 4.1.1 . In Excel 2016 (Windows 7) when I replace WebHelpers.UrlEncode from WebRequest.cls by WorksheetFunction.EncodeURL - source - everything works like a charm. |
@justinmassiot, Lines 997 to 1007 in f400fbc
The first approach is non-standard and not supported by some (most?) servers, see Wikipedia. I've implemented the second approach (not pushed to GitHub yet), and it works like a charm, except that I haven't tested it with surrogates. It may require additional code. Also, there are other places where UTF-8 support is required. The solution should be universal. @timhall, what do you think? |
Also, I wonder whether |
So, is it solved? |
Function URLEncode in the WebHelpers module do not convert correctly Unicode characters!
When sending json with non-Unicode characters it will broken.
I suggest to use this function instead
The text was updated successfully, but these errors were encountered: