-
-
Notifications
You must be signed in to change notification settings - Fork 413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[New bug] Issues with some weird characters inserted into DB #3423
Comments
I can't reproduce it on r22493 (x64). dbcon = dbConnect("mysql", "dbname=testdb;host=localhost", "root", "")
dbExec(dbcon, "SET NAMES utf8mb4")
dbExec(dbcon, "CREATE TABLE IF NOT EXISTS `table` (message TEXT)")
dbExec(dbcon, "INSERT INTO `table` (message) VALUES (?)", "😁") Another testCode: dbcon = dbConnect("mysql", "dbname=testdb;host=localhost", "root", "")
dbExec(dbcon, "SET NAMES utf8mb4")
function query(...)
local queryHandle = dbQuery(dbcon, ...)
if (not queryHandle) then
return nil
end
local rows = dbPoll(queryHandle, -1)
return rows
end
iprint(query("SELECT message from `table`"))
iprint(query("SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%'")) Result:
|
Hmm, I wonder if the MySQL version running on the server being outdated could be causing the problem with the updated version of MTA. What MySQL version do you have running? |
I couldn't reproduce the issue. I've used MySQL Community Server 8.4.0 with a |
The commit 31c68fd is unrelated. I simply couldn't connect to my local MySQL server without this option. |
8.3.0, but I just tested on a very old version (5.7.23) and it's ok too. |
I know why this couldn't be reproduced as I just tried the same code in a script and it worked fine, but when done through runcode it fails. This is just so weird though, that this code in runcode would work fine on... now this gets even more weird as I can now reproduce this bug on older versions that I'm sure worked fine, I'm going to try older client version to see if the problem is that the MTA client is sending garbled characters. I just tried with the oldest available nightly, 22388 and that also gives the error. I've found a possible clue: But when the same character is done in a script: So this bug maybe have nothing to do with MySQL but when a player enters certain characters in MTA in their console, GUI, etc they are handled incorrectly. I've tried in client side runcode and client side file and got: |
This could be why: Whereas that executed in a script file: On the client that 1 character is being split into 2. I just wish there was a way to fix the MySQL insertion errors, like what about utf8_bin? |
For your information, the MTA client has a broken UTF16 to UTF8 conversion when the UTF16 string contains surrogate pairs. Use the function below to fix the broken UTF8 string: function utf8_decode_utf16_surrogate_pairs(text)
local characters = {}
local highSurrogate = 0
local length = 0
for position, codepoint in utf8.next, text do
if highSurrogate > 0 then
if codepoint >= 0xDC00 and codepoint <= 0xDFFF then
codepoint = (highSurrogate - 0xD800) * 0x400 + (codepoint - 0xDC00) + 0x10000
length = length + 1
characters[length] = utf8.char(codepoint)
end
highSurrogate = 0
elseif codepoint >= 0xD800 and codepoint <= 0xDBFF then
highSurrogate = codepoint
else
length = length + 1
characters[length] = utf8.char(codepoint)
end
end
return table.concat(characters)
end |
I see [mysqld]
collation_server = utf8mb4_unicode_ci
init_connect='SET NAMES utf8mb4'
character_set_server = utf8mb4 |
Thanks for that code botder, it works for hack fixing this bug. |
If my function worked for you, then this isn't an actual issue with the database nor the libraries used, but with user input conversion to UTF-8. Your database protected you from inserting garbage UTF-8 bytes. |
Describe the bug
After updating MTA server I started getting warnings about unable to insert into MySQL such as:
[script] FAIL: (1366) Incorrect string value: '\xED\xA0\xBD\xED\xB8\x81' for column 'msg' at row 1 [Query:INSERT INTO my_history (time, acc, msg) VALUES (1717270392, 'Arran', '😁')]
I narrowed it down to r22396 that has MySQL changes: https://buildinfo.multitheftauto.com/index.php?Revision=22396&Branch=
Steps to reproduce
Note that I have the MySQL table set to utf8mb4 and have also tried passing charset=utf8mb4 with dbConnect but it makes no difference. Only using the revision before this fixes it. I checked what gets inserted in the previous version and MySQL workbench shows: í ½í¸� for 😁
Version
Windows 64 bit v1.6-release-22396
Additional context
No response
Relevant log output
No response
Security Policy
The text was updated successfully, but these errors were encountered: