Skip to content

fix(scripting/mono-rt2): Fix UTF8 encoding issues for surrogate pairs and Korean characters#3267

Closed
poco8537 wants to merge 1 commit intocitizenfx:masterfrom
poco8537:fix-mono-v2-utf8
Closed

fix(scripting/mono-rt2): Fix UTF8 encoding issues for surrogate pairs and Korean characters#3267
poco8537 wants to merge 1 commit intocitizenfx:masterfrom
poco8537:fix-mono-v2-utf8

Conversation

@poco8537
Copy link
Contributor

@poco8537 poco8537 commented Mar 20, 2025

Goal of this PR

Ensure that UTF8 encoding in the Mono v2 scripting environment correctly handles surrogate pairs and Korean characters, preventing string corruption during native interop.

How is this PR achieving the goal

This PR updates the UTF8 encoding functions to correctly handle surrogate pairs by encoding valid pairs as 4-byte sequences and replacing invalid or missing surrogates with the replacement character (U+FFFD), ensuring that Korean and other Unicode characters are encoded properly.

This PR applies to the following area(s)

ScRT: C#

Successfully tested on

Game builds: ..

Platforms: Windows, Linux

Checklist

  • Code compiles and has been tested successfully.
  • Code explains itself well and/or is documented.
  • My commit message explains what the changes do and what they are for.
  • No extra compilation warnings are added by these changes.

Fixes issues

fixs #3263

@github-actions github-actions bot added ScRT: C# Issues/PRs related to either C# scripting runtimes triage Needs a preliminary assessment to determine the urgency and required action labels Mar 20, 2025
@prikolium-cfx prikolium-cfx added the status:needs planning Hard issues that take time as there's no 'one' obvious way. label Apr 4, 2025
@prikolium-cfx
Copy link
Collaborator

Thank you for the contribution. We need some time to verify that this change doesn't break anything else.

@prikolium-cfx prikolium-cfx self-requested a review April 4, 2025 12:20
@DaniGP17
Copy link
Contributor

DaniGP17 commented Apr 4, 2025

I said in the original issue(#3263) that maybe the cause was CString but looking into it in depth I think this won't fix it

@poco8537
Copy link
Contributor Author

poco8537 commented Apr 4, 2025

I said in the original issue(#3263) that maybe the cause was CString but looking into it in depth I think this won't fix it

When I tested it, it worked fine.
image

@DaniGP17
Copy link
Contributor

DaniGP17 commented Apr 4, 2025

When I tested it, it worked fine.

Oh, sorry, when I checked it I thought it was unrelated. It seems I didn't investigate well enough, and when I saw that the "Code compiles and has been tested successfully." check was unchecked, I thought you hadn't tested it.

@poco8537
Copy link
Contributor Author

poco8537 commented Jul 1, 2025

Any updates?

@DaniGP17
Copy link
Contributor

DaniGP17 commented Jul 1, 2025

Any updates?

Mono v2 is pending some important changes to fix bugs and implement the new msgpack which includes the definition of the CString type, so that change will no longer be useful when @manups4e PR is merged.
Then I guess it would be better to do another PR in the cfx msgpack repository.

@poco8537 poco8537 closed this Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ScRT: C# Issues/PRs related to either C# scripting runtimes status:needs planning Hard issues that take time as there's no 'one' obvious way. triage Needs a preliminary assessment to determine the urgency and required action

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants