Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support unicode characters longer than a byte #1

Closed
microbit-carlos opened this issue Jan 17, 2019 · 1 comment
Closed

Support unicode characters longer than a byte #1

microbit-carlos opened this issue Jan 17, 2019 · 1 comment

Comments

@microbit-carlos
Copy link
Collaborator

This bug has been "ported" from the PythonEditor: bbcmicrobit/PythonEditor#60

At the moment the String.charCodeAt() method is used to get the user script character codes, and it's return value is stored in a Uint8Array.
If the value returned is larger than a byte, as it would happen with some UTF-8 characters, then some of that information is lost and the wrong character is encoded into the hex file.

The code is in the src/common.ts file:
https://github.com/microbit-foundation/js-microbit-micropython-fs/blob/ee7081de3b29faf91407eb3535bf0b8cc29fb334/src/common.ts#L7-L18

https://github.com/microbit-foundation/js-microbit-micropython-fs/blob/ee7081de3b29faf91407eb3535bf0b8cc29fb334/src/common.ts#L22-L31
@microbit-carlos microbit-carlos mentioned this issue Jan 17, 2019
Merged
microbit-rosslowe added a commit that referenced this issue Feb 15, 2019
@microbit-rosslowe
Create a test for unicode characters
70c954f 
Create a new test for issue #1 , testing if strToBytes() supports characters longer than 1 byte.
@microbit-rosslowe microbit-rosslowe mentioned this issue Feb 15, 2019
Merged
microbit-rosslowe added a commit that referenced this issue Feb 19, 2019
@microbit-rosslowe
Fix encoding for >1 byte characters
e906516 
- Properly encode >1 byte characters, closes #1 (adds text-encoding as a dependency)
- Add test for bytesToStr()
- Fix test for strToBytes(); add 3 byte character test
@microbit-carlos
Copy link
Collaborator Author

Fixed in #5 #6, thanks Ross!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@microbit-carlos