Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run code directly from dev server cause Non-Ascii characters garbled #13

Closed
ITJesse opened this issue Aug 15, 2019 · 17 comments · Fixed by #17
Labels
bug

Comments

@ITJesse
Copy link

@ITJesse ITJesse commented Aug 15, 2019

image

@RageBill

This comment has been minimized.

Copy link

@RageBill RageBill commented Aug 15, 2019

I have the same problem when running in dev environment. However, once I have my debugger turned on, the app runs normally. This is a very weird error regarding Non-ASCII characters.

@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Aug 15, 2019

@RageBill When you turn the debugger on, the code is running on the debugger, not on your phone.

@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 15, 2019

@ITJesse For the broken text, do you have any special properties, like custom font?
How about to share some code here?
E.g.

<Text style={{ fontSize: 24 }}>Non-ASCII, e.g. 天気</Text>

And it looks like the text on your UI bottom sheet is correct, what is the difference between these text?

At last, what is the React Native version you used?

@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Aug 15, 2019

@Kudo I am using styled-components for styling.
The bottom sheet is a part of native, not RN.

React Native: 0.59.10

@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Aug 15, 2019

I can confirm that every Non-ASCII character is broken.

@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 15, 2019

I've tried on 0.59.10 but works for me.
Let me try styled-components.

@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 15, 2019

@ITJesse
This works for me.

import styled from 'styled-components/native';

const StyledText = styled.Text`
  color: palevioletred;
`;

type Props = {};
export default class App extends Component<Props> {
  render() {
    return (
      <View style={styles.container}>
        <StyledText>天気</StyledText>
      </View>
    );
  }
}
@RageBill

This comment has been minimized.

Copy link

@RageBill RageBill commented Aug 16, 2019

Not sure if it helps, in my scenario, it works fine on my production build and on React Native Debugger. However, it just doesn't work on development build on my Android emulator. The broken characters in my case are Chinese (I think OP's ones are Japanese) and these Chinese characters were returned from an API call.

My RN version is 0.60.4 and my Android emulator is API 26 Nexus 5X.

@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Aug 16, 2019

@Kudo Thanks for your help. I will do more tests on how to reproduce this problem this weekend.

@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 17, 2019

I've also tested an API returned text and seems works for me.
Looking forward to seeing if there are reproducible test cases.

const StyledText = styled.Text`
  color: palevioletred;
  font-size: 24px;
`;

type Props = {};
export default class App extends Component<Props> {
  state = {
    text: '',
  };

  componentDidMount() {
    fetch(
      'https://postman-echo.com/get?text=%E4%B8%AD%E6%96%87%E5%A4%A9%E6%B0%97',
    )
      .then(resp => resp.json())
      .then(resp => {
        this.setState({text: resp.args.text});
      });
  }

  render() {
    return (
      <View style={styles.container}>
        <Text>
          {global._v8runtime != null ? global._v8runtime().version : 'Not V8'}
        </Text>
        <StyledText>{this.state.text}</StyledText>
      </View>
    );
  }
}
@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Aug 20, 2019

I still cannot figure out why the Non-Ascii is broken in some of my projects.
I will close this issue now, and keep update if I found the reason.

@ITJesse ITJesse closed this Aug 20, 2019
@t0m0120

This comment has been minimized.

Copy link

@t0m0120 t0m0120 commented Aug 20, 2019

@ITJesse

Only debugging is same issue.

Template literal may be the cause.

const Text = styled.text``

<Text>{`日本語テキスト\n↑JapaneseText`}</Text> // garbled 
<Text>日本語テキスト\n↑JapaneseText</Text> // good
template literals not template literals
image image
@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 20, 2019

Thanks @t0m0120, but I am still unable reproduce the problem even for template literal.

Let's trying to dig out more information.

  1. Is the problem reproducible on Android emulator or only on real device?
  2. What's your Android system language?
  3. What's is your react-native version and react-native-v8 version? Please check the real version from yarn.lock.
  4. On the same device which had the problem, could you reproduce the problem for a simple test case?
    E.g. react-native init RN059 --version react-native@^0.59.10, adding react-native-v8 and use my sample code above.
  5. Verify the text from bundle, is the text a normal UTF-8 text? (Download bundle from wget 'http://localhost:8081/index.bundle?platform=android&dev=true&minify=false')

@ITJesse As there are people reported such problems, I am going to leave this issue opened and hopefully we could find the root cause and fix it.

@Kudo Kudo reopened this Aug 20, 2019
@Kudo Kudo added the bug label Aug 20, 2019
Kudo added a commit that referenced this issue Aug 29, 2019
`v8::String::ExternalOneByteStringResource` does not support UTF-8 string.
(V8 supports UTF-16 and Latin-1 only).
Instead of converting UTF-8 to UTF-16 from V8Runtime, intent to use
`v8::String::NewFromUtf8()` existing API.
The drawback is one time buffer copy, but it seems we don't have any
other alternatives.
Fixes #13 #14 #16
@Kudo Kudo closed this in #17 Aug 29, 2019
Kudo added a commit that referenced this issue Aug 29, 2019
`v8::String::ExternalOneByteStringResource` does not support UTF-8 string.
(V8 supports UTF-16 and Latin-1 only).
Instead of converting UTF-8 to UTF-16 from V8Runtime, intent to use
`v8::String::NewFromUtf8()` existing API.
The drawback is one time buffer copy, but it seems we don't have any
other alternatives.
Fixes #13 #14 #16
Kudo added a commit that referenced this issue Aug 29, 2019
`v8::String::ExternalOneByteStringResource` does not support UTF-8 string.
(V8 supports UTF-16 and Latin-1 only).
Instead of converting UTF-8 to UTF-16 from V8Runtime, intent to use
`v8::String::NewFromUtf8()` existing API.
The drawback is one time buffer copy, but it seems we don't have any
other alternatives.
Fixes #13 #14 #16

(cherry picked from commit f6b452b)
@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Aug 29, 2019

Publish react-native-v8 0.59.10-patch.2, 0.60.4-patch.2, 0.60.5-patch.0 which all include #17 fix.
Please choose a version to see if the new version did help for the issue.
Thanks for all your help.

@t0m0120

This comment has been minimized.

Copy link

@t0m0120 t0m0120 commented Sep 3, 2019

@Kudo
try 0.59.10-patch.2
No longer garbled in debug mode
Thank you !!

@Kudo

This comment has been minimized.

Copy link
Owner

@Kudo Kudo commented Sep 3, 2019

@t0m0120 Awesome! Thanks for your confirmation.

@ITJesse

This comment has been minimized.

Copy link
Author

@ITJesse ITJesse commented Sep 11, 2019

@Kudo Thanks for your awesome work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.