"Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8"

[ghost]

I've installed flutter on my linux distro (arch). I ran
flutter doctor

Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8.

Then I ran
sudo flutter doctor

[✓] Flutter (on Linux, channel master)
• Flutter at /home/albert/flutter
• Framework revision ca0201d (3 days ago), 2017-03-03 18:47:00
• Engine revision 342a311
• Tools Dart version 1.23.0-dev.2.0

[✗] Android toolchain - develop for Android devices
✗ Android SDK not found. Download from https://developer.android.com/sdk/
(or visit https://flutter.io/setup/#android-setup for detailed instructions).

[✓] Android Studio (version unknown)
• Android Studio at /opt/android-studio
• Gradle version 3.2

[✓] IntelliJ IDEA Community Edition (version 2016.3)
• Dart plugin version 163.13137
• Flutter plugin version 0.1.11.1

[✓] Connected devices
• SHIELD Tablet K1 • 0424916012921008a8b8 • android-arm • Android 7.0 (API 24)

I (think) have correctly installed the android sdk tools/platform-tools. I've set the path variables

echo $PATH

/home/albert/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/android-sdk/platform-tools:/opt/android-sdk/tools:/opt/android-sdk/tools/bin:/usr/lib/jvm/default/bin:/opt/opencascade/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/opt/android-sdk/tools:/opt/android-sdk/platform-tools:/home/albert/flutter/bin

yet nothing will change the little ✗ to a ✓

I've tried ignoring the doctor and just run the sample flutter app from my intellij editor, but I get the same
"Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8."

Going manually to the project directory and running
sudo flutter run

Unable to locate a development device; please run 'flutter doctor' for information about installing additional components.

[eseidelGoogle]

@johnmccutchan I'm not sure where flutter doctor might be crashing here, but it's possible --bug-report could collect the necessary information to diagnose. @tvolkert would know.

[tvolkert]

Yeah a bug report should have the info. @ajdewit1994 can you run flutter doctor --bug-report and attach the generated bugreport.zip? Note that it will contain info about your local environment, including paths in your file system, so you may want to inspect it to make sure there's nothing sensitive you wish to remove before attaching.

[ghost]

bugreport_01.zip
Here it is, thank you.

[tvolkert]

From a cursory look, it looks like it goes through the following process in attempting to find your Android SDK:

1. It looks in /root/Android/Sdk ($HOME is set to /root) and finds nothing
2. It looks for aapt in your $PATH and finds nothing
3. It looks for adb in your $PATH and finds one in /usr/?/? (I'm assuming /usr/local/bin)
4. It tries to validate that /usr is a valid Android SDK root, and that validation fails.

The quick fix may just be to put /opt/android-sdk/platform-tools and such before /usr/* in your PATH...

[tvolkert]

I also filed #8618 for fix this on the Flutter side.

[ghost]

Thank you very much @tvolkert and @eseidelGoogle

It works. I've prepended the android-sdk path to the PATH variable and I get the last remaining check. Good night!

[tvolkert]

@ajdewit1994 in the morning, if you're so inclined, could you run without sudo and reproduce the original "Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8." error? If so, if you could attach a bug report for that run as well, we could investigate that problem too.

Thanks!

[ghost]

Hello @tvolkert
It won't make a bug report without sudo. Maybe I've accidentally installed flutter while being logged in as root, I do that sometimes. I'll check later today.

[ghost]

I've redownloaded flutter, ran flutter doctor again, but in the end, it still gives me the Dart_NewStringFromUTF8 error.

[tvolkert]

The bug report will be written to your current directory, so try flutter doctor --bug-report again from a directory that you write access to.

[ghost]

bugreport_01.zip

Here it is.

[ghost]

Is this issue still active? I can't work with flutter anymore since the update where it's not possible anymore to run flutter with sudo. It's sort of blocking me now.

[tvolkert]

@ajdewit1994 sorry - I'll have a look today (I got distracted by another issue).

[tvolkert]

@ajdewit1994 it looks like the most recent bug report was still obtained while running as root. Can you do the following:

• git pull to get the latest code
• flutter doctor to see if you still get the "Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8." error
• If you do, flutter doctor --bug-report to grab a fresh bug report (not as root)

Thanks!

[ghost]

Hi @tvolkert. I hope I don't get on your nerves, was just wondering if anything had happened.

Unfortunately, it won't generate a bug report.

Don't stress it, I commented the "sudo check" out in the flutter file which allows me to start using flutter.

[tvolkert]

Ah the plot thickens! How about trying to run (not as root): flutter doctor --verbose and pasting the output here? Something's breaking before we're able to write out the bug report.

p.s. not getting on my nerves at all! :-)

[tvolkert]

Also: the terseness of that error message may be caused by #8742

[Hixie]

@tvolkert can a recording help us here?

[tvolkert]

See the last few comments - it's crashing before getting to write out a bug report, one possible reason for which would be a VM crash. I'm hoping a --verbose run might let us know approximately where this is happening.

@rmacnak-google do you know if there's a flag to flutter tools we can pass to enable VM trace statements to be logged to stdout?

[rmacnak-google]

Not stdout, but there's a VM flag --systrace_timeline that causes timeline events to go to systrace on Android/Linux. I don't think it will be very helpful here because that's an API error, not a Dart exception, so we really want the C backtrace when it happens.

[ghost]

The --verbose tag still gives me
Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8.

[Hixie]

With no other output at all? Fascinating!

Can you try going to the root of your flutter checkout and running the following? It will dump a LOT of output, if things go well. It may not dump much at all if they don't. It would be very useful to see what it does dump.

bin/cache/dart-sdk/bin/dart --checked --trace-loading --trace-compiler --print-stacktrace-at-throw --verbose packages/flutter_tools/bin/flutter_tools.dart doctor --verbose

[ghost]

Uh, ok. I don't have the bin/cache/ so I used the /opt/dart-sdk/bin/dat

Results in the following

http://pastebin.com/4gK8W734

[tvolkert]

It looks like you may have accidentally typed "/packages/..."

Basically, the "packages/flutter_tools/bin/flutter_tools.dart" should be the path to your flutter_tools.dart (either relative from your current directory, or absolute).

Can you try sending it again? :-)

[ghost]

Sorry for the late answer, I've been a bit busy lately.

@tvolkert https://gist.github.com/ajdewit1994/067b233b0c5b05375e2b96d9ce1d97b3

@Hixie https://gist.github.com/ajdewit1994/b37e5c764c08ba7b1d7aa056fcee2086
That's the result of ls -alR > contents.txt because it was so large.

[Hixie]

What's the full path to your project directory?

[tvolkert]

Just summarizing what I think we know at this point:

• Running as non-root...
  • you don't have a <flutter_root>/bin/cache directory, so you tried to use /opt/dart-sdk/bin/dart
  • with that dart binary, you get the "Dart_NewStringFromUTF8" error, with a trace report here. My guess is that we shouldn't bother trying to track this specific error down because since you're running an external Dart binary, it's not the version that Flutter expects, and all bets are off
• Running as root...
  • You get the directory structure listed here, and everything works, modulo the fact that we actively try to block you from running as root, and you had to locally hack around that restriction.

If all that I say above is correct, then here's where I propose we go from here:

1. Ignore the "Dart_NewStringFromUTF8" error for the reason I state above
2. Stop trying to diagnose any problems with you running as root, since it's not a supported path, and in the meantime, you're able to run as root as a temporary work-around
3. We try to figure out why you don't have a <flutter_root>/bin/cache directory when running as non-root.

Before we start talking solutions, does all sound about right?

[Hixie]

I don't understand how there could be a bin/cache directory as non-root but there is one as root.

[eseidelGoogle]

If the permissions/ownership on $FLUTTER_ROOT/bin don't allow creating cache that could happen. Especially if the flutter wrapper script doesn't catch/report a cache creation error.

[Hixie]

@ajdewit1994 Is your flutter clone when running as root in a different directory than the one when running as your main user, or are they the same directory?

[Hixie]

@tvolkert If I understand correctly, he's getting the UTF8 error even when he just runs "flutter doctor", which would be downloading Dart into bin/cache and using that, it wouldn't ever use the system local version.

[Hixie]

@ajdewit1994 To make further progress could you let us know:

• Is your flutter clone when running as root in a different directory than the one when running as your main user, or are they the same directory?

• What is the full path to your project directory when running as non-root?

Thanks!

[ghost]

Hello,

Thank you for your patience and sorry for the delay, I've been busy lately.

To answer the questions:

What's the full path to your project directory?

That's ~/Documents/flutter\ projects/test\ project/

" ... " Before we start talking solutions, does all sound about right?

Yes

@ajdewit1994 Is your flutter clone when running as root in a different directory than the one when running as your main user, or are they the same directory?

No, they are the same directory `~/flutter/

[Hixie]

@ajdewit1994 If you don't mind, can you give the full path? (i.e. expand the ~ part?) If you don't want to make that public you can e-mail it to me privately at ianh@google.com.

Can you paste the output of running ls -al . bin bin/* in ~/Documents/flutter\ projects/test\ project/?

Do you get the same problems if you move your project into ~/test ?

What happens if you run flutter doctor as you (not as root) but from inside ~/bin/flutter ?

[ghost]

the ~/ == /home/albert/

output of running ls -al . bin bin/* gives zsh: no matches found: bin/*

after copying the test\ project to the home folder of my user, I tried running flutter run again, same error, tried sudo flutter run and got it running on my tablet. Difference from previous tests, this time, the images in the test project are actually loaded onto the tablet (they didn't show previously).

To your last question, did you mean ~/flutter/bin? In that case, same error.

[Hixie]

Do you get the same failures if you run under bash instead of zsh?

Can you dump your entire environment? (output of env)

What is the output of cd ~/flutter && git fetch && git rebase && flutter doctor?

Thanks for still helping us try to track this down. I'm at an absolute loss as to what the problem could be.

[ghost]

Bash gives the same result

Output of env: https://gist.github.com/ajdewit1994/8001135d2309b93b1ffad92aebfd57c8

Output of cd ~/flutter && git fetch && git rebase && flutter doctor still gives the dart....UTF8 error.

[jason-simmons]

I can reproduce something like this by running Flutter inside a directory with a non-UTF8 name:

  python -c "import os; os.mkdir('a\x81')"
  cd a*
  git clone https://github.com/flutter/flutter.git
  cd flutter
  bin/flutter

Result:

Downloading Dart SDK 1.23.0-dev.11.7...
######################################################################## 100.0%
Building flutter tool...
Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8.

I'm not sure if this is the same issue though. If I run dart flutter_tools.dart on with --trace-compiler etc. all I get is:

Compiling function : 'dart:_builtin_::__getPrintClosure@13225868' @ token 118, size 7
--> 'dart:_builtin_::__getPrintClosure@13225868' entry: 0x7fdada54a0c0 size: 175 time: 389 us
Dart_NewStringFromUTF8 expects argument 'str' to be valid UTF-8.

I get the same output if I run any other Dart script from this directory.

[jason-simmons]

I looked into this some more, and it seems that Dart generally assumes that filesystem paths are in UTF-8. Dart's runtime libraries are taking results from getcwd()/readdir()/etc. and feeding them to Dart_NewStringFromUTF8.

In the --trace-compiler log in this bug, it looks like Dart is exiting from AndroidStudio._allLinuxOrWindows() in packages/flutter_tools/lib/src/android/android_studio.dart. In particular, it fails after getting the homeDirPath. The next thing we do after that is a listSync() on the homeDirPath.

@ajdewit1994 If you run ls -a $HOME is there anything with a name that is not UTF-8?

[Hixie]

I bet what's happening is we're (or probably, something in the core libraries is) looking for some file in your home directory, and when you run as root the home directory is a different directory without any non-UTF8 characters, which is why it runs fine as root.

[ghost]

Jesus, I feel dumb. You are right. A few months ago, I downloaded some stuff from a german university in a zip file, which, after I unzipped it, threw an encoding error. I didn't think much of it, threw it away since I couldn't read it. For some reason apparently, it puked something in my home folder.

I'm very sorry for the inconvenience. I didn't know that the source of error would be something like this.

It now runs flutter doctor without any trouble.

[Hixie]

@ajdewit1994 not your fault at all, we should not crash just because there's a badly encoded filename!

[tvolkert]

Ok, I filed dart-lang/sdk#29451 to track the source of the crash. I'm gonna close this issue out now.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.