io_win32 does not handle non-ascii chars in paths.

[ml232528]

protoc.exe --cpp_out=. ./helloworld.proto
If the path is in Chinese, this problem will arise.
The GetCurrentDirectoryA function returns the GB18030 encoding
Using MultiByteToWideChar (CP_UTF8, 0, s.c_str (), s.size (), result.get (), len + 1) error ，CP_ACP should be used。

file ：src\google\protobuf\stubs\io_win32.cc

[xfxyjwf]

Can you avoid non-ASCII characters in your dir path? The protobuf code base assumes UTF-8 for char encoding so it likely doesn't work well with non-ASCII characters in non-UTF-8 environment.

[liujisi]

@laszlocsomor Could you please take a look at this?

[laszlocsomor]

I can take a look later today, have another important bug on my plate.

[laszlocsomor]

Indeed, this is a bug, I could repro it, and it works on Linux and doesn't on Windows.
Let me take a closer look.

[laszlocsomor]

@pherl : I can't self-assign this bug. Could you assign it to me please?

[liujisi]

Hmm, looks like I can only assign the issue to one of the owners of the project.

BTW, what do you think about the other issue about wildcast no longer works on windows protoc (#3957)?

[raphael3207]

3

[laszlocsomor]

@pherl : I'm still trying to understand whether the other issue is related to this one or not. Maybe you can tell this: do you know how the CLI expands wildcard patterns to a list of filenames? (I could figure it out but asking is easier. ;)

Also, it seems that already the main method receives a bad octet-stream for non-ascii charcters.
I modified https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/main.cc to print the arguments as string and as octet-stream. Even though I seem to be able to write non-ascii text into cmd.exe, the protoc.exe's main method already receives question marks.

C:\tempdir>protoc.exe --cpp_out c:\tempdir\cyr\Привет\foo hello_there.proto
DEBUG[main] argv=4
DEBUG[main] argv[0]=(protoc.exe)
  0x70 0x72 0x6f 0x74 0x6f 0x63 0x2e 0x65 0x78 0x65
DEBUG[main] argv[1]=(--cpp_out)
  0x2d 0x2d 0x63 0x70 0x70 0x5f 0x6f 0x75 0x74
DEBUG[main] argv[2]=(c:\tempdir\cyr\??????\foo)
  0x63 0x3a 0x5c 0x74 0x65 0x6d 0x70 0x64 0x69 0x72 0x5c 0x63 0x79 0x72 0x5c 0x3f 0x3f 0x3f 0x3f 0x3f 0x3f 0x5c 0x66 0x6f 0x6f
DEBUG[main] argv[3]=(hello_there.proto)
  0x68 0x65 0x6c 0x6c 0x6f 0x5f 0x74 0x68 0x65 0x72 0x65 0x2e 0x70 0x72 0x6f 0x74 0x6f
c:\tempdir\cyr\??????\foo/: Invalid argument

[laszlocsomor]

For the record, my edit is:

$ git d
diff --git a/src/google/protobuf/compiler/main.cc b/src/google/protobuf/compiler/main.cc
index 680d642..78d32c7 100644
--- a/src/google/protobuf/compiler/main.cc
+++ b/src/google/protobuf/compiler/main.cc
@@ -48,6 +48,14 @@
 #endif  // ! OPENSOURCE_PROTOBUF_CPP_BOOTSTRAP

 int main(int argc, char* argv[]) {
+  fprintf(stdout, "DEBUG[main] argv=%d\n", argc);
+  for (int i = 0; i < argc; ++i) {
+    fprintf(stdout, "DEBUG[main] argv[%d]=(%s)\n ", i, argv[i]);
+    for (char* p = argv[i]; *p; ++p) {
+      fprintf(stdout, " 0x%2x", *p);
+    }
+    fputc('\n', stdout);
+  }

   google::protobuf::compiler::CommandLineInterface cli;
   cli.AllowPlugins("protoc-");

[laszlocsomor]

FYI, I have a bugfix that I'll send a PR for shortly, that fixes the original issue in this thread:

C:\tempdir\cyr\Привет\foo>dir /b foo

C:\tempdir\cyr\Привет\foo>protoc-dev.exe --cpp_out foo hello.proto

C:\tempdir\cyr\Привет\foo>dir /b foo
hello.pb.cc
hello.pb.h

[laszlocsomor]

Created a PR with my bugfix that I mentioned in my previous comment: #3978