Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vsxmake project doesn't like unicode characters #1689

Closed
SirLynix opened this issue Sep 21, 2021 · 11 comments
Closed

vsxmake project doesn't like unicode characters #1689

SirLynix opened this issue Sep 21, 2021 · 11 comments
Labels
bug improvement
Milestone
v2.5.8

Comments

@SirLynix
Copy link
Member

Describe the bug

I think there's two issues here:

First one: create a project with a unicode name (i'm using é here, which is common in french), generate a vsxmake project and:
image
("introuvable" means "not found" here).

Second: create a project with a regular name in a path containing a unicode character (for example: D:/xmakeunicode_é/project) and try to build it:

1>------ Début de la génération : Projet : xmakeunicode, Configuration : debug x64 ------
1>$xmake config -y  -P . -p windows -m debug -a x64 -o "build"
1>xmake.lua not found, scanning files ..
1>EXEC : error : project not found!
1>C:\Program Files\xmake\plugins\project\vsxmake\vsproj\Xmake.targets(98,5): error MSB3073: La commande "..." s'est arrêtée avec le code -1.

Building fails (project not found) as xmake works in another folder:
image

Expected behavior

That unicode characters are properly handled with Visual Studio.

Error output

Here's the log with verbose/diagnosis enabled:

L'opération de génération a démarré...
1>------ Début de la génération : Projet : xmakeunicode, Configuration : debug x64 ------
1>
1>MSBuild Properties:
1>  Xmake Props:
1>    XmakeTarget: xmakeunicode
1>    'XmakePlat|Mode|Arch': 'windows|debug|x64'
1>    XmakeBasename: xmakeunicode
1>    XmakeFilename: 
1>    XmakeKind: binary
1>    XmakeDefault: true
1>    XmakeCudaVersion: 
1>    XmakeWindowsSdkVersion: 
1>    XmakeMfcKind: 
1>    XmakeDefines: 
1>    XmakeLanguages: 
1>    XmakeSubSystem: console
1>    XmakeIncludeDirs: 
1>    XmakeLinkDirs: 
1>    XmakeSourceDirs: 
1>    XmakePrecompiledHeader: 
1>    XmakeRunEnvs: PATH=;C:\VulkanSDK\1.2.170.0\Bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Oculus\Support\oculus-runtime;C:\Program Files (x86)\Razer\ChromaBroadcast\bin;C:\Program Files\Razer\ChromaBroadcast\bin;C:\Program Files (x86)\Razer Chroma SDK\bin;C:\Program Files\Razer Chroma SDK\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\.cargo\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Lynix\AppData\Local\GitHubDesktop\bin;C:\Program Files\CMake\bin;C:\Program Files (x86)\Touch Portal\plugins\adb\platform-tools;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files\xmake;C:\Users\Lynix\.cargo\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Lynix\AppData\Local\GitHubDesktop\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\xmake;C:\Program Files\xmake;C:\Users\Lynix\.dotnet\tools
1>  Xmake Path:
1>    XmakeProgramDir: C:\Program Files\xmake
1>    XmakeProjectDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeScriptDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeBuilDir: D:\xmakeunicode_é\xmakeunicode_é\build
1>    XmakeTargetDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>    XmakeConfigDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeConfigFileDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>    XmakeRunDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>  Xmake Resolved Path:
1>    XmakeProgramDir: C:\Program Files\xmake\
1>    XmakeProjectDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeScriptDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeBuilDir: D:\xmakeunicode_é\xmakeunicode_é\build\
1>    XmakeTargetDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    XmakeConfigDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeConfigFileDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    XmakeRunDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>  Xmake Flags:
1>    XmakeWarning: true
1>    XmakeVerbose: true
1>    XmakeDiagnosis: true
1>    XmakeRebuildFile: false
1>    XmakeCleanAll: true
1>    XmakeCommonFlags: 
1>    XmakeConfigFlags: 
1>    XmakeBuildFlags: 
1>    XmakeCleanFlags: 
1>  Xmake Internal:
1>    _XmakeCommonFlags: -y -D -v -P .
1>    _XmakeConfigFlags: -p windows -m debug -a x64 -o "build"
1>    _XmakeBuildFlags: -w
1>    _XmakeBuildFileFlags: -w
1>    _XmakeCleanFlags: -a
1>    _XmakeExecutable: "C:\Program Files\xmake\xmake.exe"
1>    _XmakeEnv: 
1>      
1>      chcp 65001 > NUL
1>      pushd D:\xmakeunicode_é\xmakeunicode_é\
1>      set XMAKE_CONFIGDIR=D:\xmakeunicode_é\xmakeunicode_é
1>      set XMAKE_PROGRAM_DIR=C:\Program Files\xmake
1>      set XMAKE_SKIP_HISTORY=1
1>      set XMAKE_IN_VSTUDIO=1
1>    
1>      echo XMAKE_CONFIGDIR=%XMAKE_CONFIGDIR%
1>      echo XMAKE_PROGRAM_DIR=%XMAKE_PROGRAM_DIR%
1>      echo CD=%CD%
1>      chcp
1>    
1>  Vs Props:
1>    'Configuration|Platform': 'debug|x64'
1>    VisualStudioVersion: 16.0
1>    MSBuildProjectDirectory: D:\xmakeunicode_é\xmakeunicode_é\vsxmake2019\xmakeunicode
1>    MSBuildProjectName: xmakeunicode
1>    OutDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    IntDir: D:\xmakeunicode_é\xmakeunicode_é\build\.vs\xmakeunicode\x64\debug\
1>    SourcePath: ;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\mfc;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\mfcm;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\atl;;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\crt\src;;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\src;;C:\Program Files (x86)\Windows Kits\10\Source\10.0.18362.0\ucrt;;;
1>    TargetName: xmakeunicode
1>    TargetExt: .exe
1>    PlatformToolset: v142
1>    CharacterSet: MultiByte
1>    UseOfMfc: false
1>    WindowsTargetPlatformVersion: 10.0
1>    PreprocessorDefinitions: _MBCS;;
1>    LanguageStandard: Default
1>    SelectedFiles: 
1>    VCTargetsPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\
1>$xmake config -y -D -v -P . -p windows -m debug -a x64 -o "build"
1>XMAKE_CONFIGDIR=D:\xmakeunicode_�\xmakeunicode_�
1>XMAKE_PROGRAM_DIR=C:\Program Files\xmake
1>CD=D:\xmakeunicode_�\xmakeunicode_�
1>Page de codes active�: 65001
1>xmake.lua not found, scanning files ..
1>EXEC : error : @programdir\core\main.lua:279: @programdir\actions\config\scangen.lua:67: project not found!
1>stack traceback:
1>    [C]: in function 'error'
1>    [@programdir\core\base\os.lua:829]: in function 'raise'
1>    [@programdir\actions\config\scangen.lua:67]: in function 'scangen'
1>    [@programdir\actions\config\main.lua:262]:
1>    [C]: in function 'load'
1>    [@programdir\core\base\task.lua:519]: in function 'run'
1>    [@programdir\core\main.lua:277]: in function 'cotask'
1>    [@programdir\core\base\scheduler.lua:365]:
1>
1>stack traceback:
1>	[C]: in function 'error'
1>	@programdir\core\base\os.lua:829: in function 'raise'
1>	@programdir\core\main.lua:279: in function 'cotask'
1>	@programdir\core\base\scheduler.lua:365: in function <@programdir\core\base\scheduler.lua:362>
1>C:\Program Files\xmake\plugins\project\vsxmake\vsproj\Xmake.targets(98,5): error MSB3073: La commande "..." s'est arrêtée avec le code -1.
1>Génération du projet "xmakeunicode.vcxproj" terminée -- ÉCHEC.
========== Génération : 0 a réussi, 1 a échoué, 0 à jour, 0 a été ignoré ==========

We see that both XMAKE_CONFIGDIR and CD are set to wrong path with invalid unicode characters.

Related Environment

  • xmake version: xmake v2.5.7
  • os: Windows 10
  • target platform:

Additional context

The project doesn't matter much, I used a default project created using xmake create
@waruqi
Copy link
Member

waruqi commented Sep 21, 2021

Can you give me your *.vcproj files?

@SirLynix
Copy link
Member Author

Unfortunately GitHub doesn't allow me to put it here as a .vcxproj so I renamed its extension to .txt
xmakeunicode_é.txt

@waruqi
Copy link
Member

waruqi commented Sep 22, 2021

This is because vcproj and sln files need utf8 encoding with bom to support unicode characters. I tried to improve it, you try again.

xmake update dev

BTW, I have switched to lua5.3 runtime on dev.

@waruqi waruqi mentioned this issue Sep 22, 2021
Closed
@waruqi waruqi added this to the v2.5.8 milestone Sep 22, 2021
@SirLynix
Copy link
Member Author

This fixed the first issue, I can now make project with unicode names which VS will recognize.

However, the second issue is still there:

L'opération de génération a démarré...
1>------ Début de la génération : Projet : xmakeunicode, Configuration : debug x64 ------
1>
1>MSBuild Properties:
1>  Xmake Props:
1>    XmakeTarget: xmakeunicode
1>    'XmakePlat|Mode|Arch': 'windows|debug|x64'
1>    XmakeBasename: xmakeunicode
1>    XmakeFilename: 
1>    XmakeKind: binary
1>    XmakeDefault: true
1>    XmakeCudaVersion: 
1>    XmakeWindowsSdkVersion: 
1>    XmakeMfcKind: 
1>    XmakeDefines: 
1>    XmakeLanguages: 
1>    XmakeSubSystem: console
1>    XmakeIncludeDirs: 
1>    XmakeLinkDirs: 
1>    XmakeSourceDirs: 
1>    XmakePrecompiledHeader: 
1>    XmakeRunEnvs: PATH=;C:\VulkanSDK\1.2.170.0\Bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Oculus\Support\oculus-runtime;C:\Program Files (x86)\Razer\ChromaBroadcast\bin;C:\Program Files\Razer\ChromaBroadcast\bin;C:\Program Files (x86)\Razer Chroma SDK\bin;C:\Program Files\Razer Chroma SDK\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\.cargo\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Lynix\AppData\Local\GitHubDesktop\bin;C:\Program Files\CMake\bin;C:\Program Files (x86)\Touch Portal\plugins\adb\platform-tools;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files\xmake;C:\Users\Lynix\.cargo\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Lynix\AppData\Local\GitHubDesktop\bin;C:\Users\Lynix\AppData\Local\Microsoft\WindowsApps;C:\Users\Lynix\xmake;C:\Program Files\xmake;C:\Users\Lynix\.dotnet\tools
1>  Xmake Path:
1>    XmakeProgramDir: C:\Program Files\xmake
1>    XmakeProjectDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeScriptDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeBuilDir: D:\xmakeunicode_é\xmakeunicode_é\build
1>    XmakeTargetDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>    XmakeConfigDir: D:\xmakeunicode_é\xmakeunicode_é
1>    XmakeConfigFileDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>    XmakeRunDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug
1>  Xmake Resolved Path:
1>    XmakeProgramDir: C:\Program Files\xmake\
1>    XmakeProjectDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeScriptDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeBuilDir: D:\xmakeunicode_é\xmakeunicode_é\build\
1>    XmakeTargetDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    XmakeConfigDir: D:\xmakeunicode_é\xmakeunicode_é\
1>    XmakeConfigFileDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    XmakeRunDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>  Xmake Flags:
1>    XmakeWarning: true
1>    XmakeVerbose: true
1>    XmakeDiagnosis: true
1>    XmakeRebuildFile: false
1>    XmakeCleanAll: true
1>    XmakeCommonFlags: 
1>    XmakeConfigFlags: 
1>    XmakeBuildFlags: 
1>    XmakeCleanFlags: 
1>  Xmake Internal:
1>    _XmakeCommonFlags: -y -D -v -P .
1>    _XmakeConfigFlags: -p windows -m debug -a x64 -o "build"
1>    _XmakeBuildFlags: -w
1>    _XmakeBuildFileFlags: -w
1>    _XmakeCleanFlags: -a
1>    _XmakeExecutable: "C:\Program Files\xmake\xmake.exe"
1>    _XmakeEnv: 
1>      
1>      chcp 65001 > NUL
1>      pushd D:\xmakeunicode_é\xmakeunicode_é\
1>      set XMAKE_CONFIGDIR=D:\xmakeunicode_é\xmakeunicode_é
1>      set XMAKE_PROGRAM_DIR=C:\Program Files\xmake
1>      set XMAKE_SKIP_HISTORY=1
1>      set XMAKE_IN_VSTUDIO=1
1>    
1>      echo XMAKE_CONFIGDIR=%XMAKE_CONFIGDIR%
1>      echo XMAKE_PROGRAM_DIR=%XMAKE_PROGRAM_DIR%
1>      echo CD=%CD%
1>      chcp
1>    
1>  Vs Props:
1>    'Configuration|Platform': 'debug|x64'
1>    VisualStudioVersion: 16.0
1>    MSBuildProjectDirectory: D:\xmakeunicode_é\xmakeunicode_é\vsxmake2019\xmakeunicode
1>    MSBuildProjectName: xmakeunicode
1>    OutDir: D:\xmakeunicode_é\xmakeunicode_é\build\windows\x64\debug\
1>    IntDir: D:\xmakeunicode_é\xmakeunicode_é\build\.vs\xmakeunicode\x64\debug\
1>    SourcePath: ;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\mfc;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\mfcm;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\src\atl;;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\crt\src;;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\src;;C:\Program Files (x86)\Windows Kits\10\Source\10.0.18362.0\ucrt;;;
1>    TargetName: xmakeunicode
1>    TargetExt: .exe
1>    PlatformToolset: v142
1>    CharacterSet: MultiByte
1>    UseOfMfc: false
1>    WindowsTargetPlatformVersion: 10.0
1>    PreprocessorDefinitions: _MBCS;;
1>    LanguageStandard: Default
1>    SelectedFiles: 
1>    VCTargetsPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\
1>$xmake config -y -D -v -P . -p windows -m debug -a x64 -o "build"
1>XMAKE_CONFIGDIR=D:\xmakeunicode_�\xmakeunicode_�
1>XMAKE_PROGRAM_DIR=C:\Program Files\xmake
1>CD=D:\xmakeunicode_�\xmakeunicode_�
1>Page de codes active�: 65001
1>xmake.lua not found, scanning files ..
1>EXEC : error : @programdir\core\main.lua:280: @programdir\actions\config\scangen.lua:67: project not found!
1>stack traceback:
1>    [C]: in function 'error'
1>    [@programdir\core\base\os.lua:829]:
1>    [@programdir\actions\config\scangen.lua:67]:
1>    [@programdir\actions\config\main.lua:262]:
1>    [C]: in function 'xpcall'
1>    [@programdir\core\base\utils.lua:280]:
1>    [@programdir\core\base\task.lua:519]: in function 'run'
1>    [@programdir\core\main.lua:278]: in function 'cotask'
1>    [@programdir\core\base\scheduler.lua:371]:
1>
1>stack traceback:
1>	[C]: in function 'error'
1>	@programdir\core\base\os.lua:829: in function 'base/os.raiselevel'
1>	(...tail calls...)
1>	@programdir\core\main.lua:280: in upvalue 'cotask'
1>	@programdir\core\base\scheduler.lua:371: in function <@programdir\core\base\scheduler.lua:368>
1>C:\Program Files\xmake\plugins\project\vsxmake\vsproj\Xmake.targets(98,5): error MSB3073: La commande "..." s'est arrêtée avec le code -1.
1>Génération du projet "xmakeunicode.vcxproj" terminée -- ÉCHEC.
========== Génération : 0 a réussi, 1 a échoué, 0 à jour, 0 a été ignoré ==========

I checked the encoding of the .sln and .vcxproj files, they are in UTF-8 with BOM as expected.

@waruqi
Copy link
Member

waruqi commented Sep 22, 2021

I removed chcp 65001 and it works now. please try it again.

    <_XmakeExecutable>"$([System.IO.Path]::GetFullPath('$(XmakeProgramDirResolved)xmake.exe'))"</_XmakeExecutable>
    <_XmakeEnv>
-      chcp 65001 &gt; NUL
      pushd $(XmakeProjectDirResolved)
      set XMAKE_CONFIGDIR=$(XmakeConfigDirResolved.TrimEnd('\').TrimEnd('/'))
      set XMAKE_PROGRAM_DIR=$(XmakeProgramDirResolved.TrimEnd('\').TrimEnd('/'))
@@ -77,7 +76,6 @@
      echo XMAKE_CONFIGDIR=%25XMAKE_CONFIGDIR%25
      echo XMAKE_PROGRAM_DIR=%25XMAKE_PROGRAM_DIR%25
      echo CD=%25CD%25
-      chcp
    </_XmakeEnv>
  </PropertyGroup>

@OpportunityLiu I don't know if the purpose of adding chcp 65001 was also to support unicode, but from the current case, if I remove it, it can support unicode well.

I'm not sure whether it will have other effects.

@SirLynix
Copy link
Member Author

I can confirm it works now, even if unicode output in VS console is still broken (but this is not really a big deal). Thanks a lot.

I'm not sure if I should close this issue now but as far as I'm concerned it works.

@waruqi
Copy link
Member

waruqi commented Sep 22, 2021

I closed it first, if there is still a problem, please reopen it.

@waruqi waruqi closed this as completed Sep 22, 2021
@OpportunityLiu
Copy link
Member

I can confirm it works now, even if unicode output in VS console is still broken (but this is not really a big deal). Thanks a lot.

@waruqi That's the reason chcp 65001 added for

@waruqi
Copy link
Member

waruqi commented Sep 23, 2021 •
edited
Loading

I can confirm it works now, even if unicode output in VS console is still broken (but this is not really a big deal). Thanks a lot.

@waruqi That's the reason chcp 65001 added for

Got it, but I don't have any other way to fix it.

@waruqi
Copy link
Member

waruqi commented Jun 3, 2022

VS console is still broken (but this is not really a big deal).

Can you try this patch? #2431 @SirLynix

@SirLynix
Copy link
Member Author

SirLynix commented Jun 3, 2022

I found no issue with this patch and it does fixes VS unicode character support!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug improvement
Projects
None yet
Milestone
v2.5.8
Development

No branches or pull requests
3 participants
@waruqi @SirLynix @OpportunityLiu