The changes of `symbol.tsv` is not applied until cleaning (Windows)

[tats-u]

Description

A clear and concise description of what the bug is.

I'm trying to modify src\data\symbol\symbol.tsv, but its changes are difficult to be reflected because they require us to clean the build.

Steps to reproduce

Steps to reproduce the behavior:

1. python build_mozc.py gyp
2. python build_mozc.py build -c Release package
3. Edit src\data\symbol\symbol.tsv
4. python build_mozc.py build -c Release package
5. Install the generated Mozc64.msi & Enable Mozc
6. Try the change

Expected behavior

A clear and concise description of what you expected to happen.

The change is applied

Actual behavior

A clear and concise description of what you actually got.

The change is not applied

Screenshots

If applicable, add screenshots to help explain your problem.

Version or commit-id

[e.g. Mozc-2.28.4960.100+24.11.oss or d50a8b9]
You can get the version string by converting "Version" or "ばーじょん".

6ba364e (master)

Environment

• OS: [e.g. Ubuntu 20.04, macOS 13.1, etc]
• IMF (for Linux) [e.g. Ibus, Fcitx, etc]
• Related Applications [e.g. Chromium 109, gedit, etc]

Windows 11 (on Windows Sandbox)
(Notepad)

Investigations

• Whether this issue happens on both Ibus and other IMF (e.g. Fcitx, uim).
  • yes, no, N/A
• Whether this issue happens on other IMEs (e.g. Anthy, SKK).
  • yes, no, N/A
• What applications this issue happens on (e.g. Chromium, gedit).
  • list of applications
• What applications this issue does not happen on (e.g. Chromium, gedit).
  • list of applications, or not-found
• (optional) What versions or commit-ids this issue did not happen on
  (e.g. Mozc-2.28.4960.100+24.11.oss).
  • list of versions or commit-ids, or no-investigation.

Additional context

Add any other context about the problem here.

We have to rebuild after cleaning once (python build_mozc.py clean) to apply the changes in symbol.tsv.

Solutions:

• Migrate to Bazel completely (Use bazel to build Mozc for Windows #948)
• Fix GYP (will not done by maintainers)

[hiroyuki-komatsu]

Hi tats-u,
Thank you for your report.

I guess we might need to modify data_manager/data_manager.gypi or data_manager/oss/oss_data_manager_base.gyp (or other .gyp / .gypi files).

https://github.com/google/mozc/blob/master/src/data_manager/oss/oss_data_manager_base.gyp
https://github.com/google/mozc/blob/master/src/data_manager/data_manager_base.gypi

Since we are migrating the build system from GYP to Bazel (#948), we would not address this limitation.
If you are willing to fix it, we would appreciate it and review your PR.

Best,

[tats-u]

@hiroyuki-komatsu Will Mozc64.msi be built by Bazel in the future, won't it?
If so I think I can wait for it and will link this issue to that one.

[hiroyuki-komatsu]

Yes, we want to build .msi files with Bazel in the future.

[tats-u]

I see.

[hiroyuki-komatsu]

We have supported Bazel build for Windows.

Please check the instruction.
https://github.com/google/mozc/blob/master/docs/build_mozc_in_windows.md#build-with-bazel-experimental

I will close this issue in a week if no issues are found.
Thank you,

[tats-u]

Without [Console]::OutputEncoding = [Console]::InputEncoding = [Text.Encoding]::UTF8, I got:

INFO: From Compiling absl/random/internal/randen_round_keys.cc [for tool]:
randen_round_keys.cc
INFO: From Compiling absl/flags/internal/commandlineflag.cc [for tool]:
commandlineflag.cc
ERROR: I:/mozc/src/gui/base/BUILD.bazel:74:12: Executing genrule //gui/base:base_files failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:base_files) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/moc.exe -p $(dirname gui/base/window_title_modifier.h) -o bazel-out/x64_windows-opt/bin/gui/base/moc_window_title_modifier.cc gui/base/window_title_modifier.h"): 謖・ｮ壹＆繧後◆繝輔ぃ繧､繝ｫ縺瑚ｦ九▽縺九ｊ縺ｾ縺帙ｓ縲・
 (error: 2)
Target //:package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 35.236s, Critical Path: 1.11s
INFO: 1346 processes: 1315 internal, 31 local.

PS> java --version
openjdk 11.0.23 2024-04-16 LTS
OpenJDK Runtime Environment Microsoft-9394293 (build 11.0.23+9-LTS)
OpenJDK 64-Bit Server VM Microsoft-9394293 (build 11.0.23+9-LTS, mixed mode, sharing)

PS> gcm java -all | select Version,Source

Version   Source
-------   ------
11.0.23.0 C:\Program Files\Microsoft\jdk-11.0.23.9-hotspot\bin\java.exe
21.0.3.0  C:\Program Files\Eclipse Adoptium\jdk-21.0.3.9-hotspot\bin\java.exe

[tats-u]

Even with [Console]::OutputEncoding = [Console]::InputEncoding = [Text.Encoding]::UTF8 I got:

INFO: From Compiling base/win32/hresult.cc:
hresult.cc
ERROR: I:/mozc/src/gui/base/BUILD.bazel:85:12: Executing genrule //gui/base:qrc_tr failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:qrc_tr) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/rcc.exe -o bazel-out/x64_windows-opt/bin/gui/base/qrc_tr.cc -name qrc_tr gui/base/tr.qrc"): 指定されたファイルが見つかりません。
 (error: 2)

Note: not yet terminated by this error.

I wonder I should remove the stale Java 11.

PS. stucked at:

[2,264 / 4,377] Compiling transliteration/transliteration.cc; 468s local

Bazel Ctrl+C handler; shutting down.
[2,264 / 4,377] Compiling transliteration/transliteration.cc; 517s local

[tats-u]

After removing Java 11 and git clean -f, stucked with Extracting Bazel installation....

[yukawa]

ERROR: I:/mozc/src/gui/base/BUILD.bazel:85:12: Executing genrule //gui/base:qrc_tr failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:qrc_tr) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/rcc.exe -o bazel-out/x64_windows-opt/bin/gui/base/qrc_tr.cc -name qrc_tr gui/base/tr.qrc"): 指定されたファイルが見つかりません。
 (error: 2)

Most likely this error means bazel failed to find bash.exe in your system. Check out the following section for details.

• https://bazel.build/install/windows#bazel_does_not_find_bash_or_bashexe

[tats-u]

I see. We should add the link to https://bazel.build/install/windows?hl=ja#install-compilers in the Wiki page.
Should Java be 11, not 21? Java 11 is more than 5 years old.

[yukawa]

Should Java be 11, not 21? Java 11 is more than 5 years old.

My understanding is that recent versions of Bazel (or bazelisk) download JRE automatically, so you in general don't need to install Java by yourself.

Note also that we will definitely make sure that our Bazel build continues to succeed on the GitHub Actions. Setting up an environment similar enough to it may help.

• https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md

mozc/.github/workflows/windows.yaml

Lines 77 to 133 in ace3145

	build_bazel:
	# https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md
	runs-on: windows-2022
	timeout-minutes: 120
	steps:
	- name: Checkout
	uses: actions/checkout@v4
	with:
	submodules: 'recursive'
	- name: Set up pip
	shell: cmd
	working-directory: .\src
	run: |
	python -m pip install six requests
	- name: Try to restore update_deps cache
	uses: actions/cache@v4
	with:
	path: src/third_party_cache
	key: update_deps-${{ runner.os }}-${{ hashFiles('src/build_tools/update_deps.py') }}
	- name: Install Dependencies
	shell: cmd
	working-directory: .\src
	# This command uses src/third_party_cache as the download cache.
	run: |
	python build_tools/update_deps.py
	- name: Biuld Qt
	shell: cmd
	working-directory: .\src
	run: |
	python build_tools/build_qt.py --release --confirm_license
	# This is needed only for GitHub Actions where free disk space is quite limited.
	- name: Delete Qt src to save disk space
	shell: cmd
	working-directory: .\src
	run: |
	rmdir third_party\qt_src /s /q
	- name: build artifacts
	shell: cmd
	working-directory: .\src
	env:
	ANDROID_NDK_HOME: ""
	run: |
	bazel --bazelrc=windows.bazelrc build --config oss_windows --config release_build package
	- name: upload Mozc64_bazel.msi
	uses: actions/upload-artifact@v4
	with:
	name: Mozc64_bazel.msi
	path: src/bazel-bin/win32/installer/Mozc64.msi
	if-no-files-found: warn

[tats-u]

I tried MSYS2 via WinGet but it wasn't be recognized.
I had to reinstall it using Scoop.

However, I confirmed the fix of this issue.