Cross-platform compatibility fixes for Windows and macOS - #43
Merged
Conversation
Path detection in Config._read_option was gated on sys.platform: the Windows branch only recognized values containing a backslash, but the .ini config files use forward slashes (e.g. ~/.ivert). On Windows those values fell through path handling entirely, leaving ~ unexpanded. Replace the two platform-specific branches with a single branch that treats a value as a path if it contains ~, /, or \, and add an _is_absolute_path helper recognizing Unix, Windows drive, and UNC roots regardless of the running OS.
signal.SIGKILL does not exist on Windows, so evaluating abs(exitcode) == abs(signal.SIGKILL) raised AttributeError whenever a validation subprocess exited non-zero. Look the attribute up with getattr and skip the OOM-killer recovery branch when it is absent: Windows has no SIGKILL and no equivalent OOM-killer, so a failed worker now surfaces as the intended RuntimeError instead of crashing. POSIX behavior is unchanged.
The bcolors utility emitted raw ANSI escape codes, which render on Linux and macOS but not on Windows consoles, where virtual-terminal processing is off by default and the codes appear as literal garbage. Enable ANSI/VT processing on import via colorama.just_fix_windows_console(), which targets modern Windows terminals and is a no-op elsewhere. It only sets the console-mode flag and does not wrap sys.stdout, so it does not interfere with loggerproc.Logger's stdout redirection (which already strips these codes from log files). The import is defensive so Linux/macOS, where colorama is unnecessary, degrade to native ANSI rendering. Add colorama as an explicit Windows-only dependency.
mp.set_start_method('spawn') raises RuntimeError if the start method was
already set in the process. Pass force=True in both validation entry
points so re-invocation (or both entry points running in one process)
cannot crash.
Text-mode open() defaults to the platform locale encoding (e.g. cp1252 on Windows) rather than utf-8, which can corrupt non-ASCII content and differ across platforms. Pass encoding='utf-8' explicitly on all text-mode opens (empty-result markers, summary stats, user config, photon-tile lists, the AWS datasource probe, and the process logger). Binary opens in pickle_blosc.py are left unchanged.
Collaborator
Author
Issue #10 progress (refs #10 — does not close it)This PR is part of the ongoing cross-platform work for #10. It lands the code-level fixes; the issue stays open pending on-hardware testing. Done in this PR (5 fixes, Linux behavior unchanged):
Still open under #10 (not in this PR):
|
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a set of cross-platform compatibility issues so IVERT runs on modern Windows and macOS as well as Linux. Each fix preserves existing Linux behavior. No functional change on Linux.
Fixes
Config path resolution (
configfile.py) — Path detection was gated onsys.platform: the Windows branch only recognized values containing a backslash, but the.inifiles use forward slashes (e.g.~/.ivert). On Windows those values fell through unhandled, leaving~unexpanded. Replaced the two platform-specific branches with one platform-agnostic branch (treats a value as a path if it contains~,/, or\) plus an_is_absolute_path()helper that recognizes Unix, Windows drive, and UNC roots regardless of OS. On Windows~/.ivertnow resolves toC:\Users\<user>\.ivert.signal.SIGKILLguard (validate_dem.py) —signal.SIGKILLdoes not exist on Windows, soabs(exitcode) == abs(signal.SIGKILL)raisedAttributeErrorwhenever a validation subprocess exited non-zero. Look the attribute up withgetattrand skip the OOM-killer recovery branch when it is absent (Windows has no SIGKILL / OOM-killer); a failed worker now surfaces as the intendedRuntimeError.Cross-platform terminal colors (
bcolors.py) — Raw ANSI escape codes render on Linux/macOS but appear as literal garbage on Windows consoles, where virtual-terminal processing is off by default. Enable it on import viacolorama.just_fix_windows_console()(no-op elsewhere; only sets the console-mode flag, does not wrapsys.stdout, so it does not interfere withloggerproc.Logger's stdout redirection).coloramaadded as a Windows-only dependency; the import is defensive so Linux/macOS degrade to native ANSI.Multiprocessing start method (
validate_dem.py,validate_dem_collection.py) —set_start_method("spawn")raisesRuntimeErrorif the start method was already set. Passforce=Trueso re-invocation cannot crash.Explicit UTF-8 encoding — Text-mode
open()defaults to the platform locale encoding (e.g. cp1252 on Windows). Passencoding="utf-8"on all text-mode opens (empty-result markers, summary stats, user config, photon-tile lists, the AWS datasource probe, and the process logger). Binary opens are unchanged.Testing
Verified on Linux: config paths still resolve to the correct absolute home-directory locations; the SIGKILL decision logic was unit-checked across POSIX/Windows exit-code cases; bcolors imports and renders (VT-enable is a no-op on Linux and degrades gracefully if colorama is absent);
set_start_method(force=True)is idempotent; and no text-modeopen()without an encoding remains.ruff checkandruff formatpass on all changed files.Not covered here (requires real Windows/macOS hardware): the shared-memory +
spawnparallel-validation path invalidate_dem.py— Windows frees namedSharedMemorysegments when the last handle closes, unlike Linux/dev/shm, so that flow should be exercised directly on the target platform.