Skip to content

Game Optimization Linux 1.6 Alpha

Latest
Latest

Choose a tag to compare

View all tags
@DevVoidPL DevVoidPL released this 17 Aug 12:35
· 7 commits to main since this release
v1.6.0-alpha
603bb1b

Game Optimization Linux 1.6 Alpha

Large alpha update focused on real game performance measurements, automatic optimization, runtime reliability and the first working version of the local Polish game narrator.

This is one of the largest updates to Game Optimization Linux so far.

A large part of this release was tested with real Steam/Proton games instead of only synthetic or automated tests.

Warning

Game Analyzer, Performance Baseline, Automatic Optimization and Polish Narrator are still experimental.

Compatibility can vary significantly between games, Proton versions, desktop environments and hardware.

The Polish Narrator in particular is an early implementation. OCR accuracy and narration latency can depend on the game, subtitle font, selected capture area and CPU performance.

Please report problems on Discord or GitHub Issues. Game-specific reports are especially useful.

Experimental Polish Narrator

Added the first working version of the local Polish game narrator.

The narrator can capture English subtitles directly from a running game, recognize them, translate them to Polish and generate Polish speech locally.

Current pipeline:

Game Window -> xdg-desktop-portal -> PipeWire -> GStreamer -> OCR -> EN/PL Translation -> Piper TTS -> Qt Audio

Added

  • New Lektor section in the application
  • Per-game narrator configuration
  • Wayland screen/window capture through xdg-desktop-portal
  • KDE/Wayland source picker support
  • PipeWire capture
  • GStreamer video transport
  • Tesseract OCR
  • Local English -> Polish translation
  • Local Polish text-to-speech
  • Piper pl_PL-gosia-medium voice
  • Direct PCM playback through Qt Multimedia
  • Configurable narrator volume
  • Configurable speech rate
  • Configurable subtitle capture region
  • Per-game narrator settings
  • Local translation cache
  • Narrator session diagnostics
  • OCR, translation, TTS and audio timing information
  • Optional narrator component downloading
  • Component integrity verification using pinned sizes and SHA256 hashes
  • Local component storage outside the base application package

The narrator does not require an online translation or cloud TTS service during normal operation.

Narrator Capture / Wayland

The original ScreenCast portal implementation exposed several real-world compatibility problems that did not appear in isolated tests.

Fixed

  • Replaced the narrator's low-level PySide6 QtDBus ScreenCast transport with dbus-next
  • Fixed incorrect D-Bus serialization of ScreenCast options
  • types, cursor_mode and persist_mode are now correctly sent as D-Bus UINT32
  • Added one persistent D-Bus connection for the complete portal session
  • Added explicit Unix FD negotiation for OpenPipeWireRemote
  • Improved PipeWire file-descriptor ownership and cleanup
  • Improved portal request cancellation
  • Added stale-session protection
  • Added handling for Session.Closed
  • Improved portal lifecycle shutdown
  • Fixed race where the portal backend could report itself ready before its event loop was actually accepting work
  • Improved portal error diagnostics
  • Removed dependency on QtDBus from the narrator capture transport

The portal flow now uses:

CreateSession -> SelectSources -> Start -> OpenPipeWireRemote

Real testing on KDE Plasma Wayland successfully reached the system window picker and captured running game windows.

Narrator GStreamer / PipeWire

  • Fixed use of the unsupported pipewiresrc on-disconnect property
  • Improved PipeWire source initialization
  • Improved handling of portal-provided stream targets
  • Improved GStreamer capture error reporting
  • Added capture resolution diagnostics
  • Added frame timing information
  • Added bounded capture processing to prevent unlimited queued work

Some games or capture sessions can still occasionally expose PipeWire/GStreamer format negotiation problems.

Narrator OCR

Major work was done to reduce false subtitle detection.

Early real-game tests showed that raw OCR could interpret textures, HUD elements and background graphics as random text.

Examples included random numbers, letters and symbols being passed into translation and speech.

Added / Improved

  • Subtitle-oriented image preprocessing
  • Grayscale preprocessing
  • Subtitle ROI upscaling
  • Improved contrast handling
  • Tesseract TSV-based OCR instead of blindly trusting plain text output
  • Per-word OCR confidence handling
  • Character-weighted phrase confidence
  • Generic garbage detection
  • Digit-ratio filtering
  • Symbol-ratio filtering
  • Isolated-token filtering
  • Subtitle text normalization
  • Temporal subtitle stabilization
  • Similarity-based OCR confirmation
  • Punctuation-insensitive phrase matching
  • Protection against repeatedly speaking an unchanged subtitle
  • Better handling when subtitles disappear
  • Raw noisy OCR observations no longer cancel already accepted translation/TTS work

The current stabilization system generally requires two sufficiently similar OCR observations before a phrase is accepted.

High-noise results are rejected before translation.

Performance

Synthetic local tests showed approximately:

  • ~10 ms subtitle preprocessing
  • ~100 ms Tesseract boundary processing under controlled test conditions

Real-game OCR latency varies depending on subtitle size, capture region and system performance.

Narrator Audio

Fixed an important Qt Multimedia lifecycle issue that caused narration to sometimes work once and then stop reading subsequent lines.

Fixed

  • Fixed QAudioSink.stateChanged handling with PySide6
  • Removed invalid conversion of QAudio::State through a Python object slot
  • Audio state is now read directly from QAudioSink
  • Fixed completed playback not releasing the current queue entry
  • Fixed pending narration sometimes never starting
  • Improved narrator audio lifecycle cleanup

Real testing confirmed that the narrator PCM format:

  • 22050 Hz
  • mono
  • signed 16-bit PCM

can be played correctly through the Flatpak audio environment on tested hardware.

Narrator Known Limitations

The narrator is functional, but still experimental.

Current real-game testing has shown the following limitations:

  • Some subtitles can still be missed
  • Short subtitles are more difficult to recognize reliably
  • OCR can still occasionally append incorrect symbols, digits or characters
  • Some game UI elements can be mistaken for subtitles
  • Recognition quality depends heavily on subtitle font and contrast
  • Recognition quality depends on the configured capture area
  • Some games work much better than others
  • Translation is not always perfect
  • Long dialogue can introduce noticeable latency
  • OCR, translation and speech synthesis performance depends partly on CPU performance
  • Slow CPUs may experience a larger delay between a subtitle appearing and speech playback
  • Some PipeWire/GStreamer capture formats may still fail on the first attempt
  • Switching games closes the existing portal capture session and requires selecting a new capture source

The narrator should currently be considered an Alpha feature rather than a replacement for native game localization.

Performance Baseline

The MangoHud-based performance measurement system received significant improvements.

Improved

  • Improved private MangoHud baseline configuration
  • Baseline measurement no longer depends on the user's normal MangoHud layout
  • Improved MangoHud CSV discovery
  • Improved support for current MangoHud log formats
  • Improved conversion of recorded values
  • Improved handling of memory and VRAM units
  • Improved representative measurement selection
  • Improved measurement segmentation
  • Improved detection of valid gameplay measurement windows
  • Improved semantic baseline states
  • Improved handling of low GPU utilization
  • Improved average FPS calculation
  • Improved frame-cap detection
  • Frame-cap detection is no longer limited to a small fixed list of common FPS limits
  • Improved bottleneck evidence
  • Improved baseline failure diagnostics

The application continues to use real recorded gameplay data rather than fabricated benchmark estimates.

Game Analysis

Game Analyzer and performance analysis received additional reliability work.

Improved

  • Better reuse of existing executable information
  • Improved game/runtime information propagation
  • Improved performance evidence handling
  • Improved bottleneck classification
  • Improved handling of insufficient data
  • Improved frame pacing analysis
  • Improved CPU/GPU interpretation
  • Improved memory-pressure analysis foundation

Game detection remains conservative.

When reliable information is unavailable, Unknown is preferred over a guessed result.

Experimental Automatic Optimization

Automatic Optimization has moved beyond the original architecture-only foundation.

The intended workflow remains:

Analyze -> Measure -> Detect Bottleneck -> Recommend -> Apply -> Measure Again -> Keep/Revert

Improved

  • Added real runtime optimization candidates
  • Added safer candidate evaluation
  • Improved evidence requirements
  • Improved recommendation confidence handling
  • Improved reversible optimization flow
  • Improved integration with measured game performance
  • Improved protection against recommendations when evidence is insufficient
  • GameMode can now be used as a real runtime optimization candidate where appropriate
  • Improved no-op handling when an optimization would not provide a meaningful change

Automatic Optimization deliberately continues to avoid:

  • fake FPS predictions
  • generic RAM booster behavior
  • drop_caches tricks
  • random sysctl collections
  • blindly applying unknown engine variables
  • applying changes only because they are technically possible

Some games will intentionally receive no automatic recommendation.

Performance / Queue Reliability

Several systems now use bounded asynchronous processing.

This is especially important for Narrator workloads where capture, OCR, translation and speech synthesis run at different speeds.

Improved

  • One active task with bounded pending work
  • Newer subtitle work can replace stale unprocessed work
  • Raw OCR noise no longer cancels already accepted narration
  • Improved stale-result protection
  • Improved cleanup when sessions stop
  • Improved thread and event-loop lifecycle handling
  • Improved shutdown reliability

Optional Components

Narrator dependencies that do not need to be part of the base application can be installed separately.

Current optional/local narrator components include:

  • English OCR data
  • English -> Polish translation model
  • Polish Piper voice

Downloads are validated against expected metadata and cryptographic hashes.

Fixed

  • Improved Hugging Face redirect handling
  • Added support for current Hugging Face/Xet delivery hosts
  • Improved rejected-host diagnostics
  • Fixed Polish voice installation failing after legitimate Hugging Face redirects

Flatpak

The Flatpak package has grown considerably because the application now contains additional local runtime infrastructure for OCR, translation, speech synthesis and related dependencies.

Large language/voice resources remain separate optional downloads where practical.

Added / Improved

  • Added dbus-next
  • Added narrator runtime dependencies
  • Added Tesseract
  • Added Leptonica
  • Added local translation runtime
  • Added Piper runtime
  • Added ONNX Runtime CPU support
  • Improved offline dependency packaging
  • Improved narrator component installation
  • No broad new host permissions were added for ScreenCast capture
  • Wayland capture continues to use the desktop portal permission model

The Flatpak remains the primary distribution format.

Controller / UI

  • Added the new Narrator page
  • Added narrator component status
  • Added narrator session status
  • Added narrator performance diagnostics
  • Added narrator source and game selection
  • Added subtitle capture-region configuration
  • Added translation profile selection
  • Added voice configuration
  • Improved status messages and errors
  • Improved long-running task feedback
  • Improved controller/runtime cleanup in several areas

Testing

This release received significantly more automated and real-game testing than previous builds.

Testing included:

  • narrator portal transport
  • D-Bus type serialization
  • portal request lifecycle
  • PipeWire Unix FD handling
  • OCR preprocessing
  • OCR confidence filtering
  • subtitle stabilization
  • translation
  • Piper TTS
  • PCM audio
  • cancellation behavior
  • session shutdown
  • MangoHud baseline processing
  • game analysis
  • QML runtime checks
  • Flatpak builds
  • real KDE Plasma Wayland capture
  • real Steam/Proton games

Automated test counts changed continuously during development as new tests were added.

Real game testing remains more important than the raw test count.

Fixed

  • Narrator incorrectly reporting a running Steam/Proton game as stopped from inside the Flatpak sandbox
  • ScreenCast CreateSession response type handling
  • ScreenCast SelectSources failing because numeric options were serialized as signed INT32
  • Portal lifecycle races
  • PipeWire remote FD ownership
  • Invalid GStreamer pipewiresrc property
  • Narrator translation/TTS being cancelled by unverified OCR noise
  • Narrator repeatedly producing garbage from unstable OCR observations
  • Narrator audio playing once and then failing to advance the playback queue
  • Narrator audio state conversion errors in PySide6
  • Polish Piper voice download redirects being rejected
  • Several narrator shutdown and stale-session cases
  • Additional performance-baseline parsing and lifecycle issues
  • Various runtime, QML and Flatpak integration problems

Known Limitations

  • Game engine/API/runtime detection is still incomplete
  • Some games still return Unknown
  • Performance Baseline can still behave differently between games and Proton versions
  • Automatic Optimization still has a limited number of safe candidates
  • Not every detected bottleneck has an automatic optimization available
  • Gamescope compatibility varies between games and systems
  • Polish Narrator compatibility varies significantly between games
  • Narrator OCR can miss subtitles or recognize additional characters
  • Narrator latency depends partly on CPU performance
  • PipeWire/GStreamer capture may still expose game/environment-specific problems
  • Some features still need broader testing on non-KDE desktops and different Linux distributions
  • This is still an Alpha release

Feedback

Real game testing is extremely useful at this stage.

If something does not work, please report it on Discord or GitHub Issues.

For game-specific problems, useful information includes:

  • game name
  • Steam AppID
  • CPU
  • GPU
  • Linux distribution
  • desktop environment
  • Wayland or X11
  • Proton version
  • screenshot
  • relevant logs

Narrator reports are particularly useful when they include the subtitle style/font and the configured subtitle capture area.

Still Planned

The following areas are not considered finished:

  • larger Automatic Optimization library
  • improved CPU/GPU bottleneck optimization
  • improved RAM/VRAM pressure handling
  • better Narrator OCR
  • lower Narrator latency
  • broader Narrator game compatibility
  • deeper engine-specific optimization
  • Deep Optimize / unnecessary game-data cleanup
  • AI Graphics Remaster
  • additional narrator languages / voices
  • broader Couch Mode improvements

This release is still experimental.

Game Optimization Linux should prefer doing nothing over applying an optimization it cannot justify.

And when a feature does not work correctly in a specific game, reports are welcome - especially during Alpha development.

Assets 3
Loading