Skip to content

v2.0.0 Release for UE5.8

Latest

Choose a tag to compare

@getnamo getnamo released this 13 Oct 04:27
· 7 commits to main since this release

A full rewrite for UE 5.8. TensorFlow runs out of process in an embedded Python with a per-project venv, through PythonCLI 0.4.0 on CLISystem 1.4.0. Both plugins are bundled.

The refactor

1.x was last updated for UE 5.3. Its component was a Blueprint on top of MachineLearningRemote, which sent everything over socket.io to a separate Python server.

  • Messages were capped at about 1 MB.
  • Floats travelled as JSON text.
  • Latent nodes hung when Python returned None or raised.
  • The embedded Python 3.7 path was broken.
  • callEvent and training-complete were never wired.

2.0 replaces the transport and keeps the API. It's now three layers, each its own plugin:

Layer Role
CLISystem 1.4 child processes with framed stdin/stdout
PythonCLI 0.4 the generic layer: an embedded CPython 3.12 and uv, a per-project venv, a shared warm Python process, and the components, streams and ML API
TensorFlow 2.0 thin: environment profiles, the TensorFlow component, texture helpers, examples
  • Control messages go as framed stdio.
  • Bulk data (render targets, audio, tensors) goes through shared memory.

Upgrading from 1.x

  • Remove MachineLearningRemote-Unreal from your project; it's no longer used.
  • upymodule.json is gone. Pick an environment profile, or list packages in Python CLI > Requirements or Content/Scripts/requirements.txt.
  • Blueprint references to the old component load through a CoreRedirect, but its variables (Tensor Flow Module, Should Train On Begin Play) need setting again.
  • 1.x's On Event is now On Custom Event.

Changes

  • Python out of process: a crash in TensorFlow never takes the editor down, and the GIL never touches the game thread. MachineLearningRemote, socket.io and UnrealEnginePython are gone.
  • Environment profiles (Project Settings > Plugins > TensorFlow):
    • TensorFlow CPU 2.21 is the default.
    • TensorFlow with CUDA, for Linux.
    • Keras 3 on PyTorch, for GPU on Windows.
    • None, to bring your own packages.
  • Setup: the first editor start installs the profile into Intermediate/PythonCLI/venv in the background.
  • Same Python API: TFPluginAPI scripts with camelCase hooks, getApi(), callEvent, shouldStop and import unreal_engine as ue keep working. snake_case hooks work too, and ue.keras.StopOnRequest / ProgressEvents stop and report Keras training.
  • TensorFlow Component (now C++): Begin Training, Stop Training, Send Json Input, Send Json Input Texture, Call Custom Function. A CoreRedirect loads 1.x Blueprint references.
  • Streams: render targets, audio and gameplay observations go to Python every frame, through shared memory without stalls.
  • TF Audio Capture is rebuilt on Unreal's AudioCapture.
  • Example map /TensorFlow/Examples/MnistExample: trains MNIST, then classifies the digit a camera sees, live.
  • Packaging: Win64 and Linux builds bundle the interpreter and, by default, the installed packages. They're installed for the target platform, so a Linux build packaged on Windows works. Install On First Run and None are the other modes.
  • Tests:
    • 30 automation tests (TensorFlow.*, PythonCLI.*) pass in the Win64 editor and in packaged Win64 builds, with Bundle and with Install On First Run.
    • In packaged Linux x86_64 builds (WSL2), 25 of them run and pass. The rest are skipped: the render-target tests need a GPU, the submix test needs an audio device, the requirements test doesn't apply to bundled packages, and the microphone test is opt-in.

Notes

  • TF1 graph and session scripts need porting to TF2 / Keras 3.
  • The last 1.x code is tagged v1.6.0beta-legacy.