pr - #24
Merged
Merged
Conversation
* ci: add GitHub Actions release workflow [no release] * ci: fix release jobs (explicit win targets, install libomp on mac) [no release] * ci: only publish release when all backend jobs succeed [no release] * ci: provision mac OpenMP off, Windows Vulkan SDK + CUDA toolkit [no release] * ci: adopt llama.cpp release patterns (7z packing, CUDA 12/13 matrix, Vulkan SDK fix, OpenMP-off CUDA build) [no release] * ci: use valid Vulkan SDK 1.4.357.0 and full CUDA versions in matrix [no release] * ci: bump cuda-toolkit to v0.2.36, CUDA 12.4/13.2 [no release] * ci: source vcvarsall before CUDA cmake so nvcc finds cl.exe [no release] * ci: fix robocopy exit-code mapping in CUDA bundle step [no release] * ci: fix CUDA artifact zip path (build/bin is 1 level shallower than CPU/Vulkan preset bins) [no release] * ci: bundle CUDA runtime from bin/lib/bin-x64 and stop double-shipping runtime DLLs in CUDA release * ci(win-cuda): build with GGML_BACKEND_DL like llama.cpp so CUDA ships as ggml-cuda.dll instead of 3 monoliths * ci: port llama.cpp release pipeline for CPU/Vulkan/CUDA/Metal (get-tag-name, GGML_BACKEND_DL CUDA, robust cudart bundling) * ci(win-cuda): pin CMAKE_CUDA_ARCHITECTURES per toolkit; use CUDA 13.3 in the matrix * ci(win-cuda): use ggml-style virtual/real CUDA archs (faster) and bundle cufft64 runtime * ci: one-command releases (publish toggle, scripts/release.sh, docs/RELEASING.md) * ci: one-click GUI releases (publish defaults on) + GUI-first releasing docs * ci: remove CLI release helper; GUI-only releasing docs * ci: semver tag-driven releases (remove b<N>/auto-push); manual version input + publish gate * ci: preserve pre-release flag and notes when attaching binaries to a release * ci: upload only real package files (.zip/.tar.gz) to the release * ci: fix boolean publish gate (inputs.publish) so manual dispatch releases publish * ci: enable native model manager in prebuilt releases Add -DAUDIOCPP_BUILD_NATIVE_MODEL_MANAGER=ON to the CMAKE_ARGS so the prebuilt binaries ship with the self-contained native UI (model downloads, dynamic model management, etc.) instead of relying on external Python. Addresses: #286 (comment) * fix: pass native model manager flags to Windows builds too The env.CMAKE_ARGS is consumed only by the Linux and macOS jobs which call cmake directly. Windows CPU and Vulkan jobs use build_windows.ps1 which has its own CLI parameters (-DeploymentBuild, -NativeModelManager) and ignores CMAKE_ARGS. Windows CUDA jobs also hardcode cmake flags. This fix: - Adds -DeploymentBuild -NativeModelManager to all build_windows.ps1 calls - Adds -DAUDIOCPP_BUILD_NATIVE_MODEL_MANAGER=ON to the CUDA cmake command * fix: add native model manager support to all build scripts Port the upstream fix (e9e8f14) to the fork: - Add --native-model-manager, --system-openssl, --boringssl-archive CLI flags - Add -DAUDIOCPP_BUILD_NATIVE_MODEL_MANAGER to cmake invocations - Print native model manager status in build output This completes the owner's request: the release CI now builds the self-contained native UI on all platforms. * fix: bundle tools/ and model_specs/ into prebuilt archives The server's model installer invokes tools/model_manager_v2.py for package downloads. Without it in the prebuilt zip, model download from the WebUI fails with 'model preparation helper was not found'. Include tools/ and model_specs/ at the archive root so the server can resolve its repository_root (traverses upward from the exe). * ci: upload raw bundle artifacts, split bin/cudart, archive only at release - Build jobs now upload the unpacked bundle (binaries + tools/ + model_specs/) instead of a zip-wrapped-in-an-artifact, so downloading an Actions artifact yields a ready-to-run folder. - CUDA jobs upload bin and cudart as two separate artifacts (the old wildcard path merged both into one ~1 GB blob). - The release job downloads per-artifact folders, creates the final zip/tar.gz archives there, and uploads those to the Release -- asset layout stays identical to v0.6.0. * ci: bundle MSVC runtime DLLs into Windows prebuilt packages Windows CPU/Vulkan/CUDA prebuilts link the dynamic MSVC runtime (default /MD) plus vcomp140.dll (OpenMP, from -DENGINE_ENABLE_OPENMP=ON). These are not guaranteed on clean/enterprise/Server/container Windows, so the packages bundle them app-locally (vcruntime140*.dll, msvcp140*.dll, and vcomp140.dll for CPU/Vulkan) from the toolchain's Redist\MSVC layout. Total added size is ~1 MB compressed - negligible vs the CPU/Vulkan/CUDA package sizes, and keeps installs self-contained (no VC++ Redistributable dependency). CUDA skips vcomp140 because the CUDA build disables OpenMP.
* feat(server): add --idle-unload-ms to auto-unload idle models The 5-minute idle unload used to live in an external Python monitor inside the audio-server wrapper, which polled the server log mtime and called /v1/tasks/unload_all_models. Move it into audiocpp_server itself: - new ServerConfig field idle_unload_ms (default 0 = disabled), parsed from server.json and overridable via --idle-unload-ms - a background thread unloads every resident non-busy model once the server has been idle that long without a model load/run; the next request reloads lazily This drops the log-mtime heuristic (which required --log trace spam) and removes the need for the external Python wrapper. * feat(server): refuse model load when host/GPU memory is insufficient Before every lazy model load, estimate the model's resident footprint (weights plus runtime overhead) and compare against free host memory and, for GPU backends, the backend device's free memory. Refuse the load with HTTP 503 insufficient_memory when estimate + configured headroom does not fit, instead of exhausting the machine (the previous failure mode was kIOGPUCommandBufferCallbackErrorOutOfMemory after models accumulated on a 16GB Mac). - add ServerConfig.min_free_memory_mb (default 512 MiB headroom), parsed from server.json and overridable via --min-free-memory-mb - add a macOS implementation of available_host_memory_bytes() using Mach VM stats (free + inactive + purgeable pages); Linux/Windows were already covered - add InsufficientMemoryError, mapped to 503 insufficient_memory * fix(server): address code review for idle unload and memory guard Independent review found no criticals; fix the actionable findings: - --min-free-memory-mb help text now matches the actual 512 MiB default - estimate_model_memory_bytes() sums directory-style model trees (with depth/file limits) instead of counting only regular files, so directory models are no longer estimated as 0 - the model load path is serialized even when max_loaded_models is 0, so concurrent lazy loads cannot both pass the memory pre-check - expose engine::core::ensure_backends_loaded() and call it before the GPU memory query so the very first load actually runs the device check - the idle-unload thread now wakes on shutdown in <=250ms slices instead of waiting out a full poll interval - document idle_unload_ms / min_free_memory_mb in app/server/README.md and example.json - add server_config_test coverage for the new fields (defaults, overrides, negative rejection) * ci(server): add cross-platform build+test matrix for memory guard Run audiocpp_server build + server_config_test on ubuntu/windows/macos to prove the idle-unload and pre-load memory-check changes compile and behave on all three desktop platforms. GPU backends are off here (the changes query memory through the backend-agnostic ggml_backend_dev_memory; existing workflows already cover CUDA/Vulkan/Metal builds). * ci(server): fix macOS build step to not run the nproc variant The Linux build step used nproc and its condition also matched macOS, so macOS ran both build steps and the nproc one stalled. Scope the nproc step to Linux and let macOS use sysctl -n hw.logicalcpu. * fix(server): address maintainer review on memory guard Address 0xShug0's review on #306: 1. Make the memory guard opt-in: min_free_memory_mb now defaults to 0 and 0 disables ensure_model_fits_memory entirely, so existing configs that never opted in see no load-behavior change. 2. Stop over-summing directory models and stop masking the real loader error. estimate_model_memory_bytes now estimates only what the loader will read: a single file, the one GGUF a directory selects (find_directory_gguf), or a full safetensors/HF tree. A directory with several GGUFs and no model.gguf is ambiguous, so the guard estimates nothing and the loader's own "contains N GGUF files" error surfaces instead of a misleading 503. 3. Resolve relative session-option paths against the model directory when model.path is a directory (parent_path only when it is a file). 4. Measure idle time from request completion, not start: run_model and run_streaming_model_impl now stamp last_activity_ms_ on completion, so a long inference is not unloaded the moment it returns. 5. Only serialize lazy loads through model_load_mutex_ when a guard needs it (max_loaded_models > 0 or min_free_memory_mb > 0); with both off, unrelated first-load requests keep their original concurrency. Verified on macOS (Apple M4): audiocpp_server + server_config_test build and pass; startup smoke shows a single-GGUF dir estimates just the selected file (10.83 GiB -> 503 when guard on), and an ambiguous multi-GGUF dir surfaces the real "contains 2 GGUF files" loader error with the guard both on and off.
Release 0.7 WIP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.