After running for an afternoon the cache for the model was still zero, even though a lot of tokens got reused.
To Reproduce
Steps to reproduce the behavior:
- I was running Qwen3.6 35B-A4B oQ8, but I've had the same issue with 27b
- Generating code with OpenCode and OpenSpec for hours and hours
Desktop (please complete the following information):
- macOS Version: 26.4.1
- oMLX Version 0.3.8, dmg or brew version appear identical
Additional context
There's a lot of errors regarding the ssd cache in the logs, especially around the eviction. I suspect the cache needed to be written then, didn't but did register that they should have been. Personally I would prefer the cache to write after either every processed pre-prompt or even better, after every batch. And as a sidenote I'd love to be able to make the batches smaller, the memory overhead from 2048 token processing is likely smashing through the limit.
Hot cache is set to roughly 4gb and ssd cache to 186gb.
Also.. Love your work Jundot! I hope this helps.
`2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Max model memory changed: 50.40GB -> 50.40GB
2026-05-05 13:58:26,457 - omlx.process_memory_enforcer - INFO - [-] - Process memory limit changed: 58.9GB -> 58.9GB
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Process memory limit updated to 58.9GB
2026-05-05 13:58:26,457 - omlx.process_memory_enforcer - INFO - [-] - Prefill memory guard: enabled
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Prefill memory guard enabled
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Hot cache max size changed: 10.00 GB -> 4.00 GB
2026-05-05 13:58:26,457 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 13:58:26,457 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 13:58:26,457 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 13:58:26,457 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 052fc2d18d735f68
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 853a9babfe976361
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c3a4438fe93b8423
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 0c853a1a33b93107
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 6c3c32832e63e9e1
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block a9ed2d3c290f4ddb
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block f43a857299d30fa7
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5e6166d3669f9ea6
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 434b48c88b2db97a
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c2656a86260e4578
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block ebffc75a5168e252
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block b2080eea42e5f862
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 38fb2587de768892
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block f210ea922075a2a4
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 64dfa86e8b541d73
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block dc007968d9e60c2d
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block aad794c4f2814892
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d532e8d521a2bdec
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58f73333469454ad
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 74d213b633f65105
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 375eaa0be22d6a41
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58cdb7029ceb2d1c
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 296e9e313078d1b1
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 2bbbc966f87a7b52
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d8ead7f107784587
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 204bad07b4fad805
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block b66d6849c45a1401
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c9c5c1548bb0429b
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7b02717b09d15def
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 10b7b0b41a7fecad
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5fc555158c51c9db
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 08faf853f9d0ee18
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d86d73abd500aaba
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 921c60ef5c46bb75
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block e30d4595667531bf
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block e2f2a6734d3408d2
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58f7297b1ef9c0e1
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 90269f5cea34dade
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block fe3dfc37863701b7
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 8d17f746c0e53bb3
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5aa9fa947b7a0743
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block de62d8e380e49914
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block da2ecce1bf1c0232
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 198565b9f26c622a
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - INFO - [-] - Flushed 33 hot cache blocks to SSD write queue
2026-05-05 13:58:27,756 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 13:58:27,773 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 13:58:27,838 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 13:58:27,847 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 13:58:37,523 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request f55d804d-53b9-40f8-beb0-595ed1bc530d
2026-05-05 13:58:37,524 - omlx.server - ERROR - [-] - Error during chat streaming: 'NoneType' object has no attribute 'abort_request'
2026-05-05 13:58:39,454 - omlx.scheduler - ERROR - [-] - Error in batch generation step: 60269743504
Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/scheduler.py", line 4018, in step
scheduled, rejected = self._schedule_waiting()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/scheduler.py", line 3378, in _schedule_waiting
del self.uid_to_request_id[temp_uid]
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 60269743504
2026-05-05 13:58:39,455 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=32.39GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 13:58:39,455 - omlx.admin.routes - INFO - [-] - Cache settings updated. Unloaded 1 models.
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Network settings updated
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Sampling defaults updated: max_context_window=262144, max_tokens=65536, temperature=0.7, top_p=0.95, top_k=0, repetition_penalty=1.0
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Idle timeout disabled
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - API key updated via admin settings
2026-05-05 13:58:39,456 - omlx.settings - INFO - [-] - Saved settings to /Users/roelvideler/.omlx/settings.json
2026-05-05 13:58:56,443 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 13:58:58,509 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 13:58:59,699 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 13:58:59,700 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 13:58:59,762 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=160, indexed=160, errors=0, total_size=22.65 GB
2026-05-05 13:58:59,762 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=160, disk_free=382.00 GB, cache_used=22.65 GB
2026-05-05 13:58:59,762 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 13:58:59,762 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 13:58:59,763 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 13:58:59,763 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 13:58:59,763 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 13:58:59,763 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 13:58:59,821 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 13:58:59,825 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 13:58:59,847 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (estimated: 20.64GB, total: 20.64GB)
2026-05-05 13:58:59,848 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 13:59:00,248 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 13:59:00,607 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 27 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 14:03:26,152 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request 61dd7177-6c93-4d0e-be1c-fb75e396fd83
2026-05-05 14:03:31,743 - omlx.scheduler - INFO - [-] - Prefill interrupted at 94208/147318 tokens: 1 request(s) aborted
2026-05-05 14:03:34,739 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 14:03:34,740 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:03:34,740 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:03:34,740 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:03:34,798 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:03:34,825 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:03:34,873 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:03:34,882 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:03:35,113 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=21.75GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 14:03:35,113 - omlx.admin.routes - INFO - [-] - Manually unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:04:09,943 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:04:16,872 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:04:23,121 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:04:24,309 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:04:24,310 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:04:24,310 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:04:24,310 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:04:24,310 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:04:24,378 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=160, indexed=160, errors=0, total_size=22.65 GB
2026-05-05 14:04:24,378 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=160, disk_free=381.98 GB, cache_used=22.65 GB
2026-05-05 14:04:24,378 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:04:24,378 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:04:24,379 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:04:24,379 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:04:24,379 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:04:24,379 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:04:25,233 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=256, max_blocks=100000, max_tokens=204800000
2026-05-05 14:04:25,234 - omlx.scheduler - INFO - [-] - SpecPrefill: draft model set with SSD cache (model_name=/Users/roelvideler/.cache/huggingface/hub/Qwen3.5-2B-oQ8-fp16)
2026-05-05 14:04:25,234 - omlx.engine.vlm - INFO - [-] - SpecPrefill: draft model loaded (/Users/roelvideler/.cache/huggingface/hub/Qwen3.5-2B-oQ8-fp16)
2026-05-05 14:04:25,292 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:04:25,295 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:04:25,298 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (estimated: 36.86GB, total: 36.86GB)
2026-05-05 14:04:25,299 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:05:26,679 - omlx.scheduler - INFO - [-] - SpecPrefill: scored 84424 tokens in 60.9s, selected 16872/84424 (keep=20%, prompt 159602 = system 75178 + conv 84424, cached 0)
2026-05-05 14:07:57,888 - omlx.scheduler - INFO - [-] - SpecPrefill: system prompt 75178 tokens full prefill
2026-05-05 14:09:06,128 - omlx.scheduler - INFO - [-] - SpecPrefill: sparse prefill 16871/84424 conv tokens in 217.9s (total 159602, cached 0, system 75178 full, conv 84424 sparse)
2026-05-05 14:09:09,189 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.0ms/1
2026-05-05 14:11:30,498 - omlx.scheduler - INFO - [-] - SpecPrefill: scored 85472 tokens in 140.3s, selected 17120/85472 (keep=20%, draft cache hit 83968, prompt 160650 = system 75178 + conv 85472, cached 0)
2026-05-05 14:13:42,486 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request 635d130c-633a-4707-8a1c-4684b6ff1965
2026-05-05 14:13:59,274 - omlx.scheduler - INFO - [-] - SpecPrefill: system prompt 75178 tokens full prefill
2026-05-05 14:14:20,411 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:14:20,413 - omlx.admin.routes - INFO - [-] - Settings changed for loaded model Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16. Reload required to take effect.
2026-05-05 14:14:27,105 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (immediate abort)
2026-05-05 14:14:27,105 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:14:27,105 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:14:27,105 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 2430348873756347
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7556cf76164d9ea1
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block fdf359fbb179e2ed
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 72d02cde747d0414
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7d1e5611e041e6ec
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 1a03c7e70cd2dd3a
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 0a230a00f788b95c
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 4e6f770aa58dff7e
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - INFO - [-] - Flushed 33 hot cache blocks to SSD write queue
2026-05-05 14:14:28,288 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:14:28,309 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:14:28,382 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:14:28,391 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:14:54,820 - omlx.server - WARNING - [-] - POST /admin/api/models/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16/unload → 400: Model not loaded: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:15:07,656 - omlx.scheduler - ERROR - [-] - SpecPrefill sparse prefill failed: 'NoneType' object has no attribute 'layers'
2026-05-05 14:15:07,656 - omlx.scheduler - WARNING - [-] - Cache corruption detected: 'NoneType' object has no attribute 'layers', clearing cache and re-prefilling...
2026-05-05 14:15:07,658 - omlx.scheduler - INFO - [-] - Cache recovery completed
2026-05-05 14:15:07,769 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16, freed=43.02GB (expected>=34.86GB), active_memory: 1.86GB (settled)
2026-05-05 14:15:07,769 - omlx.admin.routes - INFO - [-] - Manually unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:15:10,109 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:15:37,586 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:15:41,645 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:15:42,836 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:15:42,836 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:15:42,905 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=193, indexed=193, errors=0, total_size=23.42 GB
2026-05-05 14:15:42,905 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=193, disk_free=381.21 GB, cache_used=23.42 GB
2026-05-05 14:15:42,905 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:15:42,905 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:15:42,906 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:15:42,906 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:15:42,906 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:15:42,906 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:15:42,962 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:15:42,965 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:15:42,970 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (estimated: 20.64GB, total: 20.64GB)
2026-05-05 14:15:42,971 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:15:43,370 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 14:15:44,027 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 27 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 14:15:55,500 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request a0da1670-15cd-40cf-bdbb-1d70cc4f73d9
2026-05-05 14:15:57,464 - omlx.scheduler - INFO - [-] - Prefill interrupted at 10240/147318 tokens: 1 request(s) aborted
2026-05-05 14:15:59,818 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 14:15:59,819 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:15:59,819 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:15:59,819 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:15:59,879 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:15:59,900 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:15:59,950 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:15:59,958 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:16:00,177 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=22.01GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 14:16:00,177 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:16:06,482 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:16:07,666 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:16:07,667 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:16:07,734 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=193, indexed=193, errors=0, total_size=23.42 GB
2026-05-05 14:16:07,734 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=193, disk_free=381.20 GB, cache_used=23.42 GB
2026-05-05 14:16:07,734 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:16:07,734 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:16:07,734 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:16:07,734 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:16:07,734 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:16:07,734 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:16:07,792 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:16:07,796 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:16:07,802 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (estimated: 36.86GB, total: 36.86GB)
2026-05-05 14:16:07,803 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:26:14,874 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 75878bf0-69d4-493a-a0ee-bd09e00d58a2: storing 157696/159709 tokens (skipping trailing partial block, 76 intermediate snapshots)
2026-05-05 14:26:15,145 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.1ms/1, store_cache_main_eval=99.0ms/1, store_cache_main_prep=103.3ms/1
2026-05-05 14:26:15,301 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 14:27:22,383 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for a3a5bd77-db9f-4e0e-8b15-589db0064019: storing 161792/163249 tokens (skipping trailing partial block, 1 intermediate snapshots)
2026-05-05 14:27:22,430 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.1ms/2, store_cache_main_eval=142.9ms/2, store_cache_main_prep=150.5ms/2
2026-05-05 14:27:45,011 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.2ms/3, store_cache_main_eval=241.3ms/3, store_cache_main_prep=249.0ms/3
2026-05-05 14:28:04,732 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.2ms/4, store_cache_main_eval=336.8ms/4, store_cache_main_prep=344.5ms/4
2026-05-05 14:28:58,534 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 376cd0a3-8bbf-43e9-9d5e-d1ab8caf1c64: storing 163840/165741 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 14:28:58,647 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.3ms/5, store_cache_main_eval=437.7ms/5, store_cache_main_prep=446.5ms/5
2026-05-05 14:29:38,379 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.4ms/6, store_cache_main_eval=532.4ms/6, store_cache_main_prep=541.2ms/6
2026-05-05 18:16:28,764 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 0219abec-8537-4c35-94d1-473b8df62a2a: storing 165888/166446 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:16:28,890 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.5ms/7, store_cache_main_eval=635.1ms/7, store_cache_main_prep=649.4ms/7
2026-05-05 18:16:51,671 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.5ms/8, store_cache_main_eval=734.9ms/8, store_cache_main_prep=749.2ms/8
2026-05-05 18:17:22,475 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.6ms/9, store_cache_main_eval=833.6ms/9, store_cache_main_prep=847.9ms/9
2026-05-05 18:17:55,147 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 95739b9f-b7ff-497d-9a84-165032575bf2: storing 167936/168182 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:17:55,269 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.6ms/10, store_cache_main_eval=935.3ms/10, store_cache_main_prep=952.1ms/10
2026-05-05 18:18:03,671 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.7ms/11, store_cache_main_eval=1039.6ms/11, store_cache_main_prep=1056.4ms/11
2026-05-05 18:18:11,176 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.7ms/12, store_cache_main_eval=1137.2ms/12, store_cache_main_prep=1154.1ms/12
2026-05-05 18:18:40,420 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 298218f9-b4f3-461d-b392-dd81dd3c5b7f: storing 169984/170643 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:18:40,538 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.8ms/13, store_cache_main_eval=1241.1ms/13, store_cache_main_prep=1260.5ms/13
2026-05-05 18:18:53,050 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.8ms/14, store_cache_main_eval=1342.2ms/14, store_cache_main_prep=1361.7ms/14
2026-05-05 18:19:17,759 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.9ms/15, store_cache_main_eval=1442.2ms/15, store_cache_main_prep=1461.6ms/15
2026-05-05 18:19:51,532 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.0ms/16, store_cache_main_eval=1544.2ms/16, store_cache_main_prep=1563.8ms/16
2026-05-05 18:20:27,163 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 24816554-3fc2-4df3-a28c-fc0f2c67da2a: storing 172032/173240 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:20:27,233 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.0ms/17, store_cache_main_eval=1570.1ms/17, store_cache_main_prep=1591.2ms/17
2026-05-05 18:20:48,454 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.1ms/18, store_cache_main_eval=1677.3ms/18, store_cache_main_prep=1698.4ms/18
2026-05-05 18:21:05,136 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.1ms/19, store_cache_main_eval=1779.2ms/19, store_cache_main_prep=1800.4ms/19
2026-05-05 18:21:23,688 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.2ms/20, store_cache_main_eval=1887.7ms/20, store_cache_main_prep=1908.9ms/20
2026-05-05 18:22:09,069 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 25bc82fd-8432-4791-a7e6-7b55b93ab011: storing 174080/175213 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:22:09,218 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.3ms/21, store_cache_main_eval=2001.6ms/21, store_cache_main_prep=2025.7ms/21
2026-05-05 18:22:10,073 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/0/05c7072110d47b7804e51e77126ecd2a54adb150b6ffb1c7259ba681e59a16fa.safetensors
2026-05-05 18:22:10,076 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 1/85 blocks, 2048 tokens
2026-05-05 18:34:09,065 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 567eb638-ac4c-49b6-a3f4-def16ffbc4a4: storing 174080/175550 tokens (skipping trailing partial block, 83 intermediate snapshots)
2026-05-05 18:34:09,374 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.3ms/22, store_cache_main_eval=2118.4ms/22, store_cache_main_prep=2147.1ms/22
2026-05-05 18:34:10,380 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/7/7cfb553cf3dd28c23254bdbb77a4abddb2fc383b13a5af8f5c7f9e3336ca2950.safetensors
2026-05-05 18:34:10,380 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 5/85 blocks, 10240 tokens
2026-05-05 18:46:14,375 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for e4446ef7-1ffd-49e3-a97a-fd7a42e10cf5: storing 174080/175755 tokens (skipping trailing partial block, 79 intermediate snapshots)
2026-05-05 18:46:14,618 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.4ms/23, store_cache_main_eval=2228.4ms/23, store_cache_main_prep=2261.4ms/23
2026-05-05 18:46:15,557 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/c/c92e8d13913e0c94d64e268a6db383e1ac29adc82494dd7ed12ef5807f4a102d.safetensors
2026-05-05 18:46:15,557 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 9/85 blocks, 18432 tokens
2026-05-05 18:46:15,557 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 3 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 18:58:26,665 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for b50aa733-5c0a-418d-83c8-851a35545e00: storing 174080/176205 tokens (skipping trailing partial block, 78 intermediate snapshots)
2026-05-05 18:58:26,914 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.5ms/24, store_cache_main_eval=2345.5ms/24, store_cache_main_prep=2384.1ms/24
2026-05-05 19:00:35,093 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/6/69a7d9c1ea51c7d2a5c594263929439142e1961d50d1c809977e2f4f0170805f.safetensors
2026-05-05 19:00:35,093 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 17/85 blocks, 34816 tokens
2026-05-05 19:00:35,093 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 7 trailing block(s) with placeholder non-sliceable state, keeping 10 block(s) (20480 tokens)
2026-05-05 19:12:30,540 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for d4133bc6-77ca-4760-b6a8-d93cb65848e4: storing 176128/176402 tokens (skipping trailing partial block, 75 intermediate snapshots)
2026-05-05 19:12:30,760 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.6ms/25, store_cache_main_eval=2456.3ms/25, store_cache_main_prep=2500.0ms/25
2026-05-05 19:12:32,935 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/c/cddf177778d4ef9c9a6e49811601334ebdb6471a21846ac74ebcc90185c181d1.safetensors
2026-05-05 19:12:32,935 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 79/86 blocks, 161792 tokens
2026-05-05 19:14:36,549 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 23d78da4-26fd-4d9d-b6f2-946a4714cdfc: storing 176128/177038 tokens (skipping trailing partial block, 6 intermediate snapshots)
2026-05-05 19:14:36,661 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.6ms/26, store_cache_main_eval=2563.1ms/26, store_cache_main_prep=2610.1ms/26
2026-05-05 19:15:02,064 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.7ms/27, store_cache_main_eval=2666.8ms/27, store_cache_main_prep=2713.9ms/27
2026-05-05 19:15:47,122 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/7/7cfb553cf3dd28c23254bdbb77a4abddb2fc383b13a5af8f5c7f9e3336ca2950.safetensors
2026-05-05 19:15:47,122 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 5/86 blocks, 10240 tokens
2026-05-05 19:28:03,077 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 8161b8f5-7856-4356-853c-bc4928a0152e: storing 178176/178817 tokens (skipping trailing partial block, 81 intermediate snapshots)
2026-05-05 19:28:03,339 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.7ms/28, store_cache_main_eval=2784.7ms/28, store_cache_main_prep=2837.7ms/28
2026-05-05 19:28:15,226 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/6/69a7d9c1ea51c7d2a5c594263929439142e1961d50d1c809977e2f4f0170805f.safetensors
2026-05-05 19:28:15,226 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 17/87 blocks, 34816 tokens
2026-05-05 19:28:15,226 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 7 trailing block(s) with placeholder non-sliceable state, keeping 10 block(s) (20480 tokens)`
After running for an afternoon the cache for the model was still zero, even though a lot of tokens got reused.
To Reproduce
Steps to reproduce the behavior:
Desktop (please complete the following information):
Additional context
There's a lot of errors regarding the ssd cache in the logs, especially around the eviction. I suspect the cache needed to be written then, didn't but did register that they should have been. Personally I would prefer the cache to write after either every processed pre-prompt or even better, after every batch. And as a sidenote I'd love to be able to make the batches smaller, the memory overhead from 2048 token processing is likely smashing through the limit.
Hot cache is set to roughly 4gb and ssd cache to 186gb.
Also.. Love your work Jundot! I hope this helps.
`2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Max model memory changed: 50.40GB -> 50.40GB
2026-05-05 13:58:26,457 - omlx.process_memory_enforcer - INFO - [-] - Process memory limit changed: 58.9GB -> 58.9GB
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Process memory limit updated to 58.9GB
2026-05-05 13:58:26,457 - omlx.process_memory_enforcer - INFO - [-] - Prefill memory guard: enabled
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Prefill memory guard enabled
2026-05-05 13:58:26,457 - omlx.admin.routes - INFO - [-] - Hot cache max size changed: 10.00 GB -> 4.00 GB
2026-05-05 13:58:26,457 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 13:58:26,457 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 13:58:26,457 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 13:58:26,457 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 052fc2d18d735f68
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 853a9babfe976361
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c3a4438fe93b8423
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 0c853a1a33b93107
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 6c3c32832e63e9e1
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block a9ed2d3c290f4ddb
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block f43a857299d30fa7
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5e6166d3669f9ea6
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 434b48c88b2db97a
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c2656a86260e4578
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block ebffc75a5168e252
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block b2080eea42e5f862
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 38fb2587de768892
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block f210ea922075a2a4
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 64dfa86e8b541d73
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block dc007968d9e60c2d
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block aad794c4f2814892
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d532e8d521a2bdec
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58f73333469454ad
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 74d213b633f65105
2026-05-05 13:58:26,458 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 375eaa0be22d6a41
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58cdb7029ceb2d1c
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 296e9e313078d1b1
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 2bbbc966f87a7b52
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d8ead7f107784587
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 204bad07b4fad805
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block b66d6849c45a1401
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block c9c5c1548bb0429b
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7b02717b09d15def
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 10b7b0b41a7fecad
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5fc555158c51c9db
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 08faf853f9d0ee18
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block d86d73abd500aaba
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 921c60ef5c46bb75
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block e30d4595667531bf
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block e2f2a6734d3408d2
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 58f7297b1ef9c0e1
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 90269f5cea34dade
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block fe3dfc37863701b7
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 8d17f746c0e53bb3
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 5aa9fa947b7a0743
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block de62d8e380e49914
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block da2ecce1bf1c0232
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 198565b9f26c622a
2026-05-05 13:58:26,459 - omlx.cache.paged_ssd_cache - INFO - [-] - Flushed 33 hot cache blocks to SSD write queue
2026-05-05 13:58:27,756 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 13:58:27,773 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 13:58:27,838 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 13:58:27,847 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 13:58:37,523 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request f55d804d-53b9-40f8-beb0-595ed1bc530d
2026-05-05 13:58:37,524 - omlx.server - ERROR - [-] - Error during chat streaming: 'NoneType' object has no attribute 'abort_request'
2026-05-05 13:58:39,454 - omlx.scheduler - ERROR - [-] - Error in batch generation step: 60269743504
Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/scheduler.py", line 4018, in step
scheduled, rejected = self._schedule_waiting()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/scheduler.py", line 3378, in _schedule_waiting
del self.uid_to_request_id[temp_uid]
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 60269743504
2026-05-05 13:58:39,455 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=32.39GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 13:58:39,455 - omlx.admin.routes - INFO - [-] - Cache settings updated. Unloaded 1 models.
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Network settings updated
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Sampling defaults updated: max_context_window=262144, max_tokens=65536, temperature=0.7, top_p=0.95, top_k=0, repetition_penalty=1.0
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - Idle timeout disabled
2026-05-05 13:58:39,456 - omlx.admin.routes - INFO - [-] - API key updated via admin settings
2026-05-05 13:58:39,456 - omlx.settings - INFO - [-] - Saved settings to /Users/roelvideler/.omlx/settings.json
2026-05-05 13:58:56,443 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 13:58:58,509 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 13:58:59,699 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 13:58:59,699 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 13:58:59,700 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 13:58:59,762 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=160, indexed=160, errors=0, total_size=22.65 GB
2026-05-05 13:58:59,762 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=160, disk_free=382.00 GB, cache_used=22.65 GB
2026-05-05 13:58:59,762 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 13:58:59,762 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 13:58:59,763 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 13:58:59,763 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 13:58:59,763 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 13:58:59,763 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 13:58:59,821 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 13:58:59,825 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 13:58:59,847 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (estimated: 20.64GB, total: 20.64GB)
2026-05-05 13:58:59,848 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 13:59:00,248 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 13:59:00,607 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 27 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 14:03:26,152 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request 61dd7177-6c93-4d0e-be1c-fb75e396fd83
2026-05-05 14:03:31,743 - omlx.scheduler - INFO - [-] - Prefill interrupted at 94208/147318 tokens: 1 request(s) aborted
2026-05-05 14:03:34,739 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 14:03:34,740 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:03:34,740 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:03:34,740 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:03:34,798 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:03:34,825 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:03:34,873 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:03:34,882 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:03:35,113 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=21.75GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 14:03:35,113 - omlx.admin.routes - INFO - [-] - Manually unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:04:09,943 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:04:16,872 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:04:23,121 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:04:24,309 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:04:24,310 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:04:24,310 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:04:24,310 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:04:24,310 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:04:24,378 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=160, indexed=160, errors=0, total_size=22.65 GB
2026-05-05 14:04:24,378 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=160, disk_free=381.98 GB, cache_used=22.65 GB
2026-05-05 14:04:24,378 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:04:24,378 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:04:24,379 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:04:24,379 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:04:24,379 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:04:24,379 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:04:25,233 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=256, max_blocks=100000, max_tokens=204800000
2026-05-05 14:04:25,234 - omlx.scheduler - INFO - [-] - SpecPrefill: draft model set with SSD cache (model_name=/Users/roelvideler/.cache/huggingface/hub/Qwen3.5-2B-oQ8-fp16)
2026-05-05 14:04:25,234 - omlx.engine.vlm - INFO - [-] - SpecPrefill: draft model loaded (/Users/roelvideler/.cache/huggingface/hub/Qwen3.5-2B-oQ8-fp16)
2026-05-05 14:04:25,292 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:04:25,295 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:04:25,298 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (estimated: 36.86GB, total: 36.86GB)
2026-05-05 14:04:25,299 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:05:26,679 - omlx.scheduler - INFO - [-] - SpecPrefill: scored 84424 tokens in 60.9s, selected 16872/84424 (keep=20%, prompt 159602 = system 75178 + conv 84424, cached 0)
2026-05-05 14:07:57,888 - omlx.scheduler - INFO - [-] - SpecPrefill: system prompt 75178 tokens full prefill
2026-05-05 14:09:06,128 - omlx.scheduler - INFO - [-] - SpecPrefill: sparse prefill 16871/84424 conv tokens in 217.9s (total 159602, cached 0, system 75178 full, conv 84424 sparse)
2026-05-05 14:09:09,189 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.0ms/1
2026-05-05 14:11:30,498 - omlx.scheduler - INFO - [-] - SpecPrefill: scored 85472 tokens in 140.3s, selected 17120/85472 (keep=20%, draft cache hit 83968, prompt 160650 = system 75178 + conv 85472, cached 0)
2026-05-05 14:13:42,486 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request 635d130c-633a-4707-8a1c-4684b6ff1965
2026-05-05 14:13:59,274 - omlx.scheduler - INFO - [-] - SpecPrefill: system prompt 75178 tokens full prefill
2026-05-05 14:14:20,411 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:14:20,413 - omlx.admin.routes - INFO - [-] - Settings changed for loaded model Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16. Reload required to take effect.
2026-05-05 14:14:27,105 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (immediate abort)
2026-05-05 14:14:27,105 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:14:27,105 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:14:27,105 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 2430348873756347
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7556cf76164d9ea1
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block fdf359fbb179e2ed
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 72d02cde747d0414
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 7d1e5611e041e6ec
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 1a03c7e70cd2dd3a
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 0a230a00f788b95c
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD write queue full, dropping evicted block 4e6f770aa58dff7e
2026-05-05 14:14:27,106 - omlx.cache.paged_ssd_cache - INFO - [-] - Flushed 33 hot cache blocks to SSD write queue
2026-05-05 14:14:28,288 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:14:28,309 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:14:28,382 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:14:28,391 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:14:54,820 - omlx.server - WARNING - [-] - POST /admin/api/models/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16/unload → 400: Model not loaded: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:15:07,656 - omlx.scheduler - ERROR - [-] - SpecPrefill sparse prefill failed: 'NoneType' object has no attribute 'layers'
2026-05-05 14:15:07,656 - omlx.scheduler - WARNING - [-] - Cache corruption detected: 'NoneType' object has no attribute 'layers', clearing cache and re-prefilling...
2026-05-05 14:15:07,658 - omlx.scheduler - INFO - [-] - Cache recovery completed
2026-05-05 14:15:07,769 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16, freed=43.02GB (expected>=34.86GB), active_memory: 1.86GB (settled)
2026-05-05 14:15:07,769 - omlx.admin.routes - INFO - [-] - Manually unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:15:10,109 - omlx.model_settings - INFO - [-] - Updated settings for model 'Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16'
2026-05-05 14:15:37,586 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:15:41,645 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:15:42,836 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:15:42,836 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:15:42,836 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:15:42,905 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=193, indexed=193, errors=0, total_size=23.42 GB
2026-05-05 14:15:42,905 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=193, disk_free=381.21 GB, cache_used=23.42 GB
2026-05-05 14:15:42,905 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:15:42,905 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:15:42,906 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:15:42,906 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:15:42,906 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:15:42,906 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:15:42,962 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:15:42,965 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16
2026-05-05 14:15:42,970 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (estimated: 20.64GB, total: 20.64GB)
2026-05-05 14:15:42,971 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:15:43,370 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 14:15:44,027 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 27 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 14:15:55,500 - omlx.engine.vlm - INFO - [-] - [vlm_stream_generate] Aborting request a0da1670-15cd-40cf-bdbb-1d70cc4f73d9
2026-05-05 14:15:57,464 - omlx.scheduler - INFO - [-] - Prefill interrupted at 10240/147318 tokens: 1 request(s) aborted
2026-05-05 14:15:59,818 - omlx.engine_pool - INFO - [-] - Unloading model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16 (immediate abort)
2026-05-05 14:15:59,819 - omlx.engine_core - INFO - [-] - Engine stopped
2026-05-05 14:15:59,819 - omlx.scheduler - INFO - [-] - Scheduler shutdown initiated...
2026-05-05 14:15:59,819 - omlx.cache.paged_ssd_cache - INFO - [-] - Shutting down PagedSSDCacheManager...
2026-05-05 14:15:59,879 - omlx.scheduler - INFO - [-] - Scheduler shutdown completed
2026-05-05 14:15:59,900 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager cleared (reset to 32 initial blocks)
2026-05-05 14:15:59,950 - omlx.scheduler - INFO - [-] - Deep reset completed - all caches cleared
2026-05-05 14:15:59,958 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine stopped
2026-05-05 14:16:00,177 - omlx.engine_pool - INFO - [-] - Unloaded model: Qwen3.6-35B-A3B-MLX-VL-oQ4-FP16, freed=22.01GB (expected>=18.64GB), active_memory: 28.00B (settled)
2026-05-05 14:16:00,177 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:16:06,482 - omlx.engine.vlm - INFO - [-] - Vision feature cache enabled (SSD: /Users/roelvideler/.omlx/cache/vision_features)
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-05-05 14:16:07,666 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-05-05 14:16:07,666 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=32, max_blocks=100000, max_tokens=204800000
2026-05-05 14:16:07,667 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Users/roelvideler/.omlx/cache
2026-05-05 14:16:07,734 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=193, indexed=193, errors=0, total_size=23.42 GB
2026-05-05 14:16:07,734 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, hot_cache=4.00 GB, existing_files=193, disk_free=381.20 GB, cache_used=23.42 GB
2026-05-05 14:16:07,734 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-05-05 14:16:07,734 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-05-05 14:16:07,734 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Users/roelvideler/.omlx/cache, max_size=185.00 GB, block_size=2048 tokens
2026-05-05 14:16:07,734 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Users/roelvideler/.omlx/cache, block_size=2048, max_blocks=100000
2026-05-05 14:16:07,734 - omlx.engine_core - INFO - [-] - Engine started
2026-05-05 14:16:07,734 - omlx.engine.vlm - INFO - [-] - TurboQuant KV cache enabled for VLM: 8.0 bits
2026-05-05 14:16:07,792 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-05-05 14:16:07,796 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/roelvideler/.cache/huggingface/hub/Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16
2026-05-05 14:16:07,802 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.6-35B-A3B-MLX-VL-oQ8-FP16 (estimated: 36.86GB, total: 36.86GB)
2026-05-05 14:16:07,803 - omlx.engine.vlm - INFO - [-] - Structured output is not available in the DMG version (xgrammar requires torch which significantly increases app size). Use the pip or Homebrew version for structured output support.
2026-05-05 14:26:14,874 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 75878bf0-69d4-493a-a0ee-bd09e00d58a2: storing 157696/159709 tokens (skipping trailing partial block, 76 intermediate snapshots)
2026-05-05 14:26:15,145 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.1ms/1, store_cache_main_eval=99.0ms/1, store_cache_main_prep=103.3ms/1
2026-05-05 14:26:15,301 - omlx.cache.paged_cache - INFO - [-] - Block pool grown: +256, total=288/100000
2026-05-05 14:27:22,383 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for a3a5bd77-db9f-4e0e-8b15-589db0064019: storing 161792/163249 tokens (skipping trailing partial block, 1 intermediate snapshots)
2026-05-05 14:27:22,430 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.1ms/2, store_cache_main_eval=142.9ms/2, store_cache_main_prep=150.5ms/2
2026-05-05 14:27:45,011 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.2ms/3, store_cache_main_eval=241.3ms/3, store_cache_main_prep=249.0ms/3
2026-05-05 14:28:04,732 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.2ms/4, store_cache_main_eval=336.8ms/4, store_cache_main_prep=344.5ms/4
2026-05-05 14:28:58,534 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 376cd0a3-8bbf-43e9-9d5e-d1ab8caf1c64: storing 163840/165741 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 14:28:58,647 - omlx.scheduler - INFO - [-] - Cache phase timings: cleanup_finished_sync=0.3ms/5, store_cache_main_eval=437.7ms/5, store_cache_main_prep=446.5ms/5
2026-05-05 14:29:38,379 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.4ms/6, store_cache_main_eval=532.4ms/6, store_cache_main_prep=541.2ms/6
2026-05-05 18:16:28,764 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 0219abec-8537-4c35-94d1-473b8df62a2a: storing 165888/166446 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:16:28,890 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.5ms/7, store_cache_main_eval=635.1ms/7, store_cache_main_prep=649.4ms/7
2026-05-05 18:16:51,671 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.5ms/8, store_cache_main_eval=734.9ms/8, store_cache_main_prep=749.2ms/8
2026-05-05 18:17:22,475 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.6ms/9, store_cache_main_eval=833.6ms/9, store_cache_main_prep=847.9ms/9
2026-05-05 18:17:55,147 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 95739b9f-b7ff-497d-9a84-165032575bf2: storing 167936/168182 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:17:55,269 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.6ms/10, store_cache_main_eval=935.3ms/10, store_cache_main_prep=952.1ms/10
2026-05-05 18:18:03,671 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.7ms/11, store_cache_main_eval=1039.6ms/11, store_cache_main_prep=1056.4ms/11
2026-05-05 18:18:11,176 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.7ms/12, store_cache_main_eval=1137.2ms/12, store_cache_main_prep=1154.1ms/12
2026-05-05 18:18:40,420 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 298218f9-b4f3-461d-b392-dd81dd3c5b7f: storing 169984/170643 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:18:40,538 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.8ms/13, store_cache_main_eval=1241.1ms/13, store_cache_main_prep=1260.5ms/13
2026-05-05 18:18:53,050 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.8ms/14, store_cache_main_eval=1342.2ms/14, store_cache_main_prep=1361.7ms/14
2026-05-05 18:19:17,759 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=0.9ms/15, store_cache_main_eval=1442.2ms/15, store_cache_main_prep=1461.6ms/15
2026-05-05 18:19:51,532 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.0ms/16, store_cache_main_eval=1544.2ms/16, store_cache_main_prep=1563.8ms/16
2026-05-05 18:20:27,163 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 24816554-3fc2-4df3-a28c-fc0f2c67da2a: storing 172032/173240 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:20:27,233 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.0ms/17, store_cache_main_eval=1570.1ms/17, store_cache_main_prep=1591.2ms/17
2026-05-05 18:20:48,454 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.1ms/18, store_cache_main_eval=1677.3ms/18, store_cache_main_prep=1698.4ms/18
2026-05-05 18:21:05,136 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.1ms/19, store_cache_main_eval=1779.2ms/19, store_cache_main_prep=1800.4ms/19
2026-05-05 18:21:23,688 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.2ms/20, store_cache_main_eval=1887.7ms/20, store_cache_main_prep=1908.9ms/20
2026-05-05 18:22:09,069 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 25bc82fd-8432-4791-a7e6-7b55b93ab011: storing 174080/175213 tokens (skipping trailing partial block, 0 intermediate snapshots)
2026-05-05 18:22:09,218 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.3ms/21, store_cache_main_eval=2001.6ms/21, store_cache_main_prep=2025.7ms/21
2026-05-05 18:22:10,073 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/0/05c7072110d47b7804e51e77126ecd2a54adb150b6ffb1c7259ba681e59a16fa.safetensors
2026-05-05 18:22:10,076 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 1/85 blocks, 2048 tokens
2026-05-05 18:34:09,065 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 567eb638-ac4c-49b6-a3f4-def16ffbc4a4: storing 174080/175550 tokens (skipping trailing partial block, 83 intermediate snapshots)
2026-05-05 18:34:09,374 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.3ms/22, store_cache_main_eval=2118.4ms/22, store_cache_main_prep=2147.1ms/22
2026-05-05 18:34:10,380 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/7/7cfb553cf3dd28c23254bdbb77a4abddb2fc383b13a5af8f5c7f9e3336ca2950.safetensors
2026-05-05 18:34:10,380 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 5/85 blocks, 10240 tokens
2026-05-05 18:46:14,375 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for e4446ef7-1ffd-49e3-a97a-fd7a42e10cf5: storing 174080/175755 tokens (skipping trailing partial block, 79 intermediate snapshots)
2026-05-05 18:46:14,618 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=3.1ms/1, boundary_capture_sync=4.5ms/1, boundary_snapshot_save=13.3ms/1, cleanup_finished_sync=1.4ms/23, store_cache_main_eval=2228.4ms/23, store_cache_main_prep=2261.4ms/23
2026-05-05 18:46:15,557 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/c/c92e8d13913e0c94d64e268a6db383e1ac29adc82494dd7ed12ef5807f4a102d.safetensors
2026-05-05 18:46:15,557 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 9/85 blocks, 18432 tokens
2026-05-05 18:46:15,557 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 3 trailing block(s) with placeholder non-sliceable state, keeping 6 block(s) (12288 tokens)
2026-05-05 18:58:26,665 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for b50aa733-5c0a-418d-83c8-851a35545e00: storing 174080/176205 tokens (skipping trailing partial block, 78 intermediate snapshots)
2026-05-05 18:58:26,914 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.5ms/24, store_cache_main_eval=2345.5ms/24, store_cache_main_prep=2384.1ms/24
2026-05-05 19:00:35,093 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/6/69a7d9c1ea51c7d2a5c594263929439142e1961d50d1c809977e2f4f0170805f.safetensors
2026-05-05 19:00:35,093 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 17/85 blocks, 34816 tokens
2026-05-05 19:00:35,093 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 7 trailing block(s) with placeholder non-sliceable state, keeping 10 block(s) (20480 tokens)
2026-05-05 19:12:30,540 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for d4133bc6-77ca-4760-b6a8-d93cb65848e4: storing 176128/176402 tokens (skipping trailing partial block, 75 intermediate snapshots)
2026-05-05 19:12:30,760 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.6ms/25, store_cache_main_eval=2456.3ms/25, store_cache_main_prep=2500.0ms/25
2026-05-05 19:12:32,935 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/c/cddf177778d4ef9c9a6e49811601334ebdb6471a21846ac74ebcc90185c181d1.safetensors
2026-05-05 19:12:32,935 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 79/86 blocks, 161792 tokens
2026-05-05 19:14:36,549 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 23d78da4-26fd-4d9d-b6f2-946a4714cdfc: storing 176128/177038 tokens (skipping trailing partial block, 6 intermediate snapshots)
2026-05-05 19:14:36,661 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.6ms/26, store_cache_main_eval=2563.1ms/26, store_cache_main_prep=2610.1ms/26
2026-05-05 19:15:02,064 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.7ms/27, store_cache_main_eval=2666.8ms/27, store_cache_main_prep=2713.9ms/27
2026-05-05 19:15:47,122 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/7/7cfb553cf3dd28c23254bdbb77a4abddb2fc383b13a5af8f5c7f9e3336ca2950.safetensors
2026-05-05 19:15:47,122 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 5/86 blocks, 10240 tokens
2026-05-05 19:28:03,077 - omlx.scheduler - INFO - [-] - Using boundary cache snapshot for 8161b8f5-7856-4356-853c-bc4928a0152e: storing 178176/178817 tokens (skipping trailing partial block, 81 intermediate snapshots)
2026-05-05 19:28:03,339 - omlx.scheduler - INFO - [-] - Cache phase timings: boundary_capture_extract=6.6ms/2, boundary_capture_sync=9.2ms/2, boundary_snapshot_save=21.9ms/2, cleanup_finished_sync=1.7ms/28, store_cache_main_eval=2784.7ms/28, store_cache_main_prep=2837.7ms/28
2026-05-05 19:28:15,226 - omlx.cache.paged_ssd_cache - WARNING - [-] - SSD cache file missing: /Users/roelvideler/.omlx/cache/6/69a7d9c1ea51c7d2a5c594263929439142e1961d50d1c809977e2f4f0170805f.safetensors
2026-05-05 19:28:15,226 - omlx.cache.prefix_cache - INFO - [-] - Partial cache reconstruction: 17/87 blocks, 34816 tokens
2026-05-05 19:28:15,226 - omlx.cache.prefix_cache - INFO - [-] - Walk-back truncation: dropped 7 trailing block(s) with placeholder non-sliceable state, keeping 10 block(s) (20480 tokens)`