Bitwiseness#14
Merged
Merged
Conversation
Much is still to be done and this isn't final
- Fix issue leftover in benchmark code that used out of date zpaq usage after a parallelization logic change - Add BinaryToken (bitwise) support natively - Make the `Ctw` model AC-CTW, and leave `FacCtw` as FAC-CTW, instead of `Ctw` being a wrapper over FacCtw, and removing convoluted AIXI logic around this. - Macroize RateBackend capabilities/runtime - For Byte models operating bitwise, add a binary tree to allow O(1) MSB and LSB lookup per-bit. - We have `BinaryTokens` - a true binary representation, for which native bit models work, and 2-symbol [0,1] byte model usage. Added `BytePrefix` as a session type, which uses the binary tree to decompose and provide per-bit probabilities. The former ensures per-bit learning, whereas the latter updates only per byte.
- Prevents RateBackendBitSession::finish() from rejecting cleanly-aligned streams by evaluating BytePrefixMass::has_partial_bits instead of failing if the prediction cache merely exists.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This introduces `supports_frozen_reset` and `begin_fresh_stream` to the `OnlineBytePredictor` trait, allowing backends like ZPAQ to gracefully signal their inability to perform in-place plugin state resets without violating the API semantic contract.
…tationarity in semantics; Also fix some outdated logic in scripts.
…provements - Add bindings for new bitwise helpers: - Expose BitOrder, BitStreamSemantics, BinaryPrediction, BytePrefixMass, and RateBackendBitSession - Add InfotheoryCtx.rate_backend_bit_session and string alias parsing for bit semantics/order - Add python tests for said new bitwise python code - In Infotheory, Add BytePrefixMass::from_log_probs - Route RateBackendBitSession prefix initialization through the shared helper
- Add `begin_stream` hooks to `RateBackendSession` and `RateBackendBitSession` - Update `MixtureRuntime` to route restarts via `begin_fresh_stream`. Mixture weights and cumulative log losses correctly reset to their priors, while delegating to each expert's supported initialization behavior (preserving fitted state on resettable experts while hard-restarting non-resettable ones). - Harden the ZPAQ memory lifecycle by guaranteeing a sequential `drop(take())` of the underlying streaming context prior to reallocation. - Update tests accordingly
…e backends - Implement `MixtureBitPrefixState` to step bit-by-bit natively through mixture experts and avoiding redundant full 256-way byte PDF materialization. - Introduce `RateBackendBitSessionCheckpoint` to enable exact reversible checkpoints for byte-packed bit sessions. - Update `api/context.rs` to support `BufferedBytePrefixKind::NativeMsb` with transactional frozen-update rollbacks.
- AIQI byte-packed validation now treats observations plus reward as one percept segment instead of requiring each to be byte-aligned separately. - ZPAQ update_and_score now reports raw compressor bit growth again in zpaq_rate.rs, so the probability floor stays confined to probability-query APIs instead of capping metric paths.
- ZpaqRateModel::fill_log_probs is now actually read-only with respect to pending speculative state - RateBackendBitSession::from_backend now rejects pathological BitStreamSemantics::BytePacked sessions cleanly instead of silently admitting ZPAQ into the O(256 * H) fallback path - added a separate capability, supports_efficient_byte_packed_bit_sessions
…x a few pre-existing bugs.
- Add lifecycle snapshot/restore coverage across mixture experts and key backends, tighten online policy stream init ordering, and keep FAC-CTW JSON canonical by omitting unset msb_first. - Enforce LF-only benchmark TSV output via script updates, .gitattributes, and tests.
…two_json.lua so it's not mistaken for label collision label collision isn't relevant to it's particular usecase as is
- Removed the hidden lifecycle methods/types from the public OnlineBytePredictor trait in crates/infotheory/src/mixture.rs. Internal rate-backend experts now use a private ExpertPredictor wrapper so compact lifecycle rollback stays available without leaking into the public trait. - Removed the dead FAC-CTW predict_update_byte_msb wrapper and its test from crates/infotheory/src/backends/ctw.rs. - Made RatePdfPredictor::from_compiled test-only in crates/infotheory/src/compression/mod.rs. - Removed the no-op ZPAQ release function and all surviving call sites across crates/infotheory/src/backends/zpaq_rate.rs, crates/infotheory/src/lib.rs, crates/infotheory/src/spec.rs, and crates/infotheory/ src/aixi/common.rs. - Documented lifecycle snapshot allocation cost for CTW/FAC-CTW and added justifications for the relevant too_many_arguments suppressions, including the switching-mixture helper that also appeared in the branch diff. - Removed redundant debug_assert! checks in crates/infotheory/src/prediction.rs; the runtime contract panic remains.
… parallel_uct cloned rollouts - Improves memory efficiency and parallel_uct performance (in theory...)
…ope hooks; update some docs for clarification.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make bitwise predictor and consumer support first-class, including bit-to-byte and byte-to-bit adaptability and interoperability.
This involves several layers:
Ctw-- which was previously AC-CTW in AIXI paths, but FAC-CTW in compression -- it's now just AC-CTW consistently, withFacCtwbeing the unambigous FAC-CTW path.This should open up the way to implement Secondary Symbol Estimation, and some more useful algorithms which wouldn't otherwise be fit for a bytewise prediction library.