[pull] main from danny-avila:main#134
Merged
Merged
Conversation
…ed Cache Reads (#296) * Fix prune calibration with cache token usage * Skip live specs when credentials are blank * Skip Anthropic live spec without credentials --------- Co-authored-by: Romuald Wandji <romualdesdras.wandji@kindredgroup.com>
…rt (#297) * feat: shared media token estimation + timed-media (video/audio) support Consolidate image/document token estimation into exported functions so consumers (LibreChat) price attachments identically to the SDK pruner, and add estimation for previously-unhandled video/audio. - Export estimateImageBlockTokens, estimateDocumentBlockTokens, estimateMediaTokensForMessage, IMAGE_TOKEN_SAFETY_MARGIN - Add estimateTimedMediaBlockTokens: Gemini-calibrated duration-from-size estimation for video (~300 tok/s) and audio (32 tok/s), flat fallback for bare URLs; wired into getTokenCountForMessage + estimateMediaTokensForMessage * fix: handle standard video/audio blocks and gate generic media by MIME Codex review on PR 297: - Add standard {type:'video'} / {type:'audio'} block shapes (accepted by the Bedrock converter) to timed-media handling, reading data (base64 string or Uint8Array), base64 url, or fileId. - Classify generic Google {type:'media'} blocks by MIME so an image/document media block is no longer mispriced as a ~30s video; non-video/audio media returns 0. * fix: native Bedrock media bytes, remote URI schemes, image_file pricing Codex round 2 on PR 297: - mediaBlockByteLength now reads the native Bedrock nested video/audio.source.bytes (Uint8Array), not just top-level data/url, so inline native media is sized instead of hitting the ~30s fallback. - base64ByteLength treats any URI scheme (gs://, s3://, file://, http(s)) as remote — a non-data URI is no longer mis-measured as base64 and clamped low. - estimateMediaTokensForMessage routes image_file through image estimation (1024 fallback) instead of the document path (2000), matching its doc. * refactor: drop dead estimateMediaTokensForMessage; count MIME-as-type media Codex round 3 on PR 297: - Remove estimateMediaTokensForMessage entirely. It had no caller (the block estimators are the shared source of truth used by getTokenCountForMessage), and keeping it created the image_file pricing divergence and the extra typed API surface Codex flagged. Deleting it resolves both. - Classify Google MIME-as-type blocks ({ type: 'audio/wav'|'video/mp4', data }) as timed media in timedMediaKind + a new isTimedMediaType gate, so they are counted by the pruner instead of returning 0. * fix: size nested video/audio data-URL payloads before falling back Codex round 4 on PR 297: mediaBlockByteLength now reads nested video/audio.{data,url} (the formatMessage media-array shape) in addition to source.bytes, so an inline nested data URL is sized from its bytes instead of the ~30s remote fallback. Remote nested URLs still fall back correctly.
* test: cover Langfuse cache usage export * fix: preserve Bedrock fresh input in Langfuse
#299) * feat(hooks): support injectedMessages in hook outputs at PostToolBatch Hooks can now return injectedMessages (InjectedMessage[]) alongside additionalContext. At the PostToolBatch boundary, ToolNode converts each entry into its own HumanMessage (role/source preserved in additional_kwargs) and appends them AFTER the consolidated hook-context message, so verbatim user speech — e.g. a mid-run steering message — keeps per-message identity and lands closest to the next model call. Eager-execution gates (prestart, early completion emission, eager event consumption) now key on HookRegistry.hasResultAlteringHooks() (PreToolUse/PostToolUse/PostToolUseFailure) instead of registry presence, so an observation-only registry (e.g. a PostToolBatch steering drain) keeps those fast paths. Exports HOOK_INJECTED_MESSAGES_CAPABLE so hosts can hard-gate drain-style hooks: a queued message must never be consumed by an SDK version that would silently drop it. * 🎯 fix: Scope Eager Gates to the Current Run's Hooks Codex P2: hasResultAlteringHooks() with no session id scans every session bucket, so another run registering session-scoped Pre/PostToolUse hooks mid-flight flipped canConsumeEagerEventExecution for this run — takeMatchingEagerEventExecution then left the eager record and re-dispatched the call, executing the tool twice. - Pass the run's hook-session id (runId / graph.runId) at every gate: isBatchSensitiveToolExecution, canEmitEarlyCompletions, canConsumeEagerEventExecution, recordEventToolPlanningTurn. - takeMatchingEagerEventExecution now checks static enablement only: a stored record means the reservation-time gates passed and the host already dispatched the execution, so consuming is the only path that cannot double-execute; PostToolUse hooks still process consumed results downstream.
* fix: honor stream delay for Bedrock * fix: pace Bedrock reasoning streams * fix: smooth Bedrock stream chunks * fix: narrow Bedrock reasoning deltas
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )