Conversation
…ke flags cbor_load pre-allocates storage for definite-length collections sized by the declared element count. A crafted input can therefore trigger a very large allocation before any data is validated. This was WAI and documented in the response to #418, but not in the library docs. - Add a warning to cbor_load's doc page explaining the risk and pointing to cbor_set_allocs (capping allocator) and the streaming decoder as the intended mitigations - Expand the cbor_set_allocs Doxygen comment to explain its role as the resource-limiting mechanism for untrusted input - Expand the cbor_set_allocs section in item_reference_counting.rst with a concrete note about the capping-allocator pattern Also clarify the CMake configuration docs: - Split getting_started.rst config table into build options and a separate test-only section; add WITH_EXAMPLES and SANITIZE which were missing - Fix SANE_MALLOC description: it gates tests that only pass when malloc refuses huge requests, and has no effect on the library itself - Fix HUGE_FUZZ and PRINT_FUZZ descriptions similarly - Add PRINT_FUZZ to the docs (it was in CMakeLists but not documented) - Update CMakeLists.txt option descriptions to match Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds examples/capped_alloc.c: a self-contained demonstration of using cbor_set_allocs with a size-capping malloc/realloc to defend against crafted inputs that declare enormous definite-length collections. Also updates: - cbor_set_allocs Doxygen doc to reference the example - doc/source/api/decoding.rst warning block to reference the example - doc/source/api/item_reference_counting.rst to show a code snippet instead of the trivial cbor_set_allocs(malloc, realloc, free) call Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace "as few as 9 bytes" / "tiny payload" with a plain statement: cbor_load allocates whatever the header declares, up to 2^64-1 elements. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous approach checked each individual malloc/realloc call against a fixed size limit. This replaces it with a global budget counter so the limit applies to total memory consumed across all libcbor allocations. Caveats the global counter as not thread-safe and notes that free() does not subtract (budget measures peak, not live bytes). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without the ability to query an allocation's size at free() time, capping_free() cannot subtract from the budget and allocated_bytes only ever grows, making it a peak counter rather than a live-bytes counter. The same problem affects realloc(): we cannot account for the old allocation being released. malloc_size() (macOS/BSD) and malloc_usable_size() (Linux/glibc) solve this. When either is available: - capping_malloc/realloc record the actual allocated size (which may be rounded up by the allocator) - capping_realloc subtracts the old size before checking the remaining budget, allowing reallocations that shrink or grow within budget - capping_free subtracts the freed size, so allocated_bytes tracks current live bytes On platforms without either function the previous conservative fallback is preserved: the budget only grows, causing cbor_load to fail sooner than strictly necessary, but always safely. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add ALLOC_SIZE support for Windows via _msize() from <malloc.h>. Restructure the allocator functions into two clean, self-contained blocks separated by #ifdef ALLOC_SIZE / #else / #endif, rather than scattering per-function #ifdefs. Each version is now easy to read in isolation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add sphinx.ext.graphviz to conf.py and graphviz apt package to the ReadTheDocs build so the diagram renders as SVG in HTML output. The new diagram fixes the layout problems of the original: - Three decoder alternatives are grouped in a dashed cluster labelled "choose one" instead of connected with lateral ↔ arrows - Client and Manipulation routines sit at the same level, connected with a dashed bidirectional edge labelled "use decoded items" - Stateless decoder sits at the bottom, with clean edges from each driver - No unexplained abbreviations (PDS, CDS, CD) or ambiguous line styles Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collapses the decoder options into a clean table comparing cbor_load (returns cbor_item_t, allocates) vs cbor_stream_decode (callbacks per primitive/chunk/boundary, no allocation, caller handles nesting). Links both functions and the streaming decoding page. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Shows the actual architecture: - Client connects to cbor_load (bytes) or cbor_stream_decode (bytes + callbacks) - cbor_load → cbor_stream_decode with a dashed "internal callbacks" arrow - cbor_load → cbor_item_t as its output - cbor_item_t ↔ Manipulation routines via dashed side edge Drops the abstract "Default/Streaming/Custom driver" cluster framing in favour of the real function names. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collections were already mentioned; strings and bytestrings have the same pre-allocation risk since their buffer is sized by the declared byte length in the header. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #423 +/- ##
=======================================
Coverage 99.94% 99.94%
=======================================
Files 20 20
Lines 1757 1757
=======================================
Hits 1756 1756
Misses 1 1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…-limits-and-test-flags
The old warning from master was retained alongside the improved one from this branch. Keep only the improved version. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tigations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add :width: 100% to pin table to page width - Narrow attribute column (20%) to give more room to content columns (40% each) - Break long single-line cells so they wrap within columns Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The RTD theme wraps all tables in .wy-table-responsive which adds overflow:auto, preventing :width: 100% from constraining column widths. Add _static/custom.css with a fixed-table class that forces table-layout: fixed and word-wrap on cells. Apply the class to the decoder comparison table via .. rst-class:: Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Nesting/Streaming: explain caller must maintain an explicit stack to track depth and accumulate children for nested structures - Best for/Default: bulleted list — trusted/bounded inputs, general data manipulation - Best for/Streaming: bulleted list — large/unbounded inputs, memory-constrained, custom structures, known restricted schemas Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cbor_load→cbor_stream_decodechain, client entry points, andcbor_item_t/ manipulation routines relationshipssphinx.ext.graphviztoconf.pyandgraphvizapt package to.readthedocs.yamlfor ReadTheDocs buildscbor_loadis built on top ofcbor_stream_decodeinternallycbor_loadwarning to cover strings and bytestrings (not just collections), and bullet-points the mitigationsexamples/capped_alloc.c: a self-contained capping allocator example usingmalloc_size/malloc_usable_size/_msizewhere available, falling back to a conservative peak counter elsewherecbor_set_allocsDoxygen doc and the decoding warningTest plan
examples/capped_allocbuilds (make capped_alloc)sphinx-build)apt_packages: [graphviz]🤖 Generated with Claude Code