Skip to content

Keep Chunk.take and Chunk.drop valid for fractional counts - #6899

Merged
tim-smart merged 3 commits into
mainfrom
audit/repro-core-chunk-fractional-count
Aug 3, 2026
Merged

Keep Chunk.take and Chunk.drop valid for fractional counts#6899
tim-smart merged 3 commits into
mainfrom
audit/repro-core-chunk-fractional-count

Conversation

@fubhy

@fubhy fubhy commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

Fractional take and drop counts produce chunks with fractional public lengths that throw RangeError when materialized.

Important

This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.

Fractional counts create malformed chunks

Module: Chunk
Audit ID: core-a-f-chunk-fractional-slice-dimensions
Severity / confidence: high / high

What happens

Fractional take and drop counts produce chunks with fractional public lengths that throw RangeError when materialized.

Why it happens

Fractional counts are stored directly as ISlice.length and ISlice.offset. Materialization passes the fractional length to new Array(self.length), which throws RangeError: Invalid array length.

Expected behavior

take and drop return valid chunks containing a count-based prefix or suffix for their accepted number argument; no integer-only precondition is imposed.

Relevant implementation

These links and excerpts are pinned to audit base c9b56ab507f224426ee8388dc450da447ec4715f.

View problematic code at packages/effect/src/Chunk.ts:728-767
export const take: {
  (n: number): <A>(self: Chunk<A>) => Chunk<A>
  <A>(self: Chunk<A>, n: number): Chunk<A>
} = dual(2, <A>(self: Chunk<A>, n: number): Chunk<A> => {
  if (n <= 0) {
    return _empty
  } else if (n >= self.length) {
    return self
  } else {
    switch (self.backing._tag) {
      case "ISlice": {
        return makeChunk({
          _tag: "ISlice",
          chunk: self.backing.chunk,
          length: n,
          offset: self.backing.offset
        })
      }
      case "IConcat": {
        if (n > self.left.length) {
          return makeChunk({
            _tag: "IConcat",
            left: self.left,
            right: take(self.right, n - self.left.length)
          })
        }

        return take(self.left, n)
      }
      default: {
        return makeChunk({
          _tag: "ISlice",
          chunk: self,
          offset: 0,
          length: n
        })
      }
    }
  }
})

View exact lines on GitHub

View problematic code at packages/effect/src/Chunk.ts:784-822
export const drop: {
  (n: number): <A>(self: Chunk<A>) => Chunk<A>
  <A>(self: Chunk<A>, n: number): Chunk<A>
} = dual(2, <A>(self: Chunk<A>, n: number): Chunk<A> => {
  if (n <= 0) {
    return self
  } else if (n >= self.length) {
    return _empty
  } else {
    switch (self.backing._tag) {
      case "ISlice": {
        return makeChunk({
          _tag: "ISlice",
          chunk: self.backing.chunk,
          offset: self.backing.offset + n,
          length: self.backing.length - n
        })
      }
      case "IConcat": {
        if (n > self.left.length) {
          return drop(self.right, n - self.left.length)
        }
        return makeChunk({
          _tag: "IConcat",
          left: drop(self.left, n),
          right: self.right
        })
      }
      default: {
        return makeChunk({
          _tag: "ISlice",
          chunk: self,
          offset: n,
          length: self.length - n
        })
      }
    }
  }
})

View exact lines on GitHub

View problematic code at packages/effect/src/Chunk.ts:408-427
const toReadonlyArray_ = <A>(self: Chunk<A>): ReadonlyArray<A> => {
  switch (self.backing._tag) {
    case "IEmpty": {
      return emptyArray
    }
    case "IArray": {
      return self.backing.array
    }
    default: {
      const arr = new Array<A>(self.length)
      copyToArray(self, arr, 0)
      self.backing = {
        _tag: "IArray",
        array: arr
      }
      self.left = _empty
      self.right = _empty
      self.depth = 0
      return arr
    }

View exact lines on GitHub

Reproduction

pnpm test --run packages/effect/test/Chunk.test.ts -t "produces valid chunks for fractional counts"

Observed failure: Materializing Chunk.take(Chunk.make(1, 2, 3), 1.5) threw RangeError: Invalid array length.

Implementation handoff

The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.

  1. Start with the pinned implementation excerpts and the Why it happens analysis above.
  2. Change the implementation so it satisfies the stated Expected behavior; do not weaken or remove the reproduction assertions.
  3. Run the focused reproduction command(s) and confirm the observed failures become passing tests:
pnpm test --run packages/effect/test/Chunk.test.ts -t "produces valid chunks for fractional counts"
  1. Run the affected package's existing tests, then the repository lint and type checks before requesting review.

Audit provenance

  • Audit base: c9b56ab507f224426ee8388dc450da447ec4715f
  • Reproduction base: c9b56ab507f224426ee8388dc450da447ec4715f
  • Findings: core-a-f-chunk-fractional-slice-dimensions
  • Initial patch: focused reproduction tests; implementation fix pending

Closes EFF-350

@github-project-automation github-project-automation Bot moved this to Discussion Ongoing in PR Backlog Aug 3, 2026
@changeset-bot

changeset-bot Bot commented Aug 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 91bf5ed

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-slopcop effect-slopcop Bot added the 4.0 label Aug 3, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — one reproduction test added to Chunk.test.ts:

  • Fractional-count reproduction: adds it("produces valid chunks for fractional counts") under the existing take describe block, asserting take(1.5)[1] and drop(1.5)[2, 3]. The test correctly surfaces the RangeError: Invalid array length caused by the internal ISlice / IConcat backing structures using a non-integer length field.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) | 𝕏

@fubhy fubhy changed the title Add reproduction for Chunk issue Keep Chunk.take and Chunk.drop valid for fractional counts Aug 3, 2026
@effect-slopcop effect-slopcop Bot added the bug Something isn't working label Aug 3, 2026
@fubhy fubhy added the audit Findings originating from the Effect runtime correctness audit label Aug 3, 2026
Comment thread packages/effect/src/Chunk.ts Outdated
<A>(self: Chunk<A>, n: number): Chunk<A>
} = dual(2, <A>(self: Chunk<A>, n: number): Chunk<A> => {
if (n <= 0) {
const _n = Math.floor(n)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should flip the variable names so the function parameter has the underscore.

@tim-smart
tim-smart enabled auto-merge (squash) August 3, 2026 21:39

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changesMath.floor fix applied to take and drop, plus naming feedback:

  • Fractional-count fix in take and drop: added const n = Math.floor(_n) at the top of both functions so slice length and offset fields are always integers. This resolves the RangeError: Invalid array length thrown during materialization of chunks produced by fractional counts.
  • Variable naming flipped per review feedback: the function parameter is now _n (underscore-prefixed raw input) and the floored local is n.
  • Reproduction test retained: it("produces valid chunks for fractional counts") asserts take(1.5) → [1] and drop(1.5) → [2, 3].

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) | 𝕏

@tim-smart
tim-smart merged commit 985de09 into main Aug 3, 2026
16 checks passed
@tim-smart
tim-smart deleted the audit/repro-core-chunk-fractional-count branch August 3, 2026 21:55
@github-project-automation github-project-automation Bot moved this from Discussion Ongoing to Done in PR Backlog Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
basic.ts 7.06 KB 7.06 KB 0.00 KB (0.00%)
batching.ts 9.86 KB 9.86 KB 0.00 KB (0.00%)
brand.ts 6.34 KB 6.34 KB 0.00 KB (0.00%)
cache.ts 10.62 KB 10.62 KB 0.00 KB (0.00%)
config.ts 20.60 KB 20.60 KB 0.00 KB (0.00%)
differ.ts 20.20 KB 20.20 KB 0.00 KB (0.00%)
http-client.ts 21.49 KB 21.49 KB 0.00 KB (0.00%)
logger.ts 10.76 KB 10.76 KB 0.00 KB (0.00%)
metric.ts 8.99 KB 8.99 KB 0.00 KB (0.00%)
optic.ts 7.18 KB 7.18 KB 0.00 KB (0.00%)
pubsub.ts 14.90 KB 14.90 KB 0.00 KB (0.00%)
queue.ts 11.58 KB 11.58 KB 0.00 KB (0.00%)
schedule.ts 10.74 KB 10.74 KB 0.00 KB (0.00%)
schema-class.ts 19.14 KB 19.14 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 28.96 KB 28.96 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 25.29 KB 25.29 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.30 KB 13.30 KB 0.00 KB (0.00%)
schema-string.ts 10.94 KB 10.94 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.17 KB 15.17 KB 0.00 KB (0.00%)
schema-toArbitraryLazy.ts 21.94 KB 21.94 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.34 KB 24.34 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.18 KB 19.18 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 19.01 KB 19.01 KB 0.00 KB (0.00%)
schema-toFormatter.ts 18.87 KB 18.87 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 22.60 KB 22.60 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.52 KB 19.52 KB 0.00 KB (0.00%)
schema.ts 18.41 KB 18.41 KB 0.00 KB (0.00%)
stm.ts 12.54 KB 12.54 KB 0.00 KB (0.00%)
stream.ts 9.79 KB 9.79 KB 0.00 KB (0.00%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 audit Findings originating from the Effect runtime correctness audit bug Something isn't working

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants