Skip to content

fix(state): persist json checkpoints as utf-8 - #7255

Closed
Ghraven wants to merge 1 commit into
crewAIInc:mainfrom
Ghraven:raven-utf8-json-provider
Closed

fix(state): persist json checkpoints as utf-8#7255
Ghraven wants to merge 1 commit into
crewAIInc:mainfrom
Ghraven:raven-utf8-json-provider

Conversation

@Ghraven

@Ghraven Ghraven commented Sep 4, 2026

Copy link
Copy Markdown

Problem

JsonProvider persists checkpoint JSON with the platform default text encoding in both the sync and async paths. The matching checkpoint readers also rely on the default encoding.

That makes checkpoint round-trips environment-dependent when serialized state contains non-ASCII text on systems whose default encoding is not UTF-8.

Before / after

Before this change, checkpoint writes and reads used the process/platform default encoding.

After this change, sync and async checkpoint writes/readers use encoding="utf-8", matching JSON's expected interoperable encoding and keeping checkpoint behavior stable across platforms.

Verification

  • python -m py_compile lib/crewai/src/crewai/state/provider/json_provider.py lib/crewai/tests/test_checkpoint.py
  • git diff --check
  • direct smoke test of JsonProvider sync + async non-ASCII checkpoint round trip via file import: passed

I also attempted:

  • python -m pytest lib/crewai/tests/test_checkpoint.py -k "JsonProviderFork or checkpoint_uses_utf8" -q

That local run was blocked by environment setup in this bare checkout: first the configured --timeout=60 option required a missing pytest-timeout plugin, then collection required repo dependencies such as jsonref. The added tests are narrow and exercise the same provider paths as the passing direct smoke test.

Closes #7256

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Thanks for the pull request.

First-time contributors need an associated open issue before we can review a PR.

  1. Open an issue with a template, or pick an existing open one.
  2. Open a new PR (or reopen this one) whose title or body mentions that issue, for example #123.

See the contributing guide.

@github-actions github-actions Bot closed this Sep 4, 2026
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 0f6a9766-5b88-4835-a8f5-c6ab45072efc

📥 Commits

Reviewing files that changed from the base of the PR and between 92eb5f9 and c470dc5.

📒 Files selected for processing (2)
  • lib/crewai/src/crewai/state/provider/json_provider.py
  • lib/crewai/tests/test_checkpoint.py

📝 Walkthrough

Walkthrough

JsonProvider now uses explicit UTF-8 encoding for synchronous and asynchronous checkpoint reads and writes. Tests verify byte encoding and round-trip preservation for non-ASCII JSON.

Changes

UTF-8 checkpoint I/O

Layer / File(s) Summary
Explicit UTF-8 checkpoint I/O
lib/crewai/src/crewai/state/provider/json_provider.py, lib/crewai/tests/test_checkpoint.py
Synchronous and asynchronous checkpoint reads and writes now use UTF-8. Tests verify non-ASCII JSON bytes and round-trip results.

Suggested reviewers: greysonlalonde

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Ghraven

Ghraven commented Sep 4, 2026

Copy link
Copy Markdown
Author

Thank you for the guidance. I created tracking issue #7256 and linked it from the PR body with Closes #7256.

Local verification so far:

  • python -m py_compile lib/crewai/src/crewai/state/provider/json_provider.py lib/crewai/tests/test_checkpoint.py
  • git diff --check
  • direct UTF-8 checkpoint smoke test against the provider

I also attempted the focused pytest target, but this bare checkout is missing the repo test/dependency setup (pytest-timeout/jsonref). I am happy to add or adjust focused tests if the maintainers prefer a different shape here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON checkpoint provider uses locale default encoding

1 participant