Skip to content

fix cached self entries fsspec#8863

Merged
Light2Dark merged 4 commits intomainfrom
sham/fix-self-entries-fsspec
Mar 25, 2026
Merged

fix cached self entries fsspec#8863
Light2Dark merged 4 commits intomainfrom
sham/fix-self-entries-fsspec

Conversation

@Light2Dark
Copy link
Copy Markdown
Collaborator

@Light2Dark Light2Dark commented Mar 25, 2026

📝 Summary

image

Some fsspec backends can return the queried directory itself as a child entry when listings are from a stale parent cache.

Backend
We check if there is an exact match & only 1 returned result when doing ls. If yes, invalidate cache, and try ls again.

The logic is intentionally simple to catch the simplest case, I've not seen more complex cases of this.

Frontend
Updated the generated GoogleDriveFileSystem(...) snippet to include use_listings_cache=False, which wouldn't cause this issue in the first place.

📋 Checklist

  • I have read the contributor guidelines.
  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • Tests have been added for the changes made.
  • Documentation has been updated where applicable, including docstrings for API changes.
  • Pull request title is a good summary of the changes - it will be used in the release notes.

@Light2Dark Light2Dark requested a review from manzt as a code owner March 25, 2026 05:27
Copilot AI review requested due to automatic review settings March 25, 2026 05:27
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 25, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Mar 25, 2026 6:00am

Request Review

@Light2Dark Light2Dark added the bug Something isn't working label Mar 25, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 25, 2026

Bundle Report

Changes will increase total bundle size by 52 bytes (0.0%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
marimo-esm 25.59MB 52 bytes (0.0%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: marimo-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/add-*.js 52 bytes 55.39kB 0.09%

Files in assets/add-*.js:

  • ./src/components/editor/connections/storage/as-code.ts → Total Size: 4.97kB

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Addresses an edge case in fsspec directory listings where some backends can return the queried directory itself as a child entry (often due to stale listing caches), which can cause recursive “folder contains itself” trees in the storage inspector UI.

Changes:

  • Backend: detect self-entry echoes in FsspecFilesystem.list_entries, invalidate relevant fsspec dircache keys, retry once, and finally filter exact self-entries.
  • Tests: add coverage for the retry-on-self-entry and exact-self-entry filtering behavior.
  • Frontend: update generated GoogleDriveFileSystem(...) snippet to set use_listings_cache=False.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/_data/_external_storage/test_storage_models.py Adds tests for retrying and filtering when a self-entry is returned by ls().
marimo/_data/_external_storage/storage.py Implements self-entry detection, cache invalidation + retry, and final filtering for fsspec listings.
frontend/src/components/editor/connections/storage/as-code.ts Updates generated Google Drive connection code to disable listing cache.
frontend/src/components/editor/connections/storage/tests/snapshots/as-code.test.ts.snap Updates snapshots to reflect the new generated Google Drive code.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@Light2Dark Light2Dark merged commit 3cf9f4a into main Mar 25, 2026
46 of 71 checks passed
@Light2Dark Light2Dark deleted the sham/fix-self-entries-fsspec branch March 25, 2026 13:12
@github-actions
Copy link
Copy Markdown

🚀 Development release published. You may be able to view the changes at https://marimo.app?v=0.21.2-dev55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants