Skip to content

Explicitly using fsspec and zarr FsspecStore causes RuntimeError "Task attached to a different loop" #3487

@dmitriyrepin

Description

@dmitriyrepin

Summary

Mixing explicit use of fsspec and zarr FsspecStore produces the following error:
'RuntimeError: Task <Task pending ... > attached to a different loop. Task was destroyed but it is pending!'

Reproducers:

  1. When one explicitly creates fsspec.AbstractFileSystem and then uses it in zarr.create_array(), he/she will observe error mentioned above
    gs_file = "gs://my_data/array.tmp"

    fs, url = fsspec.url_to_fs(gs_file)
    store1: fsspec.AbstractFileSystem = fs.get_mapper(url)
    arr1 = zarr.create_array(store1, shape=(2, 2), dtype='int32', overwrite=True)
    arr1[:] = np.array([[1, 2], [3, 4]], dtype='int32')
  1. When one explicitly uses fsspec functionality and then uses zarr.create_array(), he/she will observe error mentioned above:
    gs_file = "gs://my_data/array.tmp"

    fs: fsspec.AbstractFileSystem = fsspec.filesystem('gs')
    info = fs.info(gs_file)

    # Create a zarr array in the cloud
    store1: fsspec.AbstractFileSystem = zarr.storage.FsspecStore.from_url(gs_file, read_only=False)
    arr1 = zarr.create_array(store1, shape=(2, 2), dtype='int32', overwrite=True)
    arr1[:] = np.array([[1, 2], [3, 4]], dtype='int32')

Suspected cause:

When one explicitly uses fsspec, it will create asyncio thread with an event loop (the fsspecIO thread will also be created).
If one later would try to use the zarr FsspecStore, which under the hood also uses fsspec, it will create the second asyncio thread with the second event loop (the zarr_io thread will also be created), which likely causes the error down the road

Image

Versions

  • zarr: 3.1.2
  • Python: 3.13.5
  • fsspec: 2025.9.0
  • gcsfs: 2025.9.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions