v0.3.0

Large folders: faster to open, filter and page, above all with a remote kernel (e.g. JupyterHub).

• Filtering and paging no longer re-sort the list: a keystroke in the filter or a page change takes a few milliseconds, even with tens of thousands of items (it was 40–200 ms).
• Smaller listings: the folder's listing is sent to the browser about 10× smaller (about 50 bytes per item, whatever the length of paths and URLs), so large folders open faster over a slow connection.
• Very large folders are handled by the kernel: for folders with more than max_browser_items items (new option, default 20,000; None: never), the kernel filters, sorts and pages the list and sends one page at a time. Typing in the filter shows its effect at once, the next page is fetched ahead, and selections across pages work as before. Regular expressions there follow Python's syntax (e.g. (?P<name>...)).

Install

pip install -U jupyter-storage-explorer==0.3.0                # the widgets
pip install -U jupyter-storage-explorer-launcher==0.3.0       # plus the JupyterLab launcher item

What's Changed

• File list: sort once per listing, with one collator by @sugibuchi in #20
• File list: send the listing as columns, without paths or URLs by @sugibuchi in #21
• File list: large folders filtered, sorted and paged by the kernel by @sugibuchi in #22

Full Changelog: v0.2.2...v0.3.0

v0.2.2

Interface improvements; no API changes since 0.2.1.

• Filters have a clear button (✕), and are cleared as you browse: the folder filter when you click a folder in the tree, the file filter whenever the list shows another folder.
• File operations block the explorer while they run: moving, deleting, pasting, creating a folder, downloading and uploading show a busy dialog (e.g. "Deleting c.txt…", with the progress of uploads) after a moment, and nothing else can be clicked meanwhile.

Install

pip install -U jupyter-storage-explorer==0.2.2                # the widgets
pip install -U jupyter-storage-explorer-launcher==0.2.2       # plus the JupyterLab launcher item

What's Changed

• Filters: a clear button, cleared on navigation; a busy dialog during file operations by @sugibuchi in #19

Full Changelog: v0.2.1...v0.2.2

v0.2.1

A packaging release: no changes to the widgets since 0.2.0.

• MIT licence, now included in both packages and shown on PyPI and GitHub.
• PyPI project pages: author, keywords, classifiers and links (source, issues, changelog), and the README's
  screenshots now display on PyPI.

Install

pip install -U jupyter-storage-explorer==0.2.1                # the widgets
pip install -U jupyter-storage-explorer-launcher==0.2.1       # plus the JupyterLab launcher item

What's Changed

• MIT licence and PyPI metadata; absolute image and file links in the READMEs by @sugibuchi in #17
• Update CLAUDE.md: testing the JavaScript, CI, file operations, Unsafe mode, conventions by @sugibuchi in #18

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Highlights

File operations, from the right-click menu:

• Copy and Paste files into a folder, or into the folder shown by right-clicking empty space in the list, also between explorers and storages. The files' URLs go to the clipboard too.
• Delete files, after a confirmation.
• Create a folder: asks for the name and checks it. On S3 and Google Cloud Storage, which have no empty folders, it says the new folder shows once a file is put in it.
• Download to local saves files on your computer (formerly Download), and Download to JupyterLab copies them into a folder of the Jupyter server: download_to_jupyterlab_path="." (where the explorer runs) by default, e.g. "~/Downloads". Each can be turned off: allow_download_to_local=False, download_to_jupyterlab_path=None.
• Drag files onto JupyterLab's file browser to download them to the Jupyter server, after a confirmation (needs jupyter-storage-explorer-launcher).

Unsafe mode for folders: moving, copying, pasting and deleting folders (with everything in them) is offered only after turning on the Unsafe mode slider in the status bar. It allows one operation: any operation turns it off again.

Filters and paging:

• Filter the folder tree, the file list, and the data browser's Select columns by name: text, or a regular expression with .*; case is ignored.
• The file list shows 100 items per page (◀ ▶ above it).

Smaller changes:

• With several filesystems, a tree root of just / is named after its filesystem, e.g. / [Scratch (memory)].
• Fix: switching to a filesystem for the first time left its tree collapsed when filesystems share their root path (e.g. DirFileSystems).
• Fix: the right-click menu's last rows were cut off in short explorers.
• Fix: dragging a file that is part of a selection failed.

Changes to know about

• Download in the right-click menu is now Download to local; for files, Copy replaces Copy URL (it also puts the URLs on the clipboard); folders keep Copy URL.
• Subclasses: _mv(source, target) and _rm(path) take recursive= for folders; new overridable hooks _cp (copying a file) and _mkdir (creating a folder). All fsspec calls on the filesystem shown go through the methods of the File operations block of StorageExplorer.

Install

pip install -U jupyter-storage-explorer==0.2.0                # the widgets
pip install -U jupyter-storage-explorer-launcher==0.2.0       # plus the JupyterLab launcher item

What's Changed

• Add CI with Codecov coverage, PyPI and Codecov badges, and update actions by @sugibuchi in #8
• Expand the tree's root after switching to a filesystem shown for the first time by @sugibuchi in #9
• Download files to the Jupyter server by dragging them onto JupyterLab's file browser by @sugibuchi in #10
• Tree: name the filesystem next to a root of just / by @sugibuchi in #11
• Right-click: "Download to local" and "Download to JupyterLab" by @sugibuchi in #12
• Delete, Copy and Paste files; group StorageExplorer's file operations by @sugibuchi in #13
• Move, copy, paste and delete folders, in a one-shot Unsafe mode by @sugibuchi in #14
• Create a folder from the right-click menu by @sugibuchi in #15
• Filters for the tree, the file list and Select columns; pages of 100 items by @sugibuchi in #16

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Highlights

• Format options for all formats. format_options (for both StorageExplorer and DataExplorer) is now one dictionary; each format uses the options it takes:

  StorageExplorer(
      "s3://my-bucket/lake",
      format_options={
          "polars_storage_options": {"aws_region": "eu-west-1"},
          "pandas_storage_options": {"anon": False},
          "polars_delta_use_pyarrow": True,
      },
  )

  polars_storage_options and polars_credential_provider now apply when polars reads Delta Lake tables, Parquet, CSV and JSON Lines (previews and Query data), and pandas_storage_options / fsspec_storage_options go into the generated Python code. See Format options.

• Format names. Formats have short names (delta, parquet, csv, json-lines, json, excel, markdown, text, image) for DataExplorer(..., format=...) and CSS, and show their full labels ("Delta Lake table") in the widgets.

• Fix: the polars code for JSON files no longer passes storage_options to pl.read_json, which does not accept it.

Breaking changes

• DataExplorer(..., format=...) and the CSS attribute data-file-format use the new names, e.g. format="parquet" instead of "Parquet"; excel-workbook is now excel.
• format_options is flat instead of keyed by format. The Delta Lake options polars_use_pyarrow and polars_pyarrow_options are now polars_delta_use_pyarrow and polars_delta_pyarrow_options. Unknown option names raise ValueError.
• The generated Python code no longer includes the explorer's own fsspec storage options; set pandas_storage_options / fsspec_storage_options / polars_storage_options instead.
• DEFAULT_FORMATS lists format classes. To add a format of your own: formats=[*default_formats(), MyFormat()].
• Custom formats: load_code(target, requirements), recipes(target) and scan(url, is_dir) have new signatures.

Install

pip install -U jupyter-storage-explorer==0.1.0                # the widgets
pip install -U jupyter-storage-explorer-launcher==0.1.0       # plus the JupyterLab launcher item

What's Changed

• Format names and labels, flat format_options, and a test suite by @sugibuchi in #7

Full Changelog: v0.0.6...v0.1.0

v0.0.6

• On PyPI: jupyter-storage-explorer and jupyter-storage-explorer-launcher are now published to PyPI, so they install with pip without downloading wheels first.
• Automated releases: pushing a version tag builds both packages, creates the GitHub release with them and publishes them to PyPI (GitHub Actions, with PyPI trusted publishing). See Releasing in DEVELOPMENT.md.
• READMEs: a screenshot of the launcher item, and more compact feature descriptions.

No changes to the widgets themselves since 0.0.5.

Install

pip install jupyter-storage-explorer==0.0.6                # the widgets
pip install jupyter-storage-explorer-launcher==0.0.6       # plus the JupyterLab launcher item

What's Changed

• Add release workflows for GitHub releases and PyPI by @sugibuchi in #6

Full Changelog: v0.0.5...v0.0.6

v0.0.5

• PySpark load code: Python code ▸ Load with pyspark for Parquet (file or folder), CSV/TSV and JSON Lines, and Load with delta-spark for Delta Lake tables (spark.read.parquet / csv / json / format("delta")). It uses the notebook's spark session (e.g. on Databricks); without one, the first part of the code creates it. PySpark is only used for this code, never for previews.
• The code dialog's first part is now "1. Set up" (installing missing packages, then setting up e.g. a Spark session).
• Spark code uses Spark's URL schemes: abfss:// for Azure, s3a:// for S3.
• README examples use full Azure URLs (abfs://container@account.dfs.core.windows.net/path).

Install the wheels attached below (the launcher with the widgets):

pip install jupyter_storage_explorer-0.0.5-py3-none-any.whl jupyter_storage_explorer_launcher-0.0.5-py3-none-any.whl

v0.0.4

Two packages from now on, released together:

• jupyter-storage-explorer: the widgets, as before.
• jupyter-storage-explorer-launcher (new): adds Storage Explorer to JupyterLab's launcher and command palette. It opens the explorer in its own tab, on the launcher's folder, with a kernel of its own that stops when the tab is closed (the tab survives page reloads). What the tab shows is set by storage-explorer/launcher.py in a Jupyter configuration directory (~/.jupyter, /etc/jupyter, ...): its launch() returns the widget, and an optional style() returns CSS, e.g. for icons. Requires JupyterLab 4; install it in the environment of JupyterLab and of the default kernel.

Also new:

• Range filters in the data browser: numbers and dates have Null / empty and Range (from and to, either may be empty); text has Contains, Regex and Range, each optionally ignoring case. (The separate =, >, <, >=, <= filters are gone; remembered filters of those types are dropped.)
• ⟳ Reload in the data browser reads the data again, keeping filters, sorting and the page.
• height="fill" for StorageExplorer and DataExplorer: as tall as the container (e.g. the launcher's tab).

Install the wheels attached below, the launcher with the widgets:

pip install jupyter_storage_explorer-0.0.4-py3-none-any.whl jupyter_storage_explorer_launcher-0.0.4-py3-none-any.whl

See the README (and launcher/README.md) for usage.

v0.0.3

• DataExplorer: the data browser on its own for one table, e.g. DataExplorer("s3://bucket/lake/sales"). Works with Delta Lake tables, Parquet files and folders, CSV/TSV, JSON Lines and Excel. If polars (or a package the format needs) is missing, it raises an error with the %pip install command. Settings are shared with Storage Explorer's Query data.
• Azure URLs name the account: Copy URL, code snippets and Query data use abfs://container@account.dfs.core.windows.net/....
• Query data reads tables with polars directly from their URL; Spark output folders with _SUCCESS and .crc files work.
• Copy path is removed (fsspec paths can differ from the storage's own paths); use Copy URL.
• File rows carry data-file-format for styling icons per format (see the README).
• Demo notebooks for each widget.

Breaking changes: StorageExplorer's public interface is now folder_url and selected_urls (read-only). Its traits and methods (current_dir, filesystem, refresh(), open_preview(), move(), ...) and the url_for= argument are no longer public; override _url in a subclass to change URLs. The module jupyter_storage_explorer.browser is renamed jupyter_storage_explorer.storage_explorer.

Install the wheel attached below with pip install jupyter_storage_explorer-0.0.3-py3-none-any.whl. See the README for usage.

v0.0.2

• Query data: with polars installed, Delta Lake tables, Parquet, CSV, JSON Lines and Excel open in an interactive data browser. Page through the rows, sort by a column, filter columns by type (null / empty, equals, contains, regex, greater / less than), hide columns and resize them. Settings are remembered per table, optionally in a JSON file (QuerySettingsStore).
• Custom icons: folder rows carry data-depth and file rows carry data-file-format, so icons can be styled with CSS. See "Customizing icons" in the README.
• The status bar is hidden while the preview panel is open.

Install the wheel attached below with pip install jupyter_storage_explorer-0.0.2-py3-none-any.whl. See the README for usage.