Repository navigation
Version 0.9.5 is a patch release on 0.9.4 that breaks with every earlier
release: no export, session file or dataset catalog written for 0.9.4 is read.
AnnData is the one data model. The prepared export format is removed
from the web app, the Python package and the R package, with the export
server, the manifests, quantization and the exports-v3 layout of the sample
datasets. prepare() and cellucid_prepare() now write a
Cellucid-ready AnnData, a standard AnnData that anndata reads and the web
app opens directly, and take a required format= argument. Session files are
format 2; a format-1 file is refused. Dataset catalogs (datasets.json)
are schema 2 and list AnnData files. The contract between the writers, the
web app and the dataset repositories is the
Cellucid-ready AnnData specification
(docs/user_guide/python_package/c_data_preparation_api/09_cellucid_ready_anndata_specification.md).
The main changes are:
- One data model:
.h5ad, AnnData Zarr v2 and v3 folders and
.zarr.zip, read by the web app's own HDF5 and Zarr readers, and MuData,
Loom, 10x Cell Ranger, MatrixMarket folders and Seurat or
SingleCellExperiment.rds/.rdafiles, each opened as AnnData; one input
format registry names them in the web app, Python and R. The Python and R
writers produce identical Zarr stores from the same input. - Open from a link: CELLxGENE, GEO, GitHub, Zenodo, Figshare, Hugging
Face, Dropbox, Google Drive, OneDrive and any web address, read in place with
range requests that follow the measured link, from one address bar that also
connects to a server; a private Google Drive or OneDrive file opens for a
collaborator after one sign-in, with no Cellucid server involved. cellucid serveand Jupyter hand the file to the browser, which reads it
itself; the server computes nothing per request. They take every format the
web app reads (AnnData, MuData, Loom, 10x, MatrixMarket, R objects) and
in-memory AnnData and MuData.- One viewer channel: a notebook viewer and Python talk over one
authenticated WebSocket; the notebook output is a plain iframe. - MuData:
prepare(data=mdata)writes a Cellucid-ready MuData, and the
modalitylaunch default names the modality a dataset opens on. - Stored outlier quantiles and an outlier filter that reads or computes
them on first use, cancellably. - Analysis, selection, figure export and edges off the main thread, every
analysis cancellable. - Sessions save analysis results, Figure Export settings and the theme,
in smaller files that restore in short tasks.
Added
-
Python:
prepare(modalities=[...])chooses which modalities of a MuData are written, in that order, as cellucid-r'scellucid_prepare(modalities = )does: several are a Cellucid-ready MuData of those modalities (the rootvar/varmaphold theirs alone); one is a plain Cellucid-ready AnnData with the root's cells, obs, embeddings, graph and identity and that modality'sX,varandobsm(it must have measured every cell;gene_expression,varandgene_identifiersthen apply to it). -
R:
cellucid_apply_session()writes a saved session's highlight groups and user-defined fields onto a Seurat object or an experiment, with the rules ofapply_cellucid_session_to_anndata(); both are held to one web-app-written session file. -
Web app, Python, R: The Cellucid-ready AnnData.
uns['cellucid']names the dataset:schema(1),id(the id sessions and community annotations key on),name,description,citation(text,url),source(name,url), the launchdefaults(embedding1d/2d/3d,knn,expression,geneIdColumn,latentKey,vectorFieldDefault,obsKeys,normalizeEmbeddings,centroidMinPoints) andoutlierQuantiles. Every key is optional; an unknown key refuses the file with a message. A server's explicit launch choices override the file's defaults, and the dataset panel shows the citation. -
Python, R: The writers store the outlier quantiles. With
latent_key,prepare()andcellucid_prepare()compute every categorical obs column's latent-space outlier quantiles (categorical, text, boolean and nullable boolean columns; a column with no category ofcentroid_min_pointscells, 10 by default, gets no entry) and store them asuns['cellucid']['outlierQuantiles'] = {latentKey, minPoints, fields: {<column>: float32[n_obs]}}, NaN where a cell has none. Python (cellucid.prepare_data._outliers), R (src/latent_outliers.c) and the web app (data/latent-outliers.js) are bit-identical, checked against a numpy conformance fixture. On Suo (562k cells, 14 categorical fields, a 24-column scANVI latent space) they add 25.5 MB and take 3 s inprepare(). -
Web app: The Session panel's Link or server address bar, and the
?data=page parameter, open data where it lives, with no sign-in: a CELLxGENE dataset (its id or Explore link, read from the CZ CELLxGENE Census copy of the latest release), a GEO series (GSE…, its.h5adsupplementary file), GitHub (gh:owner/repo/path,github.comtree, blob and Raw links,raw.githubusercontent.com, Git LFS files read frommedia.githubusercontent.com), Zenodo (zenodo:<record>, record pages, file links and DOIs; a concept DOI opens the latest version), Hugging Face (hf:<user>/<repo>[@rev][/path], pinned to the commit) and Dropbox file links, or any web address of an.h5ad, a.zarr.zip, a Zarr folder or a catalog folder. A GitHub link without@branchreads the repository's default branch, whatever it is called. Single files are read in place with HTTP range requests in the reader worker, so only what a view needs is downloaded. Each provider is one module behind one contract (assets/js/data/sources/providers/). -
Web app: The Folder… item of the Open… menu, and a folder dropped on the page, open an AnnData
.zarrstore folder, asprepare()andcellucid_prepare()write it, a Cell Rangerouts/folder or a MatrixMarket folder. -
Web app: AnnData files written before anndata 0.8 open: anndata 0.7 and 0.6 files and SeuratDisk
Convert()output (0.7 dataframes with categories under__categories, Zarr stores without a root.zattrs, 0.6 compoundobs/var/obsmwith categories inuns,raw.X/raw.var, h5sparse-marked matrices, and every column a SeuratDisk file leaves out of itscolumn-order). Each is read as a view of the current layout, so data mapping, annotation and sessions see the same dataset; files written by the real anndata 0.7.8 and 0.6.22 read value-for-value like their current-layout twins. -
Web app: A session opens its own dataset. The session manifest records where the dataset came from (
source:{kind: 'sample'|'link'|'remote'|'file', …}, or null for a notebook); Load State reads it first and, when another dataset or none is open, opens that one, then restores. Only the fingerprint decides whether the session applies. -
Web app: Every analysis can be cancelled. Differential expression, Marker genes (and the heatmap genes it reads afterwards), Correlation, Gene signature, Quick insights, Detailed and the outlier quantiles run as one kind of cancellable job (
app/jobs/cancellable-job.js): a box shows what the run covers, its progress, bytes read, rate and time left, with Cancel. Closing the window or switching dataset cancels the run. A cancelled run keeps nothing, and its reads stop in the reader worker unless another caller still waits for them. Cancelling one analysis worker task no longer ends the whole worker pool. -
Web app: The Legend's outlier control is shown for every categorical field and says what it can do: no latent space (with a Choose latent space button that opens Data mapping), no category large enough (the fewest cells it needs), not stored (what the first use reads, in MB), reading (progress with Cancel) or failed (why; moving the slider tries again). Data mapping marks the stored latent space "outliers stored".
-
Python:
CellucidSessionBundlereads session format 2:dataset_source,required_fields, andcellucid.session_codecs(decode_cell_setfor all five highlight encodings,digest_cell_set,decode_user_defined_codes,decode_typed_json), vectorized with numpy.apply_cellucid_session_to_anndatarefuses an AnnData missing any obs column or gene the session requires, naming all of them, before any write. -
Python:
cellucid servegains--normalize-embeddings/--no-normalize-embeddingsand--centroid-min-points N, the launch choicesserve_anndataandshow_anndataalready took. -
R:
cellucid_transition_drift(transition_matrix, embedding, normalize_rows)computes each cell's drift in an embedding from a cell-cell transition matrix (CellRank's, for example), operation for operation as Python'scompute_transition_drift(). A shared conformance fixture (transition_drift_conformance.json) holds both to the same drift: bit for bit where the arithmetic is exact, otherwise within the forward-error bound. -
Datasets: The five samples in
cellucid-datasetsare Cellucid-ready AnnData Zarr v2 folders,anndata/<id>.zarr/, written byprepare(format="zarr")and listed byanndata/datasets.json(schema 2, defaultsuo, each with a format-2 starting view). They hold the same cells, genes, fields and coordinates as before (every 3D cell-order digest is unchanged), raw counts as float32 instead of 8-bit quantized values, the latent space with its stored outlier quantiles, and Pancreas's velocity under scVelo's names. Sizes: Suo 215.5 MB, Garcia 107.2 MB, He 32.3 MB, Kanemaru 50.2 MB, Pancreas 3.8 MB; no file over 1.6 MB. The app's sample catalog ishttps://theislab.github.io/cellucid-datasets/anndata/. -
Demo datasets:
cellucid-demo-custom-datasetspublishes its three datasets as acatalog/of Zarr v2 folders with a schema-2datasets.json, and the 2-D dataset as an.h5adand a.zarr.zipinfiles/, all written byprepare()and rebuilt byte for byte bygenerate_datasets.py --check. The README describes publishing with.nojekylland the link forms. -
Demo datasets:
cellucid-demo-custom-datasetsadds a multimodal sample in two forms:catalog/synthetic-citeseq.zarr/, a Cellucid-ready MuData Zarr (60 cells, modalitiesrnaandprot) written byprepare(data=mdata, format="zarr"), andcatalog/synthetic-citeseq-seurat.rds, the same cells as a Seurat v5 object (assaysRNAandADT, reductionspcaandumap, graphRNA_snn,misc$cellucid). The generator writes the.rdswithout R, as R serialization format 3 in gzip; R 4.5.3'sserialize()of the object SeuratObject 5.4.0 builds from the same values is byte for byte the same. Both are listed in the catalog, so its Sample dataset list shows an R object. -
Annotation: Community annotation keeps every dataset separate. User files are version 2 (the profile once, then one section per dataset id) and every merge in
merges.json(version 2) names its dataset; schemasuser-v2andmerges-v2. Repositories on the version-1 schemas are refused with a message. -
Docs: "Host a Dataset on GitHub" (why a Zarr folder is the way to publish on GitHub,
.nojekyll, measured speed of GitHub Pages andraw.githubusercontent.com, GitHub's limits and Git LFS), "Which format?" inprepare(),?cellucid_prepare, the specification and the preparing guides, and a Memory budgets section in the developer docs. -
Docs: Tutorials, each run end to end before publishing: "Tutorials: open published data as it is" (a GEO MatrixMarket sample by link, a Seurat
.rdson Zenodo by link, a Cell Rangerouts/folder from disk); Python "CITE-seq as a MuData: prepare and show" and "Serve Loom, 10x and MatrixMarket files"; R "Explore a Seurat object from R" (cellucid_view(), commands, a selection back in R), "Share a SingleCellExperiment on GitHub" and "CITE-seq: a Seurat object with RNA and ADT". The example-link tables of "Open from a link" tag each file Python or R, and list an R object and a GEO MatrixMarket sample. -
Web app, Python: Two data-mapping kinds: Values, whether each expression matrix (
X,raw.X, each layer) holdslog1pvalues and in which base, and Highly variable genes, the true/falsevarcolumn that marks them (raw.varforraw.X). They are chosen under Data mapping › Genes and saved in sessions. The defaults are Scanpy's: withuns['log1p'],X, every layer and a non-integerraw.Xarelog1pin itsbase(none is e), else every matrix is as stored;var['highly_variable']when it is true/false, else none. A default read from Scanpy's records is marked guessed. The data mapping (sessions, the Jupyter wire,cellucid.data_mapping.validate_data_mapping) gains the keyslog1p({"<matrix>": {"base": null|number}}) andhighlyVariable, pinned in the shared conformance fixture. Analyses read them through the gene loader:log1p({matrix})andhighlyVariableGenes(), which reads the column on first use. The sample presets incellucid-datasetscarry the two keys. -
Web app: A categorical obs column opens in the colours its file stores the way Scanpy keeps them,
uns['<column>_colors'](one hex colour per category, assc.pl.umap(adata, color="leiden")leaves it), read as Scanpy reads it: a shorter list is not used, a longer one gives its first colours. The colours are read with the column's categories when it is first shown. Without them, or once categories are merged, the colours are Cellucid's palette. A session records a colour only where it differs from the one the field opens with. -
Web app: Every opening shows its progress from the first moment: a link (
?data=or the address bar, every provider), a file, a.zarr.zipor a folder, a sample, a catalog dataset, acellucid serveserver and a notebook. While no dataset is shown, a panel over the empty viewer names what is opened and the step under way (Finding the file on NCBI GEO…, Reading the file's structure…, Reading embedding X_umap…), with a progress bar, how much has arrived, how fast and how long is left when that is known, and Cancel. While another dataset stays on screen, the same appears as a notification with Cancel. A request made again after a failure shows as Retrying… with why. Cancel stops the opening where it is (its reader worker and every request end) and keeps the dataset on screen; how an opening ended is kept in the notification center. Every layer that opens data reports into one opening (data/opening-progress.js, shown byapp/opening-progress-view.js), so a link, the file it names and the dataset change that draws it show as one. -
Web app: Loom, 10x Cell Ranger and MuData files open directly, read in place like an
.h5ad(only what a view needs), through layout views that present each file as AnnData (data/anndata/views/): no conversion step. Loom (.loom, Loom 2.0.1 and 3.0.0, loompy 2 and 3, velocyto, SCope/pySCENIC, loomR's orientation):/matrixand the layers asXandlayers(a transposed view), cell attributes asobs, two-dimensional ones (X_umap) and coordinate columns (_X/_Y,_tSNE1/_tSNE2,umap_1/umap_2) and SCope's compound embeddings asobsm,col_graphsasobsp, Loom 2's XML character references decoded. 10x Cell Ranger HDF5 (.h5, Cell Ranger 1.2 to 9, one group per genome ormatrix/features):Xis the stored matrix as it is (CSC over barcodes is CSR over cells), var columns named asscanpy.read_10x_h5names them (gene_ids,feature_types,genome). A matrix has no embedding, so a lone.h5is refused saying to open its Cell Rangerouts/folder (Open… › Folder…, or a dropped folder), which opens with Cell Ranger's ownanalysis/projections (X_umap,X_tsne,X_pca) and clusterings (graphclust,kmeans_<k>_clusters) matched by barcode;molecule_info.h5is refused naming the matrix to open. MuData (.h5muwith its 512-byte user block, and MuData Zarr v2 and v3; mudata 0.2, 0.3 and 0.4): the globalobs,obsmandobsp, each modality's obs columns and embeddings as<modality>:<key>(mudata 0.4 no longer copies them to the globalobs), a MuData sharing features instead of cells refused. A Loom file with no embedding is refused with the two lines that compute one in Python and in R. The file picker, drops, links and the format registry list.loom,.h5and.h5mu. A corpus written by loompy 2.0.17 and 3.0.8, as Cell Ranger writes (with scanpy's Cell Ranger 1.2 test files), and by mudata 0.2.4, 0.3.2 and 0.4.1 (scripts/formats-corpus/,tests/fixtures/{loom,tenx,mudata}/) is read value for value against what loompy, scanpy and mudata read (tests/formats-corpus-conformance.test.mjs). -
Web app, Python, R: Modalities: a dataset of several feature spaces over the same cells (a MuData's modalities, the feature types of a 10x matrix such as Gene Expression and Antibody Capture, the genomes of a Cell Ranger 2 file) opens with its gene-expression modality, and Data mapping › Genes shows one Modality select above Expression for it (nothing changes for a dataset of one feature space). Gene names, expression, its
log1pand the highly variable genes are then the modality's; gene search, colour by gene, Marker genes, Differential expression and Dot plot read it. A modality that measured only some cells reads NaN for the others (not 0), through MuData'sobsmap. The 10x feature types are column windows of the one matrix and share its gene index. The data mapping (sessions, the Jupyter wire,cellucid.data_mapping.validate_data_mapping) gains the keymodality, present only for a multimodal dataset, so every existing session, preset and fixture is unchanged; the launch defaults gainmodality. The shared conformance fixture holds the rules (defaultModality, the modality a dataset opens with: the first namedrna,gex,gene expression,gene_expressionorsct, else the first;mappingModality, a mapping's modality checked against the dataset's), pinned in all three repositories; Python addsdefault_modality()andrequire_mapping_modality(). Every multimodal store is presented in MuData's layout (/mod/<name>,mod-order,obsmap), which is the one contract a writer of several modalities follows. -
Web app: Seurat and SingleCellExperiment objects open directly from
.rds,.rdaand.RDatafiles: no conversion step and no R. The web app reads R's serialization itself (assets/js/formats/rds/: versions 2 and 3; gzip natively, bzip2 and xz with its own decoders, no WebAssembly; every item type R writes, including closures, byte code, environments, references, S4 objects and the ALTREP compact sequences, deferred strings and wrappers), in one pass over the file that keeps only what is shown (scale.data,commands,tools, neighbours, loadings and models are passed over; a Seurat v5 layer, whose name comes after its values, is passed over while the file streams and read in a second pass only if it turns out to be needed), into a store kept in the browser's origin-private file system for the tab's life (memory where there is none), opened as AnnData (data/anndata/transcode/). An uncompressed.rds(compress = FALSE,readr::write_rds()) is not copied: its matrices are read in place, from disk or a link. Seurat (SeuratObject 3.0 and later,Assay,Assay5with split layers joined,SCTAssay, Signac'sChromatinAssay):meta.dataandactive.identas obs, every reduction asX_<name>,graphsasobsp, the default assay'sdata(elsecounts) asXwith the other as a layer, variable features ashighly_variable, other assays as modalities (the default assay first), Visium and FOV positions asspatial,misc$cellucidasuns['cellucid']. SingleCellExperiment (1.8 and later) and SummarizedExperiment:colData,reducedDims(X_<name>, case kept),logcounts(elsecounts) asX,rowData,altExpsas modalities,colPairsasobsp,metadata$cellucid. Asave()file of several objects keeps all of them in the one pass and the opening asks which to open (one button per object); choosing another later in the tab reads nothing again. Refused with the R line that fixes it: matrices kept outside the file (BPCells, HDF5Array/DelayedArray,SaveSeuratRds(move = TRUE)), Seurat v2 objects, h5Seurat files, objects with no embedding, zstd, ASCII and native-binary saves. On a 100,000-cell Seurat object (2.45 GB gzip, 5.3 GB decompressed, counts and data with 1,500 genes per cell,scale.data, PCA, UMAP, an SNN graph) the pass takes 1.3 times plain decompression (16.6 s in Node on an M1 Pro), keeps 3.5 GB on disk and peaks at 277 MB of memory; reopening it in the tab takes milliseconds. The opening shows the pass as one step with bytes, rate, time left and Cancel; a cancelled or failed pass leaves nothing on disk, and a closed tab's stores are removed at the next start. -
Web app: MatrixMarket folders open with Open… › Folder… or a drop: a count matrix (
.mtx,.mtx.gz) with its feature and barcode lists as Cell Ranger 2 and later, STARsolo, kb-python, alevin-fry and GEO uploads lay them out, its orientation read from its size, written once into a cells × features matrix as the entries stream. A matrix that arrives in an order Cellucid cannot keep (a MatrixMarket file in no order, Seurat v5 split layers, an RdgTMatrix) is put in order in blocks of 4 Mi entries in browser storage, never whole in memory; one written by feature (kb-python, alevin-fry) is kept as it arrived, as acsc_matrix. Several feature types (features.tsvcolumn 3: Gene Expression with Peaks or Antibody Capture) open as one modality per type, Gene Expression first, as a 10x.h5does. The folder's own cell tables give the embedding: Cell Ranger'sanalysis/projections and clusterings, and any.csv/.tsvtable of the cells (UMAP_1/UMAP_2-style columns become embeddings, the rest obs fields; a clustering column such asCluster,seurat_clusters,leiden,louvainorkmeans_10is categorical even when its labels are numbers); without one the folder is refused with how to compute one. A Cell Rangerouts/folder that also holds the.h5opens through the.h5, in place. -
Web app: The Dataset box says what became of an R object or a MatrixMarket folder read once into the browser: one line under its format, Read once into this browser's storage for this tab, Opened from this browser's storage, read earlier in this tab or Read once into memory for this tab (no origin-private file system, as in a Firefox private window), with · N parts not read when something was left out, and in Details the rows Kept (where, until the tab closes) and Not read (one line per part, named by its R path and its size in the file:
assays$RNA@scale.data (15 KB): Cellucid does not use it.). The R parser records the bytes each dropped part spans and whether it held anything, so the emptyscale.data,commandsandneighborsevery Seurat object carries are not listed; a Seurat v5scale.datalayer is listed too. The Read row of such a dataset says Once, from start to end, and its origin line names the format alone (R object (.rds) · this computer). -
Web app: The R and MatrixMarket corpora (
tests/fixtures/rds/corpus.zip): objects written by SeuratObject 4.1.4 and 5.4.0 and SingleCellExperiment 1.32 in every compression and both serialization versions, SeuratObject's ownpbmc_small, refusal cases, and MatrixMarket folders in each producer's layout (scripts/rds-corpus/,scripts/mtx-corpus/), read value for value against what R's accessors and scanpy read (tests/rds-corpus-conformance.test.mjs); every transcoded store also reads in anndata and mudata (scripts/rds-corpus/verify_with_python.py). The parser is checked item type by item type, at every split of its input (tests/rds-parser-contract.test.mjs), and the kept stores' lifetime with an origin-private file system on disk (tests/transcode-space-contract.test.mjs). -
Web app: Figshare, Google Drive and OneDrive open from a link (the address bar or
?data=), read in place with range requests (Google Drive, OneDrive and private sharing). Figshare: an item page (figshare.com or an institution's portal), a Figshare DOI or any DOI that resolves to one, orfigshare:<id>[v<n>][/<file>]; the page asksndownloader.figshare.comfor the file's signed S3 link (its bot check refuses requests from a reader worker in some browsers) and the reader reads S3 with it for the link's stated ten seconds, then asks the page for the next with a one-byte request; private links (figshare.com/s/…) and bare download links are refused with the reason. Google Drive: a file shared with anyone opens with no sign-in through the Drive API and Cellucid's public, referrer-restricted API key, its details and first bytes asked for together (one round trip); a private file shows "Private file on Google Drive. [Sign in to Google]", signs in with thedrive.filepermission only and opens Google's file picker already showing that file, so one click on Select grants it. OneDrive and SharePoint (personal, work and school accounts): every file opens after Sign in with Microsoft (OneDrive serves nothing anonymously; PKCE in the page, no client secret, no library), withFiles.Read, andFiles.Read.Allasked for only for a colleague's file (which many organisations let only an administrator approve); a share link is redeemed durably, so sessions and?data=name the file asonedrive:<drive>/<item>; the short-lived download address is renewed through Microsoft Graph once for all reads waiting on it, and a file changed meanwhile is refused. Folder links are refused (share one.h5ador.zarr.zip). Sign-ins live in the tab'ssessionStoragethrough one store shared with the GitHub sign-in of community annotations, never in a link, a session, the console, analytics or an error; an expired one shows a calm Sign in again under the address bar and as a notification, and the reads waiting on it resume. A read behind a sign-in or an expiring link asks the page for its URL and headers once for all parallel reads (httpByteSource's newaccess), and any redirect that states its lifetime (X-Amz-Expires) is reused for that lifetime and no longer. Google answers only the origins Cellucid registers (www.cellucid.com and a localcellucid serveon port 8765,provider-config.jsregisteredOrigins), for public files and sign-ins alike; on any other page Cellucid says where Drive works before asking Google (a local server on another port: start it without--port; a notebook viewer on another port: stop the server holding 8765 and show the data again; another host: www.cellucid.com). Microsoft sign-in is refused up front, with the reason, on a plainhttp://page of another computer and inside another site's page (a notebook atlocalhostshowing a viewer at127.0.0.1, whose storage the browser keeps apart from the sign-in window); a sign-in window found closed offers the step again, and a sign-in finished in it after all still counts until the step ends. The site's privacy policy iswww.cellucid.com/privacy.html, linked as Privacy in the sidebar footer; the public client IDs, API key and project number are inassets/js/data/sources/provider-config.js(registered for www.cellucid.com andhttp://127.0.0.1:8765/http://localhost:8765), and a copy of the app without them says "not configured on this site yet". Checked against the real services: public Drive files open with no sign-in and are read in place, and Microsoft's sign-in page accepts all three return addresses. Google holds back downloads without a sign-in from a network that made many of them (an HTML "unusual traffic" page, HTTP 403, without CORS headers, while the file's details still answer and signed-in requests go through; seen three times on 2026-10-25, each after some 200–400 MB from one address within minutes, whether in 30 requests or 400, for about 50 minutes): its head read is asked once, and a failure confirmed with one byte (Google sends its page after about seven seconds, so this is told in about eight seconds); a file whose first bytes are held back opens after "Google Drive sends this file only with a Google sign-in for now. [Sign in to Google]" and the picker, and a read held back later says so, in place of the generic "no connection, or no CORS headers" (registerSourceAccess'sunreadable). A Drive file granted to Cellucid is read with the sign-in while one is held, public or not.drive.google.com/u/<n>/uc?id=…links (as gdown writes them) are Drive links, and the address of a OneDrive or SharePoint page that is not a share link is refused with how to copy the file's link. The Dataset box's read line ("5.5 MB read in 43 requests") is shown for a file read at an address with a query string (Google Drive's…/files/<id>?alt=media, a Dropbox?dl=1link): the box names a file by its address without the query, and the Transfers log now finds its entry by that address. The documentation lists public Google Drive files (Python and R) with what each first view read. -
Web app: Google Drive and OneDrive in every browser. Time spent in a sign-in window, a consent screen, Google's file picker or the choice of an R file's object never counts as a stalled download: every wait on the person registers itself (
utils/person-wait.js), the startup deadline stands still meanwhile and starts over once the person has answered, and the opening goes on by itself after the sign-in; a data link's stall says "while opening the data link". Signed in to Google, a Drive link to a file not yet chosen for Cellucid (thedrive.filesign-in reads only those) opens Google's picker on it at once and is read with the sign-in; closing the picker reads a public file with the key unless Google holds it back, and a read held back later says to open the file again with the Google account. A Drive download held back after the file opened is told in about 9 s, as at opening: the first read that gets no answer asks Drive, from the file's access, for the file's details and one byte, and the details answering while the byte fails ends the read in Drive's words (unreadable.confirm()insources/access.js, asked by the reader worker withsource-unreadable); a check that does not confirm the hold leaves the read's failure as it was. Signed Drive reads go to one stable address per request in flight (lane=<n>, a parameter Drive ignores) under the HTTP cache mode, so Chromium and Firefox preflight each address once: on a Drive-shaped host (0.65 s a request, 0.2 s a preflight) a signed first gene takes 5.3 s, as with no sign-in. The head a provider reads beside a file's details answers every read that starts inside it, so an R object from Drive or OneDrive is not read twice (the test kit's Seurat.rds: 9.4 MB in 3 requests). A OneDrive item is asked for whole, since Microsoft Graph leaves the download address out of an answer whose fields are selected ($select), even when the selection names it; an answer without what reading needs is said in Graph's terms ("Microsoft Graph described … without a download address"). Sign in with Microsoft opens its window empty, on Cellucid's own address, in the click, and sends it to Microsoft once the attempt's PKCE verifier is made, so Safari on a Mac with Microsoft's single sign-on extension (Company Portal) shows Microsoft's ordinary sign-in page for any account instead of handing the window to the extension; a second click brings back a sign-in window still open, and a pop-up blocker's closed window is said as a blocked window. Sign-in failures are said for what they are, with the next step: an organisation that lets only an administrator approve Cellucid (AADSTS90094,90095,65001, naming the organisation), an account it did not assign (AADSTS50105), its sign-in policy (AADSTS53003), a decline (AADSTS65004, which is also how Microsoft reports leaving its "Need admin approval" page, so both are named), a site registration Microsoft or Google does not accept, a Google Workspace administrator's block (admin_policy_enforced), and Google's window closed (Google shows "Access blocked" there). Where the same account would fail again the button offers Use another account, and the line links the administrator steps in the documentation. -
Web app: Composition (Analysis → Cells) asks whether cell-type proportions differ between conditions, accounting for samples. It counts the cells of each type per sample in one pass in a worker (a sample spanning conditions counts once per condition) and, with a condition, tests every type across the samples: logit proportions
(count + 0.5) / (total + 1)with Welch's t-test or one-way ANOVA (after propeller, without its empirical-Bayes moderation), ranks of proportions (Wilcoxon or Kruskal-Wallis) or centred log-ratios, with Benjamini-Hochberg or Bonferroni over the types. Without samples it runs χ² tests (Fisher's exact test for a sparse 2 × 2 table) and says the cells were the replicates. It shows stacked bars per sample or condition, per-sample points per type, a table, the results as CSV and the counts as CSV; the statistics match SciPy on the shared fixture. -
Web app: Dot plot (Analysis → Genes) draws, for each gene of one or more named gene sets and each group of a categorical field, the mean expression as color and the fraction of cells expressing as dot size, as scanpy's
pl.dotplot(or as a matrix plot, or as a stacked violin of each gene's distribution per group): scaled 0–1 or to z per gene, a free expression cutoff, the mean of expressing cells only, swapped axes, groups ordered by size or a dendrogram, Reds, Viridis or Blues. Only the listed genes are read, each summed per group in one pass over its non-zero entries in a worker; display changes never read again. Genes the dataset lacks are left out on a chip. It exports the long table (group, gene, gene_group, mean, fraction_expressing, n_cells) and opens Gene signature with its sets (Score as signature). Means and fractions match scanpy'sdot_color_dfanddot_size_dfon the shared fixture. -
Web app: Marker genes, Differential expression and Dot plot have an Expression matrix option: the matrix the data mapping shows (default), or
X, a layer orrawof the same file for that run alone, without changing what is shown. Each matrix's genes are named on its own axis (rawhasraw.var), read through the samevar/{axis}/{names}routes as a mapped matrix on every way of opening a dataset (a file, a link,cellucid serve, a notebook); a gene the matrix lacks is named as not found; Automatic fold change follows that matrix's ownlog1p. The choice is part of the marker cache key, the result line names it, and sessions save it with the options and the result. -
Web app: Differential expression has a Replicates option: with a sample field the run is a pseudobulk test, each sample's cells of a group one observation (its mean expression, from one pass over each gene's non-zero entries) and Welch's t between the two groups' samples, as
scipy.stats.ttest_ind(equal_var=False)on the per-sample means; no variance moderation. Each group needs cells in at least 2 samples. -
Web app: Clicking a gene in a Marker genes or Differential expression result colours the cells by it through the sidebar's gene selector (Coloring & filtering → Gene expression), whose search box then names it.
-
Web app: Analysis forms share one layout: the essentials, then one collapsed More options holding a collapsed group per topic, each counting the options changed from their defaults, with Reset to defaults; then Run, with the reason above it when it cannot run. Every option has an (i) tip and a default stated in the documentation; the options and which groups were open are saved in sessions. After a run, a change that would change the numbers dims the result with Stale · Run; display options (filters, ranking, labels, plot, rescaling) redraw it without computing again. Correlation and Gene signature run on Run instead of on every change.
-
Web app: The site has terms of use (
www.cellucid.com/terms.html: BSD 3-Clause license, no warranty, your data stays yours, other services keep their own terms) and an accessibility statement (accessibility.html: what works, the WebGL plot's limits, how to report a barrier), reading pages in the privacy policy's design, linked as Terms beside Privacy and Accessibility beside the licence in the sidebar footer; the privacy policy now names Google Analytics' two cookies./.well-known/security.txt(RFC 9116) names the security policy's contact, itsExpiresrenewed a year after each build byscripts/generate-web-asset-inventory.mjs. -
Python, web app: The viewer channel. A notebook viewer opens one WebSocket to its server,
<server>/_cellucid/ws, named byGET /_cellucid/protocol({"channel": "_cellucid/ws", "commands": [...], "binaryCommands": [...], "events": [...]}), and sayshellowith its viewer id and token; the server checks theHostandOriginheaders (the server's own host, a loopback name or anallowed_hostsname), compares the token in constant time and answerswelcome. Commands are JSON text frames, and a command that names cells (highlight,setVisibility) one binary frame: auint32header length, the JSON header padded to a multiple of 4 bytes, then the cells as little-endianuint32, strictly ascending, which the viewer reads in place and checks as strictly as the hosts check its binary events (a host whose protocol document lists nobinaryCommandsis told to upgrade). Commands are queued until the viewer connects and sent in order, never twice; events are JSON text frames, and a selection (uint32 cell indices) and a session bundle (streamed to a temporary file) binary frames. Close codes: 1001 stopped, 4401 unauthorized, 4409 replaced by a newer page of the same viewer, 1008 protocol violation. The server pings every 20 s; the viewer reopens a dropped socket once, at once, and one health request tells a stopped host from a proxy that does not pass WebSockets. The rules are one document,viewer-protocol.json(web appdata/viewer-protocol.js), kept byte for byte in cellucid-python and cellucid-r. Two million selected cells reach a hook in 89 ms. -
Python:
cellucid serve,serve_anndata(),show_anndata()andAnnDataViewertake every format of the input format registry by path:.h5ad, AnnData Zarr (folder or.zip),.h5mu, MuData Zarr,.loom, 10x.h5files and Cell Rangerouts/folders, MatrixMarket folders (with GEO'sGSM…_prefixes) and.rds/.rda/.RData(gzipped or not);anndata.file.kindand the healthformatcarry the registry id. A folder of files is served file by file and listed in/_cellucid/datasets(file.files: [{path, size}]); the health and infon_cells/n_genesarenullfor a format Python does not read. An in-memoryMuDatais written once to a temporary Cellucid-ready.h5mu. -
Python:
prepare(data=...)takes an AnnData or a MuData whole. Every component it holds (obs, X with var, obsm embeddings and vector fields,obsp['connectivities'],uns['cellucid']) is the default, an explicit argument replaces one component, and the output is byte-identical to passing the components by hand.embeddings,dataset_idanddataset_nameare required only withoutdata. The caller's arrays are never changed: a matrix that needs canonicalising (sorted indices, summed duplicates, dropped stored zeros) is copied first, and value ranges are checked without whole-matrix temporaries, so peak memory beyond the input is about one converted matrix. A backed AnnData (backed="r") is prepared in two passes over row blocks of its file, never held whole. -
Python:
prepare()writes a Cellucid-ready MuData from a MuData:format="h5mu", or MuData Zarr with"zarr"/"zarr.zip". Each modality is a Cellucid-ready AnnData, the shared cells, embeddings and graph are at the root, and modalities are written one at a time to bound memory (30,000 cells, 20,000 genes and 200 proteins: 4.8 s and 316 MB peak to a Zarr folder). New optional extra:pip install "cellucid[mudata]". -
Python: New launch default
modality(prepare(modality=...),uns['cellucid']['defaults']['modality'],serve_anndata/show_anndata(modality=...),cellucid serve --modality): the modality a multimodal dataset opens on. -
Python:
apply_cellucid_session_to_anndataapplies a session to a MuData; the added columns go to the globalmdata.obs, and each modality's arrays are placed among the cells only when read. -
Web app: GEO links open MatrixMarket samples: a series'
GSM…_matrix.mtx.gzwith its_features/_genesand_barcodesfiles is one dataset, opened by its matrix (geo:GSE…/GSM…_matrix.mtx.gz), read from the sample's own folder with the sizesfilelist.txtstates.GSM…sample accessions are accepted. An incomplete or ambiguous sample is refused naming the file it lacks or the two that compete. -
Web app:
.rds.gz,.rda.gzand.RData.gz(GEO's gzipped R objects) open from links, providers, drop and the File… picker; the content decides whether a file is gzipped once or twice, and it is decompressed as it streams. A host that sends an R file asContent-Encoding: gzipeven for a part of it (NCBI's GEO file host does, forsaveRDS()'s default gzip) cannot be read in parts by any browser: its file is downloaded whole once, decoded by the browser, and the notification says why (GSE327442's 25 MB Seurat object opens in 7–10 s in Chromium, Firefox and WebKit). -
Web app: A folder of MatrixMarket files sharing a sample prefix (an extracted GEO
_RAW.tar) opens with Open… › Folder…; with several samples, Cellucid asks which to open.
Changed
-
Python, R, web app: A host program opens its viewer at
?viewer=true&viewerId=…&viewerToken=…(it was?jupyter=true), and/_cellucid/infonames the program serving it,runtime:"python"(cellucid serve, a viewer outside a notebook),"jupyter"(a viewer in a notebook kernel) or"r". The viewer shows it in the Dataset box ("h5ad file · Jupyter", "Seurat object (.rds) · R") and in the dataset list ("R data"); its startup steps name "the viewer's host". The page keys and the runtime names are part of the shared viewer protocol file (page,runtimes; SHA642c04e1…in all three repositories). The old key opens nothing. -
Python: A notebook viewer (
show_anndata,AnnDataViewerwithport=None) serves on 8765, the one local port Google sign-in accepts, or on the next free port when another server holds it, saying so; it used a port the operating system picked, where Google sign-in never worked. cellucid-r'scellucid_view()already did this. -
Python:
prepare(format="h5ad")(and.h5mu) compresses chunks on every core, written with HDF5's direct chunk write in grid order, so the file is laid out as HDF5's own writer lays it out: 1M cells (150 values each, 1.15 GB of X) in 13.3 s at 4.5 GB peak (63.9 s when HDF5 compresses one chunk after the other on one thread), 2M cells in 22.1 s, 200,000 cells in 2.8 s. -
Python: Applying a session copies a continuous field's missing values (NaN) as the viewer copied them, and accepts field names and highlight fields with
:(ATAC peaks, MuData<modality>:<column>columns), as the viewer does; both were refused. -
Python, development: The
devextra brings themudataextra, so a development install, CI and the release gate run the MuData tests instead of skipping them. On Python 3.11, where pip installs mudata 0.3, the suite (tests/conftest.py) sets mudata'spull_on_update=False, so every MuData it builds or reads behaves as under mudata 0.4.tests/test_minimum_python_grammar.pyparses every Python file with the grammar of the oldest supported Python and refuses the f-strings only Python 3.12 accepts. -
Web app, Python, R: One input format registry decides what a file or folder is, everywhere (
assets/js/data/input-formats.js). Each format has a stable id (h5ad,zarr), the name endings of each layout it arrives in (file:.h5ad;zip:.zarr.zip;folder:.zarr), container signatures and folder markers. The File… picker'sacceptlist, a drop, the address bar's routing, the link providers (GitHub, Zenodo, GEO, Hugging Face, any web address), the dataset catalog, the served-store validator, the server health checks and the reader worker all ask it; the rules each of them spelled out before are gone. A name decides whether a file is taken; its content decides what it is, so a misnamed file (an.h5adsaved as.zip, a Zarr ZIP saved as.h5ad) opens as what it is, and an HDF5 file with a user block of up to 2 KiB is recognised by its signature.tests/fixtures/input-formats.jsonis the registry's document; cellucid-python ships the same bytes (src/cellucid/input_formats.json), from whichcellucid serveclassifies a path (classify_anndata_path) and names itskind, and cellucid-r ships them asinst/input_formats.json, whichcellucid_view()andcellucid_serve()read; the three repositories pin one SHA-256. Every message that names what Cellucid opens (a link that is not a dataset, a catalog path, a Drive, OneDrive or Dropbox folder, a listing with nothing to open) lists the registry's single-file formats (openableNamesText). -
Web app: One data source reads every dataset file,
AnnDataFileSource(data/anndata-file-source.js), addressed by one scheme,anndata://<dataset id>/; it replacesH5adDataSourceandZarrDataSourceand theirh5ad://andzarr://schemes; the data layer addresses an open file by its AnnData elements (obs,var,obsp,obsm/{i}/{cols},outliers/{l}/{f}, …) instead of export file names. Its loaders areloadFile(a local or remote file),loadFolderandloadRemoteFolder; the format is a fact of the open dataset (format,describeStorage():{format, layout, zarrVersion, remote, url, name, bytes}). A dataset that states no id is still<format>_<name>(h5ad_<file>,zarr_<store>), so every saved session keeps matching. A local file's source line names its format (AnnData (.h5ad),AnnData Zarr). -
Web app: Opening has one entry,
openDatasetStore({source, expect})indata/anndata/open.js, over one file{file}, a picked folder{files, rootName}or a web folder{url, name}; it opens the container and presents the root through a layout view when one recognises it (presentStore,data/anndata/views/), or transcodes a single-stream format into an AnnData Zarr store (data/anndata/transcode/). The view nodesolder-layouts.jsis built from are shared indata/anndata/views/view-nodes.js, with transposed, row-window, column-window and computed arrays for the layout views. -
Web app: The usage analytics' local load methods are named by how a dataset arrived, whatever format it holds:
local-user-file,local-user-zipandlocal-user-folder(they werelocal-user-h5ad,local-user-zarr-zipandlocal-user-zarr-folder);data_loadreports the input format it was read as (data_format:h5ad,zarr,loom, …) on success. The Dataset box names the format (Loom (.loom),MuData Zarr v3 folder,Cell Ranger outs/ folder) and, for a multimodal dataset, its modality. -
Python, breaking:
prepare(out_path, *, format, embeddings, dataset_id, dataset_name, …)writes a Cellucid-ready AnnData and returns its path.formatis required, with no default:"h5ad"(one file, for single-file hosts such as Zenodo, Figshare, Hugging Face, S3 or a lab server, local files,cellucid serveand Jupyter),"zarr"(a.zarrfolder of small files, the way to publish on GitHub: no file over 100 MB, no LFS) or"zarr.zip"(the Zarr store in one file).out_pathmust end in the format's suffix.zarr_format(2 or 3; 2 when None) applies to the Zarr formats only. New arguments:citation_text,citation_url,latent_key(anembeddingskey),normalize_embeddings. Every obs and var column is stored;obs_keyssets which columns the viewer lists. -
R, breaking:
cellucid_prepare(out_path, format, embeddings, dataset_id, dataset_name, …)writes the same Cellucid-ready AnnData as a Zarr store:formatis"zarr"or"zarr.zip", required, matched asmatch.arg()matches;zarr_format = 3writes Zarr v3. There is no"h5ad": the package writes no HDF5 and depends on no HDF5 library. -
Python, R: The layout both writers share:
Xgene-major (CSC from sparse input; dense stays dense) in float32, or float64 when the genes kept hold integer counts above 2^24;obs/varwith anndata's own encodings and pandas' category code widths (int8 below 127 categories, int16 below 32,767);X/indicesas<i4unless the axis it indexes has 2^31 entries,indptras<i8from 2^31 stored values; values that float32 rounds to zero dropped rather than stored as zeros; every embedding whole inobsm(float32, unnormalized);obsp['connectivities']as symmetric float32 CSR; chunks of 2^16 values forXand 2^18 for arrays read whole; deflate level 6, with byte shuffle on every multi-byte array except the expression values: in Zarr v2 numcodecs'shufflefilter and zlib, in Zarr v3 only codecs of the v3 specification, theblosccodec (byte shuffle, zlib level 6) on shuffled arrays and gzip on the others (numcodecs.shuffleis outside the specification; on He the v3 store is 29.00 MB, against 37.62 MB unshuffled); consolidated metadata. A.zarr.zipstores chunks first in grid order and the consolidated metadata last, with fixed timestamps and permissions, so it opens with the read of its end and the same input gives the same bytes. Python's.h5addeclares every element before writing values, so a reader gets the whole tree and the values opening needs from its first 512 KiB (431 KB on Suo); the file reads the same in anndata 0.8 to 0.12, scanpy and h5py. R and Python write identical Zarr v2 and v3 stores (every metadata document and every value), checked by cellucid-r'stests/testthat/test-cross-writer.R; R writes every NaN and NA as numpy's canonical quiet NaN. -
Web app, Python, R, breaking: A vector field binds by scVelo's naming on any embedding:
obsm['<field>_<basis>']is the field<field>overobsm['X_<basis>'](velocity_umapoverX_umap,velocity_branchesoverX_branches), the longest existing basis winning, and its id is<field>(velocity), so one field spans every basis.prepare(vector_fields=...)andcellucid_prepare(vector_fields = ...)write each array under that key, with as many columns as the embedding; arrays of another width, colliding keys and keys the viewer would read otherwise are refused before anything is written.vector_field_defaultnames the field and must be drawn on an embedding the viewer shows at open. The rule is pinned in the shared data-mapping conformance fixture (webparseVectorFieldKey, Pythoncellucid.data_mapping.parse_vector_field_key, R.parse_vector_field_key). -
Python, breaking:
add_transition_drift_to_obsm(adata, T, basis=..., field_prefix=...)computes the drift overobsm['X_<basis>'](every column) and storesobsm['<field_prefix>_<basis>'];dimis removed (usebasis="umap_3d").compute_transition_drifttakes an embedding of any width, and a zero row sum is refused. -
Python:
cellucid serve <file>,serve_anndata()andshow_anndata()hand the AnnData store to the viewer, which reads it in place with range requests, as it reads a file or a link. The server serves the store's bytes at/_cellucid/anndata/<dataset id>.h5ad(or.zarr.zip, or.zarr/<key>) with exactRange,HEADand strongETaganswers, over HTTP/1.1 with one thread per connection andsendfile, and names the dataset, its store and the launch choices in/_cellucid/datasets. Name and id come from the store'suns['cellucid'](--dataset-name/--dataset-idare needed only when it has none), and every launch choice is checked against the store before the server binds. An in-memory AnnData, or a store with a codec the browser cannot decode, is written once to a temporary.h5ad, one matrix at a time, and removed when the server stops. On a 562k-cell × 8,192-gene.h5ad: startup 0.16 s instead of 3.0 s, a UMAP about 40 ms instead of 170–250 ms, 100 genes 0.1–0.6 s instead of 115 s. -
Python, breaking:
AnnDataVieweris the one notebook viewer class (cellucid.jupyter.BaseVieweris removed; same methods and signature). -
Web app, breaking:
?github=owner/repo/pathis replaced by?data=gh:owner/repo/path, and the GitHub control by the Link or server address bar; the source typegithub-repoislink.?exportsBaseUrl=is?sampleCatalog=, and thecellucid-exports-base-urlmeta element iscellucid-sample-catalog. -
Web app, breaking:
datasets.jsonschema 2:{"schema": 2, "default": "<id>", "datasets": [{"id", "path", "session"?}]},patha Zarr folder, an.h5ador a.zarr.zipbelow the catalog folder,session({path, bytes, sha256}) the dataset's starting view. The sample list and every folder link read this one format through one loader; listing reads each file's metadata only, and a catalog link withoutdataset=opens its default. A catalog in another schema is refused. -
Web app, breaking: Session format 2. Files start with
CELLUCID_SESSION/2\n; any other file is refused before anything is read ("This file is not a Cellucid session file of format 2 (it does not start with CELLUCID_SESSION/2)."), in the web app, the Jupyter upload andCellucidSessionBundlealike. Sessions now save Figure Export settings, the docked analysis panel with analysis results (differential expression, marker genes, gene signature and correlation, as typed columns; floating windows carry theirs), per-view centroid toggles and the app theme, and no longer save derived labels, duplicated cameras, antialiasing or per-cell gene caches. Highlights are stored in the smallest of sorted gaps, a bitset, runs or relative to their category (checked by count and digest on restore); user-defined codes raw, as runs or bit-packed. The fingerprint records the dataset's identity id instead of how it was opened, and the manifest lists every obs field and gene the session uses (requires), so a dataset lacking any is refused naming every one. Restores request every field together, rebuild each kept view once and finish in background tabs and off-screen notebook frames. Suo (562k cells), a heavy session of 7 views: file 317 → 193 KB, save 189 → 101 ms, restore 1.89 → about 0.83 s, with no task over 50 ms. -
Web app, breaking: A data mapping is always
kind: "anndata"; theexportkind is deleted and a mapping or session naming it is refused. -
Web app: Own HDF5 and Zarr readers (
assets/js/formats/) under one AnnData reader (assets/js/data/anndata/) replace the vendored h5wasm and the two separate loaders. They read Zarr v2 and v3 (sharded included), from a folder, a.zarr.zipor a web host; Zarr codecs gzip, zlib, Blosc, Zstandard, LZ4, shuffle and CRC-32C; HDF5 deflate, shuffle, Fletcher-32 and LZF. A codec or HDF5 filter outside these (Blosc or Zstandard inside an.h5ad, bz2 in Zarr) is refused naming it and the dataset, with how to save the file again. A Zarr store on a web host that hides paths starting with_or.(GitHub Pages without.nojekyll) is refused naming.nojekyllinstead of read with empty cell and gene names. Suo (562k cells × 8,192 genes, CSR.h5ad) read locally in Node, against h5wasm: open 210 → 27 ms, first gene 1.9 → 0.85 s, neighbour graph 7.1 → 0.54 s. -
Web app: Reading follows the measured link (
formats/io/link.js): every byte reader measures round trip and bandwidth as it reads, and request size, gap merging, read-ahead and the number of requests in flight follow the measurement and the host's protocol (six to an HTTP/1.1 host; a host that hides its protocol, as GitHub does, starts at HTTP/2's 100 streams until a burst shows otherwise). An.h5ad's size probe carries its first 512 KiB in the same round trip; range reads bypass the HTTP cache so Chromium sends a file's ranges together; a link delivering at least 64 MB/s, a local file and a picked Zarr folder are read ahead like a disk. Suo.h5adover a simulated 80 ms / 50 MB/s link: probe and open 98 ms (1.2× the round-trip bound), dataset ready in the app 1.3 s. -
Web app: An
.h5adon a web host is opened with 16 KiB metadata pages, since anndata writes each element's headers within a few KiB beside its data: a 1M-cell.h5adreads 1.8 MB before its first view (1.4 s at 80 ms and 2 MB/s). -
Web app: A part of an unfiltered HDF5 chunk (anndata's default sparse
.h5ad) is fetched as the bytes it needs, not the whole chunk: 1,000 cells of a 1M-cell CSR.h5adat 30 ms and 5 MB/s read 4.4 MB (the whole chunks are 121 MB). -
Web app: Over a slow link, the first gene of a CSR
.h5adkeeps the matrix values it reads (its entries spread over the whole matrix), so later genes read nothing (a 1M-cell file at 5 MB/s: the second gene in 6 ms, not 133 MB in 27 s). A cell subset weighs the bytes it would fetch by the link: an analysis of two large groups reads from the gene index in memory (5.5 s) instead of 400 MB of rows (117 s). -
Web app: Opening a file reads no category dictionary. Every obs column is described from its metadata, with its category count, and a categorical's categories (and the colours Scanpy stored for them) are read when the column is first shown. A GEO file whose barcodes are stored as a categorical (21,751 categories, 1.1 MB) no longer reads them before its first view.
-
Web app: Every request into a Zarr store bypasses the HTTP cache (
cache: 'no-store'), as range reads of a file do: WebKit never finished delivering to the reader worker the bodies of store chunks served from its cache, and a store's metadata is never paired with chunks cached from an earlier version. -
Web app: A refusal written for the person (a record with several files, a file with nothing to plot, a folder link with no
datasets.json, an AnnData too old to read) is shown as it is instead of a generic network sentence. A folder link withoutdatasets.jsonsays so for every host: "The linked data could not be opened: https://example.org/atlas/ holds no datasets.json, so it is not a dataset catalog. Link to the folder that holds datasets.json …". -
Web app: Analysis computes each gene from its non-zero entries, read as
{length, indices, values}and never installed in field state: Wilcoxon and Welch statistics treat the zeros as one rank block, bit-identical to the dense kernels for differential expression and Wilcoxon markers (Welch marker p-values within 2.2e-9 relative). Differential expression sends the groups once and each gene as its column; a cell subset is read by whichever of rows or columns costs fewer bytes. The worker pool runs one worker per core less the page and reader (no cap of 8). In Chromium on Suo: marker discovery bycell_type(69 groups × 8,192 genes, Wilcoxon) 12.3 → 0.86 s with no long task; differential expression of one 266-cell type against the rest 14.1 → 2.1 s. -
Web app: Gene clustering in the Genes panel is exact agglomerative clustering on the analysis workers (Müllner's nearest-neighbour-candidate algorithm, double precision; the same merges as before, trees equal to scipy's to 5e-15): 3,000 genes × 69 groups 5.0 s on the main thread → 0.15 s. Memory is 8 bytes per gene pair (was about 104), so the default budget fits about 11,580 genes instead of 2,270. Gene signatures read each gene once as its sparse column (100 genes 450 → 32 ms), CSV export of page data and signatures is written by the workers, categorical page data is returned as codes, and Correlation computes pages concurrently and refuses a categorical X or Y.
-
Web app: Box and violin points, and the volcano plot's genes, are one image per trace drawn by a worker at exactly the positions Plotly lays out, instead of one SVG element per point; hover and SVG/PNG export are unchanged. Suo, 544,130 cells in two pages: box with all points 5.0 → 0.13 s, violin 5.2 → 0.23 s, hover no longer blocks for 0.7–0.8 s.
-
Web app: Figure export computes bounds, draws the point passes, writes vector circles and encodes PNGs in a worker; the page only composes. Where a browser's workers have no WebGL2 -- WebKitGTK and WPE (WebKit bug 209809), Safari 16.4–16.6 and 17.0–17.1 on macOS Ventura (WebKit bug 183720) -- the worker says so once and the same code draws the point passes on the page. Output is byte-identical apart from the timestamp, except that Full Vector now honours a depth sort at every point count (it skipped one above 200,000 points). Suo: main-thread long tasks of up to 804 ms → none; 600 dpi PNG about 1.1 → 0.6 s; Full Vector SVG peak memory +239…+581 MB → +15…+114 MB.
-
Web app: The lasso projects the pane's visible cells once per press and tests only the cells a new vertex can flip, bit for bit the full even-odd test: 0.06–0.11 ms per pointer move on Suo with a 900-vertex lasso (was 0.93–1.3 s). Proximity re-tests only the radius band that changed; KNN expands on the neighbour lists the reader worker built. Selections, highlights, undo history and the Python selection wire share one representation, an ascending
Uint32Array; selected cell sets are identical before and after over 510 exact comparisons. -
Web app: The reader worker builds a neighbour graph's page payload as it checks the matrix: edges arrive already in the renderer's fixed random order as 12-byte records, with one ascending neighbour list per cell, and a transform-feedback pass gathers edge positions on the GPU, so a filter change gathers nothing on the CPU. With edges on (Suo, 6.3M edges, K = 250,000), a legend click takes 6–8 ms instead of 19–57 ms; frames are pixel-identical. Embeddings and vector fields are checked, normalized and bounded in the worker (2D embedding main-thread time 71.6 → 7.3 ms), and no spatial tree is built on the main thread.
-
Web app: Colouring and filtering touch each cell once, with byte-identical output over 45 interactions. Main-thread time on a 562,000-cell dataset: picking a loaded gene 18 → 5 ms, a legend click 12 → 6 ms, an outlier-slider step 22–24 → 9–12 ms, keeping a view 56 → 40 ms. A view whose filter inputs match an array another view holds reuses it, and GL error reads happen only after new storage is allocated, so restoring a 7-view session runs with no task over 50 ms. Smoke rebuilds only when its inputs change.
-
Web app: Outlier quantiles are read from the file, one field at a time, on the filter's first use; for any other file or latent space they are computed in the reader worker, every field asked for together in one pass over the latent space, which is then kept within its budget. Quantiles that no longer fit the file's categories are computed instead. The latent space is no longer downloaded when a dataset opens. Sessions never drop an outlier threshold silently: one that cannot be restored is left at 100% and named.
-
Web app: Memory budgets are disjoint shares of one allowance, half the estimated device memory (
navigator.deviceMemory, raised past an 8 GiB report by four times the JavaScript heap limit; 8 GiB where neither is reported). Decode workers are sized by cores and that budget; switching expression matrices keeps each built gene index while they fit, so switching back reads nothing; the default analysis budget is a sixteenth of device memory. -
Web app: A server or notebook dataset (
cellucid serve,serve_anndata,show_anndata) is opened with the browser's AnnData reader, as a linked file is, under the server's dataset id, name and launch defaults; its dataset list is described from metadata and a dataset is opened when chosen. Theremote://andjupyter://schemes and the server-side mapping routes are gone. -
Web app: Sessions and community annotations follow the dataset, not the route: a dataset opened from a sample, a link, a server or a local file is the same dataset. A local file is named by
uns['cellucid']['name'], elseuns['title'], else its file name. -
Web app: The sidebar gene search and the analysis gene selector share one windowed list (
app/ui/components/windowed-listbox.js) with the same keys: arrows wrap at either end, Home/End jump to the ends, and every option carriesaria-posinset/aria-setsize. Field renames, deletions and restores touch only what changed (Suo: gene delete 20 → 2.3 ms). -
Datasets: The datasets are built by
scripts/build_atlas.pyandscripts/build_pancreas.pythroughscripts/pinned_build.py, which pins the environment, inputs, analysis and the producer modulesprepare()executes, reads every staged store back with anndata, and compares its digest withsources/<id>.json(schema 4). Starting views were re-captured on each store as format-2 sessions; newscripts/capture_session_preset.mjs,scripts/verify_catalog.mjsandscripts/local_site.mjs. CI opens every store with anndata 0.12.19. -
Reproducibility:
cellucid-reproducibilitystates that every recorded run measured the export-based Cellucid (web0ab545aed, cellucid-python 0.9.1) and what a re-run on the current version needs; its setup checks out the measured commits. Nothing was re-run. -
Web app: The Session panel is redesigned. The Dataset box describes only the dataset that is open, rewritten on every dataset change: its name, cells and genes, format and host (
Zarr v2 folder · theislab.github.io,h5ad file · this computer), its own description (two lines, More), how much has been read in place, and a collapsed Details with source and citation links, file, size, obs fields, embeddings, layers, raw, neighbour graph, latent space and vector fields; nothing is cut mid-word. One Open… button opens a File… / Folder… menu (a browser dialog picks files or a folder, never both) in place of the H5AD, Zarr ZIP and Zarr folder buttons. One Link or server address bar with Open replaces Server and Open from link; what is typed decides (a port,host:portorlocalhostis a server, a provider form or a file address is a link, any other web address is a server when one answers/_cellucid/health), and a connection shows as one line with Disconnect. The Transfers block is removed: how much has been read is one line in the Dataset box, and a retried read is a notification. The sample list shows—while a file, a single-dataset link or a server is open, and lists a connected catalog's datasets, named "Dataset from ", instead of mixing sources. Data mapping no longer opens on its own: the best-guess notification's Show opens it, its rows are one flat list, and arrays that are off and wider than three columns, or cannot be drawn, are listed under a collapsed Other arrays. Without a latent space the outlier filter is one Choose latent space button; with one, the slider and at most one short line. A Zarr v2 folder chosen in Safari, whose folder dialog leaves out hidden files, is refused with how to open it instead (drag the folder onto the page, or its.zarr.zip). -
Web app: While the first dataset is still opening, the sample list already offers every entry of
datasets.jsonunder the name its path gives it, the one being opened marked · opening…; names and cell counts follow once the files are read. -
Web app: A dataset that does not open is said once, in one notification of one sentence (Could not open "Garcia".), its raw error folded under the notification's Details and in the console; the explanation and Try again stay next to the selector. Opening and connection failures keep their raw error under Details too, never in the sentence.
-
Web app: The Highlighting tool's gesture line (and Color by a field first…) and the Antialiasing line (Chosen automatically: …) are the first line of their (i) tips; under the Tool buttons the sidebar shows only what a gesture did.
-
Web app: The Signed in to Google line with Sign out goes when the sign-in expires: the token store says when a sign-in is stored, removed or expires.
-
Web app: A link to a host that ignores range requests opens: its file is downloaded once into memory, with its progress shown, and opened from there. A host that does not let other sites read its files says so and what to do: "
<host>did not let this site read<file>: it does not allow other websites to read its files (CORS), or it could not be reached. Download the file and open it from your computer instead." -
Web app: Opening a link starts the AnnData reader worker while the link is resolved, so the worker and its modules are loaded by the time the file is found. A CELLxGENE link probes the file in the Census release the browser read last beside the release list, so it usually opens one round trip sooner; a newer release is followed. A Zenodo link that names its file asks for the file's first bytes beside the record, so it opens one round trip sooner too. Opening a link again in the same page (Disconnect and open, a session reopening its dataset) asks its provider nothing and starts from the first bytes read the first time.
-
Web app: Analysis windows and floating sidebar panels resize from all four edges and corners, by mouse, pen, finger and keyboard (arrow keys on the focused edge, 16 px or 64 px with Shift, Home/End to the least and largest size;
Alt+arrows move the window). They are held only to their content's least size and the viewport: the 500 px height cap and the twice-the-opening-width cap are gone. Edges snap to other windows, the sidebar, the middle of the free area, the screen and the content height, with a guide line, and show the size while they move. A double-click on an edge fits it; Maximize (or a double-click on the header) fills the free area beside the sidebar. A new window of an analysis opens at the size the last one was given. On a screen 720 px wide or less, analysis windows are bottom sheets at 40%, 75% or 100% of the screen, with a grip, a collapsing header and a tab strip for the others. -
Web app: The expanded analysis view moves and resizes by mouse, pen and finger (it was mouse-only) from every edge and corner, its splitters and bottom-right corner answer arrow keys, a double-click on its header maximizes it, its least size is its content's (not 600 × 500), and its size and splits are remembered per analysis. On a screen 768 px wide or less it fills the screen with a touch grip on its split.
-
Web app: The Analysis panel groups its rows: Genes (Marker genes, Differential expression, Dot plot, Gene signature), Cells (Composition) and Explore (Quick, Detailed, Correlation).
-
Web app, breaking: The
analysis/windowssession chunk is{windows, sizes}: each window records its own box (not the header-high box of a collapsed window, which it used to save), whether it is open and its stacking layer, andsizesholds the remembered window and expanded-view sizes per analysis. A chunk without them is refused. -
Web app, development: The web contract suite needs Node.js 22.15 or newer (
engines.node); its Zstandard contracts compress their inputs withnode:zlib's Zstandard encoder, which Node.js 20 (end of life) lacks. CI runs Node 22, 24 and 26. -
Web app: Marker genes compares every group of a field against the rest or a chosen reference group, for every group or a chosen subset, inside all cells, the visible cells or a page. A group under Min cells per group is never an error: it is skipped and listed (default), pooled with the other small groups, or tested anyway and flagged low n, and chips on the result name them. The test follows scanpy's
rank_genes_groups: Wilcoxon (score = the z of the rank sum, with AUROC), Welch's t-test or the t-test with overestimated variance; p-values exact or continuity-corrected (SciPy, R) or scanpy's normal approximation with or without tie correction; Benjamini–Hochberg or Bonferroni over the genes tested; fold changes as stored or withlog1pundone (automatic when the data mapping marks the matrixlog1p). The result keeps every gene's statistics for every group, so direction, filters (adjusted p, |log2FC|, % in group, % in reference), ranking (score, adjusted p, log2FC, AUROC, % difference) and the number shown re-apply without testing again; it shows as a list, a heatmap or a clustered heatmap. Genes can be excluded by family (mitochondrial, ribosomal, hemoglobin) or pattern, or limited to the highly variable genes or a list. It exports the shown markers and every gene as CSV with scanpy's column names plusauroc,small_groupandtested, and the markers as GMT; Score as signature and Dot plot hand them on. The statistics match scanpy and SciPy on the shared fixture. -
Web app: Differential expression compares two pages or two sets of categories of a field (disease against healthy), optionally within the visible cells, a page or categories of another field, with Swap A and B. A group under Min cells per group is tested and flagged low n instead of refused; only overlapping groups and a group under 2 cells disable Run, with the reason. It has Marker genes' test options, free adjusted-p, |log2FC| and % expressing filters that redraw the volcano without testing again, and exports scanpy's column names with
pct_nz_group,pct_nz_reference,auroc, the group means and counts. -
Web app: Gene signature scores one or more named gene sets — typed, pasted with any separators, imported (
.txt,.csv/.tsvin long or wide shape,.gmt) or taken from Marker genes — by mean, median, mean z-score, scanpy's module score (tl.score_genes: control genes drawn from expression bins with a seed) or UCell (ranks, with a max rank). A name is matched exactly, or else to the one gene matching it ignoring case; names the dataset lacks are left out and listed, never an error. Rescaling (z-score, 0–1), the plot and the layout of several sets redraw without scoring again. The CSV ispage,set,cell_index,score. Module score matches scanpy and UCell a numpy reference on the shared fixture. -
Web app: Gene-streaming analyses (Marker genes, Differential expression, Gene signature) size themselves: a run reads ahead as many genes as the workers consume while one gene is fetched, from the measured link and each gene's measured size and test time, within the analysis memory budget, and adapts while it runs (more read-ahead when the workers wait on reads, kept only when it makes the run faster; less on a throughput drop or memory pressure). Analysis workers start one at a time, so building the pool never holds the page. How many requests go out at once is the reader's scheduler's to decide. Nothing it decides changes a statistic.
-
Web app: One design for every screen of the site: the startup failure screen, the code-load failure screen, the no-JavaScript screen, the 404 page, the privacy policy and the sign-in return page are the app's brand card (wordmark, heading, one sentence, clear actions) in the theme chosen in the app; the standalone pages load a small
page.cssinstead of the whole app stylesheet. The design principles are inassets/css/README.md. -
Web app: The startup failure screen says what kind of problem it was (the data could not be opened, a server did not answer, this browser cannot run Cellucid, Cellucid could not start), folds the exact message (address, host, status) under Details, and offers Try again and Open Cellucid without this link. A module that fails to download shows the same screen, not only a red status line.
-
Web app: Data a link names that does not open no longer stops Cellucid. A
?data=link, a?remote=server, a?dataset=or a catalog's default dataset that cannot be read leaves the app running without it: a panel over the empty viewer says the data could not be opened, with the exact message under Details (and in the notification log), Try again, and Open without this link, which removes those settings from the address. The full-screen card is kept for the app itself failing (no WebGL2, its code did not load, its own server). The panel says the reason itself, as its notification does ("This 10x Cell Ranger .h5 file has no cell embedding …"; "… holds no datasets.json …"; "The owner of x.rds has turned off downloading for it"), and when asking again cannot change the answer (a file or folder that is not there, a refusal, a file Cellucid does not read), Open without this link comes first, before Try again. Google Drive and OneDrive refusals and a Drive read held back are shown as written wherever they arrive. -
Docs: The documentation site takes the app's look: Inter, the app's ink and cyan colours in light and dark, square surfaces, the app's favicon, and the wordmark without a duplicate "Cellucid" beside it.
-
Docs: Every screenshot is recaptured from the current app by its scenario, with a
captures.jsonrecord naming the clean web commit it shows, so none is left without provenance; captions and alt text describe what each image shows. New figures for the R objects, Loom, Cell Rangerouts/, MatrixMarket and.h5muModality pages, the opening progress and failure panels, the Google Drive and OneDrive sign-in lines, Composition and Dot plot, the R viewer, and the phone layout. The JupyterLab images are replaced by the viewershow_anndata()serves and a transcript of a realviewer.debug_connection()report, and the Firefox and WebKit captures are gone. The screenshot tool takes a per-scenarioviewport(withtouchfor a phone) and grows a crop to hold the whole drawn pointer. -
Docs: A new home page: the tagline and one sentence, Open Cellucid and the 60-second tour as calls to action at a documentation size, the formats it opens, a fresh light and dark capture of the Suo sample in 3D, the three ways in (web app, Python, R), four features with their own captures, compact capability and learning cards, and citation and contribution; it has no sidebars. The outdated close-up screenshot is gone. The images are WebP captures from new
home-*scenarios of the screenshot tool, which now writes WebP (format: 'webp', a fixedquality) as well as PNG, with provenance incaptures.json. -
Web app: One
favicon.ico(the root one) for every page. -
Web app: A published sample view that cannot be applied says so in one sentence; the reason is under the notification's Details.
-
Python, R, web app: Highlights and visibility changes carry their cells in a binary frame, not a JSON list.
viewer.highlight_cells()andviewer.set_visibility()take a list of ints or a one-dimensional NumPy integer array (the fast form), in any order, each once; the cells are checked and sorted once and written straight into the frame.set_visibility([])is refused (passNonefor every cell). Two million cells are a 7.6 MB frame instead of 15 MB of JSON: from NumPy, Python builds it in 3 ms (the JSON took 0.35 s) and the viewer has applied the highlight 0.22 s after the call; ten million cells, 13 ms (1.7 s) and 0.86 s. R builds the frame of ten million cells in 16 ms (the JSON took 4.2 s). -
Python:
viewer.send_message()returnsTruewhen the command was sent andFalsewhen it waits for the viewer to connect; it raises only afterviewer.stop().get_session_bundle()raises at once when the viewer reports it could not capture the bundle, or sent an invalid one. Hook callbacks run on the viewer channel's thread, one event at a time and in order; a hook that raises is logged and the channel stays open. The event is recorded inviewer.stateand handed towait_for_event()before any hook runs, and the other hooks still run: one buggyon_messagehook no longer drops the event, soget_session_bundle()no longer times out behind it (several failures are logged together, as oneExceptionGroup).debug_connection()reportschannel_connected; thedebug_snapshotevent no longer hasparentOrigin, and no event carriesviewerId. -
Docs: "Architecture: message routing (HTTP vs postMessage)" is "Architecture: the viewer channel" (
e_jupyter_hooks/05_architecture_viewer_channel). -
Web app: In Full quality, the highlight pulse redraws only the layer over the points. While highlights pulse, each pane's base layer (backdrop, grid, every point, the vector-field overlay) is kept and every pulse frame draws the highlights, edges, centroids and orbit anchor over a copy of it, pixel for pixel the frame drawn whole (
rendering/pulse-layer-cache.js;tests/browser/highlight-pulse-layer-parity.spec.mjs, one view and a grid, three engines). GPU time of a pulse frame (Chromium, Metal, antialiasing off): 10M cells with 20,000 highlighted 9.97 → 0.48 ms, with 5M highlighted 38.9 → 24.1 ms; 1.5M cells with 15,000 highlighted 4.91 → 0.69 ms. With antialiasing on, a multisampled pane keeps its own samples instead (WebGL2 cannot copy one multisampled buffer into another): the ring pass marks the samples it covers in the stencil, and a pulse frame resets and draws again only those, every point still submitted but only those under a ring shaded (rendering/pulse-sample-mask.js; the same spec, the pulse moved between two phases, three engines). 10M cells, antialiasing on: 20,000 highlighted cells in one region 40.3 → 7.5 ms, 300,000 in one region 70.0 → 15.7 ms, 20,000 scattered over the embedding 37.2 → 25.3 ms, 1M scattered 44.3 → 45.0 ms (their rings cover the embedding). -
Web app: Applying a highlight costs the highlighted cells, not the dataset: the page counts no longer copy a page's cells into an array to count them, nor to digest them (5M of 10M cells: 85 ms of the apply); 1M highlighted cells of 10M apply in 24–26 ms (57 ms).
-
Web app: Startup no longer runs as one ~90 ms task: it yields between its first stages (the interface, the WebGL programs), and the page's first layout (about 25 ms) is done once, where the welcome dialog takes focus, instead of also being forced by checks that measured the page (the dialog's focus check and the notification center's empty card stack). Chromium: no main-thread task over 50 ms while the app starts. A PNG figure's pixels are read in bands of rows with a yield between them, so the read-back of a 300 dpi figure is no longer one long task.
-
Web app: Publishing a large dataset no longer holds the main thread. Its GPU stores are allocated in a task of their own just before the publication, once the GPU has done the work queued before (a WebGL allocation is checked by a
getErrorround trip that waits for the GPU: up to 30 ms of the publishing task), so the publication reads nothing back; its positions and colors go to the GPU with the first frame (the colors were sent twice); and each per-cell array is written once: the first visibility pass adopts the new dataset's all-shown visibility, clearing the active field leaves gray colors gray and republishes no unchanged transparency, the highlight list of a dataset with no highlight is empty instead of being derived from ten million bytes, a fresh mask is proven binary as it is made, and the 0/1 per-cell bytes (validation, the R8 alpha texture of the live and kept views) are read four cells to a word. The first slice of positions for the spatial-tree worker is copied in its own task. Chromium (Metal), 10,000,000 cells in 3D: opening 227–237 ms → no task over 50 ms, switching to it 337–349 ms → none (largest 45 ms). -
Web app: Coloring by a category of a large dataset no longer holds the main thread: before the field is shown, its per-category totals and its centroids for the shown representation are computed in slices of about 8 ms (the same additions in the same order, so the same centroids), the codes are proven against their categories once when they load, the colors are written without the transparency the visibility pass writes next, and the visible count of each category is read from the totals when the only filter is the field's own (a category is then all shown or all hidden). Hiding or showing one category reads no mask when every cell is shown. Chromium, 10,000,000 cells: coloring by a 30-category field 187–195 ms → no task over 50 ms; hiding one category in the legend 319–332 → 38 ms; showing it again 232–237 → 42 ms.
-
Web app: A 10M-cell tree landing no longer holds the main thread: the spatial-tree worker sends the tree as typed arrays (a structured clone of its 38,568 node objects took 35 ms in one task; now 10–11 ms with the install), and the store's adoption of the tree's draw order copies its source words on the GPU, drawn from the store into the source texture, instead of uploading 120 MB of positions and 40 MB of colours from the page (adoption 32–35 → 14–19 ms). Chromium (Metal), 10M cells: no task over 50 ms from opening to the tree in use (was a 77–90 ms frame of two tasks, once one 101 ms task).
-
Web app: A highlight of one cell list costs that list: a page with one enabled group is painted (runs of consecutive cells as one fill) and published from the group's own list, counted as its length, and a cell list is proven ascending once (
app/utils/cell-set.js), so the page selectors and analysis no longer walk it on every change; the highlight layer writes each record's color as one word, in a packing loop of its own. Chromium, 10,000,000 cells, highlighting 5,000,000: 114 → 47 ms, next frame 72 → 30 ms; all ten million: 177 → 72 ms, next frame 164 → 30 ms. A highlight of many cells is drawn from the view's point store, already on the GPU in draw order, with a colour word a slot written on the GPU from the highlight and visibility bytes, whenever that writes fewer bytes than records of its own (the CPU gathered ten million positions at random: 72 ms, and a new 160 MB store was cleared by the GPU process before it could be checked: 54 ms); the pulse frame of ten million highlighted cells takes 9 ms of GPU time instead of 54 (tests/browser/highlight-store-draw-parity.spec.mjs, three engines). -
Web app: Opening an analysis loads Plotly at the page's next idle moment, so its evaluation (one script, about 190 ms of main thread) runs while the analysis is set up instead of inside the first run: a differential-expression run started two seconds after its panel opened no longer has the 234 ms task (Chromium, 1.5M cells).
-
Web app: A MatrixMarket folder opens in about one and a half times the time plain inflate of the same bytes takes: the entries are parsed in blocks of about 1 MiB on the decode workers into typed arrays and collected a block at a time, a large gzip text or R file is decompressed on a worker of its own (
formats/rds/gunzip-worker.js;DecompressionStreaminflates on the calling thread), the transcode space is written 1 MiB at a time, and the cell tables are read while the matrix streams. 100,000 cells (44M counts): Chromium 1.6 s (1.5× plain inflate), Firefox 1.7 s (1.5×), WebKit 1.0 s (1.33×); 500,000 cells in Chromium 7.6 s.npm run bench:formatsruns the decode workers onworker_threads(Node, 100k cells: 1.80×). An entry whose row or column is not a whole number is refused. -
Web app: One shared module (
utils/exact-record.js) holds the record and key checks the app's modules each defined for themselves (about forty copies, with different messages and different ideas of a record):requirePlainRecord/requireKeys/requireExactKeysfor data from JSON, a session or a worker,requireObjectRecord/requireObjectKeysfor an owner the app built,requireClonedRecordfor worker payloads. A key mismatch reads the same everywhere: "… must contain exactly: a, b, c (missing: b; unexpected: d)." The community-annotation wire contract keeps its own path-addressed messages, the ones the annotation repository's validator writes. -
Web app: One formatter for sizes and durations (
utils/format-units.js): the notification center wrote "2 MB" and "850ms" where the dataset card and read progress wrote "2.0 MB" and "850 ms". Every message now writes "512 B", "4.2 KB", "12 MB" and "850 ms", "4.2 s", "3 min 5 s", and no unit rounds into the next one ("1024 KB", "1000 ms", "60.0 s"). -
Web app: A file whose bytes are not HDF5 names the format it was opened as: "The selected file is not a valid HDF5 file, so it cannot be a Loom (.loom) file. Write it again with loompy …"; a damaged file or Zarr store gives its own format's route to write it again.
-
Web app: The viewer channel's modules and names are neutral:
data/viewer-host-source.js(ViewerHostDataSource),app/viewer-host-command-handler.jsandapp/viewer-host-pointer-delivery.js(they werejupyter-*,JupyterBridgeDataSource, for a channel that serves Python, Jupyter and R hosts), and their messages say "Viewer host …". Thejupytersource type of sessions and analytics, and thejupyterruntime, are unchanged. -
Web app: The "Link or server" ⓘ tip names Figshare, Google Drive and OneDrive, and every other link provider (a contract holds it to the provider registry). The cell-info tip and page state the spatial-index threshold the viewer uses (views of 250,000 cells or more, built in a worker); they said "above two million cells" and "above 256 MB of positions no background build".
-
Web app, development:
npm run bench:formatsmeasures every non-AnnData format (Loom, a Cell Rangerouts/folder, a MatrixMarket folder, Seurat and SingleCellExperiment.rds, a Seurat.rda,.h5mu, MuData Zarr) against the same data as a Cellucid-ready.h5ad: first view, first gene, peak memory, and a stream format's pass against plain inflate of its own bytes (scripts/benchmark-formats/). -
Web app, development: A monthly CI job (
.github/workflows/r-corpus-drift.yml, also on changes to the R readers; locallysh scripts/rds-corpus/drift.sh <Rscript>) writes the R object corpus with the current CRAN SeuratObject and Matrix and Bioconductor SingleCellExperiment, reads every file with the transcoders against what R read back, and fails on a slot those packages add or remove (scripts/rds-corpus/known-slots.json). -
Python, R, web app: An h5Seurat file is refused the same way everywhere, in the words of the input format registry (
refusal, in all three repositories'input_formats.json): "This is an h5Seurat file (SeuratDisk), which Cellucid does not open. In R, read it with seu <- SeuratDisk::LoadH5Seurat(path) and save it with saveRDS(seu, "object.rds"), then open the .rds, or show it with cellucid::cellucid_view(seu)."cellucid serve x.h5Seuratandcellucid_serve()refuse it before serving. -
Python: Stopping a server closes the connections it still holds, so a viewer started next on the same port never answers from the previous dataset.
-
Web app: A gzip or zlib Zarr chunk, and a ZIP entry, are decompressed with their input fed in 32 KiB slices, so the size the format declares bounds the output while the chunk decodes, in every engine (WebKit's
DecompressionStreaminflates everything written to it at once). The session codec and the format readers share this one input feeder (formats/io/inflate.js). -
Web app: A read's progress counts the bytes of every request as they arrive, so the progress, rate and time left of a long request (an 850 KB gene-name heap over a slow link, an R object or a MatrixMarket folder read once) stay live; the reader worker posts a running read's progress at most every 250 ms and always posts the last change of an interval, and the bytes of an attempt that fails are taken back. A GEO sample file its host sends compressed is downloaded with its bytes counted.
-
Web app: Picking or dropping a file, or making a connection, is always a new opening, so the file already open opens again (a newer copy with the same dataset id is shown); only choosing a listed dataset again keeps what is open. An
.rdsopened again in the tab reads what was kept the first time. -
Web app: A format refusal's advice is what can be done from where the file was opened (the reader worker's
openis toldfromLink): a 10x Cell Ranger.h5with no embedding opened from a link says to download the wholeouts/folder and open it from this computer, or to compute an embedding (a link opens one file, and no provider shares a folder Cellucid reads); a linked molecule table says to linkfiltered_feature_bc_matrix.h5instead; a 10x MatrixMarket matrix from a GEO sample, which has noouts/folder, says to compute an embedding; and anouts/folder whoseanalysis/has no projection says Cell Ranger ran without its secondary analysis. -
Web app: An address typed without
http://on this computer (127.0.0.1:8000/atlas/), or a server's address, that answers but is no Cellucid server (a static web host serving a folder: no/_cellucid/health, or a web page there) is said to be no Cellucid server, with how to open such a folder: the address of itsdatasets.json. -
Web app: The startup phase that opens the dataset is published as
dataset(data-startup-phase, and thestartup_phaseanalytics field), notidentity. -
Web app, development:
npm run bench:anndatabenchmarks the AnnData reader, andscripts/benchmark-real-links.mjsopens files from this computer through the File… and Folder… buttons ({local}/path), serves a link over HTTP/2 (--link-protocol h2), and reports each step's peak browser memory and main-thread long tasks. -
Web app: Opening a link (
?data=, the address bar) or a server reads no sample file: the dataset list names the samples fromdatasets.json(one request) and reads their files (title and cell count, about 85 requests) only once the list is reached for. A page opened on a sample, or with nothing connected, lists them as before. -
Web app: The address bar names a link the page was opened with in its provider's form, as one opened from the bar is named: a pasted Google Drive link becomes
?data=gdrive:<id>, a Figshare pagefigshare:<id>, a OneDrive share linkonedrive:<drive>/<item>, a DOI of another publisher the Figshare item it names, and a Dropbox link its share link (dropbox:https://www.dropbox.com/…).
Removed
-
Python: The prepared export format and everything that wrote or served it:
prepare()'slatent_space,out_dir,obs_categorical_dtype,var_quantization,obs_continuous_quantization,compression,var_gene_name_columns,source_citation(nowcitation_text/citation_url),created_atandcentroid_outlier_quantilearguments;Embedding(store the whole array and choose columns withembedding_defaults={"3d": ("X_pca", [0, 1, 2])});generate_datasets_manifest();serve(),CellucidServer,show()andCellucidViewer; the prepared-export branch ofcellucid serve; the export lock, transaction journal, manifests, quantization and payload helpers. -
Python:
AnnDataAdapterand the server-side virtual export it computed (manifests, payload and mapping routes, server-side gzip, centroids at startup), the backed read mode,serve_vector_fields/cellucid serve --vector-fields(the viewer reads a vector field when its overlay is shown),/_cellucid/health'sis_backed, andcentroid_outlier_quantilefromserve_anndata,show_anndataandcellucid serve. -
R:
cellucid_prepare()'slatent_space(name an embedding withlatent_key),out_dir,obs_categorical_dtype,var_quantization,obs_continuous_quantization,compression,var_gene_name_columns,source_citation,created_atandcentroid_outlier_quantilearguments,cellucid_embedding(), and the export lock and transaction journal. -
Web app: Prepared exports (
dataset_identity.json, the obs, var and connectivity manifests, quantized and gzip payloads), catalogs of exports,state-snapshots.json, the Prepared button and the prepared-folder drop; with them gene download-ahead and its payload cache, the quantization worker, the data-mapping "Read from file" section and the analyticsdata_versionparameter. -
Web app:
defaults['centroidOutlierQuantile'](a file or server naming it is refused as an unknown default) and the per-centroid outlier array of views and kept-view snapshots; the marker cache's IndexedDB tier (cellucid_marker_cache) and itsmaxAgeDays/cacheVersionoptions; the bulk gene-expression cache and its API (getGeneExpressionSubset,fetchBulkGeneExpression); the data layer's session cache export and import. -
Datasets:
exports-v3/incellucid-datasets(299 MB of prepared exports and their snapshots) and the eight preparation notebooks;exports/and the Zarr v3 folder incellucid-demo-custom-datasets. -
Web app: The per-file notices "Opened H5AD directly in the browser…" and "Opened a Zarr … directly in the browser…", the loaders' own download notifications and their
showProgressoption, andZarrDataSource.loadFromFileMap(nothing called it): the opening progress shows every file. -
Web app: The analysis Performance settings (batch size, memory budget, network and compute parallelism, and their presets); Marker genes' mode select (Ranked, Clustered, Custom genes), its Use cached results checkbox (the cache is always on, marked Cached with Recompute) and its ranked and heatmap CSV files; Gene signature's sum; the run-cost line under the run button.
-
Python, web app: The notebook
postMessagecommand path (the script in the notebook output,AnnDataViewer.viewer_origin) and the routesPOST /_cellucid/events,/_cellucid/selectionand/_cellucid/session_bundle: commands, events, selections and session bundles travel on the viewer channel, and the server answers everyPOSTwith 405. The viewer's health poll every 3 s is gone: the host closing the channel is the stop signal. -
Web app: The remote source's unused update socket (
ws_portin/_cellucid/info) and the connection-loss recovery that only it could trigger; a server advertisingws_portis refused.
The Fixed section is omitted here because GitHub limits a release page to 125,000 characters.
The complete list of additions, changes and fixes for 0.9.5, for the web app, the Python package and the R package, is in CHANGELOG.md. The R package is released as cellucid-r v0.9.5.