Skip to content

GPTchatly/Super-Fast-Gallery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Super fast gallery

A static-site photo gallery for very large image collections. The build pipeline turns a folder of photos into a self-contained static deploy that can be served from any CDN. The runtime is a WebGL2 sprite renderer that keeps a justified grid of thousands–millions of thumbnails scrolling at 60 fps, with a two-stage lightbox showing the full-resolution original and a download button.

There is no server in production — Vercel, CloudFlare Pages, Netlify, or any static host can serve the output directly.


Quick start: rendering your own photos

1. Drop your photos into  ./photos/
2. npm install              (first time only)
3. npm run build
4. npm run dev              (preview at http://localhost:3000)

Supported formats: JPEG, PNG, WebP, AVIF, TIFF, HEIC. Any subdirectories under ./photos/ are scanned recursively. Files are ordered by lexicographic filename — name them 001-foo.jpg, 002-bar.jpg, etc. if you want a specific order.

Building writes everything into public/galleries/demo/. Visit http://localhost:3000 to see the index of all galleries.

Building under a different name

npm run build -- --id my-trip --title "Iceland 2024"

This produces public/galleries/my-trip/. The index page at http://localhost:3000/ links to every gallery in public/galleries/.

Other build flags

npm run build -- \
  --source ./somewhere/else \         # default: ./photos
  --out ./public/galleries/my-trip \  # default: public/galleries/demo
  --id my-trip \
  --title "My Trip" \
  --lods h50,h100,h200,h400 \         # which LOD thumbnails to generate
  --concurrency 8 \                   # parallel Sharp workers
  --no-originals                      # skip the full-resolution download set

--no-originals halves disk usage and skips the lightbox download button. Useful for very large galleries (>50k photos) where you don't want to store the original bytes twice.

Re-indexing

You can add, remove, or rename photos and rebuild. Hashes are cached in .hash-cache.json, so unchanged photos aren't re-processed.

Re-indexing shifts photo IDs (everything is keyed by lexicographic order from scanner.ts). The build emits a fresh buildId that cache-busts every cached browser asset on the next page load, so the viewer always sees a consistent state.


How it works

Build phase (src/builder/)

photos/  →  scanner  →  thumbgen  →  packer  →  atlas  →  manifest
                              ↓                              ↓
                         raw thumbnails             _imagemap.bin
                                                    _sprite.json
                                                    _originals.bin
                                                    sprite-*.avif

           originals/  ←  copyOriginals (parallel)
            (untouched bytes + sha-1 hash per file)

For each LOD level (h50, h100, h200, h400), every photo is resized to that height (preserving aspect ratio) and shelf-packed into 2048×2048 AVIF atlas pages. Photo positions inside each atlas are recorded in a binary file (_imagemap.bin) that the viewer reads with zero-copy typed-array views.

A second pass copies every original file byte-for-byte into originals/{paddedImageId}.{ext}, hashes it, and records hashes + filenames in _originals.bin. These hashes form per-image cache-bust tokens so re-deploying with unchanged photos hits the CDN edge cache 100% of the time.

The build runs each stage with pLimit worker parallelism (default 6). Builds are atomic: output goes to a temp directory that's renamed into place at the very end, so a failed build never leaves you with a half-published gallery.

Runtime (public/viewer/)

viewer.js  ──┬─→  binary-parser.js   (zero-copy view over _imagemap.bin)
             ├─→  layout.js          (justified grid, Uint32/Uint16 SoA arrays)
             ├─→  webgl-renderer.js  (WebGL2 instanced quads, mipmaps, RGB8)
             ├─→  atlas-cache.js     (LRU with master-LOD pinning)
             │      └─ atlas-decoder-worker.js   (off-thread AVIF decode)
             ├─→  lod-selector.js
             ├─→  originals-loader.js (lazy parse of _originals.bin)
             └─→  lightbox.js         (preview→original two-stage display)

Per frame the renderer:

  1. Binary-searches the visible scroll range over typed-array coordinates (microseconds even for 1M images).
  2. For each visible image, picks the best loaded LOD — target first, falling back to coarser LODs all the way down to a pinned master atlas, so every cell always shows something during scroll. Google Photos-style progressive sharpening.
  3. Buckets visible sprites by bitmap, writes instance data directly into a preallocated Float32Array, then issues one drawArraysInstanced per atlas. Zero per-frame heap allocations.
  4. During very fast scrolls, suspends new atlas loads (they'd scroll past before they finish decoding anyway).

The atlas cache adapts its size to navigator.deviceMemory and the user agent — small on mobile, larger on desktop — and decodes AVIF in a worker pool so scrolling never stalls on the main thread.

Lightbox

Clicking a photo:

  1. Immediately shows the 1920px JPEG preview (in images/) for instant first paint.
  2. Lazily fetches _originals.bin on first click of a session.
  3. Begins loading the full original from originals/. Fades it in over the preview when ready.
  4. Lights up a download button — same-origin <a download="…"> so the user's chosen filename is preserved.

If the original is in a format the browser can't decode (HEIC on Chrome, TIFF, RAW), the preview stays shown and a banner suggests downloading to view.


Directory layout after a build

public/galleries/{id}/
├── index.html
├── gallery/                     # WebGL renderer data
│   ├── _sprite.json             # manifest (buildId, level definitions, UI config)
│   ├── _imagemap.bin            # binary layout index (zero-copy parsed)
│   ├── _originals.bin           # lazy per-image hash/filename/size
│   └── sprite-h{N}-*.avif       # 2048² sprite atlas pages
├── images/                      # 1920px JPEG previews (lightbox first paint)
└── originals/                   # untouched source files (display + download)

What gets cached, by which layer

Asset URL pattern Cache-bust Edge TTL
index.html …/index.html n/a no-cache
_sprite.json gallery/_sprite.json none (60s revalidate) 60s
_imagemap.bin gallery/_imagemap.bin?v={buildId} global buildId 1 year, immutable
_originals.bin gallery/_originals.bin?v={buildId} global buildId 1 year, immutable
Atlas pages gallery/sprite-*.avif?v={buildId} global buildId 1 year, immutable
1920px previews images/*.jpg?v={buildId} global buildId 1 year, immutable, s-maxage=1y
Originals originals/*.{ext}?h={contentHash} per-image content hash 1 year, immutable, s-maxage=1y

The cache-bust scheme means re-deploys never invalidate originals at the edge unless a photo's actual bytes changed — critical for galleries with hundreds of GB of originals.


Deployment

The build output in public/ is a fully static site. Any CDN works.

Vercel

git add public/ vercel.json .gitignore
git commit -m "build gallery"
git push                          # auto-deploys

The committed vercel.json sets outputDirectory: public, marks the project as not a framework (so Vercel doesn't try to treat src/ as a serverless function), and installs the cache-control header rules.

If Vercel's dashboard has a Framework Preset set from when you first imported the project, change it to "Other" in Settings → General.

Alternative — bypass dashboard auto-detection entirely:

npm install -g vercel
npm run build
vercel build
vercel deploy --prebuilt

CloudFlare Pages / Netlify

These read the _headers file at the deploy root (public/_headers, auto-generated by the build). Connect the repo, set the build command to npm run build:galleries (a no-op) and the output directory to public/.

For CloudFlare specifically: HEIC/TIFF aren't in CF's default cacheable extension list. If your gallery has those formats and you want them edge-cached, add a CloudFlare Cache Rule for /galleries/*/originals/* setting Cache Eligibility to "Eligible for cache".

Self-hosting

npm run dev runs an Express dev server at http://localhost:3000 that mirrors the production cache headers. Suitable for testing; for production behind nginx/caddy, just serve the public/ directory directly with the same Cache-Control headers as _headers describes.


Scripts

Command What it does
npm run build Build the demo gallery from ./photos/public/galleries/demo/
npm run build -- --id X --title Y Build a named gallery
npm run build -- --no-originals Skip the full-res originals copy (half the disk)
npm run build:watch Rebuild on photo changes
npm run dev Local dev server at port 3000
npm run seed Generate 50 synthetic test photos into ./photos/
npm run seed:large Generate 500 synthetic test photos
npm run build:galleries No-op build command for static-site CDNs (Vercel, etc.)

Performance notes

For a 100k-image gallery on a 4 GB mobile device:

Value
JS heap (steady state) ~5 MB
Per-frame heap allocations 0
VRAM (8 atlases @ RGB8 + mipmaps) ~96 MB
Layout computation ~10 ms (one-time, on resize)
Visible-rect culling O(log n) binary search
Atlas decode off main thread (worker pool of 3)
_originals.bin parse lazy, on first lightbox click
_imagemap.bin parse zero-copy (only aspects precomputed)

Tested on galleries up to 100k images. Scaling beyond 1M images is plausible but requires moving layout to a worker (it's currently O(n) single-pass on the main thread).


Requirements

  • Node.js ≥ 22 (uses --experimental-strip-types to run TypeScript directly)
  • Modern browser with WebGL2 + module Workers (Chrome 80+, Safari 15+, Firefox 114+)

Project structure

src/
├── builder/         # build pipeline (Node only)
│   ├── build.ts        # CLI entrypoint
│   ├── scanner.ts      # filesystem scan + Sharp metadata
│   ├── thumbgen.ts     # parallel raw-RGB LOD thumbnails
│   ├── packer.ts       # shelf-packing for sprite atlases
│   ├── atlas.ts        # composite atlas pages to AVIF
│   ├── originals.ts    # hash + copy untouched source files
│   └── manifest.ts     # emit _sprite.json + _imagemap.bin
├── server/
│   └── server.ts       # Express dev server (NOT used in production)
└── shared/
    └── types.ts        # TypeScript types shared between build and runtime

public/
├── viewer/          # browser runtime (served as static JS modules)
└── galleries/{id}/  # build output, one subdirectory per built gallery

scripts/
└── seed.ts          # synthetic photo generator for testing

About

Pre-rendered very fast static-site photo gallery with a WebGL2 sprite renderer for very large image collections. Build pipeline turns a folder of photos into a self-contained CDN deploy; runtime scrolls a justified grid of thousands of thumbnails at 60 fps with a full-resolution lightbox + download.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors