Skip to content

4.0.0: Redesign, Gallery, MineBench for iPhone, Limited Free Generations & More

Latest
Latest

Choose a tag to compare

View all tags
@Ammaar-Alam Ammaar-Alam released this 28 Aug 19:29
· 31 commits to master since this release
4.0.0
7bf991a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

MineBench 4.0 is our largest release yet!
Explore and share AI-built worlds in the new Gallery, generate with Gemini 3.7 Flash free for a limited time – no API key required – and take MineBench anywhere with the new iPhone app. Accounts now save generations automatically and unlock personal model rankings, while a faster Arena, stronger global ratings, and an upgraded renderer make every comparison feel better.

Highlights

  • Explore the new Gallery. Discover Top and New community prompts, inspect builds from every angle, compare up to four examples, vote for your favorites, reuse any prompt, and move between Gallery entries with on-screen arrows or the Left and Right Arrow keys. (#122, #130)
  • Generate free with Gemini 3.7 Flash. Signed-in users can select Gemini 3.7 Flash in Generate without adding a Gemini or OpenRouter API key for a limited time. Your own keys still work and take precedence. (#130)
  • MineBench for iPhone. Download MineBench free on the App Store for native blind Arena voting, leaderboards, Sandbox comparisons and imports, a Metal-powered viewer, and GIF, video, and build exports.
  • Never lose a generation. Signed-in Generate jobs keep running if you leave the page. Every result is saved privately under Account → Builds, where you can inspect, retry, stop, download, remove, or publish it under your name or anonymously. (#122)
  • Accounts and personal rankings. Sign in to keep your builds and see which models you personally prefer based on your public Arena votes. Anonymous voting remains available without an account. (#118)
  • A much faster Arena and Leaderboard. Background preparation, build caching, more efficient rendering data, and cached ranking snapshots make comparisons ready to inspect and vote on much sooner. (#106–#114)
  • Sharper builds and a quieter interface. Smoother lighting, crisper textures, better distant detail, stronger mobile layouts, and a more consistent visual system keep the builds at the center of the experience. (#99, #100, #105)
  • More rigorous rankings and more models to test. Public ratings now use a joint Bradley–Terry fit with confidence intervals and stable unique ranks. Sandbox supports arbitrary OpenRouter models, and GLM 5.3 Flash joins the benchmark. (#96, #113, #123)
  • Private evaluation workspaces. Teams can privately generate, compare, manage, and export results for unreleased or fine-tuned checkpoints without affecting MineBench's public rankings. (#98, #115–#117, #121)

Detailed Changes

Gallery

  • Added a public Gallery with Top and New discovery views for community-submitted prompts and builds.
  • Added permanent, shareable detail pages with previous and next controls that preserve the active Gallery order.
  • Added compact lifetime and hosted generation totals to user profiles in Gallery administration.
  • Added Left and Right Arrow keyboard navigation while keeping text fields, dialogs, menus, and modified shortcuts undisturbed.
  • Added prompt voting, contributor attribution, anonymous publishing, reporting, reusable prompts, and lightweight visual previews.
  • Added full build inspection plus opt-in comparison for two to four examples and comparison GIF export.
  • Added public nicknames and per-contribution anonymous attribution without exposing account, credential, or private storage details.
  • Added focused moderation, appeals, duplicate-prompt handling, official-prompt selection, and vote-integrity controls.

Free Gemini 3.7 Flash and Saved Builds

  • Added a limited-time hosted Gemini 3.7 Flash offer for signed-in Generate users, with no API key required.
  • Kept user-supplied Gemini and OpenRouter keys authoritative; normal key-based generation remains available after the hosted offer is exhausted.
  • Allowed failed hosted Gemini generations to retry without connecting an API key or counting the retry as another hosted generation.
  • Kept the hosted credential server-only and reused the existing encrypted, per-job secret lifecycle rather than storing a reusable key on user accounts.
  • Made signed-in Generate jobs durable so provider work continues after the browser tab closes.
  • Saved successful results privately under Account → Builds with live status, optimized previews, full viewers, source downloads, cancellation, retry, removal, and direct Gallery publishing.
  • Kept each selected model as an independent job so multi-model generations remain separately recoverable and manageable.
  • Kept saved results private until their owner explicitly publishes them to Gallery.
  • Updated the Privacy Policy to explain temporary credential handling and hosted generation through OpenRouter and the underlying model provider.

MineBench for iPhone

  • Launched MineBench on the App Store.
  • Added a native Metal renderer with per-corner ambient occlusion, texture mipmapping, visible-face culling, and orbit, pan, and pinch-to-zoom controls.
  • Brought blind Arena matchmaking, progressive build loading, and touch-friendly voting to iPhone.
  • Added multi-model Sandbox comparison, local build imports, prompt inspection, and cache-first Leaderboard access.
  • Added fixed-orbit GIF capture, H.264 video loops, and canonical build export through the native share sheet.
  • Added Dynamic Type, VoiceOver, Reduce Motion, automatic light and dark appearance, and an offline FAQ fallback.
  • Added public, cacheable web data routes for the native client while preserving server-computed rankings and blind-delivery contracts.

Accounts and Personal Rankings

  • Added public registration, email confirmation, sign-in, password recovery, and sign-out.
  • Added Google, GitHub, Discord, and X OAuth entry points with identity linking through Supabase Auth.
  • Kept anonymous Arena voting fully available. When a user signs in, eligible public votes from the current browser session are attached to that account automatically.
  • Added a private account home with saved builds and personal model rankings derived from the user's own eligible public votes.
  • Reused the public Leaderboard's model, record, and vote language so personal and global rankings remain easy to compare.
  • Added responsive account navigation and a clear signed-in state on desktop and mobile.
  • Kept private evaluation votes out of public accounts, personal rankings, and public rating inputs.

Arena and Delivery Performance

  • Unified public build normalization across Arena, Sandbox, Leaderboard, Gallery, and the native client.
  • Added checksum-addressed mesh caching for authoritative Sandbox and Leaderboard builds.
  • Prepares the next full Arena matchup in the background during reveal, then reuses in-flight or completed work when the user advances.
  • Prioritizes visible builds over queued background work while retaining already-started preparation.
  • Replaced hot-loop block-object allocation with packed buffers and precomputed face, material, texture, emissive, and tint data.
  • Added the identity-free MBF1 mesh-facts format for large builds, avoiding repeated neighborhood work while preserving blind delivery.

In Alpha measurements, visible-work prioritization reduced average time-to-vote from 868 ms to 619 ms and p95 from 3.15 s to 2.14 s. Native MBF1 delivery reduced 150k–300k-block mesh p50 from 754 ms to 166 ms, and 300k–1m-block mesh p50 from 1.26 s to 266 ms.

Leaderboard and Ratings

  • Replaced pairwise incremental public ratings with a global Bradley–Terry fit over eligible public A/B and tie outcomes.
  • Added rating uncertainty, 95% confidence intervals, confidence labels, and stable unique ranks to Leaderboard and model detail responses.
  • Preserved the hard boundary that prevents private evaluations from influencing public ratings, counters, coverage, snapshots, or eligibility.
  • Shares one ranking snapshot between the page and API, backed by Vercel Runtime Cache and in-process miss coalescing.
  • Includes ranking data in the initial page response and refreshes it in the background.
  • Added an accurate, responsive, reduced-motion-aware Leaderboard skeleton.
  • Reduced built Leaderboard HTML from 798 KB to 191 KB uncompressed in local production-build measurements; a local Runtime Cache hit returned the same ranking snapshot in roughly 0.24 ms versus roughly 999 ms for a cold computation.

Web Interface and Rendering

  • Replaced heavy card stacks with a quieter hairline surface system across Arena, Leaderboard, Sandbox, Gallery, FAQ, and account flows.
  • Balanced Arena desktop and mobile layouts so the comparison stage and primary vote controls stay within the working viewport more reliably.
  • Eliminated dark-mode first-paint flashing and added a stable viewer backdrop that keeps dark blocks legible.
  • Added smooth four-corner ambient occlusion across all six block faces without adding another GPU vertex attribute.
  • Rebuilt the texture atlas with edge-replicated gutters, trilinear mipmapping, and anisotropic filtering to reduce distant texture shimmer.
  • Added diagonal flipping under uneven corner lighting to avoid visible crease seams.
  • Reworked the footer, mobile safe-area spacing, document height behavior, and global feedback treatment.

Sandbox, Models, Discovery, and Support

  • Added arbitrary OpenRouter model IDs to Sandbox while keeping custom credentials and catalog-backed models isolated.
  • Added GLM 5.3 Flash with direct Z.AI and OpenRouter routing, maximum reasoning, and the model's documented output cap.
  • Improved Gemini reliability through OpenRouter by sanitizing unsupported schema features and using JSON mode for Gemini requests. (#128, #129)
  • Retries OpenRouter requests without strict parameter routing when a provider returns the supported fallback condition.
  • Added canonical hyphenated model URLs with permanent redirects from legacy internal keys.
  • Added dynamic model metadata, Dataset and ItemList structured data, canonical sitemap entries, and stricter crawler rules for private and internal routes.
  • Added a contact and feedback form with categories, optional reply email, validation, honeypot protection, rate limiting, and direct FAQ and issue-tracker paths.
  • Corrected the DeepSeek V4 Pro Preview label and GLM 5.3 benchmark cost metadata.

Private Evaluations

  • Added invite-only organization workspaces for configuring endpoints, generating private cohorts, inspecting builds, and reviewing results.
  • Interleaves private checkpoints against eligible public models in the blind Arena without exposing checkpoint identity or contaminating public rankings.
  • Added organization roles, exact-email invitations, retention and purge controls, immutable cohort records, deidentified exports, and monotonic evaluation closure.
  • Added selectable generation concurrency, corrected upload controls, and checkpoint editing without requiring delete-and-recreate workflows.
  • Added accurate failed-generation states, provider error extraction, build-level error callouts, and retry actions.
  • Allows draft checkpoints with partial or failed attempts to be safely reconfigured until they participate in active matches or votes.
  • Hardened private endpoints against private-network routing and unencrypted remote URLs.

Reliability, Privacy, and Observability

  • Restored blind binary artifact delivery and hardened gzip stream decompression.
  • Added end-to-end timing across build resolution, transfer, decode, atlas readiness, meshing, Three.js assembly, first render, reveal, and vote readiness.
  • Added privacy-safe metrics and structured delivery logs for Arena, Sandbox, Leaderboard, saved generations, worker availability, and queue health.
  • Kept prompts, model identity, build IDs, tokens, checksums, storage paths, and account identifiers out of client performance metrics.
  • Added structured Arena vote request logging for restricted operational abuse investigation without persisting raw IP addresses in account records.
  • Centralized safe error formatting so internal server and credential details do not leak into user-facing feedback.
  • Added terminal cleanup and worker-health protections for durable generations, including bounded provider requests and credential deletion.

Changelog

Full Changelog: 3.14.0...4.0.0

Contributors

Ammaar-Alam
Assets 2
Loading