Skip to content

Walk-in portal immersives - #265

Merged
ruofeidu merged 34 commits into
google:mainfrom
salmanmkc:demo/portal-walk-in-pr
May 9, 2026
Merged

Walk-in portal immersives#265
ruofeidu merged 34 commits into
google:mainfrom
salmanmkc:demo/portal-walk-in-pr

Conversation

@salmanmkc

@salmanmkc salmanmkc commented May 9, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #261. Same portal demo, but you can now walk into a portal and end up inside the world.

Each scene has a matching immersive sphere with true 3D raymarching, not the 2D parallax disc. Walk through the disc and the world swaps to the immersive shader; walk back out the other side and you're back in the lobby. Bidirectional, so the entry isn't blocked behind you.

A few other things picked up along the way:

  • per-eye stereo parallax on the disc (was flat before)
  • the lobby pauses non-active portals while you're inside one
  • forest immersive mirrors the cinematics from the disc (owl, comet, lightning, aurora) so going in doesn't feel like a different scene
  • cosmic ring is now a drifting rainbow, the rest still use their 2-color bands
  • lava: cone-shaped volcanoes with proper 3D bombs, dropped a 2D explosion artifact from the preview
  • cyberpunk: night park with neon billboards
  • underwater: warmer sand + a school of Dory tang & clownfish

Most of the perf cost is the immersive shader. The exit-snapshot RT is now 256² and runs every other frame, and the hot per-frame allocations in the gallery are gone. Still GPU-bound on M1 Chrome but should be fine on the Galaxy XR target.

npm run servehttp://localhost:8080/demos/portals/

Demo: https://youtu.be/TMl-VkBOv50

salmanmkc added 30 commits May 9, 2026 03:09
Walk through the cosmic portal disc to enter a full 360° space scene.
Exit portal renders the simulator room in real-time with dampened parallax.
Fade-to-white transition masks the scene switch on entry/exit.

- CosmicImmersive: inverted sphere with 3D noise (no UV seam), raymarched
  planets/sun/gas-giant, octahedral star mapping
- PortalGalleryScene: entry/exit detection, render-to-texture exit portal,
  fade sphere (speed 26), portal pickup fix (disabled fade sphere raycast)
PortalGalleryScene now tracks per-portal immersive instances via a
name→class map, so any scene with an Immersive class becomes walk-through.
Entry/exit/fade infrastructure is now portal-index driven instead of
hard-coded to the cosmic portal.

ForestImmersive: inverted sphere with twilight sky, ring of pine
silhouettes around the user (cylindrical azimuth space, no UV seam),
3D-positioned drifting fireflies, crescent moon, distant lightning,
ground fog.
Inverted sphere with depth-gradient water, surface caustics + sun disc
when looking up, volumetric sunbeam shafts, ocean floor with fbm haze,
drifting jellyfish at varied 3D positions, passing whale shark on a
slow arc, rising bubble columns, marine snow particles.
Inverted sphere with smoky red sky, drifting ash clouds, distant volcano
silhouettes with glowing crater + lava streaks down the slopes, ash plume
with internal underlight, lava bombs on parabolic trajectories with
trailing smoke, fast-rising ember columns in 3D, lava ground beneath the
user with hot crack patterns, occasional lightning flash in the plume.
Inverted sphere with smoggy purple sky + magenta horizon, drifting smog,
ring of skyscraper silhouettes around the user with random window grids
(warm/cyan/magenta lights, occasional flicker), antenna top glows,
hover-car streaks crossing the sky as fading neon trails, holographic
billboard with scanlines, rain streaks, wet-street reflections beneath
the user with puddle-modulated neon mirror.
Trees in the Forest immersive felt 2D because they used cylindrical
azimuth projection — moving past them did nothing. Now they're 18
fixed-position pines (two concentric rings, 4-13m out) raycast as
trunk cylinders + stacked canopy spheres, so walking actually moves
you between them.

Also:
- Drop camQuat from uViewRotation so view rays stay in portal-local
  space (previously double-applied head rotation made trees swing
  with your head).
- Lower ground to y=-1.6 so eye level matches portal entry — no more
  spawning with your head in the dirt.
- Forest portal disc now uses the same raycast trees and ground, so
  the preview matches what you see after walking through.
- Bump EXIT label panel + font size so it's actually readable.
Same root cause as Forest: uViewRotation was portalQuatInv * camQuat,
which double-applied the camera rotation to vWorldDir (already in
world space) and made the whole ocean swing with your head when you
walked around. Just use portalQuatInv to convert world → portal-local.

Underwater's jellyfish/whale/bubbles are already 3D raycast against
fixed positions, and the user is meant to float mid-water (ground at
y=-6, surface at y=+8), so no other changes needed.
Disc preview was 2D parallax (small fish school, bell silhouettes)
while the immersive has raycast jellyfish, a passing whale shark,
rising bubble columns, and a sandy floor. So you'd see one thing
through the portal and a different thing once you walked in.

Move the same raycast helpers into the scene so the disc renders
the actual scene from your camera's perspective.
Same root cause as Forest and Underwater: uViewRotation was
portalQuatInv * camQuat, which double-applied the camera rotation
to vWorldDir (already in world space) and made the whole scene
swing with your head — most noticeable when walking toward the
exit portal, where the universe felt like it was sliding away.
Just use portalQuatInv to convert world → portal-local.
… school

- Clownfish school: full primitive raycast (body + tail peduncle + dorsal +
  pectorals + forked caudal + eye + mouth + bands), slow drift, gentle wiggle.
- Blue tang (Dory) school: same multi-primitive treatment with blue body,
  yellow tail and palette mark.
- Coral reef: vertical trunks + 6 tilted radiating branches per cluster,
  bumpy fbm surface texture for organic look.
- Mirrored to UnderwaterScene disc preview.
- Fix uViewRotation to use portalQuatInv only (was doubling cam rotation).
- Replace cylindrical volcano silhouettes with stacked-ellipsoid raycast
  volcanoes that parallax correctly as the user walks.
- Add foreground lava rock field (8 ellipsoid rocks with crack glow) for
  near-field depth cue.
- Mirror the same 3D volcanoes and rocks into the LavaScene disc preview.
…ards

- Fix uViewRotation to use portalQuatInv only (matches Forest/Underwater/Lava).
- Replace cylindrical building masks with a 3D ring of 14 raycast box
  skyscrapers at fixed world positions; varying width, depth, height; lit
  window grid with flicker; antenna glow on top face. Walks parallax correctly.
- Add 3 raycast neon billboard slabs (magenta / cyan / amber) with scanlines.
- Mirror 3D city ring into the CyberpunkScene disc preview, replacing the
  layered 2D skyline.
…spurts

Replace the stacked-ellipsoid volcanoes with a true ray-cone shape so the
silhouette reads as a volcano. Crater is a cracked basalt crust with
glowing molten fissures; slopes have layered strata, ashy upper gradient,
and a few discrete lava channels trickling down. Cones are flatter and
wider so they look like stratovolcanoes rather than steep spikes.

Add an animated ash plume column anchored to each apex world position,
plus 5 staggered lava bombs arcing out of each crater on parabolic
trajectories. Mirrored between immersive and disc preview so both views
match.

Drop the ground plane in the immersive to y=-1.6 so the user spawns at
standing height instead of half-buried in lava.
Walk through any gallery portal disc from either face to enter, and
walk back through from either face to exit. A 600ms transition
cooldown prevents the immediate reverse-cross from snapping us back.

Drive-by fixes while in the same file:
- Restore disc renderOrder to 1 (matches Portal._buildDisc) on exit
  so the disc no longer sorts behind other portals' rings/halos.
- Stop running _captureExitSnapshot every frame in gallery mode; the
  snapshot is only visible while inside an immersive, so the extra
  per-frame render pass was wasted.
Street life additions (mirrored across immersive + disc preview):
- 8 ground cars on two opposing circular lanes with white headlight
  / red taillight faces.
- 4 hovercars looping above with cyan headlights, magenta taillights,
  cyan underglow.
- 12 pedestrian silhouettes drifting along the sidewalk.
- 4 traffic lights cycling red/yellow/green.
- Lit shop fronts (warm yellow / pink / cyan windows + sign band) on
  the lower 10% of every building.

Neon night park directly in front of the user (azimuth -π/2, radius
10m, inside the street ring): holo-grass slab, three holo-trees with
slowly cycling magenta↔cyan canopies, a colour-drifting holo-statue,
and two warm lamp posts. Cars, hovercars, and pedestrians skip the
park wedge so nothing drives through it. Buildings 10 and 11 are
omitted from the city ring so the park sits in an open plaza.

Wet street reflection plane was at y=0 while the buildings, cars, and
pedestrians sit on y=-1.6, leaving the user floating above the visible
street level. Aligned the reflection plane to y=-1.6 so spawn height
matches the rest of the city.
When the gallery now lets you walk through a portal disc from either
face, the immersive needs to know which side you came from. Each
immersive's show() takes a fromSide argument; back-entry rotates the
entry matrix 180° around Y so the user spawns looking into the scene
instead of at the wall behind it.
Sand reads as golden tan now instead of murky teal. The water surface
at y=8 has a slow sin-based UV wobble so caustics drift instead of
sitting flat. Far geometry fades into a deeper teal with view
distance, giving the world some depth past the first reef.
Each eye now gets its own uCamLocal via onBeforeRender, so XR
ArrayCamera sub-cameras shoot independent rays through the disc.
Before this both eyes used the mono camera position, which made the
scene inside the portal look flat — no real depth, just a window
texture. Now distant objects in cosmos / underwater / forest etc.
hold their parallax as you move your head.
ashPlume() and lavaBombs() were leftover from before the 3D plume
loop and per-volcano spurt loop replaced them — both were still being
called and doing redundant work. Also moved v2 in the gallery shader
to match the immersive position so the preview lines up with what
you walk into.
Trees lean and vary in canopy size from a per-tree hash so the
silhouette looks organic instead of cloned. Slow sin sway makes the
canopies breathe. Three soft moonbeams slant through gaps in the
trees. A small winding creek scrolls along the forest floor. Also
fixed the page title which was still saying Cosmic Portal.
A huge ringed gas giant now hangs on the horizon, just the limb
showing so it reads as something massive nearby instead of a planet
floating in front. Nebula gets composited from 3 depths so it has
volume instead of looking like wallpaper. The supernova, meteor
shower, rocket flyby and wormhole that were only firing in the
gallery preview now also play inside the immersive.
The huge gas-giant limb and the existing small Saturn were both on
the right side, making it look like one planet was clipping through
the other. Limb is now on the left so they sit on opposite horizons
and the composition reads as two distinct worlds.
The two ringed planets were both warm Jupiter palettes which made
them feel like the same planet at different scales. The horizon limb
is now blue-white with a pale storm and an icy ring; the small
Saturn keeps its warm tan and red spot. Cleaner read at a glance.
The flat screen-space arc of bombs was sitting on top of the actual
3D spurt loop and looked tacked on. Removing it means the preview
just shows the per-volcano spurts coming out of the craters.
Three depth layers of fbm noise was the heaviest hit in the
fragment shader and the far layer barely added anything visually
behind the closer two. Cuts noise samples per pixel by roughly a
third, near + mid still give a sense of volume.
This reverts commit 4c456b6170c1c5f55388cb53aaa2fc3d67516348.
@salmanmkc

Copy link
Copy Markdown
Contributor Author

Yay thanks for the quick review!

Comment thread demos/portals/PortalGalleryScene.js Outdated
export class PortalGalleryScene extends xb.Script {
portals = [];
labels = [];
immersives = []; // index → ImmersiveInstance | null

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably use -> and avoid unicode for comments?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, thanks!

Comment thread demos/portals/PortalGalleryScene.js Outdated

// Immersive worlds (one per scene that supports walk-in).
for (let i = 0; i < SCENES.length; i++) {
const Cls = IMMERSIVE_BY_NAME[SCENES[i].name];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: more descriptive name for Cls

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also good point, thanks!

Comment thread demos/portals/scenes/LavaScene.js Outdated
vec3 apexW = vec3(2.0, 4.4, -16.0);
// Sample 14 horizontal slices from the crater upward; each slice tests
// distance from the vertical column through the apex.
for (int i = 0; i < 14; i++) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: avoid magic number, use kNumSlices

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks again, updated!

- replace unicode arrow with -> in immersives comment
- rename Cls to ImmersiveClass for clarity (matches ComponentClass / ModelClass convention used elsewhere)
- pull the 14-slice loop bound out into a named const kNumSlices in the lava ash-plume shader
@salmanmkc

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback, just addressed it!
Should i be putting this now in a sample you think?

@ruofeidu

ruofeidu commented May 9, 2026

Copy link
Copy Markdown
Collaborator

I think you can list it as an advanced demo now. Thanks Salman!

@ruofeidu
ruofeidu merged commit b3d2b0a into google:main May 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants