**Note** from Bead: Snow Globe Harvard Spheres · [canonical source](https://redfish.acequia.io/guerin/.agents/6a8f0a99-c255-4a5c-9c4a-9097d0ee9cd8/2026-06-11/notes/01-snow-globe-viewer-plan.md) · session 2026-06-11 · discussion: Talk: Snow Globe Harvard Spheres
A 3D model rendered so it appears to sit **inside** the physical sphere, like a miniature in a snow globe — instead of the current content model where imagery is a skin **on** the sphere's surface. Three stages of view-dependence: 1. **Sweet-spot rendering** — fixed virtual eye; the illusion is exactly correct from one place in the room, gracefully approximate elsewhere. 2. **Motion parallax** — detect a viewer's eyes and move the virtual eye with them. 3. **Multi-viewer segmentation** — when people stand around the sphere separated by more than a threshold sphere angle, partition the globe's faces into angular segments, each rendered for the viewpoint at the center of its viewer cluster.
## 1. The core geometry Work in the **sphere frame**: origin at globe center, radius `R` (the physical display surface), virtual scene fitted inside radius `r ≤ R` (model + pedestal + snow particles). For an eye at `E` (outside the globe) and a surface point `P`, the snow-globe illusion requires the pixel displayed *at* `P` to show the virtual interior's radiance **along the ray E→P**. Equivalently: the sphere surface is a window; each viewer-visible texel is the scene as seen from `E` through that texel. **Implementation is a clean two-pass GPU pipeline:** 1. **Pass 1 — CubeCamera at E.** Render the interior scene (model, snow, pedestal) into a cubemap whose center is `E`. A cubemap lookup is purely directional from its center, so this captures exactly the field of radiance leaving the scene toward `E`. 2. **Pass 2 — equirect shader.** For each output texel `(λ, φ)` of the 2:1 equirectangular frame: compute `P = R·(cos φ cos λ, sin φ, cos φ sin λ)`, then sample `cubemap(normalize(P − E))`. That mapping is *exact* (not an approximation) for every texel on the hemisphere facing `E` — which is all that viewer can physically see. Texels on the far side are invisible to that viewer and are free real estate: by default render them for the **antipodal eye** (so the sphere looks right from both sides with two implicit sweet spots), which is also the degenerate K=2 case of stage 3. Anything three.js loads works as the interior scene: GLB models, point clouds, procedural terrain. The snow is a cheap particle system constrained to the globe interior (Perlin drift, settle on floor, optional "shake" burst triggered from the controller). > **Revision (same day):** the CubeCamera pipeline above is demoted to *v0 scaffold / WebGL2 fallback*. The architecture of record is **per-texel chord rays on WebGPU** with Gaussian-splat and SDF evaluators — each sphere texel is a window pixel whose ray traverses a chord of the globe; view dependence is the transmissive dual of specular. See [04-chord-ray-rendering-webgpu-3dgs.md](04-chord-ray-rendering-webgpu-3dgs.md). Multi-viewer segmentation (stage 3) becomes nearly free under that formulation.
## 2. Architecture — ride the existing CMS untouched The survey ([00-sphere-namespace-survey.md](00-sphere-namespace-survey.md)) shows `type:"iframe"` playlist items get a full 200%-width layer in the existing viewer. So: - **`snow-globe.html`** — a single-file three.js page that renders the equirect output full-frame. Configured by query string: `?model=<glb-url>&scale=…&eyeDist=…&eyeAz=…&snow=…` (or `?config=<json-url>` for richer scenes). - **Playlist item** — `{ "type": "iframe", "url": ".../snow-globe.html?model=…", "title": "Snow Globe — <model>", "duration": 120 }`. Enters through the normal propose→approve flow. **Zero changes to sphere-viewer, controller, or admin.** - **Realtime** — `snow-globe.html` joins the same acequia group (`sphere-control`) itself and maps the existing `xRotation` state to a **model turntable** (spinning a snow globe spins what's inside; the sweet-spot eye stays put). Controllers work day one with no modification. - Tracking state (stages 2–3) rides the same group: a tracker page publishes `viewers: [{id, az, el, dist, t}]` patches; `snow-globe.html` consumes them. Staging: prototype lives in this bead's `artifacts/` (or a non-deploying `dev/` workshop per the epic/step methodology); deployment to `guerin/apps/sphere/` is Stephen's move, per standing namespace discipline.
## 3. Stage 1 — sweet spot - Default eye: equatorial elevation, distance ≈ 2–4× R (typical standing viewer at the Cabot installation), azimuth chosen for the dominant approach direction. - `eyeAz` is a config/state value, so the sweet spot can be *aimed* from a controller before tracking exists. - **Registration**: displayed `λ=0` vs. physical room azimuth must be calibrated once — ship a calibration mode that renders a labeled meridian grid; an operator notes which displayed meridian faces the room's reference direction; store the offset in a config sidecar next to the item JSON. - Far hemisphere: antipodal-eye render (default), or a slow ambient (starfield / falling snow only).
## 4. Stage 2 — eye detection and motion parallax - **Sensor**: one webcam near the sphere (later: a ring of 2–3 for full 360°). Browser-side **MediaPipe Face Landmarker** on a small tracker page (`snow-globe-tracker.html`) running on the projection PC or any spare device — on-device inference, no frames stored or transmitted (public-space privacy posture; only derived angles leave the page). - **Estimate**: face center → bearing in camera frame → calibrated camera→sphere transform → viewer azimuth/elevation; interocular pixel distance → rough range. Eye position, not gaze — parallax needs the eye location only. - **Publish**: throttled (~10–15 Hz) group-state patches `{viewers:[…]}`; smooth in the consumer with a one-euro filter; `E` follows the smoothed viewer. - Single tracked viewer = stage 1 with a moving sweet spot. Lost tracking → ease `E` back to the default sweet spot over ~2 s.
## 5. Stage 3 — multi-viewer segmented faces When viewers are distributed around the sphere: - **Cluster**: merge viewers within threshold sphere angle `Θ` (start ~45°); cluster eye = circular mean of members (weighted toward closer viewers). Yields K cluster eyes `E₁…E_K` (cap K at 3–4). - **Render**: one CubeCamera pass per cluster per frame → K cubemaps. - **Composite (the segmentation)**: in the equirect shader, each texel `P` belongs to the cluster whose center azimuth is angularly nearest — an **angular Voronoi partition** of the globe's surface. Sample that cluster's cubemap with `normalize(P − E_k)`. Each person standing in front of "their" segment sees a correct snow globe. - **Seams**: fall mid-between clusters, i.e. where neither viewer looks head-on. Blend the two nearest clusters across a ~10° band. Add **hysteresis** on cluster membership and lerped cluster centers so segments never pop when someone shifts weight. - **Degenerate cases**: K=1 → stage 2; everyone within Θ → one cluster (correct); opposite sides → the antipodal default, now tracked.
## 6. Performance budget - K cubemap renders × 6 faces/frame. At K≤2, 512–1024² faces are comfortable on any modern GPU; for K=3–4 drop face resolution or round-robin cubemap refresh (the nearest/most-active cluster updates every frame, others at half rate). Snow particles and a single GLB are light; the equirect composite is one full-screen pass. - Unknown: the projection PC's GPU (current viewer is DOM/video only — never exercised WebGL). **First on-site test should include a WebGL2 capability + fps probe.**
## 7. Open questions (need facts from the room, → `uploads/`) 1. Projection-system input spec: exact resolution, full equirect vs. visible portion, polar warp behavior (plan.md handled >±85° with static overlays for maps — is that a hardware limit?). 2. Sphere mounting height and typical viewer distance → default `eyeDist`, `eyeEl`. 3. Tracking camera: hardware available? mounting ring? Harvard privacy/IRB posture for a camera in a public space (even with on-device inference). 4. Room-azimuth registration offset (one-time calibration). 5. Projection PC GPU capability. 6. First model: lunar surface? (notes.md lead), Harvard campus massing, the bonobo habitat from `harvardviz.live/models/bonobo/`?
## 8. Epics (per the epic/step methodology) - **E1 — Sweet-spot snow globe**: scene prototype (GLB + snow + pedestal) → cubemap→equirect pass → sphere-frame config + `xRotation` turntable via group → ship as one iframe item → on-sphere verification (incl. WebGL probe). - **E2 — Calibration**: meridian-grid mode, registration offset sidecar, eye-aiming from controller. - **E3 — Single-viewer parallax**: tracker page (MediaPipe), `viewers` group state, smoothing, dropout easing. - **E4 — Segmented faces**: clustering with threshold + hysteresis, angular-Voronoi per-texel eye selection with seam blending, perf tuning. (Near-free under the chord-ray renderer — see E6.) - **E5 — Companion controller protocol**: media declares what runs on the controller screen while it plays; `companion` field on item JSON, item-scoped group state. Spec: [02-companion-controller-protocol.md](02-companion-controller-protocol.md). The snow globe is the pilot companion (shake / aim-eye / turntable). - **E6 — Chord-ray renderer (WebGPU + 3DGS)**: shared chord ray-gen module, splat evaluator (ray-traced Gaussian splatting), SDF evaluator (snow, pedestal, calibration grid), capture-to-globe content pipeline. Supersedes E1's cubemap evaluator. Spec: [04-chord-ray-rendering-webgpu-3dgs.md](04-chord-ray-rendering-webgpu-3dgs.md). E1 is independently shippable and demos the whole idea; the others each add capability without touching the CMS. Known defects in the existing controller/admin flows that any new item tooling must not inherit are catalogued in [03-controller-flow-audit.md](03-controller-flow-audit.md).