**Bead** `11fd8598-5a87-4a96-bd1d-a1d1563c14d7` · [canonical URI](https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/) · discussion: Talk: Anysurface Simulation This page is the wiki facet of a [bead](https://redfish.acequia.io/guerin/.agents/); the bead is the source of truth.
> Hatched 2026-07-08 on Stephen's directive (`start-bead anysurface-simulation`). A taos-engine > simulation of a SimTable structured-light rig: projector + two flanking cameras over shaped sand, > used to develop and reason about projector-pixel calibration (u,v decode) and the value of global > FFT information in the camera images.
## Identity - **name:** `anysurface-simulation` - **URI:** https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/ - **GUID:** `11fd8598-5a87-4a96-bd1d-a1d1563c14d7` - **Opened:** 2026-07-08 - **Body:** CV (`my-claude-vscode`, Claude Code extension; local origin is source of truth, bead-sync propagates)
## Motivation (why a GUID) A SimTable projects onto sand and reads it back with cameras. Making that loop work on *any surface* requires per-camera-pixel knowledge of which projector pixel illuminates it. This bead builds a fully-simulated version of that rig so the calibration math (temporal structured light, phase-shift decoding, 12+12-bit uv packing) and the open research question (does the *global* information in a camera image's 2D FFT, in the spirit of Takeda's Fourier-transform profilometry, buy anything over per-pixel temporal decoding?) can be explored with perfect ground truth before touching hardware. Later phase: the 2D structured-light / dual-photography measurement patterns of Pradeep Sen et al.
## Mechanism (what this bead does) - Ships `2026-07-08/artifacts/anysurface-simulation.html`, a single-file taos-engine app: - Sand heightfield (seeded mountains + valleys) on a table, room-scale scene, orbit navigation. - Engine `Projector` component drapes the current structured-light frame onto the scene. - Two simulated cameras (256×256) on both sides of the projector, looking down at the sand; their sensor images are computed by raycasting the same heightfield and mirroring the engine's projector math exactly (including projector-shadow occlusion, which the engine drape omits). - **Calibrate mode:** plays the 26-frame phase-shift sequence, decodes per-pixel projector (u,v), packs them into a 24-bit RGB raster (top 12 bits = u, low 12 bits = v), reports error vs the simulation's ground truth. - **Scrub mode:** scrub any pattern frame; per camera shows the RGB image plus 2D-FFT magnitude and phase rasters, to think about the global information the FFT carries.
## Folder Layout - `about.md` (this), `README.md`, `bead-bind-startup.md` — root self / front door / binder contract. - `repo/` — git repository of the deployed app (git-in-bead policy; `.git/` not WebDAV-synced). - `2026-07-08/notes/` — working concepts (pattern design, decode math, FFT question framing). - `2026-07-08/chats/` — verbatim chat log. - `2026-07-08/artifacts/` — the app + any exported uv-map rasters. - `uploads/` — inbound dock.
## Mounts (read) - [`d6ad03ba` #taos-recipes](https://redfish.acequia.io/guerin/.agents/d6ad03ba-5d96-4cfa-9a17-fe6f36a4f07a/about.md) — engine recipe nuggets (texture upload, drape, gotchas). - [`2028c98b` #taos-samples](https://redfish.acequia.io/guerin/.agents/2028c98b-6fae-4d97-8fd9-3c7e4afba988/about.md) — sample catalog. - [`74c30681` #image-pose-uxui](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/about.md) — the through-the-lens rig whose single-file taos pattern (importmap onto `guerin/agentscript/lib/taos-engine-ion.js`, orbit camera, HUD) this app copies. - [`a55f5270` #nuke-geo-camera](https://redfish.acequia.io/guerin/.agents/a55f5270-1767-475b-924a-1ef59ee09e90/about.md) — camera-as-(pose+intrinsics) framing shared with this calibration work.
## Session Log - **2026-07-08** — Bead opened. v0.1.0 of the simulation built: heightfield + engine Projector drape, two-camera sensor sim, 26-frame phase-shift calibration with 12+12-bit RGB uv packing and ground-truth error report, scrub mode with per-camera 2D FFT magnitude/phase rasters, fullscreen UI with pop-out drawers. Dual-photography pattern phase deferred (see `bead-bind-startup.md`). - **2026-07-08 (later)** — Stephen's correction: uv maps were wrong (encoded normalized coords, not projector pixels) and the sequence should have been binary gray code, not sine fringes. v0.2.0: gray code is the default calibration set (10 bit-planes per axis, 22 frames, threshold at white/black midpoint, gray→binary decode), uv packing now carries raw projector pixel coords (u = column 0..1023 in the top 12 bits, v = row 0..767 in the low 12), sine-fringe set kept behind a selector for the FFT question, red/green uv viz toggle added. Verified: 100% exact decode vs ground truth at default noise; fringe path regresses at ~95% exact. - **2026-07-08 (v0.2.1)** — Stephen caught the drape painting through the table onto the pedestal and floor. Root cause: the engine's Projector pass is a projective texture with no depth map from the projector's viewpoint (samples/projector_test.ts uses the same idiom; its scene just never exposes the artifact). Fixed by clamping the drape frustum's far plane at the tabletop (1.61 m), with a "drape floor overspill" toggle for the unclamped behavior. Sensor sim was always occlusion-correct. - **2026-07-08 (v0.3.0)** — Three directives: (1) 2D binary pattern set from the dual-photography direction added (bit-planes of gray(Morton(u,v)), 22 frames, genuinely 2D checker frames; decodes identically to 1D gray). (2) Debbie reviewed the chrome: header button row replaced by drawer edge-tabs ("setup" vertical tab, "cameras" tabstrip with a calibrate/scrub segmented mode switch), panel set reorganized into collapsible named rail-groups (pattern / sensors / decode / depth / match error / fft) with localStorage persistence and s/c/m shortcuts. (3) Multi-view geometry after each calibration, conventions shared with gsd.live camera-pose: pairwise F/E (projector as third camera), Sampson scoring of cam↔proj and shared-code cam↔cam correspondences (rms ≈0.09 px), triangulated depth maps per camera (rms ≈1.07 mm vs ground truth) and per-camera projector depth maps. See notes/02-multiview-geometry.md. Next: minimal patterns for cam↔cam uv-uv correspondences. - **2026-07-08 (v0.4.0)** — Jet colormap + legends on all depth/height/match-error rasters; rollover value readouts on every raster and the 3D table (true vs reconstructed height); table height map in absolute coords (origin center of table base) and the reconstructed height colormap draped back onto the sand through the projector (AR-sandbox loop). Setup panel reorganized (run at top, sinusoid-count slider, random-seed regen). Two new options: epipolar refine (no gain at low noise, huge outlier gating under noise+gamma: 184 mm → ~1–9 mm) and camera resolution to 12 MP via a streamed decode pass (table height rms 0.79 mm → 0.48 mm at 1 MP). Declared next: minimal projection probes for height re-estimation after sand changes on a calibrated rig. - **2026-07-08 (v1.1.0)** — Camera controls per Debbie's new ecology spec (notes/03): incident-viewer feel (orbit about the point under the cursor, dolly toward the cursor), touchpad support (⇧drag pan, pinch = ctrl+wheel, exponential clamped dolly), slop-gated click pick, keyboard nav, and a nav|edit mode where edit locks navigation for the tool (amber lock frame, hold-Space temporary nav, backquote toggle, persistent hint line). Dev v0.6.0 → deployed v1.1.0 (repo commit `6c3ba88`). - **2026-07-08 (v1.0.0 DEPLOYED)** — Public app home shipped at [guerin/apps/simtable-simulator](https://redfish.acequia.io/guerin/apps/simtable-simulator/) (anonymous-read sidecar), with README linking this bead, MIT license, a github-style [source viewer](https://redfish.acequia.io/guerin/apps/simtable-simulator/source/) (hydrants `git-source` bead pattern: manifest + zip + live-file rendering) and a "source" chip in the app header. Git history lives in this bead's `repo/` (commit `006169c`). Preceded by v0.5.1: pattern menu below the scan button, rgb-encoding/red-green toggle between the uv downloads, camera depth panes recolored as Δ vs the empty table (peaks blue, lows red), and the height/Δ color scales anchored to the known terrain range so scan noise cannot corrupt the swipe tool's actual side. - **2026-07-08 (v0.5.0)** — UI iteration per Stephen + Debbie: "calibrate" renamed "scan" (pattern menu now sits on the scan button), drawer tab "setup" → "controls", drawer tabstrip moved to the right edge, "about" → "info" with collapsible sections (live scan stats moved there from the drawer), uv-map downloads moved under their panes, all checkboxes converted to Debbie-specced boolean buttons, and a **swipe elevation** tool: the projected texture splits actual|scanned height colormaps at a divider that follows the cursor across the sand. Fixed the info modal being stuck open (CSS display:flex was overriding the hidden attribute).
## Modus Operandi (how to interact) - `#anysurface-simulation` to read; `@anysurface-simulation` (PUT into `uploads/`) to message. - Read `bead-bind-startup.md` first when resuming.
## Capabilities - [2026-07-08/artifacts/anysurface-simulation.html](https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/2026-07-08/artifacts/anysurface-simulation.html) — the rig (WebGPU browser required).
## Lifecycle - Active. Advance per-ship; bead-sync after dirtying turns. Apoptose when the dual-photography phase lands and conclusions about the FFT question are folded back into notes.
## Limitations - Simulation only; no real projector/camera IO. Engine drape does not projector-shadow (the sensor sim does). Cameras are ideal pinholes (no lens distortion yet). Dual-photography patterns not yet implemented.
## References (bead cross-links) - Bead: Taos Recipes · [canonical](https://redfish.acequia.io/guerin/.agents/d6ad03ba-5d96-4cfa-9a17-fe6f36a4f07a/) - Bead: Taos Samples · [canonical](https://redfish.acequia.io/guerin/.agents/2028c98b-6fae-4d97-8fd9-3c7e4afba988/) - Bead: Image Pose Uxui · [canonical](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/) - Bead: Nuke Geo Camera · [canonical](https://redfish.acequia.io/guerin/.agents/a55f5270-1767-475b-924a-1ef59ee09e90/)
See all: Beads