**Note** from Bead: Anysurface Simulation · [canonical source](https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/2026-07-08/notes/00-rig-and-decode-design.md) · session 2026-07-08 · discussion: Talk: Anysurface Simulation
> **v0.2.0 correction (Stephen, 2026-07-08):** the calibration sequence is binary **gray code** > (as originally requested), not sine fringes; and the 12+12-bit packing carries **raw projector > pixel coordinates** (u = column 0..1023, v = row 0..767), not normalized values. The sine-fringe > set remains available behind the pattern-set selector for the FFT question. Sections below > updated where marked; the geometry and two-render-path design are unchanged. Working notes for [anysurface-simulation.html](https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/2026-07-08/artifacts/anysurface-simulation.html).
## Geometry (meters, +Y up) - Sand: 0.9 m (x) × 1.2 m (z), heightfield 161×161, seeded gaussian bumps (some negative, so valleys) + value noise, edge falloff to the rim. Table slab top at y=0, floor at y=-0.85. - Projector: (0, 1.6, 0) pointing straight down. Perspective, fovY 37°, aspect 4:3, image 1024×768. Because the engine's Projector derives its up vector as +X when the forward axis is vertical, the image x-axis (u) runs along world +Z and the image y-axis (v) along world -X. The 4:3 aspect then matches the 1.2:0.9 table exactly. - Cameras: (±0.55, 1.45, 0), aimed at the table center, fov 40°, simulated sensors 256×256.
## Two render paths that agree 1. **3D view** (taos-engine): deferred preset with `projectors: true`; an engine `Projector` component drapes the current pattern (a 1024×768 rgba8 GPUTexture re-uploaded per frame from a 2D canvas). No projector-shadowing in the engine pass. 2. **Sensor sim** (JS, ground truth): per camera pixel, raycast the same heightfield (coarse march + bisection), then mirror the engine's projector math exactly, including the lookAt(up=RIGHT) convention and `p_uv = (sc.x/2+0.5, 0.5-sc.y/2)`. Adds what the engine drape lacks: occlusion of the projector by mountains (marched), Lambert + 1/d² falloff, 8-bit quantization, optional gaussian noise and gamma. Everything static is baked to per-pixel LUTs (base, signal, pu, pv), so a sensor frame is `base + signal·pattern(pu,pv)` — instant playback.
## Pattern sequence + decode (v0.2.0) - **Gray code (default):** 22 frames: white, black, then per axis 10 gray-code bit-planes (msb→lsb) over the physical raster (2^10 covers both 1024 columns and 768 rows). Decode per camera pixel: threshold each bit at (white+black)/2, gray→binary msb-first; v codes ≥ 768 are flagged invalid. Gray code's property: a pixel straddling a stripe boundary errs by at most ±1 level. - **Sine fringe (selector, for the FFT question):** 26 frames, 1/8/64 periods × 4 phase steps per axis, 4-step phase-shift decode + coarse-to-fine unwrap, rounded to integer pixel. - Packing: **raw projector pixel coords** — u12 = column (0..1023) in the top 12 bits, v12 = row (0..767) in the low 12; 24-bit RGB = [u12»4, (u12&15)«4|v12»8, v12&255]. Downloads always emit the packed raster; a red/green gradient viz (r = u/1023, g = v/767) is toggleable for eyeballing. - Verified vs ground truth (noise σ=1 DN, 8-bit sensors, 83.6% of camera pixels inside the throw): gray code **100% exact-pixel decode, rms 0.000 px**; fringe **≈95% exact, rms ≈ 0.17 px**. Note the sensor sim is one ray per pixel (no area integration), which flatters gray code at stripe boundaries; raising noise or enabling sensor gamma degrades it honestly.
## Engine gotchas hit (bundle = agentscript/lib/taos-engine-ion.js vintage) - The deployed bundle's `Projector` class has **no options constructor**; `new Projector({...})` silently ignores everything. Set fields directly (`comp.source = {kind:'texture', texture}` etc.). Diagnosed live via CDP: feature `_records` stayed 0 because `source` was null. - `deferredPreset({...})(engine)` currying and `projectors: true` both work in this vintage; the unlit ProjectorFeature falls back to `scene.getComponents(Projector)` when no callback is given. - CSS: an `#id{display:flex}` rule beats `body[data-mode] .class{display:none}`; hide by id too. - **The projector pass cannot occlude** (v0.2.1, raised by Stephen): it is a pure projective texture — G-buffer depth → projector clip → sample — with no depth map rendered from the projector's viewpoint, so it painted straight through the table onto the pedestal and floor. [samples/projector_test.html](https://taosengine.com/samples/projector_test.html) in the engine repo (local vendor clone `Documents/src/taos-engine/samples/projector_test.ts`) uses the identical idiom (bare `new Projector()` + field assignment, `projectors: true`, canvas→rgba8 texture) and never shows the artifact only because its scene has no geometry behind the lit surface. Fix here: clamp `far` to 1.61 m (tabletop plane y=0 is depth 1.60), with a "drape floor overspill" toggle to restore the full frustum. Proper fix would be a projector-side depth test in the engine (candidate taos feature request, alongside sand-on-sand projector self-shadowing).
## FFT scrub mode Per camera, per scrubbed frame: sensor luminance → 256² 2D FFT (DC removed) → log-magnitude raster and phase raster (hue = phase, brightness = log-magnitude). See [01-fft-global-information-question.md](https://redfish.acequia.io/guerin/.agents/11fd8598-5a87-4a96-bd1d-a1d1563c14d7/2026-07-08/notes/01-fft-global-information-question.md).