**Note** from Bead: Image Pose Uxui · [canonical source](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/2026-07-03/notes/02-fft-coherence-deep-think.md) · session 2026-07-03 · discussion: Talk: Image Pose Uxui
The frame Stephen gave: *look at the FFT of the saved views and the "FFT light field" around where those images will be calibrated; visualize the field and gradients that might drive camera-pose transformations as the camera finds coherence in the scene — and coherence with each other.* This note is the conceptual working-through; the rendered sketches are at [fft-coherence-sketches.html](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/2026-07-03/artifacts/sketches/fft-coherence-sketches.html).
## 1. The structural insight: pose error factors into two very different parts For a camera viewing a mostly-distant scene, a small pose error decomposes: | pose components | image-space effect | FFT status | |---|---|---| | **heading, pitch** (pan/tilt) | ≈ global image **translation** | exactly what **phase correlation** solves, globally, in one shot | | **roll** | global image **rotation** about the principal point | **Fourier–Mellin**: rotation of the magnitude spectrum → translation in log-polar | | **fov** (zoom) | global image **scale** | Fourier–Mellin: scale of the spectrum → the other log-polar axis | | **position x,y,z** | **parallax** — per-pixel, depth-dependent flow | *not* a global spectral transform; FFT breaks here (and that break is itself a signal) | So the 4-DOF subspace (heading, pitch, roll, fov) is *exactly* the domain where Fourier methods are classical, correspondence-free, and globally convergent — and the 3-DOF position subspace is where they hand off to the campanile pins / LM solver ([solver.js](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/repo/solver.js)). This factorization is the architecture. It also matches the calibrator's UX ladder: snapshot-view gets position roughly right by human navigation; FFT can then *lock rotation+fov automatically*; pins refine position. A refinement worth carrying: pan/tilt is only *approximately* translation on a planar image (it's a homography for wide FOV). Reprojecting both photo and render to an **equirectangular patch about the current view direction** makes pan/tilt an *exact* translation and roll an exact shift too — i.e. the natural home of this method is **the panosphere**, the same surface `#astronomical-pano` (9943a c1c) uses for the sky. One sphere, two hemispheres, and now the FFT alignment lives on it natively.
## 2. The landscape is not a metaphor — one inverse FFT evaluates ALL poses in the slice The thing that makes this more than "use FFT as a score": for two images f (photo) and g (render at current pose), the normalized cross-power spectrum's inverse FFT — the **phase correlation surface** — evaluates the coherence of *every* translational offset simultaneously, in O(N² log N). The pan/tilt slice of Stephen's "FFT light field around the pose" is not sampled pose-by-pose; **it comes out whole, as a terrain**, from one FFT round trip: - the **peak location** = the pan/tilt correction (a Newton jump, not a gradient step); - the **peak sharpness** (peak-to-sidelobe ratio, PSR) = a native **confidence** measure ("locked / drifting / lost" — the honest headline the Tufte panel wants); - the **whole surface** = the local coherence landscape, directly renderable as the visualization Stephen asked for. Phase correlation *whitens* the spectrum (keeps phase, discards magnitude), which is why it's robust to the photo-vs-render appearance gap (lighting, season, exposure) — the exact gap we face matching real photos against Google 3D tiles renders.
## 3. Frequency bands sculpt the basin: annealing as a first-class visual Band-limiting the cross-power spectrum before the inverse FFT reshapes the landscape: - **low band** → a wide, smooth bowl — huge basin of attraction, low precision; - **high band** → a needle peak — precise but with false-peak risk and a tiny basin; - **mid band** → the trust region for photo↔render matching (very low frequencies carry illumination differences; very high carry sensor noise and micro-texture the render won't have). So coarse-to-fine is not a heuristic bolted on — it is literally **annealing the coherence landscape** from bowl to needle, and it can be *shown* (the landscape morphing as the band ring sweeps outward). This is the transition Sketch 2 renders.
## 4. Coherence with each other: the constellation and the Hubler wires Pairwise phase correlation between *views* (photo↔photo, render↔render, photo↔render where frusta overlap) measures **relative** rotational offsets. Each measured offset is a soft edge in a **pose graph**; the joint solve relaxes all poses toward mutual coherence — bundle adjustment with dense spectral constraints instead of sparse pins. Per feedback_every-constraint-is-soft: the constellation of poses is the current minimum of a collective error surface, and every pairwise coherence is relaxable, reweightable (by PSR). The Hubler resonance is real, not decorative: each pairwise-coherence edge is a **wire whose conductance grows as the poses align** (the correlation peak sharpens with better overlap → higher PSR → higher weight → stronger pull). Alignment strengthens the wire; the wire drives alignment. The three saved views self-assemble into a rigid constellation exactly the way Hubler's wires self-assemble toward the attractor. For the three saved `v2_shots` specifically: their saved poses are ground truth (they were *captured* in-engine), so mutual coherence is first a **self-consistency check** (do measured spectral offsets agree with the known pose deltas?) and then the calibration mechanism for real photos with uncertain poses.
## 5. Where FFT ends: parallax as signal, and the hand-off to the LM solver Camera *translation* makes near and far content shift differently — global phase correlation smears or splits (two layers → two peaks). But **windowed / patch-wise phase correlation** turns the failure into information: a grid of per-patch correlation peaks is a flow field, and its *structure* diagnoses the error: - **uniform flow** → rotational error → fix with the global FFT lock; - **structured disagreement** (near patches move more than far; radial divergence = dolly; shear = lateral) → **position error**, with the pattern indicating the direction. The hand-off options to the existing solver: feed patch-flow vectors into LM as dense micro-correspondences (thousands of soft micro-pins, same machinery as `solvePose`), or let the multi-view constellation triangulate position from mutual rotations, or keep FFT strictly on rotation+fov and leave position to the human + pins. That fork is Sketch 4's question. (The cv-harness bead `b5221927` already has Lucas–Kanade optical flow — the image-domain twin of patch phase correlation; worth a cross-cite when we implement.)
## 6. What the three saved views give us (extraction plan) `repo/dev/through-the-lens-test-v2.html` persists `v2_shots` in **localStorage** (not OPFS): three street-level snapshots `{url: <JPEG dataURL>, pose}` captured at `CAP_SPOTS` around the office, looking at the building — non-coplanar, overlapping frusta: an ideal constellation testbed. Extraction requires the app's *origin* (localStorage is origin-scoped), so the exporter is a sibling page: [repo/dev/export-shots-to-bead.html](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/repo/dev/export-shots-to-bead.html) — open it wherever the v2 app was used, it reads `v2_shots`, previews the three frames + poses, and PUTs `shot-N.jpg` + `poses.json` into `2026-07-03/artifacts/saved-views/` in this bead. Until that runs, the sketches use a synthetic parallax-layered scene as a stand-in (the FFT machinery is identical).
## 7. The sketches (the questions, not the answers) All four live in [fft-coherence-sketches.html](https://redfish.acequia.io/guerin/.agents/74c30681-2b0c-48b2-b2a8-9e21190a4926/2026-07-03/artifacts/sketches/fft-coherence-sketches.html), computing real 128×128 FFTs in-page. Per the [sketch-ux method](https://redfish.acequia.io/skills/sketch-ux/SKILL.md): each names its question, shows the transition, and carries an in-sketch ballot that PUTs a `sketch-answer` request into this bead's `request/` dock. 1. **The compass** — the correlation landscape + correction arrow, live while you drag the render pose. *Question: when coherence knows the answer, does the camera SNAP, GLIDE, or just show a COMPASS the human follows?* Transition: perturb → measure → enact chosen behavior. 2. **The annealer** — the same landscape under a sweeping frequency band: bowl → needle. *Question: which bands do we trust across the photo↔render appearance gap — fixed mid-band, an annealing schedule, or PSR-weighted all-bands?* Transition: the anneal sweep while the pose settles coarse-to-fine. 3. **The constellation** — three overlapping views, pairwise-coherence edges (PSR-weighted), perturb one and relax. *Question: may mutual coherence MOVE saved poses (anchored? full democracy?) or only diagnose tension?* Transition: perturb → tension → relax → settle. 4. **The parallax split** — rotate vs translate the camera; global peak stays sharp vs smears, patch-quiver stays uniform vs shears. *Question: how does FFT hand off position to the LM solver — rotation-only, patch-flow-as-micro-pins, or constellation triangulation?* Transition: toggling motion type, watching both instruments respond.
## 8. Relation to the existing solver (the composition, if this pans out) Total energy = pin reprojection (campanile, sparse, human-authored) + spectral coherence (dense, automatic, PSR-weighted) + priors (soft, per solver.js). The FFT layer contributes (a) a global initializer for rotation+fov with a huge basin, (b) a dense regularizer during drags (the "coherence wind" the camera feels), and (c) a per-pose confidence instrument (PSR). Nothing replaces the pins; the pins stop being responsible for what the spectrum already knows.