**Note** from Bead: Time Line Web Component · [canonical source](https://redfish.acequia.io/guerin/.agents/ade9cea6-f50e-4922-8266-4bdd14ed9c73/2026-06-21/notes/06-plume-pipeline-flow-and-fork.md) · session 2026-06-21 · discussion: Talk: Time Line Web Component
> **Two lenses, held together** (Stephen's ask): **uxui-senior-designer** + **senior-architect**, > sketch-led (Buxton: sketches not prototypes; Ervin: a drawing *asks*, it does not assert; Tufte > deferred to refinement). The job is to make the **flow** of a new capability legible — frame-diff + > learned plume segmenter + optical flow → **per-camera plume mask** → **multi-camera coarse 3D plume** > → render in **taos-engine** + **incident-viewer** — and to foreground the **questions and > alternatives**, then give a **fork recommendation** and a **smallest demonstrable slice**. Each box is > a question wearing a wireframe; fork the one you like, don't approve the picture. Agent-agnostic; > cross-bead links are absolute URIs.
## Framing (one paragraph) The camera-grid already carries everything the *input* side of this needs: ~10 fixed AlertWildfire PTZ cameras, each frame stamped with a pose `{ts, az, el, fov, lat, lon}`, a scrubbable UTC timeline, a live Leaflet map with az±fov/2 **frustum wedges** that sweep as you scrub, a per-mouse **UV→heading projection ray** (level-mount `uvToAzEl`), and an annotation→STAC scaffold where a **raster-paint mask is already the right primitive** (note 03 Sketch 4B: an alpha-mask PNG as a STAC asset `role:"annotation-mask"`). The new idea replaces the *hand* in "hand-paint the smoke" with an **automatic** plume mask (frame-diff to isolate change, a learned detector/segmenter to label it *plume*, optical flow to give it motion), and then asks the genuinely new question: **multiple masks + known poses → a coarse 3D plume** we can drape into the two 3D renderers we already ship. So most of this is **not invention, it is wiring** — the mask is an annotation asset, the poses are frustums we already draw, the renderers already exist (taos-engine has the projector + a Lagrangian smoke sample; incident-viewer is the posed-billboards-over-terrain-on-a-master-clock app). The one truly new organ is the **mask→3D solver**, and the design bet there is *how coarse is coarse enough*.
## Data-flow sketch (the spine) ``` ┌─ per camera c, per frame t ─────────────────────────────────────────────┐ │ │ │ raw frame ──►[ frame-diff ]──►[ plume segmenter ]──► PLUME MASK(c,t) │ │ I(c,t) Δ vs ref/prev (learned, labels α-PNG, image │ │ = motion/change "this is smoke") space (u,v) │ │ └────────►[ optical flow ]──────────────────────► FLOW(c,t) │ │ (Farneback/RAFT) uv velocity │ │ field │ └──────────────┬───────────────────────────────────────────────┬──────────┘ │ mask + the frame's POSE frustum (az,el,fov, │ │ lat,lon — already in hand in camera-grid) │ ▼ ▼ ┌─ MULTI-CAMERA FUSION @ t ───────────────┐ (flow feeds time: │ N masks back-projected as cones into a │ advect / track, │ shared voxel volume → SILHOUETTE │ sketch 4) │ INTERSECTION (visual hull / space-carve)│ │ = COARSE 3D PLUME(t) (voxels or a few │ │ posed billboards / a height column) │ └───────────────┬─────────────────────────┘ │ ONE representation, written as a STAC asset │ (plume:volume) on the master clock ┌─────────────┴──────────────┐ ▼ ▼ ┌─ taos-engine ─┐ ┌─ incident-viewer ─┐ │ voxel / particle│ │ posed volume / │ │ smoke over DEM │ │ billboards over │ │ (Lagrangian │ │ terrain, driven by │ │ sample exists) │ │ the master clock │ └─────────────────┘ └────────────────────┘ ``` The seam to notice: **everything left of "FUSION" is the annotation→STAC plane already sketched in note 03** — a mask is `imageAnno/<imgId>.png`, `role:"annotation-mask"`, `type:"image/png"`, image-space truth + the frame pose stored alongside. Auto-masking just **writes the same asset a model produced instead of a finger painted.** Everything right of FUSION is the **engine-kit plane** (convergence note 01 Sketch C) — one plume representation, two renderers. The new organ is the box in the middle.
## SKETCH 1 — Pipeline placement & the per-camera artifact · *which substrate runs the CV?* **Bet (lean):** the per-camera artifact is a **STAC `plume-mask` asset** (α-PNG, image-space) **plus a companion `plume-flow` asset** (a small velocity field), keyed to the frame exactly like a pose sidecar — `imageAnno/<imgId>.plume.png` next to `imagePose/<imgId>.json`. The mask *is* note-03's raster-paint case, produced by a model. **Where the model runs** is the fork: ``` 1A in-browser (WebGPU) 1B node/bead worker 1C offline batch frame-diff + flow as WGSL a daemon animates a bead, precompute every compute (taos-engine pulls frames, runs the frame's mask once, already has a compute layer: segmenter (ONNX/torch), commit masks as STAC gpu_cull, splat training, PUTs mask+flow sidecars assets in the catalog; voxel mesher); segmenter via back to the commons the app only ever READS ORT-web / tfjs-WebGPU (bead-as-webdav-handler them (cache-as-state, 792ebed9 dock lane: PUT decentralized-cache uploads/ → 202) 4456cd24) ``` - **Question it foregrounds:** *is plume-masking a live, in-session compute, or a pre-baked catalog layer the app consumes?* The answer changes who needs a GPU and whether a citizen phone can play. - **1A** keeps the loop tight and "live" (scrub → mask recomputes in front of you) and compounds the GPU-ABM / agentscript-webgpu bet (`https://redfish.acequia.io/guerin/.agents/f4ef67b7-001f-412c-ba90-f93ceba92bda/about.md`), but ships a learned segmenter into every browser and burns battery. - **1C** is the most Acequia-honest (the mask is just another **STAC asset** in the catalog, read like perimeters or poses; offline-/partition-tolerant; a $0 client). Cost: not "live," and someone owns the batch job. **This is the least-action default** — frame-diff + flow are cheap and classical; the learned segmenter is the only heavy piece and it wants a node, not a browser tab. - **1B** is the middle path and the one that matches the ecology's grain: a **bead worker** (the animator rung of bead-as-webdav-handler) pulls frames, runs the model, and PUTs masks back — async dock, eventual-consistency, citizen-P2P-friendly. **Lean: 1C for the demo, 1B for the system** (the same job, just triggered live vs batch).
## SKETCH 2 — Mask → coarse 3D · *THE load-bearing reconstruction bet* Each camera gives a 2D silhouette + a frustum (az/el/fov from a fixed lat/lon). This is exactly the **shape-from-silhouette / visual-hull** setup — but with smoke, which is **translucent and airborne**, not an opaque solid object. ``` 2A visual hull / space-carve 2B columnar / height-field 2C particle re-seed (silhouette intersection) (smoke as a vertical column) (emit & advect) cam A cam B map (top-down) cam masks seed \ / ┌──────────────┐ a particle emitter \ / voxels kept only │ ▓▓ plume │ each pixel- at the triangulated ┌──╳──┐ where EVERY cam's │ ▓▓▓▓ column │ column has a base; flow fields │░░▓▓░│ cone says "smoke" │ ▓▓▓ height │ base (ground bias advection; │░▓▓▓▓│ → a fat blob in │ from mask │ triangulated) the LAGRANGIAN │░░▓▓░│ the air │ top edge │ + a top smoke sample in └─────┘ └──────────────┘ (mask apex) taos drives it carving in a coarse voxel grid 2.5D, cheap, reads like a physically-plausible (say 64³ over the bbox) terrain drape; ignores but hardest to tie overhangs/lean to the masks ``` - **Question it foregrounds:** *what is "coarse enough," and how much does it lean on calibration?* A wildfire plume located to **±100–300 m and a plausible height/lean** is enormously useful for situational awareness (where is the column, which way is it blowing); we do **not** need a survey-grade volume. That tolerance is the whole argument for "coarse." - **2A (lean for the demo)** is the textbook multi-view answer and needs **only silhouettes + poses** — which is precisely what masks + frustums are. With ~10 cameras you get a genuinely 3D over-determined carve; with 2–3 you get a fat lozenge that still pins the **base location** by ray-intersection. It degrades gracefully and its accuracy scales smoothly with how many cameras + how good the calibration. - **2B** is the cheapest *useful* thing and maps perfectly onto taos's DEM-drape idiom: triangulate the plume **base** (where the bottom of each camera's mask ray hits the ground), then raise a column whose height = how far up the mask reaches. It's 2.5D and wrong about lean/overhang, but it's a one-evening build and reads instantly on the map. - **2C** is the prettiest and the realtime.earth tech-moat direction (Lagrangian smoke already exists in taos, `fire-in-taos`), but it couples reconstruction to a simulation and is the hardest to keep *faithful to the observed masks*. Defer. - **Calibration dependence (the honest caveat):** carving quality is a direct function of pose accuracy. Today's `uvToAzEl` is **level-mount** (no roll/cant/distortion). The carve will be *blurry but unbiased* under level-mount and **sharpens** the moment the **φ–θ panosphere calibration** lands (cant via up-vector + k1,k2) — `https://redfish.acequia.io/guerin/.agents/9943ac1c-caa1-479d-b17e-89ef7ca326b6/2026-06-20/notes/01-treehouse-lunar-eclipse-calibration.md`. Per memory's *every-constraint-is-soft*: the carve is the current min of an error surface; better calibration just reweights it. **Don't block the carve on perfect calibration** — ship it level-mount, let it tighten. **Lean: 2A as the reconstruction bet** (visual-hull space-carve over a coarse voxel grid), with **2B as the always-available fallback** when too few cameras see the plume to carve. 2C is a later epic.
## SKETCH 3 — Render targets · *one representation, two consumers — which plane carries it?* ``` the coarse PLUME(t) as ONE artifact ─────────────┐ (STAC asset: plume:volume │ · option α: a sparse voxel set / occupancy grid │ engine-kit (convergence note 01, over a geo bbox (carve output, native) │ Sketch C): the SHARED taos helpers · option β: a few posed billboards (the │ — DEM sampler, ecefDelta, projector, camera masks themselves, draped at depth) │ the Lagrangian smoke sample — are the · option γ: a height-field column (2B)) │ common kit BOTH renderers call. ───────────────────────────────────────────────┬─┘ The plume is DATA on the data-plane; │ │ each renderer is a projection of it. ┌───────────┴───────────┐ │ ▼ ▼ ┌─ taos-engine ──────────┐ ┌─ incident-viewer ───────┐ │ voxels → instanced │ │ posed billboards over │ │ smoke particles, OR │ │ terrain on the MASTER │ │ the Lagrangian sample │ │ CLOCK (it already rides │ │ seeded at the carve; │ │ the AIR7 track over UTC); │ │ has a compute layer │ │ the plume is just another │ │ (voxel mesher exists) │ │ time-indexed STAC layer │ └────────────────────────┘ └─────────────────────────┘ ``` - **Question it foregrounds:** *is the plume a renderer-specific object, or a data-plane artifact both renderers adapt?* Per convergence note 01's **C2 (data-as-spine)** + **catalog `90f902e0` duck-typing**: make the plume a **STAC `plume:volume` asset** and let each renderer hold its own read-time adapter (taos wants voxels/particles; incident-viewer wants posed billboards or a draped height-field). **No copies, two projections** — the same Plan-9 per-caller namespace move catalog already uses for `incidents.json` vs `imageMeta` vs `imagePose/<id>.json`. - **Lean:** carry it on the **DATA plane (plane 0) as a STAC asset**, rendered through the **engine-kit (plane 3) shared taos helpers**. taos's existing **Lagrangian smoke sample** (from `fire-in-taos`, `https://redfish.acequia.io/guerin/.agents/b6fcda63-c171-4293-9d99-75297f4205bb/about.md`) is the obvious taos consumer; incident-viewer's **posed-billboards-over-terrain-on-master-clock** (`https://redfish.acequia.io/guerin/.agents/be9c3efd-ed6e-440e-9e88-653db538931d/about.md`) is the obvious viewer consumer. Both already exist — this is wiring an adapter, not building a renderer.
## SKETCH 4 — Time · *per-frame reconstruction, or an interval-tracked object?* This is note-03 Sketch 3 (instant / interval / track) made mechanical by **optical flow**. ``` 4A per-frame carve 4B flow-tracked object 4C keyframed track master cursor drives a carve once, then ADVECT the carve at sparse keyframes; fresh carve at each t: plume by the optical-flow flow interpolates between. scrub ──► re-mask ──► re- field between frames (cheap, ◆····◆·······◆ the carve. Honest, stateless; temporally coherent, smooth plume is an animated every frame independent. growth); flow = the motion Feature with timestamps. ├──●──────────────────┤ prior that links instants Richest; couples to the cost: recompute each scrub ├─[◆══════════◆]────────┤ time-line-web-component t0 "this plume" t1 bead. ``` - **Question it foregrounds:** *is the plume a thing-that-persists (an object with a lifespan) or a per-exposure observation?* Optical flow is what makes 4B/4C *possible* — it's the temporal glue. - **Couples to:** the master **time-cursor plane** (convergence note 01 Sketch B) and the time-line-web-component work (this bead's `2026-06-19` chat). Incident-viewer already **is** a UTC master clock; the plume should be a layer it scrubs, not its own clock. - **Lean:** ship **4A** (per-frame carve driven by the master cursor — stateless, demoable, honest), with **flow staged in as 4B** for temporal smoothing once the carve is trusted. 4C is a later epic and the natural home for the *enjoyable* "scrub-to-grow" UX below.
## SKETCH 5 — UX flow · *the designer lens: how does it feel seamless and enjoyable?* The friction to kill: today a user would have to **switch apps** (camera-grid to annotate, calibrator to pose, taos/incident-viewer to render). The seamless flow keeps them on **one timeline** and lets the 3D **assemble in front of them as they scrub.** ``` ┌─ PLUME STUDIO (or camera-grid in "plume mode") ───────────────────────── [3D ▸][✕] ┐ │ ┌─ camera wall (masks light up live) ─┐ ┌─ MAP / 3D ─────────────────────────┐ │ │ │ [A]▓plume [B]▓plume [C] · [D]▓ │ │ ▲A ▲B the frustums that SEE │ │ │ │ [E] · [F]▓plume [G] · … │ │ \ / the plume glow; their │ │ │ │ each tile auto-shades its smoke │ │ \ /░░░\ cones INTERSECT into a │ │ │ │ mask (confidence = opacity) │ │ ────╳(░░░░)── a coarse blob that │ │ │ └──────────────────────────────────────┘ │ \░░░/ THICKENS as more cams │ │ │ │ ‾‾ vote (live assembly) │ │ │ confidence shading: faint = model-unsure; │ [ 2D map ⇄ 3D fly ] toggle, no app │ │ │ solid = multi-camera-agreed │ switch — same canvas tilts to 3D │ │ ├──────────────────────────────────────────────────────────────────────────────────┤ │ timeline (the spine — drives EVERYTHING): │ │ ├──────────●═══════════════╪──────────────┤ ◀ scrub: masks re-mask, blob │ │ plume-density lane ▁▁▂▅█▅▂▁ the carve grows/shrinks under the cursor │ └─────────────────────────────────────────────────────────────────────────────────── ┘ ``` - **The loop (what makes it enjoyable):** 1. **Scrub to a plume time** — a **plume-density lane** on the timeline (▁▂▅█) shows *when* smoke is visible, so you scrub *toward* the event instead of hunting. (This lane is the auto-mask analogue of note 04's "pano density lane.") 2. **Masks light up per camera, automatically** — each tile shades its smoke; **confidence = opacity** (the designer's honesty move: the user *sees* how sure the model is). Hand-correct any mask with the note-03 paint tool — auto and hand are **the same primitive**, so correction is one brushstroke, not a mode change. 3. **Watch the coarse 3D assemble** — the carve **thickens as more cameras vote** (live assembly feedback). This is the delight: the 3D plume *condenses out of the cameras' agreement* in real time as you scrub, like focus pulling. Frustums that see it glow; their intersection is the blob. 4. **Fly it** — a **2D-map ⇄ 3D-fly toggle on the same canvas** (no app switch; the map tilts up into the taos/incident-viewer 3D), and you fly around the column over the terrain on the same clock. - **Question it foregrounds:** *what is the single, continuous gesture-spine?* — **the scrub.** Everything (re-mask, re-carve, render) hangs off the **one master cursor**; the user never leaves the timeline. "Enjoyable" = **scrub-to-grow** (the plume condensing as cameras agree) + **confidence shading** (you trust what you see) + **no app-switch** (map tilts to 3D in place). - **Couples to sibling note 05** (the UX critique, written in parallel): note 05 audits the *current* inspector/annotation surface; this Sketch 5 is the *forward* flow that surface grows into when masks go automatic. **Read 05 for what to fix in the existing chrome; read this for where that chrome is heading.** If 05 lands a finished inspector layout, this loop should adopt its toolbar/affordance decisions verbatim rather than re-deciding them.
## SKETCH 6 — Fork or not · *new app, or a mode/plane on the existing apps?* ``` 6A NEW APP "plume-studio" 6B MODE/PLANE on existing apps forks camera-grid, pulls in the camera-grid gains a "plume" plane; calibrator + a taos render pane; masks are STAC assets (catalog 90f902e0); one bespoke app combining all 3 incident-viewer + taos read them via their own adapters; convergence kit ┌── plume-studio ──┐ (app-shell A + namespace-bus D) carries │ grid + calib + │ the composition. No new app — a new │ taos render │ LAYER that several apps light up. └───────────────────┘ grid──┐ calib──┐ taos──┐ iv──┐ risk: a 4th app to └──STAC plume asset──┘ maintain; re-duplicates (apps compose by URIs, chrome the kit was meant not by linking code) to de-duplicate ``` **Recommendation (least-action): 6B with one carve-out — do NOT fork a standalone app; add a *plume plane* that lights up across the existing apps, and let the *fork energy* go into a thin `taos-render-pane` web component the camera-grid embeds.** The argument: - The whole convergence thesis (note 00/01) is *apps compose by sharing URIs and shelf parts, not by cloning into a new monolith.* A new "plume-studio" that re-bundles grid + calibrator + taos render would **re-duplicate exactly the chrome the app-shell kit (`f80f1929`) exists to extract** — it spends action it should save. - The pieces already live on **separate planes that already converge**: the **mask** is a STAC annotation asset (plane 0, note 03's raster case); the **pose/calibration** is the calibrator + the φ–θ panosphere (`9943ac1c` / `74c30681`); the **carve** is the one new organ; the **render** is taos + incident-viewer (plane 3 engine-kit). Wiring them as a **plane** means each app gains plume power **without a fork.** - **What "forks" is small and justified:** camera-grid is 2D-Leaflet; it has no 3D pane. The *one* new surface worth building is an **embeddable `taos-render-pane`** (the 2D-map ⇄ 3D-fly toggle from Sketch 5) so the carve can be flown **without leaving the grid.** That's an **engine-kit (Sketch C) extraction** — a reusable component — not a new app. **If Stephen does want a named app anyway** (e.g. for the **Community Resilience Architect** to demo a single "see the smoke in 3D" tool in a command post), then **"plume-studio" composes:** camera-grid (input wall + timeline + frustums) ⊕ camera-calibrator (`9943ac1c`, pose/φ–θ) ⊕ the mask→carve organ (new) ⊕ a taos/incident-viewer render-pane (`c66cbd1d` / `be9c3efd`) — **assembled from the app-shell kit (`f80f1929`) so it's shelf-parts, not a clone.** Even then, the *data* stays on the commons as STAC assets so the other apps can still read it. **Net: build the plane; if you must name an app, assemble it from the kit — never a from-scratch monolith.**
## Smallest demonstrable slice (what to prototype first) Deliberately the cheapest cut that exercises **every** seam — one plume time, a handful of cameras, **no learned model, no flow, no live compute**: 1. **Pick one plume time** in the existing camera-grid extent where **2–3 cameras** clearly see the same smoke column (the Sandy archive already has such a moment). 2. **Hand-paint the masks** using the note-03 raster-paint tool (this *is* the per-camera artifact; it stands in for the segmenter, so the demo needs **zero ML**). Each mask = `{u,v}` α-region + the frame's pose already in hand. 3. **Naïve space-carve (Sketch 2A) in plain JS:** a coarse voxel grid (e.g. 32³–64³) over the cameras' shared bbox; keep a voxel iff **every** masked camera's frustum-ray says "inside the silhouette" (level-mount `uvToAzEl` for the ray — already shipped in note 04). With only 2–3 cameras this is a fat lozenge, but it **pins the base** by ray-intersection — and that's the point: prove the geometry end to end before adding cameras or ML. 4. **Render the carve once in taos** as instanced voxel boxes (or seed the existing Lagrangian smoke sample at the carve centroid) over the Taos/Sandy DEM. **No incident-viewer adapter yet** — one renderer proves the representation. 5. **Drape the carve's footprint on the Leaflet map** (cash in the frustums you already draw) so the "2D echo" exists at zero extra cost, and add a **plume-density pip** on the timeline at the chosen time. That slice proves: **mask (hand) → pose-ray → voxel carve → taos render → map echo**, with **no ML, no flow, no calibration upgrade, no new app.** Each deferred piece then layers on as its own step: **learned segmenter** (replace step 2's hand), **optical flow** (Sketch 4 temporal glue), **φ–θ calibration** (sharpen the carve), **incident-viewer adapter** (second renderer), **node/bead worker** (Sketch 1B, take masking off the browser), and the **scrub-to-grow live-assembly UX** (Sketch 5).
## Open questions for Stephen 1. **Pipeline substrate** (Sketch 1): live in-browser WebGPU (1A), a bead worker that PUTs mask sidecars (1B), or pre-baked STAC mask assets the app only reads (1C)? Lean: **1C for the demo, 1B for the system.** Where do the learned segmenter weights live, and who runs them? 2. **Reconstruction bet** (Sketch 2): is **visual-hull space-carve (2A)** the target, with a **height column (2B)** fallback — and what is "coarse enough" *for you* (±100 m base + plausible height/lean)? Or is the real prize the **Lagrangian particle (2C)** physical model, accepting it's a later epic? 3. **Plume artifact form** (Sketch 3): a STAC `plume:volume` asset (sparse voxels / posed billboards / height-field) read by per-renderer adapters (catalog duck-typing) — agreed? What does the asset's `type`/encoding look like (a voxel-occupancy binary? a small GeoJSON-ish volume? reuse the TOA-raster RGB-pack idiom from fire-progression)? 4. **Time model** (Sketch 4): per-frame stateless carve (4A) first, flow-advected object (4B) next, keyframed track (4C) later — confirm the staging. Does the plume bind to the **incident-viewer master clock** or keep its own? 5. **Fork** (Sketch 6): build the **plume *plane*** across existing apps + a thin embeddable `taos-render-pane` (the recommendation), or do you want a **named "plume-studio" app** assembled from the app-shell kit for a CRA/command-post demo? (Even if named, data stays on the commons.) 6. **Calibration coupling** (Sketch 2 caveat): ship the carve **level-mount now** and let the **φ–θ panosphere** (`9943ac1c`) sharpen it later (per *every-constraint-is-soft*) — or hold the carve until cant+distortion are solved? Lean: **ship level-mount, tighten later.** 7. **Governance** (note-03 Q4/Q5 carried forward): are auto-generated masks **bead-local + offered** by default like poses, or written into `incidents/**`? And are masks **attributable** (model id + version + confidence as `properties`, the auto analogue of `createdBy`) so a model-mask and a hand-correction are distinguishable in the commons?
## References (bead cross-links) - Bead: Agentscript Webgpu · [canonical](https://redfish.acequia.io/guerin/.agents/f4ef67b7-001f-412c-ba90-f93ceba92bda/) - Bead: Astronomical Pano · [canonical](https://redfish.acequia.io/guerin/.agents/9943ac1c-caa1-479d-b17e-89ef7ca326b6/) - Bead: Ants In Taos · [canonical](https://redfish.acequia.io/guerin/.agents/b6fcda63-c171-4293-9d99-75297f4205bb/) - Bead: Incident Viewer Epic 4 · [canonical](https://redfish.acequia.io/guerin/.agents/be9c3efd-ed6e-440e-9e88-653db538931d/)