**Note** from Bead: Time Line Web Component · [canonical source](https://redfish.acequia.io/guerin/.agents/ade9cea6-f50e-4922-8266-4bdd14ed9c73/2026-06-21/notes/03-image-annotation-uv-vector-stac-sketches.md) · session 2026-06-21 · discussion: Talk: Time Line Web Component
> Role: **uxui-senior-designer**, brought into the camera-grid working copy (this bead). Method: > sketch-ux (Buxton: sketches, not prototypes; Ervin: "what a drawing is" — it asks, it does not > assert; Tufte deferred to refinement). The point of this note is to make **3–4 different design > bets visible** and to foreground the load-bearing **questions**, not to converge. Every box below is > a *question wearing a wireframe*. Fork the question you like; don't approve a sketch.
## What's on the ground right now (so we sketch against the real app, not an imagined one) From `repo/camera-grid/` (read 2026-06-21): - A **canvas UTC scrubber** (`#timeline-canvas`) with one frame-density lane per camera + a yellow playhead. Scrub drives `updateWall()` → each tile swaps to its `nearestFrame(cam, playhead)`. - A **wall of camera tiles** (`.cam-tile`), each carrying a per-frame pose `{ ts, az, el, fov, url }` derived from AlertWildfire telemetry (`camAzimuth / camElevation / camViewWidth`). Cameras are **fixed-position, variable-orientation (PTZ)**. - A bare **lightbox** (`#lightbox` / `#lightbox-img` / `#lightbox-caption`) — today it just zooms the frame and shows `name · UTC · az° fov°`, then closes on click. **This is the surface Stephen means by "inspect an image."** It is currently a dead-end (view-only). It is also where ALL of this lands. - A **Leaflet map** with camera markers; az wedges are explicitly deferred (`syncMapAz()` is a stub "// az wedges in S3c"). So the map is *ready to receive* a wedge/ray but draws none yet. - Pose primitive is the santafe `imagePose`/`imageMeta` **sidecar** keyed by image id (firewatch bead note 00): `imagePose/<id>.json = {Latitude,Longitude,Elevation,heading,pitch,roll,fov,fovy,far}`. STAC oriented-imagery is the destination; sidecar is source-of-truth, STAC Item *binds* it. So the design problem sits **inside the lightbox**: promote it from a viewer to an **inspector + annotator**, while the timeline behind it stays scrubbable, and emit a **URI** a STAC Item can hold. The naming: I'll call the user-drawn mark a **UV mark** (Stephen's "UV vector") — *UV* because it is authored in the **image's (u,v) pixel plane**. Whether it *stays* in (u,v) or gets lifted to a world ray is exactly Sketch 2's bet.
## SKETCH 1 — Tool placement & geometry-type picker: "floating toolbar over the frame" **Bet:** the inspector is a focused modal-ish surface; tools live in **one toolbar pinned to the frame**, and geometry type (point / polyline / polygon / paint) is a **mode you select once** then draw freely (like a vector editor). Timeline stays visible as a thin strip so you never lose the clock. ``` ┌─ INSPECT ──────────────────────────────────────────────────────── [STAC ▸] [✕] ┐ │ │ │ ╔══════════ tool rail (image-anchored) ═══════════╗ │ │ ║ [ • pt ] [ ⌇ line ] [ ▱ poly ] [ ✎ paint ] | [ ⟲ undo ] [ ⌫ ] [ 🎨 ] ║ │ │ ╚════════════════════════════════════════════════════════════════════════╝ │ │ │ │ ░░░░░░░░░░░░░ CAMERA FRAME @ T (large) ░░░░░░░░░░░░░░ │ │ ░░░░░░░░░ ___ ░░░░░░░░░░░░░ │ │ ░░░░░░░ / \ <- user traces a smoke column │ │ ░░░░░░ ( poly ) as a polygon, vertex by vertex │ │ ░░░░░░░ \ ___ / ░░░░░░░░░░░░░ │ │ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │ │ │ │ Sandy-15801 · 2026-08-14 21:32Z · az 212° fov 41° [ label: "smoke ▼" ] │ ├──────────────────────────────────────────────────────────────────────────────── ┤ │ timeline (still live): ├──────●───────────────────────┤ ◀ scrub me │ └──────────────────────────────────────────────────────────────────────────────────┘ ``` **Question it foregrounds:** *Is annotation a MODE or a free gesture?* A toolbar = explicit modes (good for paint, which needs brush size; good for discoverability). The cost: a wall operator under time pressure wants to *drop a spot fast* — 2 clicks (pick "pt", click frame) may be one too many. **Fork A1 (radial / press-and-hold):** no persistent toolbar; long-press on the frame pops a small radial — `• ⌇ ▱ ✎` around the cursor — release on the type, drag to draw. Fewer chrome pixels, more of the frame visible, but paint's brush-size/opacity has nowhere to live. **Fork A2 (smart default + escalate):** single click = drop a **point** (the 80% case: "smoke here"). Click-drag = **polyline**. Double-click-to-close = **polygon**. Paint is the only thing behind an explicit button. Bets that *geometry type is inferred from gesture*, not pre-selected. Risk: a mistaken drag becomes a 2-vertex line; needs forgiving undo.
## SKETCH 2 — THE load-bearing bet: image-(u,v) space vs world-ray space This is the "two hemispheres of one panosphere" / through-the-lens question. Where does the mark *live*, and where do you *see* it? ### 2A — Mark lives in normalized image (u,v); map is a passive echo ``` INSPECTOR (you draw HERE, in pixels) MAP (echo only) ┌───────────────────────────────┐ ┌───────────────────────┐ │ frame @ T │ │ ▲ cam │ │ ___ │ pose @ T │ /·\ <- a single │ │ / smoke \ (u,v) polygon │ ──────────▶ │ / · \ wedge, │ │ ( 0.41,0.22 … ) │ projects │ / · \ no shape │ │ │ ONLY a │ (just "this cam, │ └───────────────────────────────┘ bearing │ that az, at T") │ stored: GeoJSON in pixel CRS, └───────────────────────┘ OR an SVG <polygon> over the image ``` **Bet:** the annotation is an **image-plane fact** ("in this frame, at these pixels, there is smoke"). It is *honest* — it claims nothing about ground position it can't support. The pose (az/el/fov at T) is stored *alongside* so anyone can reproject **later**, with a better terrain model, without re-drawing. Cheap to build: it's drawing on a `<canvas>`/SVG sized to the image. The map just shows the **camera's view wedge** (which we owe it anyway — `syncMapAz()` stub). **Cost:** the mark has no footprint on the map. "Where is the fire?" is answered only as "somewhere along this bearing." Two cameras seeing the same smoke = two bearings; their **intersection** (the triangulation that actually locates the fire) is a *future* solve, not something this UI does yet. ### 2B — Mark is immediately lifted to a world ray/footprint, drawn on the map ``` INSPECTOR (draw HERE) MAP (the mark is REAL here) ┌───────────────────────────────┐ ┌───────────────────────────┐ │ frame @ T │ per-vertex │ ▲ cam │ │ ___ │ ray-cast │ \ ___ │ │ / smoke \ │ ──────────▶ │ \ /░░░\ <- ground │ │ ( drag a vertex... ) │ through │ \(░░░░░) footprint │ │ ...map polygon MOVES live ◀──┼── pose+DEM │ \░░░░/ (clipped │ └───────────────────────────────┘ │ frustum ‾‾ to FOV) │ stored: GeoJSON in EPSG:4326 └───────────────────────────┘ ``` **Bet:** the annotation *means* a place on the earth, so commit to it now: each image vertex is a ray from the camera (using `az/el/fov` + a ground/DEM assumption) → lon/lat. The map polygon is the truth; the image is just the *authoring lens*. This is the through-the-lens calibration idea made interactive — and it makes **triangulation** natural later (two cameras → two footprints → overlap). **Cost:** every vertex now depends on a **terrain/altitude assumption**. Flat-ground = wrong on slopes (and a smoke column is *above* ground — a polygon traced around airborne smoke has no single ground depth). You're committing to a geolocation you may not trust. Honesty problem inverse of 2A. ### 2C (the dual-pane compromise) — author in image, **live-preview** in world, store BOTH ``` ┌───────────── image (author) ──────────┐ ┌──────── map (preview/QA) ────────┐ │ ___ │ │ ▲ cam │ │ / smoke\ ← drag vertices here │ │ \ ___ │ │ ( · · · ) │ │ \ /ghost\ ← updates live, │ │ │ │ (·····) shown dashed = │ │ [ depth: ◉ horizon ○ DEM ○ 2km ] │ │ \···/ "projected, not │ └────────────────────────────────────────┘ │ ‾‾ surveyed" │ stores imageGeometry (u,v, authoritative) └──────────────────────────────────┘ + worldGeometry (derived, tagged provenance:"projected", with the depth choice) ``` **Bet:** refuse to choose. The **image (u,v) is source-of-truth**; the world footprint is a *derived, clearly-marked-provisional* projection (dashed = "don't trust as survey"), with a **depth selector** (horizon-plane / DEM / fixed range) so the user states their assumption explicitly. Matches the bead's "sidecar is source, STAC binds/mirrors" stance: image-geometry is the sidecar fact; world-geometry is the projection. Costs more UI (two live panes + a depth control) but is the most *epistemically clean*. **Question 2 foregrounds:** *what is an annotation a claim ABOUT?* — pixels (2A), ground (2B), or pixels-with-a-stated-projection (2C). This decision cascades into the export schema (Sketch 4).
## SKETCH 3 — Time binding: is a UV mark an instant or an interval? The mark is drawn on **one frame at T**. But a smoke column *persists*. How does the scrubber show it? ### 3A — Point-in-time observation (pinned to one frame) ``` timeline: ├─────────◆─────────────────────────────────┤ └ mark lives at exactly T (this frame). Scrub away → mark is not shown (it's a fact about THAT frame). A "◆ annotation pip" sits on the lane at T; click it to jump+reopen. ``` **Bet:** simplest + truest to provenance — an annotation is evidence tied to *one exposure*. Multiple frames of the same smoke = multiple marks (a track). Cost: no notion of duration; a persistent feature is N separate marks the UI doesn't relate. ### 3B — Interval observation (mark spans [T0, T1] on the timeline) ``` timeline: ├──────[◆════════════════◆]──────────────────┤ T0 "smoke column" T1 Drag the mark's endpoints ON the scrubber to set when it starts/ends. While playhead ∈ [T0,T1], the mark renders (re-projected per current frame's pose). ``` **Bet:** an annotation is a *feature with a lifespan*; you draw once and declare "this is present from 21:30 to 22:10Z." Maps onto STAC's `start_datetime`/`end_datetime`. Cost: the mark was drawn against *one* frame's pose — showing it on frames at other T's means re-projecting (only coherent in a world-space mark, Sketch 2B/2C), or showing the same (u,v) shape on other frames (cheap but pose drifts under PTZ, so the pixels may no longer cover the smoke). ### 3C — Keyframed track (instant marks that interpolate = the editorial answer) ``` timeline: ├───◆·····◆·······◆──────────────────────────┤ T0 T1 T2 each ◆ = a mark on a real frame; between them the feature is INTERPOLATED (a moving/growing polygon). This is a smoke-column-over-time as an animation, like a video keyframe track. ``` **Bet:** the honest unit is the per-frame mark (3A), but the *useful* unit is the track; so let marks on successive frames **link into a keyframed feature** (a `Feature` whose geometry has timestamps). Richest, most expensive; probably a later epic. Foregrounds: *do we ever need a moving annotation, or is firewatch fundamentally "spot the smoke in this frame"?* **Question 3 foregrounds:** instant vs interval vs track — and it's coupled to Sketch 2 (only world-space marks reproject cleanly across time).
## SKETCH 4 — The URI / STAC handoff: what artifact, and how does the UI say "this is now an asset"? Stephen: exported "as a **URI that can be added to a STAC** [item/asset]." Two clean cases, and the UI must make the handoff legible. ### 4A — Vector marks → a GeoJSON Feature(Collection), one asset href ``` [STAC ▸] panel slides from the right: ┌─ EXPORT ───────────────────────────────────┐ │ Frame: Sandy-15801 @ 2026-08-14T21:32Z │ │ STAC Item id: sandy-15801-1755... │ │ │ │ Annotations (2): │ │ ◆ polygon "smoke" image+world ✎ 🗑 │ │ • point "spot" image ✎ 🗑 │ │ │ │ Asset to write: │ │ role: annotation │ │ type: application/geo+json │ │ href: imageAnno/<imgId>.geojson [copy] │ │ │ │ [ Preview JSON ] [ ✓ Add as asset ] │ └─────────────────────────────────────────────┘ ``` The artifact is a **GeoJSON FeatureCollection** sidecar (parallel to `imagePose/<id>.json` → `imageAnno/<id>.geojson`). Each feature carries `properties: { kind, label, imageGeometry:[u,v…], createdBy, datetime, projection:"horizon|dem|range" }`. The STAC Item gains an **asset**: `{ roles:["annotation"], type:"application/geo+json", href:"…/imageAnno/<id>.geojson" }`. "Add as asset" = the handoff; the **href IS the URI** Stephen asked for. ### 4B — Raster paint → a PNG mask as `data:` URI **or** a written sidecar PNG ``` paint case: ┌─ EXPORT (raster) ──────────────────┐ ┌─────────────────────┐ │ ✎ painted mask 1920×1080 · 14 KB │ │ ░░░▓▓▓▓▓░░░░░░░░░░░ │ flatten → │ │ │ ░░▓▓▓▓▓▓▓░░░░░░░░░░ │ alpha mask │ store as: │ │ ░░░▓▓▓▓░░░░░░░░░░░░ │ (smoke=α) │ ○ data: URI (inline, < ~32 KB) │ └─────────────────────┘ │ ◉ sidecar PNG imageAnno/<id>.png │ one paint layer = one mask │ │ │ asset role: "annotation-mask" │ │ type: image/png │ │ [ ✓ Add as asset ] │ └─────────────────────────────────────┘ ``` **Bet on the raster handoff:** a painted region is an **alpha mask PNG**, registered as a *separate* STAC asset (`role:"annotation-mask"`, `type:"image/png"`) so vector and raster annotations coexist on one Item. Small masks can ride inline as a `data:` URI (truly "an annotation = a URI"); larger ones become a written sidecar PNG href. The mask is still **image-space** (it overlays *this* frame); its georeference is the same pose, applied per-pixel only if/when needed (drape on terrain in the 3D view). **Question 4 foregrounds:** *one asset of mixed geometry, or one asset per kind?* And *for raster, is the URI an inline `data:` blob (self-contained, no write) or a file href (needs a PUT to the sidecar tree)?* The latter reopens firewatch note-00 Q6 (where do derived sidecars get written — bead vs `incidents/**`). I'd keep annotations **bead-local + offered** by default, same rule as poses. **Legibility of "now it's a STAC asset":** whichever artifact, the moment of handoff should flip a visible state on the mark — e.g. the annotation row gains a small **`{} STAC`** chip and the mark outline goes from dashed (draft) to solid (committed). The user should never wonder whether their trace is "saved as catalog" or "just on screen."
## How these compose (the seams between the sketches) - Sketch 1 (tooling) is mostly independent — pick A2 (gesture-inferred) for speed or the toolbar for paint; either feeds the same geometry. - Sketch 2 is the **spine**. 2C (author-in-image, preview-in-world, store both) makes Sketch 3B/3C (reproject across time) and Sketch 4A (`imageGeometry` + derived `world`) *possible*; 2A makes them harder; 2B makes the image plane vestigial. - Sketch 4's schema is literally the serialization of Sketch 2's decision. Decide 2 first.
## Open questions for Stephen 1. **Image-space or world-space, or both?** (Sketch 2.) This is THE fork — it sets the schema, the map behavior, and how time-binding can work. My lean: **2C** (image is source-of-truth, world is a marked-provisional projection with an explicit depth assumption) — most honest, matches the sidecar-is-source stance, and doesn't strand us when triangulation/DEM arrive. 2. **Instant, interval, or track?** (Sketch 3.) Is firewatch annotation fundamentally "mark the smoke in THIS frame" (3A) or "this feature persists T0→T1" (3B)? My lean: ship **3A** (instant, a pip on the lane) first; reserve interval/track for a later epic once world-space marks exist. 3. **Tool affordance:** persistent toolbar (Sketch 1), gesture-inferred type (A2), or radial (A1)? My lean: gesture-inferred for point/line/poly + an explicit Paint mode (paint genuinely needs brush/opacity chrome). 4. **Asset granularity & raster URI:** one mixed-geometry GeoJSON asset + a separate mask asset (4A+4B)? And for masks: inline `data:` URI vs written sidecar PNG? Couples to firewatch note-00 Q6 (write target for derived sidecars). 5. **Annotation sidecar location / naming:** `imageAnno/<imgId>.geojson` next to `imagePose/`, bead-local + offered (default), or bound under `incidents/**`? (Same governance gate as poses.) 6. **Who is the author, and is it signed?** Should each feature carry `createdBy` (a parciante identity) so annotations are attributable/trustable in the commons, per the auth stance? Cheap to add now, expensive to retrofit.
## Smallest buildable slice (what to scaffold first) Promote the **lightbox into an inspector with ONE geometry type, image-space only, GeoJSON export** — deliberately the cheapest cut through all four sketches: 1. **Inspector chrome:** keep `#lightbox` open while the timeline behind it stays scrubbable (it already is a fixed overlay; just stop it closing on every click and add a `[✕]` + a `[STAC ▸]` button). Overlay a transparent `<svg>`/`<canvas>` sized to `#lightbox-img`. 2. **One tool — the point** (Sketch 1, the 80% case): click the frame → drop a `•` at normalized (u,v). Show it; allow undo. No mode picker yet. 3. **Image-space store (Sketch 2A as the floor):** keep the mark as `{u,v}` plus the frame's pose `{az,el,fov,ts,lat,lon}` already in hand. **Don't** reproject yet — but **draw the camera's view wedge on the Leaflet map** (cash in the `syncMapAz()` stub) so the "world echo" exists at zero geolocation risk. 4. **Export one asset (Sketch 4A):** a button that serializes a one-feature GeoJSON (`imageAnno/<imgId>.geojson`) and shows the **href URI** + a copyable STAC `asset` JSON snippet. Offer it (don't auto-write), per bead policy. The mark row flips to a `{} STAC` chip on export. That slice exercises every seam — inspect, draw, image-space truth, wedge-on-map, URI-as-asset — with **no terrain math, no time-interval model, no raster pipeline**. Polyline/polygon/paint, world-ray projection (2B/2C), and interval/track time-binding (3B/3C) then layer on as separate steps once Stephen picks the forks above. (Tufte-grade polish — wedge styling, mask compositing — waits for refinement; this is the sketch stage.)