**Artifact** from Bead: Incident Cataloging · [canonical source](https://redfish.acequia.io/guerin/.agents/c38c1239-bfd3-44dd-8d97-1a0aa39ac8da/2026-06-08/artifacts/misb-viewer-data-formats.md) · session 2026-06-08 · discussion: Talk: Incident Cataloging
**Subject:** https://redfish.acequia.io/guerin/apps/viewer-3d/index-misb.html **Local source:** `redfish.acequia.io/guerin/apps/viewer-3d/` **Snapshot:** 2026-06-08 · viewer version `v034` **Purpose:** First-pass documentation of every data format this app consumes, as input to designing a view/controller-agnostic **event catalog** format. Companion to [santafe-alert-live-events-structure.md](https://redfish.acequia.io/guerin/.agents/c38c1239-bfd3-44dd-8d97-1a0aa39ac8da/2026-06-08/artifacts/santafe-alert-live-events-structure.md) (the `alertLive` posed-photo event family).
## 1. What this app is A multi-aircraft Full-Motion-Video (FMV) viewer. It places **aircraft camera frustums / panospheres** in a 3D globe (Google Photorealistic 3D Tiles + Cesium terrain via `3d-tiles-renderer`), textures each with its MP4 video, and drives position/orientation from **MISB ST 0601** sensor telemetry. A master UTC timeline scrubs all loaded aircraft synchronously. Entry point `index-misb.html` loads one ES module: `js/main-misb-multi.js` (the multi-aircraft build; `main-misb.js` is the older single-aircraft variant). Three.js + the tiles renderer come from CDN via an import map.
## 2. The format stack (4 layers) ``` Layer A CATALOG incidents.json incident → folder* → aircraft[] (the manifest) Layer B TELEMETRY (disk) *_telemetry.json MISB ST 0601, TWO on-disk dialects *_klv.json Layer C TELEMETRY (mem) normalized "sample" parser output, dialect-agnostic Layer D MEDIA .mp4 (+ video texture) pixels, mapped onto a panosphere ``` The app's own boundary of interest is **Layer A** (what to load) and **Layer B/C** (how to pose it). Layer D is opaque media referenced by URL. This layering is the key input to the catalog-format goal: the catalog (A) is already nearly independent of the renderer.
## 3. Layer A — Catalog manifest: `incidents.json` Local file, fetched by `LayersPanel.loadManifest('incidents.json')`. A nested tree; **aircraft are the leaves**, everything above is grouping. ```jsonc { "incidents": [ { "id": "2025Palisades", "name": "2025 Palisades Fire", "folders": [ // recursive grouping nodes { "id": "air-intel", "name": "Air Intel", "folders": [ { "id": "ts", "name": "MISB Videos", "aircraft": [ // LEAF list { "id": "N57B_223016", "name": "N57B 22:30:16Z", "tailNumber": "N57B", "video": "https://guerin.acequia.io/.../VIDEOCLIP_223016230.mp4", "telemetry": "https://guerin.acequia.io/.../VIDEOCLIP_223016230_telemetry.json", "startUTC": "2025-01-07T22:29:06Z", "duration": 52 } ] } ] } ] } ] } ``` **Node types** | Node | Required fields | Role | |------|----------------|------| | incident | `id`, `name`, `folders[]` | top-level event grouping | | folder | `id`, `name`, and `folders[]` *or* `aircraft[]` | arbitrarily-deep grouping (recurses) | | aircraft (leaf) | `id`, `name`, `tailNumber`, `video`, `telemetry`, `startUTC`, `duration` | one playable FMV clip | - **`folders` recurse** to any depth; `LayersPanel.getAllAircraft()` walks the tree and flattens all leaves. - **`video` / `telemetry`** are absolute URLs and may point at **different hosts** per clip (`guerin.acequia.io` for Palisades, `simtable.acequia.io` for Sandy in the same manifest). The catalog is host-agnostic by construction. - **`startUTC`** (ISO-8601 Z) and **`duration`** (seconds) are catalog-level hints for the timeline; the authoritative time range is recomputed from telemetry at load. - **`tailNumber`** is the only domain-specific (aviation) field; everything else is generic "a posed, timed media clip." > Observation: this manifest is already a generic **catalog of timed, geo-located media leaves under a recursive folder tree**. It carries no rendering/controller config — that's all hard-coded in the viewer. Good starting point for the agnostic catalog.
## 4. Layer B — On-disk telemetry: MISB ST 0601, two dialects Each aircraft leaf points at one telemetry JSON, extracted from the MP4's MPEG-TS **KLV** metadata stream. There are **two on-disk schemas** in the wild, and `MISBTelemetryParser` accepts both (`normalizeKLVPacket` bridges them). ### 4a. "Palisades" dialect — long names, top-level envelope, `samples[]` ```jsonc { "source_file": "..._223016230.bin", "format": "MISB ST 0601", "sample_count": 3127, "duration": 51.951841, "samples": [ { "precision_time_stamp": 1736288946876095, // µs since Unix epoch "platform_heading_angle": 212.226, // deg "platform_pitch_angle": -0.048, "platform_roll_angle": 3.442, "sensor_latitude": 34.0558, "sensor_longitude": -118.5041, "sensor_true_altitude": 4737.04, // m MSL "sensor_relative_azimuth_angle": 89.936, // gimbal, body-frame "sensor_relative_elevation_angle": -44.405, "sensor_relative_roll_angle": 0.0, "slant_range": 6392.48, // m "frame_center_latitude": 34.0764, "frame_center_longitude": -118.5434, "frame_center_elevation": 0.032, // m "sample_time": 0.0, // s, video-relative "date_time": "2025-01-07T22:29:06.876095+00:00", // ISO, UTC "image_source_sensor": "FLIR 380X-HDC ZTV", "sensor_horizontal_fov": 41.913, "sensor_vertical_fov": 23.577, "target_width": 4896.77, "tag_75": "480e", "tag_78": "0b1e", "uas_ls_version_number": 4, "checksum": 36057 } // ... ] } ``` ### 4b. "Sandy" dialect — short names, no envelope, `packets[]` ```jsonc { "packets": [ { "precision_time_stamp": 1779142821416078, "platform_heading": 74.428, "platform_pitch": 0.639, "platform_roll": -1.769, "sensor_lat": 34.1889, "sensor_lon": -118.7477, "sensor_alt": 3908.07, "sensor_azimuth": 270.988, "sensor_elevation": -33.210, "sensor_roll": 0, "slant_range": 5403.08, "frame_center_lat": 34.2276, "frame_center_lon": -118.7595, "frame_center_elev": 813.22, "image_source_sensor": "FLIR 380X-HDC IR", "sensor_hfov": 39.431, "sensor_vfov": 22.179, "target_width": 3872.28, "videoOffsetSec": 0, // s, video-relative (== sample_time) "uas_ls_version": 4, "checksum": 53245 } // ... ] } ``` ### 4c. Field-name crosswalk (the normalization map) `normalizeKLVPacket()` maps Sandy → Palisades long names so the rest of the parser is schema-agnostic: | Concept | Palisades (long) | Sandy (short) | |---|---|---| | envelope key | `samples[]` + `source_file`/`format`/`sample_count`/`duration` | `packets[]` (bare) | | time (abs) | `precision_time_stamp` (µs) + `date_time` (ISO) | `precision_time_stamp` (µs) only; `date_time` derived | | time (video-rel) | `sample_time` (s) | `videoOffsetSec` (s) | | platform attitude | `platform_heading_angle` / `_pitch_angle` / `_roll_angle` | `platform_heading` / `_pitch` / `_roll` | | sensor position | `sensor_latitude` / `_longitude` / `sensor_true_altitude` | `sensor_lat` / `_lon` / `sensor_alt` | | gimbal (body-rel) | `sensor_relative_azimuth_angle` / `_elevation_angle` / `_roll_angle` | `sensor_azimuth` / `sensor_elevation` / `sensor_roll` | | FOV | `sensor_horizontal_fov` / `sensor_vertical_fov` | `sensor_hfov` / `sensor_vfov` | | frame center | `frame_center_latitude` / `_longitude` / `_elevation` | `frame_center_lat` / `_lon` / `_elev` | | range / target | `slant_range` / `target_width` | `slant_range` / `target_width` | | sensor name | `image_source_sensor` | `image_source_sensor` | **Split-packet merge:** In the Palisades dialect, position and sensor/FOV info arrive as *separate KLV packets sharing one `precision_time_stamp`*. `parseSamples()` first-pass merges all raw records by timestamp before converting; samples lacking `sensor_latitude` after merge are dropped.
## 5. Layer C — In-memory normalized "sample" `parseSamples()` emits a renderer-facing object (camelCase, dialect-erased). This is the **de-facto internal telemetry contract** every view/controller in the app reads: ```js { sampleTime, timestamp, dateTime, // s (video-rel), µs (abs), ISO latitude, longitude, altitude, // sensor/aircraft position; altitude = MSL m relativeAltitude, // always null (MISB has no AGL) yaw, pitch, roll, // ABSOLUTE world orientation, deg (see §6) platformHeading, platformPitch, platformRoll, // aircraft body attitude, deg horizontalFOV, verticalFOV, // deg frameCenterLatitude, frameCenterLongitude, frameCenterElevation, // ground aim point slantRange, targetWidth, // m sensorName // string } ``` - **Lookup:** `getSampleAtTime(videoTime, interpolate=true)` binary-searches by `sampleTime` and **lerps** numeric fields; angles use wrap-aware interpolation (`interpolateAngle`, shortest path across ±180). - **`dateTime` is NOT interpolated** (start sample's value is kept) — a minor wart.
## 6. Coordinate & angle conventions (critical, easy to get wrong) - **Positions** are geodetic (lat°, lon°, **MSL altitude in m**). The scene converts via `SceneManagerMISB.geodeticToLocal()` ↔ `localToGeodetic()`. Three.js world is **Y-up**; geodetic→local handles the ECEF/ENU framing. - **Gimbal angles in MISB are RELATIVE TO THE AIRCRAFT BODY FRAME**, not the world. The parser composes absolute orientation: - `yaw = normalize(platform_heading + sensor_relative_azimuth)` (0–360) - `pitch = platform_pitch + sensor_relative_elevation` - `roll = platform_roll + sensor_relative_roll` - Verified empirically (`check_bearing.py`): bearing to `frame_center` matches `platform_heading + sensor_relative_azimuth`. - **`frameCenter*`** is the ground point the sensor is aimed at — used for "Look at Frame Center" camera mode and to draw the frustum's footprint.
## 7. Time model Three clocks, reconciled at load: | Clock | Units | Source | |---|---|---| | absolute UTC | µs since epoch | `precision_time_stamp` (+ ISO `date_time`) | | video-relative | seconds | `sample_time` / `videoOffsetSec`; first sample = 0.0 | | wall display | UTC + Pacific | `TimeUtils` formats both; PLAN.md §UTC | - `Aircraft.startUTC = Date(firstSample.dateTime)`, `endUTC = startUTC + (lastSample.sampleTime − firstSample.sampleTime)·1000`. - Timeline (master) is in **UTC ms**; per-aircraft `videoTime = (utcTime − startUTC)/1000` seeks each `<video>` and queries telemetry. Aircraft only render when `utcTime ∈ [startUTC, endUTC]`. - Global timeline range = **union** of all loaded aircraft ranges.
## 8. Layer D — Media & hosting - **Video:** MP4 referenced by absolute URL. Loaded into a hidden `<video>` (`muted`, `preload=auto`, **`crossOrigin='anonymous'`** set before `src` so WebGL can sample it without tainting the canvas). Mapped as a live texture onto each `Panosphere` (radius/opacity user-controlled). - **CORS:** both `guerin.acequia.io` and `simtable.acequia.io` serve videos with `Access-Control-Allow-Origin: *`. This is a hard requirement for the texture path. - **Terrain basemap:** Google Photorealistic 3D Tiles (`GOOGLE_API_KEY`) / Cesium World Terrain (`CESIUM_ION_TOKEN`) in `js/config.js` — environment, not event data. (Note: live API keys are committed in `config.js`.)
## 9. Toward a view/controller-agnostic event catalog What this app teaches the catalog design: 1. **The catalog is already separable.** `incidents.json` describes *what exists* (timed, geo-located media leaves in a recursive grouping tree) with **zero rendering or controller config**. The agnostic catalog can be a generalization of this node tree. 2. **Leaves are heterogeneous in format but uniform in role.** A "leaf" here is `{id, name, time-range, geo-anchored media URL(s), payload-type}`. The MISB aircraft leaf and the `alertLive` posed-image record (see companion artifact) are both "a thing with a pose/position, a time, and a media reference" — the catalog should carry a **`type` discriminator** (`alertLive`, `misbFmv`, …) and a **typed payload**, exactly as `event.json` already does with `"type": "alertLive"`. 3. **Normalize at the edge, not in the catalog.** The app proves the pattern: keep multiple on-disk dialects (long/short MISB), normalize to one in-memory contract at load. The catalog should reference raw payloads by URL + declared format, and leave normalization to per-type adapters (the role `MISBTelemetryParser` plays). 4. **Pose is the common currency.** Both families ultimately produce `{lat, lon, alt, heading/yaw, pitch, roll, hFOV, vFOV}` per renderable item (MISB per-sample; `alertLive` per-image `imagePoses`). A shared **Pose schema** is the natural join between the two event families and the thing every view/controller consumes. 5. **Hosting is orthogonal.** Media and telemetry live on whatever host; the catalog only holds URLs. Same as the `alertLive` event, whose images live off-event on `guerin.acequia.io`. **Crosswalk to the `alertLive` event family** (companion artifact): | | MISB FMV (`incidents.json`) | alertLive (`event.json` + collections) | |---|---|---| | catalog unit | aircraft leaf | event (`buooas8…`) | | time | continuous telemetry stream (µs samples) | static `lastupdate` stamps; no playback timeline | | pose | per-sample, interpolated | per-image `imagePoses/` (solved, locked) | | media | MP4 video (live texture) | still JPEGs on `guerin.acequia.io` | | ground tie | `frameCenter` aim point | `geoPoints` ↔ `tiePoints` correspondences | | camera presets | none (live follow) | `scenePresets` (Digit1/2 bookmarks) | | view/controller config in data? | none (hard-coded) | partial (`scenePresets` are view state) | The two already share the **Pose** shape and the **off-host media URL** pattern. A unified catalog = a typed tree of leaves, each leaf = `{id, name, type, timeRange?, poses[], media[], groundRefs?, presets?}`, with per-type adapters supplying the rest. That is the next artifact to draft.
## Appendix — file map of the app | File | Role | |---|---| | `index-misb.html` | entry; DOM + import map; loads `main-misb-multi.js` | | `js/main-misb-multi.js` | app orchestrator (multi-aircraft) | | `js/main-misb.js` | older single-aircraft build | | `js/ui/LayersPanel.js` | reads `incidents.json`, builds tree, flattens leaves | | `js/ui/TimelineControl.js` | master UTC timeline / scrubber | | `js/core/AircraftManager.js` | load/select/fly/update all aircraft | | `js/core/Aircraft.js` | one clip: `<video>` + parser + panosphere | | `js/telemetry/MISBTelemetryParser.js` | **the format authority** — both dialects → normalized samples | | `js/telemetry/TelemetryParser.js` | non-MISB (drone) telemetry variant | | `js/rendering/Panosphere.js` | video-textured sphere + frustum | | `js/rendering/SceneManagerMISB.js` | 3D-tiles globe, camera, geodetic↔local | | `js/config.js` | Google/Cesium tokens, default tile source | | `incidents.json` | **the catalog manifest** | | `check_bearing.py` | offline verification of the gimbal-angle math | | `PLAN.md` | original multi-aircraft implementation plan |