**Note** from Bead: Doorbell Listener · [canonical source](https://redfish.acequia.io/guerin/.agents/883c513f-002f-4156-a845-87322fcf3929/2026-06-11/notes/01-doorbell-platform-roadmap.md) · session 2026-06-11 · discussion: Talk: Doorbell Listener
Workshop note for bead `883c513f-002f-4156-a845-87322fcf3929`. Captures Stephen's 2026-06-11 design brief: grow the single-file FFT detector into a trainable, logged, two-way **butler intercom** with a companion **doorlock**, expressed in the acequia ecology (groups/WebRTC, capability chains, agent-as-resource, bind/mount). The deliverable app lives at [`guerin/apps/doorbell/`](https://redfish.acequia.io/guerin/apps/doorbell/); this note is the plan, not the build. Open questions are flagged ⟶.
## Where we are (Epic 0 — DONE) `index.html` = a hand-tuned **spectral matched filter**: byte-FFT (4096), pick the loudest bin in a frequency window, gate on *tonality* (peak − band-average > 35) and *duration* (~150 ms), debounce 1.5 s. Calibrate-to-my-bell nudges the window+threshold to one observed ring. No learning, no memory, no persistence. Good enough to fire; too brittle to trust (other tonal sounds false-positive; one calibration sample only). The roadmap below is roughly the build order. Epic 1 is the next step Stephen named.
## Epic 1 — Train on *this* doorbell (record many clips, learn the bell) **Goal:** capture many recordings of a specific doorbell in the app and build a profile that recognizes *that* bell, stored locally, selectable, retrainable. **In-app training flow** 1. **Record mode** — "Teach me your doorbell." User rings it N times; the app auto-segments each ring (onset detection → ~1–3 s window) and stores the clips. 2. **Negatives** — capture/ship a small negative set (speech, TV, clatter, silence, other chimes) so the recognizer learns *contrast*, not just the positive. 3. **Build profile** — compute the bell's signature from the positives (see ladder). 4. **Profile library** — many named profiles (front door, back gate, Mom's house), keyed in IndexedDB; one active at a time, or run several in parallel. 5. **Active learning** — confirmed/【false】 events from the log (Epic 2) feed back as new labeled exemplars; profiles improve with use. ### Detection sophistication ladder (the "FFT vs sophisticated training" question) | # | Approach | What it learns | Cost / deps | Specific-bell fit | |---|----------|----------------|-------------|-------------------| | 0 | **Energy + tonality window** (current) | nothing — hand-tuned | trivial, none | poor | | 1 | **Spectral fingerprint / template** — mean magnitude spectrum or set of dominant peak freqs + ratios; runtime = cosine similarity vs template | the bell's *timbre* | cheap, no deps, in-browser | good, few samples | | 2 | **DTW over the spectral-feature sequence** — dynamic time warping of live features against recorded exemplars | the bell's *melody/ding-dong shape over time* | cheap, ~100 lines, no deps | **excellent** for a chime | | 3 | **Feature classifier** — MFCC/centroid/rolloff/flux/chroma/ADSR per frame → small logreg/MLP/k-NN, positives vs negatives | a decision boundary | moderate; TF.js or hand-rolled; needs negatives | good, needs more data | | 4 | **Audio-embedding few-shot** — pretrained log-mel CNN (YAMNet-class) → embedding per clip; recognize = nearest-prototype from the few positives | general audio manifold; few-shot | heavier (model ~MBs, TF.js/ONNX in-browser) | **best accuracy/sample** | **Recommendation — climb, don't leap.** Ship **(1) fingerprint** first as the immediate, dependency-free upgrade over the current heuristic (recordable entirely in-app). Then add **(2) DTW** to capture the *two-tone melody* — this is what makes "my specific ding-dong" robust and is still pure JS, no model. Treat **(4) embedding few-shot** as the high-accuracy option later, behind a toggle, accepting a model dependency. Skip a from-scratch big-data trainer — a doorbell is a **few-shot, one-class** problem (you have a handful of positives of *one* bell), so template/DTW/prototype methods dominate; you don't need, and can't easily get, a large labeled corpus. ⟶ **DECISION (2026-06-11, Stephen):** start Epic 1 at rung **4 — embedding few-shot** (over my fingerprint+DTW rec). Engine = **YAMNet** (MobileNetV1; 1024-d embedding per ~0.48 s frame + 521 AudioSet scores + log-mel spectrogram), run **in-browser via TF.js**. Few-shot recipe: mean-pool a clip's frame embeddings → one 1024-d clip vector; average the positive clips → a **prototype**; recognize live by **cosine(window-embedding, prototype) ≥ threshold** (and above the nearest-negative margin). De-risk: model hosting — the canonical TFHub/Kaggle URLs are auth-gated (probed 403/404), so we **self-host** the model. Build runs in the epic/step workshop at [`2026-06-11/dev/`](https://redfish.acequia.io/guerin/.agents/883c513f-002f-4156-a845-87322fcf3929/2026-06-11/dev/). ⟶ Eval harness: even pure-JS rungs need a tiny **labeled eval set** (held-out positives + negatives) to compare rungs honestly (precision/recall, false-fires/hour). Build it from Epic 2's log.
## Epic 2 — Event log + optional recordings (IndexedDB `doorbell`) **Goal:** durable history of detections; optionally keep the audio. - **DB `doorbell`**, object stores: - `events` — `{id, ts, profileId, confidence, peakHz, durationMs, source}` - `recordings` — `{id, eventId, blob, mime, startTs, durationMs}` (MediaRecorder webm/opus of the ~2–4 s around the event); **opt-in toggle** "log audio". - `profiles` — Epic 1's trained signatures (templates/exemplars/embeddings). - **Log UI** — list events, play back recordings, confirm/reject (→ active learning), export (JSON + clips), retention policy / cap, clear. - Confirmed events become new positives; rejected ring-alikes become new negatives — closes the loop with Epic 1. This is also the eval-set source. ⟶ Privacy: audio logging is sensitive (a hot mic at a door). Default **off**; explicit per-profile consent; local-only unless the user shares; clear retention.
## Epic 3 — Two-way intercom (mic + speaker + camera chat) over acequia groups **Goal:** on detection, open an A/V channel between the **door** (camera+mic+speaker) and a **butler** (remote person/agent). **Substrate (grounded in the platform docs):** - **acequia Groups** = membership, presence, signaling, shared state. The door app and butler(s) join a group; signaling (SDP/ICE) and call state ride `group.setState()` / `stateChanged` and the discovery-server WS relay. See [`group-shared-state-webrtc.md`](https://acequia.io/documentation/platform/projects/group-shared-state-webrtc.md): peer-elected leader, WebRTC-primary with WS fallback, version-reconciled state. - **WebRTC media** rides the same peer connections already used for data/state — add `addTrack` audio+video to the existing `connectToPeer` channels. - **Call model:** door publishes `setState({ ring: {ts, profileId, callId} })`; butlers see it via `stateChanged`; a butler answers → media negotiation over the group; door shows the butler's stream, butler sees/hears the door. ⟶ One door, many butlers: ring is a group broadcast; first-to-answer wins, or a roster/escalation (Epic 5). Leader election + 64 KB state cap are fine for call signaling (small); media is peer-to-peer, not through state.
## Epic 4 — `doorbell.agent` (agent-as-resource, WebDAV interface) **Goal:** when this bead is **referenced/energized**, a `doorbell.agent` handler activates and **presents a WebDAV-navigable interface** — the geo.camera / `supportedCapabilities.json` pattern, applied to the door. - The door is a **resource at a URI**, not a static page (per the ecology's *agent-as-file ducktyping* / *agent-as-bead* notes). Verbs: - `PROPFIND` → state + advertised capabilities (is someone there? last ring? available media? lock present?) via a `supportedCapabilities.json`-style document — **negotiated at interaction time** (OPTIONS/PROPFIND), not a pinned spec. - `GET` → current snapshot / status / latest frame. - `PUT`/route → ring, answer, hang up, (delegated) unlock. - **Energization = referenced GET** (polarized-link / Hubler framing): the agent runs while something points at it; no inbound charge → it apoptoses (correct). The app at `guerin/apps/doorbell/` is the human face; `doorbell.agent` is the machine face of the same resource. - **Permissions:** capability **chains** rooted in the door-owner's key ([`capability-delegation.md`](https://acequia.io/documentation/platform/projects/capability-delegation.md)). "A Camera app sharing a video stream delegates read access via chains rooted in the camera owner's keys" — same model here: a butler gets a scoped, short-TTL chain token to view the door stream; the **headgate** (non-renewal) revokes it.
## Epic 5 — Remote butler routing (realtime.earth / anysurface) **Goal:** a butler chooses *where* the stream renders and *which* devices capture — on their phone, **projected on a wall** (anysurface), using any available mic/camera on the **realtime.earth / anysurface** network. - This is **bind/mount + URI-as-resource** (the ecology's *uri-bind-mount* note): the stream is a resource; render **sinks** (phone screen, projector/any-surface) and capture **sources** (any mic/camera on the mesh) are resources too. The butler *composes a namespace* — "this door camera → that wall surface," "answer with the kitchen mic" — Plan-9 per-caller composition. - realtime.earth = the realtime+spatial substrate; anysurface = the any-surface display/projection mesh. A butler's session binds chosen source/sink resources from those meshes into the call. - **Escalation/roster:** unanswered ring routes to the next butler / next surface (ties to apoptosis: a ring is a signal with a TTL; unanswered → escalate or expire). ⟶ Need to confirm the realtime.earth / anysurface device-discovery + binding APIs (not yet read; likely parallels Groups + the SW VFS mount table in the platform docs).
## Epic 6 — `doorlock` companion app (capability-delegated unlock) **Goal:** a sibling app that **unlocks the door**, triggerable from the intercom by an authorized butler. - **Security is the whole story.** Unlock = a **write capability** on a lock resource, granted by a capability **chain** rooted in the owner's key, **attenuated** (unlock only, not configure), **short-TTL** (minutes), **depth-limited**, **revocable** by non-renewal. Exactly `capability-delegation.md`'s model; never an ambient/long-lived credential. - **Resource shape:** `doorlock.agent` presents a WebDAV interface like `doorbell.agent`; unlock = an authenticated `PUT`/route carrying a valid leaf JWT whose scope covers the lock path and whose chain verifies to the owner root. - **Coupling:** doorbell (sense) and doorlock (actuate) are sibling beads/agents over the same group — the classic sensor→actuator pair; the butler in the loop authorizes the actuation. Could later support policy-based auto-unlock (recognized face/voice + policy) — high-risk, explicitly deferred. ⟶ Physical bridge: what actually throws the bolt (smart-lock API / GPIO / relay) is out of scope here; `doorlock.agent` is the capability-gated front end to whatever hardware bridge exists.
## Dependency order & ecology map ``` Epic 0 (done) ─ detector v1 └─ Epic 1 train-this-bell ─┐ └─ Epic 2 log/record ─┴─ active-learning loop, eval set Epic 3 intercom (groups+WebRTC) ─┐ └─ Epic 4 doorbell.agent ───┼─ Epic 5 butler routing (bind/mount, realtime.earth/anysurface) └─ Epic 6 doorlock (capability chains) ``` Ecology cross-refs (Stephen's design notes under bead `874fce5b…`): **paths-as-event-bus** (ring = PUT/observe on a group path) · **agent-as-bead / agent-as-file** (doorbell.agent, doorlock.agent) · **uri-bind-mount** (source/sink composition, Epic 5) · **polarized-links** (energization by reference, Epic 4) · **apoptosis** (ring TTL, escalation, token headgate) · **advanced-wave** (a butler answering = value out + attention/reputation in).
## Methodology This is multi-epic; per the **epic/step dev methodology** skill, when we start building Epic 1 it gets a non-deploying `apps/doorbell/dev/` workshop (STATE / WORK-QUEUE / journal / decisions / step-NN), with the deployable app staying at its app URI.