**Note** from Bead: Stac Manager Nextgen · [canonical source](https://redfish.acequia.io/guerin/.agents/b73189b7-e605-4473-9789-20bcff57ba35/2026-06-21/notes/01-storage-architecture.md) · session 2026-06-21 · discussion: Talk: Stac Manager Nextgen
> Prepared by the senior-architect role binding the [`b2d9f6e1` senior-architect](https://redfish.acequia.io/guerin/.agents/b2d9f6e1-4c3a-4e58-8a1f-7d0c5b9e2a64/) frame (data/STAC plane + namespace-bus plane). > Design survey foregrounding questions, not a settled spec.
## 0. The reframe (one abstraction, not three) Three things are tangled today: (1) `loadStac(urls)` does raw network-only `fetch(catalog.json)`; (2) the EGP bookmarklet's `storagePut/List/Mkdir` already proved "path is the address, backend is a swappable transport" (FSA local | WebDAV+Bearer); (3) `geo.camera/acequia` already does WebDAV signaling + IndexedDB identity + SW-mediated background work, and `storage.js` has a FileCache/Settings layer. Unify: **a catalog is a STAC tree read from a StorageLocation**, and a StorageLocation is *any* of {domain/URI, local FS folder, OPFS}. Same `get/put/list` verbs across all three (project_agent-as-file-ducktyping, project_uri-bind-mount). **acequia.js + acequia-sw.js** are the transport that makes a domain/URI location behave like a local one by staging it through OPFS + a named Cache.
## 1. The StorageLocation model ```js StorageLocation = { id, label, kind, // 'webdav' | 'http' | 'fsa' | 'opfs' root, // webdav/http: origin+base; fsa: IndexedDB key of FileSystemDirectoryHandle; opfs: subdir cacheName, // named Cache bucket: "stac::<id>" auth, // {mode:'bearer', tokenRef} | {mode:'cookie'} | {mode:'none'} (tokenRef = handle into acequia.keys, NOT raw string) sync, // {mode:'cache-first'|'network-first'|'offline-only', ttlSec, pinned} writable, added, lastUsed } ``` Four `kind`s collapse to two physical capabilities the driver must expose: a **byte-range GET** and (optionally) **PUT/MKCOL/LIST**. Everything else is policy.
## 2. Minimal driver interface (generalize EGP's storagePut/get/list/mkdir) ```js interface StorageDriver { get(path, {range, signal}) -> Response // real Response so .json()/.blob()/stream work list(path, {depth=1}) -> [{name, path, isDir, size, mtime}] // PROPFIND | dir.entries() | OPFS iterate stat(path) -> {size, mtime, etag} | null put(path, data, {contentType}) -> {etag} // optional; throws NotWritable mkdir(path) -> void ; remove(path) -> void capabilities() -> {writable, range, list, background} } ``` - **fsa** — `getFileHandle().getFile()` / `dir.entries()` / writable stream (EGP `localPut` generalized; Chromium; handle persists in IndexedDB). - **opfs** — same shape rooted at `navigator.storage.getDirectory()`; always available, origin-private. The **staging tier** (§4). - **webdav/http** — fetch with the location's auth; but the UI calls the **acequia-routed** driver (§3) so reads go cache→OPFS→network and writes queue for sync-back. `loadStac` change: `const json = url => driver.get(url).then(r=>r.json())` instead of bare `fetch`. Listing catalogs = `driver.list()` at root + `loadStac` per `catalog.json`. Items read uniformly because `get()` returns a `Response` on every backend. Events on the StorageManager (EventTarget): `location-added/removed`, `sync-progress {id,done,total,bytes}`, `sync-complete`, `offline-ready`, `auth-required`, `quota-pressure`.
## 3. Domain/URI routed through acequia.js + acequia-sw.js *"domain uri can be handled by acequia.js and acequia-sw.js to send it to OPFS and then named cache for later cache syncing."* ``` UI (STAC-manager) driver.get('cameras/.../catalog.json') │ acequia.js (main thread): resolve location by URL prefix; attach bearer (from acequia.keys); fetch()→intercepted │ ┌───────────────── acequia-sw.js (service worker, scoped to registered origins) ─────────────────┐ │ 1. caches.match("stac::<id>") ── HIT ─────────────────────────────────────────────┐ │ │ │ MISS │ │ │ 2. OPFS read opfs:/staged/<id>/<path> ── HIT (offset-write ranges; moov faststart) ┤ │ │ │ MISS │ │ │ 3. network fetch(origin,+auth) → stream to client; tee→OPFS; on complete→caches.put │ │ └────────────────────────────────────────────────────────────────────────────────────┴───────────┘ │ │ Response to UI OPFS + named Cache "stac::<id>" ``` **Read path (cache-first):** named Cache → OPFS staged → network. Small JSON (catalog/collection/item docs) → directly in the **named Cache** (atomic, fast). Large binaries (FMV `.mp4`, imagery) → **range-fetched into OPFS** with offset writes (the reference_acequia-sparse-fetch-byte-offset-moov pattern: HTTP Range + MOOV faststart as the chunk map + OPFS offset-writes), **promoted to the named Cache only when complete** (Cache never holds a torn object). **Why both tiers:** OPFS = mutable, range-addressable scratch for partial/streaming downloads (survives reload). Named Cache = immutable, atomic, request-keyed snapshot = the unit of "this catalog is offline-ready"; per-location `cacheName` lets you pin/evict a whole catalog (`caches.delete('stac::<id>')`). **Sync-back (writable):** UI `put()` → online: PUT through + update Cache. Offline/fail: write OPFS `opfs:/outbox/<id>/<path>` + register Background Sync (`acequia-syncback`); SW `sync` handler drains outbox (PUT each, move outbox→staged, `caches.put`). Generalizes geo.camera's SW background-poll from "poll for peers" to "drain outbox + refresh pinned catalogs."
## 4. acequia-sw.js responsibilities (net-new; generalizes geo.camera service-worker.js) 1. **Scope registration** — main thread `postToSW({type:'register-location', loc})`; SW keeps the origin/prefix→{cacheName, sync, tokenRef} map and rehydrates it from IndexedDB on activate (SW state is ephemeral). Only registered prefixes are intercepted. 2. **`fetch` handler** — the cache→OPFS→network ladder of §3 incl. **Range** support (serve from OPFS slices or forward range upstream). 3. **Auth injection** — attach `Authorization: Bearer <token>` from the `tokenRef` capability on network fall-through; SW holds a **short-TTL** materialized token, not a long-lived secret. 4. **Promotion** — tee network → OPFS; on completion `caches.put`; emit `sync-progress`/`offline-ready`. 5. **Background sync** — drain write outbox; refresh `pinned` catalogs so an offline analyst has fresh data. 6. **Quota** — watch `navigator.storage.estimate()`; evict LRU non-pinned Cache buckets, then OPFS staged ranges; emit `quota-pressure`. 7. **Client messaging** — reuse geo.camera's `postMessage` channel. - **Path collision:** `/sw.js` is a reserved acequia-platform path (santafe.live CLAUDE.md). Name it `acequia-sw.js`, register with explicit app-path `scope`, confirm no fight with the platform SW (open Q5).
## 5. Offline / PWA - `sync.pinned=true` → fully walk+stage on add (or "Make available offline"): tree JSON→Cache, binaries→OPFS→promote. Then the manager opens with no network. - App shell (HTML/JS/CSS/Leaflet) in its own `app-shell` precache bucket. - `offline-ready {id}` flips the location badge to "✓ offline"; `caches.keys()` on boot shows what's mirrored before any network. - FSA/OPFS are inherently offline; only webdav/http need the staging machinery.
## 6. Credential placement (per location) Per feedback_acequia-as-not-no-as, feedback_browser-is-a-parciante-node, reference_token-minting-pattern — keys in nodes not the commons; placement = scope × blast-radius × revocability: - Each `webdav` location carries `auth.tokenRef` = a **handle into `acequia.keys`** (capability object), not a raw bearer in plain IndexedDB. The token is **scoped to that location's subtree** and **attenuated** via `/auth/create-token` — minimal blast radius, cascade-revocable. - SW gets a short-TTL materialized token (or re-requests per fetch-batch); the main thread is the parciante node holding the durable capability. - `cookie` mode for same-site origins (per `storage.js` AcequiaAuth). Read scopes default anonymous where the origin allows (santafe.live, `/taos`) — token only attaches on 401 or for writable locations.
## 7. Composition with the namespace-bus + open questions A StorageLocation is a **binding** (URI→resource) and the registered prefix set is a **mount table** (project_uri-bind-mount); many backends can resolve the same logical catalog path — the SW route map is the per-caller composed namespace (project_distributed-origin-architecture). The [`4c6470f9` bead-orchestrator](https://redfish.acequia.io/guerin/.agents/4c6470f9-b74a-4aba-b436-5e998f469cce/) is the natural owner of a shared location registry + leader election for the authoritative mirror. **Open questions for Stephen:** 1. **Cache vs OPFS cut** — JSON→named Cache, binaries→OPFS-staged-then-promoted (proposed)? Or everything in OPFS with Cache only as the offline snapshot? 2. **Registry per-device or commons-synced?** Promote `storage-locations.json` into the commons (under the user's `.agents/` or an acequia doc) so locations follow the user across devices, owned by `bead-orchestrator`? Raises sync/leader questions. 3. **Sync-back conflict policy** — ETag/If-Match optimistic concurrency vs last-write-wins? Or is the manager **read-mostly** (foragers produce; manager reads/pins; write-back is the rare pose-edit)? 4. **One acequia-sw for all apps, or per-app?** Single platform SW with per-app cache buckets (orchestrator vision) vs per-app SWs sharing the OPFS tier — weigh scope/`/sw.js` collisions. 5. **How far to push range-fetch now?** Whole-object staging v1 (simple) + range-staging later for FMV scrub, or range-first because video dominates payload? ### Seed code (real, on disk) vs net-new - Real: EGP `storagePut/...` (`35e928bc` note 00 §4); `geo.camera/acequia/acequia.js` (WebRTC+WebDAV class); `geo.camera/acequia/service-worker.js` (background poll to generalize); `guerin.acequia.io/acequia/storage.js` (IndexedDB cache/settings/auth). - **Net-new / partly aspirational:** `acequia.org/guerin/acequia.js` re-exports a richer module (`acequiaStore`, `postToSW`, `routes`, `getKeys`) **not present in the tree**; `acequia-sw.js` does not exist. Build the driver/SW layer on the geo.camera class + `storage.js`; don't assume the richer module is implemented.
## References (bead cross-links) - Bead: Senior Architect · [canonical](https://redfish.acequia.io/guerin/.agents/b2d9f6e1-4c3a-4e58-8a1f-7d0c5b9e2a64/) - Bead: Orchestrator · [canonical](https://redfish.acequia.io/guerin/.agents/4c6470f9-b74a-4aba-b436-5e998f469cce/)