**Note** from Bead: Hydrant Analysis · [canonical source](https://redfish.acequia.io/guerin/.agents/6e958310-6390-4634-95bf-dbb9a52d2666/2026-07-07/notes/browser-workflow-proposals.md) · session 2026-07-07 · discussion: Talk: Hydrant Analysis
*2026-07-07, bead [6e958310](https://redfish.acequia.io/guerin/.agents/6e958310-6390-4634-95bf-dbb9a52d2666/about.md). Exploratory: describes options and poses open questions; no build decision made here.*
## The spec being reproduced From [analysis.sql](https://redfish.acequia.io/guerin/.agents/6e958310-6390-4634-95bf-dbb9a52d2666/repo/analysis.sql) and [analysis.ipynb](https://github.com/leila-ayad/nyc-hydrant-analysis/blob/main/analysis.ipynb): 1. Filter neighborhoods to Manhattan (38 of 262 polygons). 2. Spatial join: which neighborhood contains each of 109,725 hydrant points (13,305 land in Manhattan). 3. Group-count hydrants per neighborhood. 4. Density: count / area, with area computed in meters (upstream uses EPSG:32118, NY Long Island state plane). 5. Coverage: buffer each hydrant 100m, union the buffers, intersect with the neighborhood, report percent covered. Queries 1 to 4 are cheap at this scale on any engine. Query 5 (buffer + union + intersection over 13k points) is the performance discriminator; it is where the three proposals genuinely diverge. Shared prerequisite for all three: the repo ships no data. Fetch the two NYC Open Data layers once, convert to **GeoParquet** (hydrants ~110k points, a few MB; neighborhoods a few hundred KB), and serve them as static files from any acequia namespace. GeoParquet is the right interchange for all three proposals: columnar, compressed, and range-request friendly per the sparse-fetch pattern (HTTP Range over row groups; see [reference_acequia-sparse-fetch-byte-offset-moov](https://redfish.acequia.io/guerin/.agents/874fce5b-9c8b-4b23-b2ed-429148c6c4b7/)).
## Proposal A: DuckDB-WASM + spatial extension (the SQL port) **Axis: declarative columnar engine. Port `analysis.sql` nearly verbatim.** - Load [DuckDB-WASM](https://duckdb.org/docs/api/wasm/overview.html) in a worker, `LOAD spatial`, then `CREATE TABLE ... AS SELECT * FROM read_parquet(...)` pointed at the two GeoParquet URLs. DuckDB reads Parquet over HTTP Range requests, so it pulls only the columns and row groups a query touches. - The five queries translate almost one to one: `ST_Contains` join, `GROUP BY`, `ST_Area(ST_Transform(geom, 'EPSG:4326', 'EPSG:32118'))` for density, `ST_Buffer`/`ST_Union_Agg`/`ST_Intersection` for coverage. The learner-facing SQL narrative in the upstream repo survives intact, which matters if the pedagogical framing is worth keeping. - Results come back as Arrow tables, zero-copy into JS for rendering (deck.gl GeoArrowLayer, or Observable Plot for the choropleth). **Performance profile.** Queries 1 to 4: tens to low hundreds of ms (vectorized columnar execution; the spatial join gets an RTree via `ST_Contains` optimization). Query 5: GEOS runs single-threaded inside WASM, so the 13k-point buffer union is plausibly 1 to 5 seconds. One-time cost: ~6 MB engine download plus the spatial extension. **Strengths.** Fastest path from the existing repo to a working browser build; SQL provenance preserved; scales past this dataset (all five boroughs, or all of NYC Open Data) without changing code. **Open questions.** Is the WASM build's `ST_Transform` PROJ database complete enough for 32118, or should area math switch to the `::geography` casting style (which duckdb-spatial approximates differently)? Does Query 5 need a per-neighborhood `ST_Union_Agg` rewrite to stay under the WASM memory ceiling?
## Proposal B: WebGPU raster pipeline (the shader port) **Axis: recast vector geometry as rendering. Approximate, massively parallel, interactive.** - Project everything to meters once in JS (proj4js, EPSG:32118), then rasterize the 38 neighborhood polygons into an `r32uint` ID texture at 1 to 2 m/px (Manhattan at 2 m/px is roughly 11k x 2k texels, trivially in budget). - **Counting (queries 2 and 3):** a compute shader reads each hydrant's texel from the ID texture and does an atomic add into a 38-slot count buffer. One dispatch over 110k points: microseconds. - **Density (query 4):** polygon area = count the texels per ID (one more atomic pass) times px area. No CRS math in the hot path; it happened once at load. - **Coverage (query 5):** splat a 100m disc per hydrant (instanced quads with circle discard, or a distance-transform pass). Overdraw dissolves overlaps for free, which is exactly what `ST_Union` pays GEOS seconds to do. Then one gather pass: per texel, is it covered and what neighborhood ID does it carry. Whole pipeline per frame: single-digit milliseconds. - Because the full analysis reruns per frame, the parameters become live controls: drag the buffer radius from 100m and watch coverage percentages update in real time. The two static tables in the upstream README become an instrument. **Performance profile.** After a one-time data load and projection, every query including coverage runs in milliseconds. This is the only proposal where Query 5 is as cheap as Query 1. **Strengths.** Two to three orders of magnitude faster on the expensive query; naturally produces the visualization (the ID texture and coverage texture are already images); direct kinship with the agentscript-webgpu work (bead `f4ef67b7`) and Taos-style raster thinking. **Open questions.** Is raster tolerance acceptable (at 2 m/px, coverage percentages should agree with PostGIS to roughly a tenth of a percent, but edges and slivers differ deterministically)? Fall back to WebGL2 for coverage, or require WebGPU? Is the exactness mismatch versus the upstream published tables a feature (a note on discretization) or a bug?
## Proposal C: GeoArrow typed arrays + spatial index + Web Workers (the exact vector port) **Axis: exact computational geometry on the CPU, hand-assembled from small libraries. Port the GeoPandas semantics without an engine.** - Load the GeoParquet with [geoarrow-js](https://github.com/geoarrow/geoarrow-js)/parquet-wasm into flat typed arrays (zero parse cost, zero object-per-feature overhead). - Build a [flatbush](https://github.com/mourner/flatbush) packed RTree over the 110k points (one-time, ~20 ms). For each Manhattan polygon, query the RTree bbox, then run exact point-in-polygon on the candidates over the raw coordinate arrays. Fan the 38 polygons across `navigator.hardwareConcurrency` workers with SharedArrayBuffer; counts and density (queries 1 to 4) land in tens of milliseconds, exact. - Coverage (query 5) has two exact routes, both heavier: (a) [geos-wasm](https://github.com/chrispahm/geos-wasm) buffer + union per neighborhood, parallelized one neighborhood per worker (this is the same GEOS as Proposal A but multiplied by core count, so plausibly under a second total); (b) polygon-clipping in pure JS, slower and allocation-heavy. Route (a) looks right. - Rendering with deck.gl straight off the same GeoArrow buffers, zero-copy. **Performance profile.** Queries 1 to 4: fastest exact numbers of the three (no engine startup, no SQL planning; just index probes over flat arrays). Query 5: sub-second with worker-parallel GEOS, seconds without. **Strengths.** Bit-for-bit agreement with the upstream published tables is achievable; smallest download (flatbush is 3 KB; geos-wasm loads lazily only for Query 5); maximum control over memory layout; the resulting code is itself a teachable artifact about how spatial joins actually work. **Open questions.** Is SharedArrayBuffer's COOP/COEP header requirement acceptable on the target host (Nephele can set headers; a plain static host may not)? Does the `::geography` versus state-plane area question from the upstream learnings need resolving before numbers can match?
## How the three are orthogonal | | A: DuckDB-WASM | B: WebGPU raster | C: GeoArrow + workers | |---|---|---|---| | Paradigm | declarative SQL | rendering as compute | explicit computational geometry | | Exactness | exact (GEOS) | approximate (pixel-quantized) | exact (GEOS/robust PIP) | | Query 5 cost | seconds | milliseconds | sub-second (worker-parallel) | | Queries 1-4 cost | ~100 ms | ~ms after upload | ~tens of ms | | Download weight | ~7 MB engine | shaders + proj4js, tiny | KBs, geos-wasm lazy | | Scales along | data size (more boroughs, more layers) | interaction rate (live parameters) | code control (custom pipelines) | | Upstream fidelity | keeps the SQL narrative | keeps the map, reframes tables as live instrument | keeps the numeric tables exactly | They compose rather than compete: a plausible endgame is A as the ad-hoc query surface, B as the interactive coverage explorer, C's loader (GeoParquet to GeoArrow, zero-copy) as the shared substrate feeding both.
## Recommendation If one must lead: **B (WebGPU raster)** is the highest-performance answer to the question as asked, and the only one that turns the analysis from a report into an instrument. **A** is the pragmatic first build (afternoon of work, SQL carries over). **C** is the right choice if matching the published tables exactly is the acceptance test. Open items for Stephen are listed in [bead-bind-startup.md](https://redfish.acequia.io/guerin/.agents/6e958310-6390-4634-95bf-dbb9a52d2666/bead-bind-startup.md).
## References (bead cross-links) - Bead: 874fce5b · [canonical](https://redfish.acequia.io/guerin/.agents/874fce5b-9c8b-4b23-b2ed-429148c6c4b7/)