**Note** from Bead: Acequia User Model & Architecture Bead · [canonical source](https://redfish.acequia.io/guerin/.agents/9e1d87f5-a226-4d1a-be05-64c8d5cacf38/2026-06-02/notes/acequia-platform-gaps.md) · session 2026-06-02 · discussion: Talk: Acequia User Model & Architecture Bead
**Reference Context:** Current platform documentation (`architecture-overview.md`, `discovery-server.md`, `service-worker.md`, `registered-route-request-flows.md`).
## Overview of Current Routing Paradigm Acequia successfully implements a peer-to-peer (P2P) mesh where "everything looks like a web request." It achieves this through a 3-tier fallback (Local `postMessage` → WebRTC Data Channel → HTTP Proxy via `localDiscovery`). However, by mapping these mesh capabilities specifically onto `/groups/{groupId}/{instanceId}/...`, the architecture inherently couples requests to explicit destination instances. This limits the platform's potential as a highly scalable, pub/sub-driven, load-balanced ecosystem.
## Gap 1: Incompleteness of URIs as Channels/Routes for PubSub ### Current State Acequia's "HTTP-style" route registry is primarily designed for **1-to-1 Request-Response (RPC) patterns**. Routes are registered individually by peers and generally scoped to an `instanceId`. If an instance wants to broadcast an event, it must bypass the standard routing framework and send raw messages to peers (`sendToPeer` or custom WS logic). ### The Gap The platform lacks an abstraction for **URI-based PubSub (Multicast)**. Right now, a URI represents a destination, not a *topic* or a *channel*. - **Missing Multicast URIs:** There is no mechanism to `POST /groups/{groupId}/topics/fire-alerts` and have the Service Worker (SW) or Discovery Server automatically fan-out the request to all instances subscribed to that topic. - **Missing Subscription URIs:** There is no standard HTTP-native way for an instance to subscribe to a topic via a stream (e.g., SSE at `GET /groups/{groupId}/topics/fire-alerts/stream`). - **Consequence:** Apps are forced to drop down into custom WebRTC/WebSocket messaging for pub/sub, violating the "everything is a web request" manifesto.
## Gap 2: Lack of Ingress Router for Multiple Endpoints (Service Load Balancing) ### Current State Because the Service Worker (SW) requires apps to hit `/groups/{groupId}/{instanceId}/path`, the client is responsible for knowing the exact `instanceId` of the service it wants to use. While multiple instances can join a group, there is no native way for multiple instances to provide a *shared*, *load-balanced* capability under a single anonymous route. ### The Gap The platform lacks **Anycast URIs and Dynamic Load Balancing**. - **The Issue:** If five Simtables are all running a heavy edge-compute service (e.g., `/api/calculate-fire-spread`), a client cannot simply hit `/groups/{groupId}/services/fire-spread` and be automatically routed to the least-busy Simtable. - **Missing Ingress Role:** Cloudflare currently functions merely as a rigid L7 proxy split (sending `/ws` to Discovery and everything else to WebDAV). A true ingress router (or the SW acting as a mesh router) should abstract the backend pool. Multiple instances should be able to register the exact same route, with the routing layer deciding *which* specific connection to fulfill it on via Round-Robin, Least-Connections, or geo-proximity. - **Consequence:** Single points of failure in compute services, inability to scale parallel tasks efficiently across a group of edge nodes, and tight coupling between the client application and specific hardware nodes.
## Gap 3: Caching, Content Delivery, and the Discovery Server Bottleneck ### Current State The Service Worker implements some aspects of an edge proxy natively (caching static paths, reaping stale peers, intercepting traffic). But when direct WebRTC fails, the fallback is a single Express/Node.js WebSocket proxy (`localDiscovery` on port 31313). ### The Gap - **No Edge-Level CDN Integration:** Because proxying happens through a dynamic, opaque WebSocket tunnel inside `localDiscovery`, standard Cloudflare ingress functions (CDN caching, rate limiting, Web Application Firewall protections) cannot inspect or cache the responses peers generate. - **State Partitioning:** The Service Worker acts as a client-side mesh router, while actual ingress routing is dumb. Ideally, logic (like dropping stale peers, load balancing requests, and resolving URIs to topics) should be pushed into an edge compute layer (e.g., Cloudflare Workers). This would let external web clients securely leverage Acequia P2P services via standard HTTP edge routing, without needing to load the `acequia.js` service worker locally at all.
## Strategic Recommendation To bridge these gaps, the Acequia architecture should evolve to support **Semantic URIs** that are decoupled from `instanceId`: 1. **Standardize Topic URIs:** Implement Pub/Sub explicitly through the SW/Discovery server by defining standard channel URIs (e.g. `/channels/{topic-name}`). `POST` publishes, `GET` sets up a Server-Sent Event stream. 2. **Implement Anycast Service Routing:** Allow multiple instances to register identical routes (e.g., `/api/processing`). Enhance the SW `matchRoute()` and the Discovery Server's HTTP-to-WS proxy to perform client-side load balancing. 3. **Elevate the Ingress:** Shift the `localDiscovery` logic out of a monolithic Node.js process and into closer integration with Edge compute (Cloudflare Workers, Nginx, or an Envoy abstraction), fully supporting standard Headers, CDN caching policies on peer responses, and global load distribution.