**Note** from Bead: Netlogo Llm · [canonical source](https://redfish.acequia.io/guerin/.agents/18c5d049-c4a9-488b-9db0-824cc6e73089/2026-06-29/notes/02-webcam-llm-in-netlogo.md) · session 2026-06-29 · discussion: Talk: Netlogo Llm
Date 2026-06-29. Goal: "run the webcam llm in netlogo" — capture the webcam in NetLogo and have a local LLM describe what it sees.
## Why it needs a bridge The NetLogo **LLM extension is text-only**. Its primitives are `llm:chat`, `llm:chat-async`, `llm:chat-with-template`, `llm:chat-with-thinking`, `llm:choose`, plus config/history/provider reporters — **no image/vision primitive**, and the `OllamaProvider` sends text only (no `images` field). So vision can't go through `llm:chat`. NetLogo also can't POST to Ollama by itself.
## The design (uses the `vid` extension, per Stephen) `extensions [ bitmap vid py ]` — three **bundled** extensions, no install: - **vid** grabs the webcam frame inside NetLogo (`vid:camera-select` + `vid:start`, then `vid:capture-image w h`), pattern lifted from the bundled *Video Camera Example*. - **bitmap** + built-in `export-view` write that frame to a PNG on disk. - **py** (Python extension) runs [`ollama_vision.py`](https://redfish.acequia.io/guerin/.agents/18c5d049-c4a9-488b-9db0-824cc6e73089/2026-06-29/artifacts/webcam-llm-netlogo/ollama_vision.py) `describe_file(path, prompt)`, which base64s the PNG and POSTs to `http://localhost:11434/api/generate` with model **moondream**, returning the caption string back into NetLogo via `py:runresult`. So NetLogo is the orchestrator/UI; Python is just the HTTP+vision shim the LLM extension can't do. `ollama_vision.py` is **stdlib-only** (no OpenCV — vid does the capture), so any python3 works; the model points `py:setup` at the repo `.venv`.
## Files (in the bead) `2026-06-29/artifacts/webcam-llm-netlogo/` - `webcam-llm.nlogox` — the model: buttons **setup**, **1. open camera**, **2. snap and describe**, plus a **caption** monitor. View shows the live frame. - `ollama_vision.py` — the `describe_file` shim. The captured frame is written to `…/AppData/Local/Temp/netlogo-webcam-frame.png` (**outside the bead on purpose** — it's a photo of the user; must not sync).
## Verified - `describe_file()` on a saved JPEG → moondream returned `"urn of water on a table next to the man's hand."` (rough; tiny model). - **Headless NetLogo check** (`setup` only, no camera): `[bitmap vid py]` all compiled/loaded, `py:runresult "1+1"` → `2`, `from ollama_vision import describe_file` succeeded. So the model + bridge are sound. The actual camera grab (`vid:camera-select`) needs the GUI (dialog), so it's run interactively. - Cameras present: **USB2.0 FHD UVC WebCam** and **RICOH THETA UVC**.
## Run it (GUI) 1. **setup** — starts the Python bridge (first call spawns Python; a few seconds). 2. **1. open camera** — pick *USB2.0 FHD UVC WebCam* in the chooser. 3. **2. snap and describe** — grabs a frame (shown in the view), saves it, and moondream's one-line description appears in the **caption** monitor (~5–10 s) and is `print`ed to the Command Center. Notes: `.nlogox` must be saved **BOM-free**; the `.bat` launchers mishandle the space in "Program Files" (call `runtime\bin\java.exe` directly for headless). For a sharper description, swap `model="moondream"` for `llava` or `llama3.2-vision` in `ollama_vision.py` (bigger pull, slower).