SE Talk Track

Most 'AI vision' demos show a static image upload. This shows live camera inference — point your phone at real objects and Workers AI finds them in about a second, running entirely at the edge. Three use cases on the same model family make a real pricing/capability tradeoff visible: DETR is 80 fixed COCO classes for $0.0000075/inference (essentially free at any volume); Moondream is open-vocabulary — describe ANY object in plain English, caption a scene, or extract structured JSON from a receipt/business card/document — for $0.30/M input + $1.00/M output tokens, about 100x more per call but far more flexible.

Demo Steps

1 Tap "Start camera" and grant camera permission — works best on mobile with the rear camera
2 Fast mode: point at a person, chair, laptop, or any of 80 COCO classes — DETR draws boxes in ~200-400ms
3 Switch to Smart mode, type a target like "red water bottle" or "coffee cup" — Moondream finds that specific object even though it's not a fixed class
4 Switch to Caption mode — Moondream describes the whole scene in a sentence, no target needed
5 Switch to OCR mode, pick a document type (receipt, business card, document, auto), point the camera at a real document and tap "Scan document" — Moondream extracts structured JSON in one shot, no continuous polling
6 Point out the stats bar: latency per frame, and real token usage for Moondream calls
7 Key message: this is near-real-time (≈1.2s polling loop) for the first three modes, not true 30fps video — round-trip AI inference latency makes 30fps unrealistic, and that's an honest tradeoff to discuss

Cloudflare Products

Workers AI@cf/facebook/detr-resnet-50@cf/moondream/moondream3.1-9B-A2B

"Isn't this just image classification?" — No: DETR and Moondream detect return bounding box coordinates for multiple objects per frame, not a single whole-image label. Moondream specifically supports open-vocabulary detection — you can ask it to find something that was never in its fixed training classes, described in plain English.

📷 Retail / Logistics / Accessibility

Live Object Detection

Point your phone camera at anything — Workers AI detects objects at the edge, near-real-time. Two models, one endpoint: fast fixed-class detection or open-vocabulary 'find my red mug'.

The Problem

""We want visual inventory checks, shelf monitoring, or accessibility scene description — but running our own vision model means GPUs, a model server, and per-region latency.""

The Outcome

Workers AI vision models, zero GPU to provision. Fast mode (DETR) for fixed classes, Smart mode (Moondream) for open-vocabulary detection.

Live demo below

Productionising this

What changes when you ship this for real

Frame rate vs cost

The 1.2s poll interval balances responsiveness against Workers AI cost. Fast mode is cheap enough to poll faster (every 500-800ms); Smart mode should stay slower given the ~100x cost difference per call.

Camera permission UX

Browsers require HTTPS for getUserMedia() and show a permission prompt on first use. Always handle NotAllowedError gracefully — users deny camera access more often on unfamiliar sites.

Bandwidth

Each frame is compressed to JPEG at 0.7 quality, ~640×480 — typically 15-40KB per frame. At 1.2s intervals that is roughly 1-2 MB/minute of upload bandwidth, fine on WiFi and most mobile connections.

Rate limiting

Add a KV-backed rate limiter per session/IP before this ships publicly — a malicious client could hammer the endpoint at a much faster interval than the UI enforces.

Coordinate normalisation

DETR returns absolute pixel coordinates relative to the exact image size sent. Moondream may return normalised (0-1) or pixel coordinates depending on model version — this demo defensively detects which convention was returned rather than assuming one.

True video streaming

For genuinely continuous video analysis (not periodic snapshots), pair this with Cloudflare Stream or Realtime SFU to pipe a live feed to a server-side decoder — a materially bigger architecture than the client-side capture loop used here.

OCR JSON reliability

The model is prompted to return JSON but nothing enforces that contract — this demo parses defensively and falls back to the raw string with a parseError flag if JSON.parse() fails. More importantly: never trust extracted totals, amounts, or dates without human verification for anything transaction-critical (invoicing, payments, tax filings) — vision-language models can hallucinate digits that look plausible but are wrong.