gobed: GPU vector search as a service
The search engine we built for app.nz, offered as an addon. A 15 MB int8 embedding model, exact and IVF/HNSW indexes on CPU, and cuVS CAGRA graph search on GPU — behind three HTTP endpoints and one injected env var. It already powers character, art, audio, and repo search across this site.
why it is fast
Static embeddings + the right index per scale
gobed embeds with static retrieval embeddings (token vectors + mean pooling — no attention pass), quantized to int8 at 512 dimensions: a 15 MB model that embeds at ~150K texts/sec on CPU and needs roughly 600 bytes per stored vector, so a million documents fit in ~600 MB. The engine then picks the index that fits your corpus:
Flat
Exact brute-force with SIMD (AVX2/arm64) dot products. Default under ~1.5K vectors — zero recall loss.
IVF + HNSW routing
K-means inverted lists with an HNSW graph over centroids. ~3× faster at scale for ~10% recall trade, tunable via nprobe.
CAGRA on GPU
NVIDIA cuVS CAGRA graph index with custom fused CUDA kernels, GPU k-means, and a pooled memory manager. Auto-enabled when CUDA is present.
| Corpus | Index | Latency | Throughput / quality |
|---|---|---|---|
| 1K docs | flat (exact) | 357 µs | ~2,800 QPS |
| 100K docs | flat (exact) | 2.23 ms | NDCG@10 > 0.99 |
| 100K docs | IVF | 417 µs | ~2,400 QPS |
| 500K docs | IVF | ~1.4 ms | ~715 QPS |
| 1M docs | GPU batch | — | ~1,050 QPS sustained |
Measured on our benchmark suite (RTX 3090 for GPU rows); methodology ships in the repo. Sub-millisecond CAGRA search at 100K+ docs is the current optimization target, not yet a shipped measurement — we publish the distinction on every number.
the addon
Three endpoints, one env var
| Endpoint | Purpose |
|---|---|
| POST /search | single or multi query, k, timeout_ms → results with similarity scores |
| POST /batch_search | high-throughput query batches |
| POST /index | index documents [{id, text}], sync or async |
| POST /batch_index | bulk ingestion |
| GET /health · /metrics · /gpu_stats | liveness, Prometheus metrics, GPU utilization |
Injected environment
GOBED_SEARCH_URLHTTP endpoint of your provisioned gobed instance (POST /search, /batch_search, /index)
GOBED_API_KEYBearer token scoped to this addon · secret
GOBED_INDEXDefault index/namespace name for this app
curl "$GOBED_SEARCH_URL/search" \
-H "Authorization: Bearer $GOBED_API_KEY" \
-d '{"query": "cozy sci-fi reading nook", "k": 5}'Shared CPU
metered
int8 CPU engine on shared workers, per-request billing
Dedicated CPU
40 credits/mo
pinned instance, ~600 bytes/vector, up to ~5M vectors
GPU (CAGRA)
from $0.228/hr
per-second billed GPU pod with cuVS CAGRA graph index
dogfood
It runs this site
gobed is not a demo we stood up for a landing page. The in-process int8 engine indexes app.nz characters, art, audio, documents, and repo code today, and the GPU sidecar serves/api/search-ais semantic character search with a SQLite fallback when it is absent — the exact architecture the addon gives you: fast path on GPU, graceful degradation, one env var to wire it.
Semantic search without the platform tax
No per-seat pricing, no minimum clusters — a metered index on shared CPU, or a per-second GPU pod when you need CAGRA throughput.