Blog
Building the AI agent cloud
Practical notes on agents, model routing, inference, deploys, GPU infrastructure, evals, and the systems work behind app.nz.
Building Polygonalize: stable low-poly video with Go, WASM, Canvas, and Three.js
How we built a free open-source image and video polygonalizer: edge-aware meshes up to 20,000 triangles, custom transparent primitives, stable video topology, Go/WASM, Canvas, Three.js, a CLI, and an app.nz serverless fallback.
Throughput on a shared GPU: fused kernels and a priority scheduler
How we serve chat, image, video, TTS, STT and a LoRA army on shared GPUs — 2× LLM tok/s (fp8+MTP), a fused multi-LoRA kernel (up to 2.4×), a 10.4× admission gate, tier-based VRAM arbitration, and cheaper building blocks (916k embeds/sec, Gemini STT).
Packaging AlayaWorld: an interactive video world model as a self-hosted Cog
Wrapping AlayaLab’s autoregressive world model (LTX-2.3-derived DiT + gemma-3-12b + Depth-Anything-3) as a Cog: camera presets, a warm engine, and an honest look at the LTX-2 Community License.
From Artificial Analysis to production: the useful eval loop
How to turn Artificial Analysis, Arena, LiveBench, SWE-bench, and open leaderboards into a runnable shortlist, workload eval, and production model decision.
Comfy models: from media evals to working GPU inference
Connect image and video benchmark evidence to hosted FLUX, SDXL, LTX, and Wan workflows, reproducible JSON, model provenance, and a global R2 weight cache.
Anatomy of an AI compute marketplace: tokens, GPU-seconds, and agent-hours on one set of rails
A deep tech dive on app.nz as a compute marketplace: 232 models across 24 providers with price-ordered self-healing failover, GPU placement that learns cold starts and arbitrages clouds under a fixed price, margins enforced as unit tests, and an agent layer that makes the whole market liquid.
The next ten years: minds as products, machine-speed markets, and why big labs are the new big tech
A ten-year thesis: AI minds with personality and skills become a product category, robots become software subscriptions, marketplaces go high-frequency and low-fee, humans and governments get augmented, and vertically integrated labs become the new big tech — with sources, and the app.nz strategy behind it.
1ms web search: gobed int8 embeddings, a self-improving index, and agent tools for flash-tier models
How app.nz search answers repeat queries in ~1ms: int8-quantized gobed embeddings at ~600 bytes/doc, a learned index fed by every search and research run, and agent tool design that cheap fast models like DeepSeek actually use well.
Five-minute training runs: capped QLoRA slices and autoresearch on RunPod
Why we cap every training run at 5 minutes: chained resumable QLoRA micro-runs, a GPU-minute budget ledger, a pod reaper, and a karpathy-style hyperparameter autoresearch loop.
Designing animation asset galleries for agents
A stable search and download contract for motion, VFX, terrain, 3D objects, REST, CLI, and MCP clients.
From reference image to indexed, downloadable 3D object
How the app.nz image-to-3D gallery connects reference images, reconstruction recipes, OBJ downloads, and production generation jobs.
Opening the app.nz motion, VFX, and terrain commons
Three pose-capture backends, retargetable character motion, configurable Three.js effects, deterministic terrain, and procedural object generators — open, searchable, and available through web, CLI, API, desktop, and MCP.
Kimi K3: the largest open model, live on the app.nz gateway
Moonshot's 2.8T-parameter Kimi K3 with a flat-priced 1M context window is available now as kimi-k3 — direct Moonshot routing, automatic failover, one metered key.
polyserve: in-process serverless that packs hundreds of apps into one process
Announcing polyserve, the open-source in-process serverless host behind app.nz: per-second billing via busy_ms and warm_s, hot-swap deploys, scale-to-zero, and autoscaling to Hetzner and RunPod GPUs.
Inside the polyserve Go host: fasthttp, unix-socket slots, and zero-drop hot swaps
How one fasthttp process serves many compiled Go apps with microsecond proxy overhead, atomic generation swaps with a 5s drain, and Postgres addons via injected DATABASE_URL.
Serving tiny ONNX models serverlessly, in-process
Why in-process beats a dedicated model server for small ONNX models: warm sessions in one asyncio host, sub-500ms cold starts, honest per-second metering, and RunPod GPU autoscaling for bigger models.
polyserve deploy docs: Go, Python, and Bun quickstarts
Per-language deploy quickstarts for polyserve spaces plus the full host control protocol: deploy, usage, health, routing, scale-to-zero, env vars, addons, and the autoscaler contract.
Interactive world models on app.nz: GPU sessions, ABot-World, and ARDY
World models need a GPU that stays up and talks over a socket. How the new session-cog protocol works, the two open-source world models you can drive today, and the harness that keeps every cog README honest with real runs.
What is an AI agent cloud?
Why app.nz is broader than a coding agent, model gateway, inference provider, or app host—and how one integrated loop changes the work of shipping AI software.
Hardening route HTML, desktop deep links, and notebook paths
A small production hardening pass: serving generated SEO HTML, keeping desktop notebook paths inside the workspace, and fixing one-pass deep-link URL encoding.
How app.nz hosted training works
The control plane behind app.nz fine-tuning: model catalogues, hardware offers, durable jobs, signed trainer specs, progress callbacks, R2 artifacts, publishing, and deploys.
Keeping training GPUs busy without wasting memory
The app.nz training worker checklist: conservative VRAM floors, bf16/TF32 defaults, efficient attention, dataloaders, direct artifact I/O, torch.compile tradeoffs, and phase-by-phase memory release.
Deploying Rust in under five seconds
Fast Rust deploys come from separating build time from release time: compile once, push a small runtime image or static bundle, then publish by swapping bytes or image refs.
How we built isolated containers for agents and CI
The app.nz isolation model: per-run Docker networks, per-step containers, service sidecars, temp Docker auth, label cleanup, timeouts, and remote machines for bigger blast-radius boundaries.
How app.nz runs GPUs on demand
Inside the GPU lifecycle: hardware catalogues with VRAM and compute capability, Cog cold starts, local GPU headroom, RunPod pods, schema introspection, metering, and idle reaping.
Why Docker cold starts are slow
A cold start is scheduling, image pull, Python import, CUDA init, weight loading, compilation, readiness, and first inference. Here is how app.nz reduces each part.
Dynamic routing across model providers
How the app.nz gateway resolves aliases, classifies auto routes, chooses BYOK or platform keys, walks fallback chains, adapts provider APIs, and meters usage from one catalogue.
Serverless vs serverful GPU inference
The GPU router behind app.nz cogs and Comfy deployments: local headroom, RunPod serverless, dedicated pods, hysteresis, single-flight cold starts, and when each tier wins.
Serving static sites from R2 and a database
The fast path behind app.nz static deploys: upload changed files, prune removed paths, index them in the database, and serve bytes directly from R2 or local storage.
Build Studio: private container builds for app.nz
How app.nz turns a Dockerfile and build context into a private registry image, with scoped Docker auth, namespaced image refs, redacted logs, and a clean release boundary.
appnz.yaml: the runtime contract
Why app.nz separates static and server runtimes with a small config file instead of guessing forever from package.json, Cargo.toml, dist folders, or Dockerfiles.
CI service containers without shared state
Inside app.nz CI: per-run Docker networks, Postgres/Redis/Mongo service aliases, per-step containers, readiness checks, timeouts, and label-based cleanup.
How coding agents become durable worker jobs
A coding-agent prompt becomes a stored task, a leased worker job, an event stream, and durable artifacts instead of a long HTTP request that disappears on failure.
Metering the model gateway without guessing
How app.nz meters token streams, image responses, media jobs, BYOK traffic, failures, and routed provider/model pairs from the same catalogue used for pricing.
BYOK provider keys in the model gateway
How app.nz lets users bring provider keys while keeping routing unified, validating providers, preferring user keys, hiding full secrets, and avoiding open-relay behavior.
Making Cog containers self-describing
How app.nz warms Cog containers, waits for real readiness, reads health and OpenAPI endpoints, and turns model schemas into typed prediction forms.
Mirroring ComfyUI models to make cold starts boring
Why app.nz mirrors popular Comfy model weights into R2 so GPU workers fetch predictable artifacts instead of relying on third-party hosts during cold starts.
RunPod serverless endpoints under the hood
How app.nz creates scale-to-zero GPU endpoints with bounded workers, queue-delay scaling, endpoint reuse, job polling, image updates, and explicit failure handling.
The lifecycle of an app.nz cloud machine
Provisioning is more than create: app.nz tracks provider ids, machine status, ready times, termination, simulated providers, and cleanup for CPU and GPU infrastructure.
Metering notebooks by the minute
How app.nz notebooks run free in the browser or on metered CPU/GPU sessions with pricing from the same endpoint, idle stop, balance stop, and durable session rows.
Queues, visibility timeouts, and dead messages
The queue primitive behind app.nz background work: receive as a lease, ack on success, retry after visibility timeout, and move poison messages to dead state.
Durable artifacts for coding agents
Why app.nz stores agent diffs, logs, previews, screenshots, and generated files as artifacts so a run remains auditable after the worker shuts down.
Cheaper GPUs by scheduling: multi-cloud arbitrage, hidden cold starts, and cogs that learn their own footprint
How app.nz cog hosting cut serving costs: community-first multi-cloud provisioning under a fixed price, serverless hedging that hides pod cold starts, and per-model learned stats (cold-start EMA, peak VRAM) that feed routing and future GPU bin-packing.
Hosted addons: Postgres with pgvector + PostGIS, and gobed GPU vector search
Attach managed services to any app or agent in one command — gobed GPU CAGRA vector search, PostgreSQL with HNSW vectors, PostGIS and graph traversal, Mongo, cache, auth, and AI monitoring — with env vars injected into every runtime.
Run cloud agent tasks on your own computers with app worker
Fire an agent task from your phone and let your desktop code it: app worker turns any machine you own into a private, user-scoped worker with local engines — our codex fork (mainline fallback), claude, cursor-agent, gemini, grok.
We open-sourced our investor room
The app.nz data room is now public pages: the 10-year vision, a full technical deep dive (architecture, ~1 ms routing, security, economics, benchmarks, roadmap, risks), all versioned in the repo.
Will it fit? An AI agent that 3D bin-packs your life into a van
A chat agent that estimates real item sizes, runs 3D bin-packing algorithms, and renders interactive three.js loading plans — with an honest verdict when it will not fit. Plus a free /api/packing/solve API.
Hosted notebooks: marimo in the browser or on cloud GPUs, billed per minute
app.nz notebooks run free on Pyodide in your browser or on per-minute CPU/GPU machines that stop themselves when idle. Plus SQLite, Parquet, and agent-trace dataset viewers with HTTP range reads.
app.nz is now an MCP server
Point Claude, Codex, or any MCP client at app.nz/mcp and your agent can run every gateway model — chat, images, embeddings — plus search the MCP directory, through one connector and one API key.
Cloud coding agents: one prompt, one pull request
Run coding agents in sandboxed cloud environments instead of on your laptop — provider-aware (Codex, Claude Code, or our runner), skill-attachable, triggerable by schedules and webhooks.
Instant static deploys: from directory to URL in one command
app.nz hosts static sites behind a URL instantly — app sites deploy uploads, diffs, and prunes a directory so a deploy is the exact state of your build, scriptable into CI or an agent run.
A searchable directory of 120+ MCP servers
What the Model Context Protocol is, how to wire a server into Claude or Codex, and a new searchable index of 120+ MCP servers filterable by category and pricing.
Test-time training is a linear attention operator in disguise
A small reproduction of a 2026 paper showing that key-value-binding TTT unrolls into a linear-attention state, plus sweeps for extra inner steps, query mismatch, gradient sign, and momentum.
An attention sink can mean two opposite things
A small reproduction of a June 2026 paper showing that the same attention stripe can be either a no-op or a broadcast channel.
The best coding models in 2026: a hands-on comparison
How to pick a coding model by the job — frontier vs agentic-mid vs fast — measured on cost per merged PR, not cost per token.
Cheap and fast: choosing budget LLMs for high-volume work
A 3-step workflow to find the cheapest model that still clears your quality bar — measured as cost per acceptable output.
How to optimize prompts with evals (LLM-as-a-judge)
Treat prompts like code: score them with an eval, then let the optimizer search for a better one against your own dataset.