Technical Deep Dive

app.nz is an integrated agent cloud: hosted git repos, autonomous coding agents, an OpenAI-compatible gateway routing 20+ model providers, GPU inference and training, app/site hosting, task queues and schedulers, web and paper search, and a dozen AI-native editors — billed from one prepaid credit balance under one API key.

The strategic claim is that agent traffic structurally prefers integrated platforms: an agent session touches repos, models, media, compute, and deploys in minutes, and consolidating that onto one account removes friction agents will not tolerate. The technical claim is that a platform seeing the whole loop can optimize it — routing every request to the best model and cheapest capable GPU inside a millisecond-scale decision budget — creating compounding margin no single-product competitor can match.

Every row is live. Links go to the product, not a screenshot of it.

The surface is wide by design: each product is a tool an agent can call, and the value of the platform scales with the number of tools reachable from one account. Live consumer proof points built on this stack: papers.app.nz, readingtime.app.nz, helix.app.nz, gpubrain.app.nz.

Control plane

A single account system owns identity, API keys, prepaid credits, spend caps, and permissions. Every product — gateway call, agent minute, GPU second, deploy, search — meters into the same ledger. This is the load-bearing decision in the company: one ledger is what makes agent spending governable and cross-product bundling possible.

Execution planes

Work runs on three planes. The request plane (gateway, search) is latency-critical, stateless, and horizontally scaled. The job plane (agents, builds, training, media renders) is queue-fed, checkpointable, and scheduled onto the cheapest capable capacity. The serving plane (deploys, hosted models, sites) is long-lived, health-checked, and migratable between providers. Separating the planes lets each be optimized on its own axis — latency, cost, and availability respectively.

Provider abstraction

Beneath all three planes sits a uniform provider layer: every model API, GPU vendor, and hosting substrate is wrapped in the same interface with live health, price, and latency telemetry. Providers are cattle. This is what makes the routing story in chapters 04–05 possible and what keeps us out of anyone's walled garden.

Frontend

The site itself is a server-rendered React app with prerendered marketing routes, lazy-loaded studio surfaces, and a screenshot harness (VisualBench) that captures every page on every change — the same harness we sell to agents for UI review, pointed at ourselves.

The gateway is an OpenAI-compatible API in front of OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Groq, Together, Fireworks, NVIDIA, OpenRouter, Fal, Netwrck, MiniMax, Z.AI, Exa, and more. Customers point an existing SDK at us and either pin a provider model or use an auto-routed lane:

Routing inputs today: task lane, live provider health, published and negotiated prices, measured latency distributions, and per-customer preferences (pin, exclude, data-locality). Fallback is automatic — a provider outage degrades to the next-best provider, not to an error. Fusion extends the same machinery to run a panel of models against one prompt, judge the outputs, and synthesize a stronger answer.

This chapter is the technical core of the ten-year plan: infrastructure decisions made per request by a learned router, not per quarter by a human.

The ~1 ms decision budget

A router is only usable if it is invisible: its decision time must vanish against the tens of milliseconds of network and the seconds of inference around it. Our design budget is roughly one millisecond per decision. That rules out calling a model to choose a model. It requires the decision to be a lookup-shaped computation: a static embedding of the request, a nearest-neighbor consultation against a routing index, and an arg-max over live price/latency/health scores — all in memory, all on CPU. Our routing research (static-embedding kNN routers trained on model-comparison outcomes) targets exactly this shape: quality within a point or two of an LLM-judge router at three to four orders of magnitude lower decision cost. The budget is a design constraint we hold ourselves to, not a marketing benchmark.

GPU placement across any provider

The same discipline applies below the model layer. A GPU job — an inference burst, a training run, a render — carries requirements (VRAM, interconnect, region, deadline) and the placer solves for cheapest capable capacity across every substrate we can reach: serverless GPU (scale-to-zero, per-second billing, cold-start cost), hosted pods (warm, predictable, reservation economics), and spot capacity (cheapest, interruptible, needs checkpointing). Because the job plane requires checkpointable workloads, the placer can chase price across providers mid-workload: pods migrate to whichever cloud is cheapest tonight, and the customer sees a smaller bill, not a migration.

The flywheel

Routing telemetry improves placement; placement volume improves capacity pricing; better pricing wins more traffic; more traffic improves telemetry. Every turn of that wheel is margin that does not depend on raising prices — it depends on making better decisions than customers would make manually, and splitting the savings with them.

Cog Studio deploys any Cog container as a scale-to-zero HTTPS endpoint with generated input forms, prediction logs, and per-second billing from $0.228/hr. Build Studio builds images on platform workers and pushes to a private registry; the CI control room watches external pipelines (GitHub Actions and friends) and lets agents repair failures. Training runs LoRA and full fine-tunes on per-second GPUs and deploys resulting weights to dedicated endpoints behind the same gateway key.

The kernel-level work matters too: our accelerated inference projects (custom Triton/CuteDSL kernels for forecasting and diffusion workloads) exist to make owned capacity cheaper per token than rented capacity — widening the arbitrage the placer exploits.

A coding agent run is: clone into an isolated workspace, plan against the task, edit, run tests and linters, capture VisualBench screenshots for UI changes, and open a PR with logs attached. Agents hold scoped credentials (repo-scoped tokens, per-run budgets) and their whole trace — every command, diff, and model call — is replayable for review.

Around the core runner: schedulers run agents on cron; auto-agents trigger on events (failing CI, new issue, queue depth); queues feed fleets of workers with retries, dead-letter, and spend caps; and skills are versioned markdown capabilities any agent can load — a searchable library of several hundred, usable on our runtime or copied out to any other.

The next runtime primitive is the agent that does not terminate. Design requirements, each of which is an infrastructure product:

Target workloads: dependency and CVE patrol, p99 defense (renegotiate a service's infra nightly via the placer), content pipelines that write/illustrate/publish continuously, and codebase gardening — the long tail of refactors no team ever schedules. Commercially, a run-forever agent is a subscription that expands with the estate it manages and does not churn while it is visibly earning its budget.

Self-organizing software is the composition of everything above into a loop: telemetry → detection → proposal → proof → gated merge → deploy → telemetry. A system that notices its own regression, writes the fix, proves it with tests and benchmarks, and ships it through a review gate a human configured once.

What exists today

Each organ of the loop is a shipped product: repos and PRs (proposal), CI and VisualBench (proof), auto-agents (detection), deploys with previews (shipping), budgets and permissions (governance). We run early versions of the loop on our own estate — agents file and fix real issues on the platform that hosts them.

What has to be true

Three hard problems stand between here and the category: verification strong enough to trust (tests, benchmarks, and visual review that catch what reviewers catch), goal stability over long horizons (constraints that survive thousands of iterations without drift), and governance that scales (review gates that stay meaningful when proposals arrive faster than humans read). Our bet is that these are platform problems — solvable with better proof machinery and better gates, not just better models — which is why a platform company gets to own the answer.

Twelve editors and studios, one substrate: video, audio, image (raster), vector, slides, sheets, docs, 3D, animation (AnimFlow), whiteboard, notebooks, and dashboards. Three design rules make them a platform rather than a bundle:

• Agent-operable. Every editor operation is exposed as a callable tool, so an agent can cut a video or restyle a deck the same way it edits code.
• One asset store. A generated image is immediately a video layer, a slide asset, a 3D texture source. No exports between products.
• One balance. Every render, generation, and model call in every editor meters into the same prepaid credits as the gateway and agents.

Media generation routes like text does: images across GPT Image, FLUX, SD3, ZImage; video across Sora, Hailuo, Seedance, Wan, LTX; music, speech, and 3D across their own provider panels — same key, same fallback, same usage logs.

Hosted git repos with branch previews, PR review queues, and pretty GitHub-style URLs; model storage keeping weights, LoRAs, and datasets next to the workers that use them; hosted datasets and notebooks for analysis; and artifacts as the shared library every editor and agent reads and writes. Data locality is a routing input: workloads follow their data, and customers can pin regions or providers for sovereignty.

Standing practices, live today (details on /security):

• Workload isolation. Agent runs and builds execute in isolated workspaces with scoped, short-lived credentials — never account-wide keys.
• Secret hygiene. Secrets are injected at runtime and scrubbed from build and CI logs; public repos can run secure CI without leaking (edukids, our open-source flagship, is the standing proof).
• Spend governance. Per-key and per-agent budgets with hard caps — financial blast-radius control as a first-class primitive.
• Auditability. Agent traces, gateway usage logs, and deploy histories are replayable and attributable to a key.
• Provider containment. Customers can exclude providers or pin data-locality per key; the gateway enforces it at routing time.
• Hardened hosts. Default-deny firewalls, integrity monitoring, and alerting on the machines we operate.

The honest gap: we are early on formal certification (SOC 2 and friends are roadmap, chapter 15). Our compensating posture is radical inspectability — public code, replayable traces, and this document.

How money flows

Prepaid credits are the unit of everything: customers top up a balance (or hold Pro/Ultra/Max plans that bundle monthly credits, reserved agent machines, and quotas) and every product meters against it. Prepay matters doubly in an agent economy — it is both working capital and the mechanism that makes autonomous spending safe to allow.

Margin sources

Cost structure

We are levered to our own thesis: the placer that saves customers money runs our fleet too, scale-to-zero keeps idle surface near-free, and agents do work that would otherwise be headcount. Small team, wide surface, usage-based revenue — the model only works if the automation story is true, which is the point.

We publish methodology and hold a rule: a number is either measured and reproducible, or it is labeled a design target. Current instruments:

• Routing evals. Static-embedding kNN routers scored against LLM-judge baselines on model-comparison datasets — quality-vs-decision-cost curves, run whenever the model catalog shifts. Target: judge-level lane assignment inside the ~1 ms budget (design target, tracked in the open research repo).
• Provider telemetry. Continuous latency, error-rate, and price sampling across all 20+ providers — the dataset the router consumes, and the basis for the model catalog's live pricing.
• VisualBench. Full-page screenshot sweeps of every route on every change, desktop and mobile — UI regression as a benchmark suite (browse it at /visualbench).
• Kernel benchmarks. Owned-kernel inference speedups (e.g. our forecasting-kernel work) measured against reference implementations before any capacity-cost claim is made.
• Agent evals. Task-completion and PR-acceptance rates on our own estate — the dogfood metric that gates what we claim agents can do.

Roadmaps are claims about the future; ours is versioned. When priorities move, the diff to this page is public like any other commit.

• Platform compression. Model labs bundle more of the loop themselves. Mitigation: multi-provider neutrality is our product — labs are unlikely to route to competitors, and customers increasingly refuse single-lab lock-in.
• Breadth vs. depth. A wide surface risks shallow products. Mitigation: shared substrate (one ledger, one asset store, one runtime) means each surface is thin UI over deep common infrastructure; dogfooding exposes shallow spots fast.
• Provider dependence. Upstream price or ToS shifts. Mitigation: 20+ providers, live re-routing, owned capacity at the margin.
• Autonomy incidents. An agent doing damage is the category's existential PR risk. Mitigation: budgets, gates, kill switches, and replayable traces are shipped defaults, not options — and we sell the governance as hard as the autonomy.
• Small-team execution risk. Real. The counterweight is the thesis itself: we compound with model capability and agent leverage, not headcount.

1. Create an account and run the loop: point an agent at a repo, watch the PR arrive, check the trace.
2. Point an OpenAI SDK at the gateway, call app/auto, and compare the usage log against any provider's list price.
3. Read the open code: the platform's public repos, the open-source flagship (edukids), and the routing research.
4. Browse VisualBench — the screenshot record of every surface, including its failures.
5. Then ask for the private annex (financials, cohorts, cap table) under NDA.

Most diligence processes reconstruct reality from a deck. Ours starts from reality: it is all running, and you have an account.