Packaging AlayaWorld: an interactive video world model as a self-hosted Cog
Wrapping AlayaLab’s autoregressive world model (LTX-2.3-derived DiT + gemma-3-12b + Depth-Anything-3) as a Cog: camera presets, a warm engine, and an honest look at the LTX-2 Community License.
Most "world model" demos are a paper, a project page, and a video you can't touch. AlayaWorld — an interactive autoregressive video world model from Alaya Lab — ships real inference code and open weights, so we packaged it as a self-hosted Cog and put it in the Open Cog catalogue alongside our other self-hosted models.
Give it one first-frame image, a camera path, and a text prompt, and it rolls out a minute-long video chunk by chunk — about 1.33 seconds per chunk — that follows the camera, with few-step distilled generation and a 3D spatial-memory cache so places you revisit stay visually consistent. It's built on a 22B-class diffusion transformer descended from LTX-2.3, a gemma-3-12b text encoder, and Depth-Anything-3 for the depth-conditioned memory.
What we actually built
`appnz-alayaworld` is a thin adapter, not a fork: cog.yaml clones AlayaWorld's own flash_alaya engine and inference.run rollout helpers at a pinned commit, unmodified. On top of that we added exactly three things upstream's CLI doesn't have:
- A warm `setup()`/`predict()` split. Upstream's
inference/run.pyis a
one-shot CLI — it loads the engine, renders one clip, exits. A Cog container serves many predictions, so we hoist the (~40s) engine load into setup() and reuse it. This also means we do not call upstream's free_generation_models() between predictions — it permanently frees the DiT and text encoder, which is correct for a CLI process about to exit and wrong for a server that has to answer the next request.
- Camera presets. AlayaWorld's own interface only accepts a pre-recorded
.pt trajectory tensor — there's no "dolly forward" API. We measured the motion statistics of upstream's bundled demo clip (a near-constant ~0.079 world-unit forward step, near-zero incidental rotation) and built nine named presets — dolly_forward, strafe_left, pan_right, and so on — calibrated to match, plus a custom option for uploading an exact trajectory. It's a best-effort heuristic, not something derived from the training distribution, and the README says so.
- Weight resolution that matches our other heavy cogs. Local dir → R2
mirror → Hugging Face fallback, the same three-step order as `ltxv23-lora-app-nz`. Nothing is baked into the image or redistributed — weights download at setup() time straight from their license holders, which matters more than usual here (see below).
The license is the interesting part
AlayaWorld's code and its merged_infer.safetensors weights are a fine-tuned derivative of LTX-2.3, so the whole thing — code and weights — is released under the [LTX-2 Community License Agreement](https://github.com/Lightricks/LTX-2/blob/main/LICENSE), not an OSI-approved license. In practice: free for research, free for commercial use under $10M annual revenue, with use-based restrictions (no deepfakes without consent, no disinformation, no weapons applications) that any redistribution has to carry forward. The gemma-3-12b-it text encoder is separately gated behind Google's Gemma Terms of Use, and Depth-Anything-3's code is Apache-2.0 with its own weight license.
We didn't relicense any of that — we can't, we don't own it. Only the adapter glue in appnz-alayaworld (predict.py, the camera-preset math, the weight resolver, the tests) is ours and MIT. Everything else is called out file by file in `THIRD_PARTY.md`, the same treatment we already give ltxv23-lora and ltx23-fast, our two other LTX-2-derived cogs.
Where it stands
The adapter, camera math, and CI contract tests are done and passing — camera presets are unit-tested against the calibration data, and the schema, license files, and build path follow the same acceptance gate as the rest of the Open Cog catalogue. What it hasn't had yet is a hosted end-to-end run: the full stack needs an 80GB-class GPU and a Hugging Face token with the gemma license accepted, so that first real rollout is pending capacity rather than something a CPU-only CI job can confirm. We'd rather say that plainly than paste in a video we haven't actually generated.
Template id alaya-world is registered in Cog Studio now, so app cogs deploy alaya-world --hardware gpu-h100 will provision it (the template's own 80GB minimum VRAM comes along automatically); the source, the camera-preset code, and the full license breakdown are on GitHub.