Comfy models: from media evals to working GPU inference
Connect image and video benchmark evidence to hosted FLUX, SDXL, LTX, and Wan workflows, reproducible JSON, model provenance, and a global R2 weight cache.
Image and video leaderboards are unusually easy to admire and unusually hard to turn into a working GPU service. A model name is not a deployment: you still need the right weights, VAE and text encoders, compatible nodes, workflow JSON, enough VRAM, storage close to the worker, and an API contract.
The app.nz eval hub now connects independent media analysis to the model routes and Comfy Studio workflows we can run.
Read the media evidence correctly
Artificial Analysis' image methodology separates text-to-image from image editing, generates at standardized settings, derives quality from blind pairwise votes, and reports provider price and recent generation time separately. Its video arena similarly separates text-to-video, image-to-video, and video editing.
That separation is valuable. Text-to-image quality says little about preserving the structure of an input image. Image-to-video says little about video-to-video editing. One “best media model” label collapses different products.
Our launch map keeps the modalities separate:
| Need | Runnable path |
|---|---|
| Fast hosted text-to-image | FLUX.1 Schnell or Fast SDXL |
| High-instruction hosted image | GPT Image 2 |
| Open SDXL photo generation | RealVisXL Comfy workflow |
| Image-guided style transfer | SDXL ControlNet workflow |
| Text or image to open video | LTX-Video workflows |
| Video to video | Wan VACE workflow |
A workflow is the reproducibility unit
For open diffusion and video models, the checkpoint alone is not enough. We keep an API-format workflow with:
- exact node types and connections;
- required checkpoint, text encoder, VAE, LoRA, and ControlNet filenames;
- exposed API inputs and defaults;
- minimum VRAM and suggested hardware;
- upstream workflow provenance and verification status.
This makes “try the model” concrete. You can download the JSON, customize it, or deploy it as a scale-to-zero endpoint. The same operations are available in the CLI and desktop MCP tools, so an agent can discover, deploy, and invoke a workflow without clicking through the UI.
Put the weights near the GPU
Downloading a large model from a third-party host during every cold start is both slow and fragile. Comfy Studio can link a public Hugging Face file or Civitai model version, retain its source metadata, and warm a SHA-256-deduplicated object in the global R2 cache.
Workers receive an ordered download plan: cached object first, original source as fallback. Different users and workflows referring to the same bytes reuse the same object. That reduces repeated egress and makes scale-from-zero behavior more predictable without pretending the original license or provenance changed.
Read the infrastructure detail in Mirroring ComfyUI models to make cold starts boring.
Test the deployment, not only the sample image
For every candidate, keep a small prompt and input suite. Record output artifacts along with cold and warm latency, queue time, GPU type, workflow version, model hash, and total cost. Include failures: faces, hands, typography, fast motion, camera cuts, long clips, and inputs near the VRAM limit.
The public arena helps choose candidates. The tested workflow proves that the candidate can be hosted. Your artifact gallery proves what it does on your work. That three-layer chain is the Comfy experience we want: discover → deploy → run → inspect, with no missing model file hidden between the arrows.