All examples
Example Cogtemplate:fastserve-qwen-27b

Prompt → reasoning, code, tool calls

Qwen3.8-27B FastServe

Qwen3.8-27B on SGLang with a C launcher that reads the GPU and picks the fastest profile for it: NVFP4 on Blackwell, FP8 on Hopper and GB300, DSPARK speculative decoding everywhere. Answers the OpenAI API and Cog predictions on one port.

Deploy & run Source
RTX 5090 32GB scale to zero Streaming textQwen3.8SGLangNVFP4DSPARKtool calling

Canonical example

Input and output, side by side

OpenAI-compatible /v1 plus Cog predictions from the same container

01 · input

Refactor this Go handler to stream results, then explain the tradeoff in two sentences.

prompt
Refactor this Go handler to stream results, then explain the tradeoff in two sentences.
system
You are a precise staff engineer.
max tokens
800
temperature
0.7
02 · Streaming text
assistant
Stream the rows as you scan them: keep the *sql.Rows open, write each JSON object into the http.ResponseWriter, and Flush() every N records. The tradeoff is that you cannot set a Content-Length or fail cleanly mid-response — once the first byte is out, an error becomes a truncated body rather than a 500.

Run the same recipe everywhere

curl -sX POST https://app.nz/api/cogs/run \
  -H "Authorization: Bearer $APP_NZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"template":"fastserve-qwen-27b","input":{"prompt":"Refactor this Go handler to stream results, then explain the tradeoff in two sentences.","system":"You are a precise staff engineer.","max_tokens":800,"temperature":0.7}}'