Building Polygonalize: stable low-poly video with Go, WASM, Canvas, and Three.js

We just launched Polygonalize, a free creative tool that turns an image or video into low-poly art. It runs locally in the browser by default, keeps one triangle topology across a video instead of rebuilding every frame, and is open source on both GitHub and app.nz source hosting.

The project started from a tiny older experiment, lee101/LoPoly: load an image, hand its pixels to a WebAssembly binary, and draw the returned triangles as SVG. That prototype proved the interaction, but its original Go source was no longer in the repository and it was never designed for video. Polygonalize is a clean implementation with a reusable Go package, tests, a documented JSON contract, a CLI, browser WASM, and an app.nz serverless fallback.

One Go engine, three ways to run it

The important boundary is the mesh, not the renderer:

image/video pixels
       │
       ▼
Go edge sampler → Delaunay / adaptive high-detail topology
       │                              │
       ├─ browser WASM ─ Canvas 2D / Three.js
       ├─ CLI ────────── SVG
       └─ app.nz API ─── JSON / SVG / PNG / WebM

video: create topology once → update colours → temporal smoothing

The Go package accepts an image.Image and returns points plus indexed, coloured triangles. That same package compiles to js/wasm for the browser and to native Go for the CLI and server. We considered executing WASM again on the backend, but native Go is smaller operationally and faster there; sharing the package gives us identical behaviour without putting a WebAssembly runtime inside another runtime.

Sampling detail where the image has detail

Uniform random points waste triangles in flat sky and miss the edge of a face. We calculate a grayscale Sobel energy map, mix it with a uniform floor, build a cumulative distribution, and sample seeded points from that distribution. The edge-bias slider controls the mix.

Those points go through a deterministic Bowyer–Watson Delaunay pass. We sort the cavity boundary before inserting new triangles: Go deliberately randomizes map iteration, and leaving that order implicit made identical seeds produce different meshes. A reproducibility test caught it before launch.

Each triangle colour is sampled at its vertices and centroid. The output is a small renderer-neutral JSON shape, so the SVG exporter, raster backend, Canvas, and WebGL preview do not each invent their own polygon algorithm.

Scaling the mesh to 20,000 triangles

Bowyer–Watson is a good fit for expressive low-detail work, but its simple incremental implementation becomes quadratic at thousands of points. Asking it for a 20,000-triangle frame would freeze a browser tab. Above 3,000 triangles, Polygonalize switches to a linear-time adaptive grid topology. Each vertex tests deterministic candidates inside its own grid neighbourhood and moves toward the strongest Sobel energy without crossing adjacent cells. The result still follows edges, covers the frame, preserves stable indices for video, and stays under the 20,000-triangle ceiling.

The renderer can draw triangles, circles, squares, diamonds, or hexagons. It can also stamp a user-supplied PNG, WebP, or SVG at each triangle centroid. Custom primitives keep their native colour and transparent pixels, but those pixels never feed back into sampling or triangulation. Source-image alpha is likewise ignored, preventing invisible PNG data from creating artificial edges.

Temporal stability is mostly a topology decision

The easiest way to polygonalize video is also the worst-looking: find new points and triangulate every decoded frame. Even a small edge movement changes the mesh, so triangles pop between unrelated shapes.

Polygonalize creates a Session on the first frame. Normalized vertex positions and triangle indices remain fixed for the clip; only sampled colours change. The stability control applies an exponential moving average to those colours. This removes structural popping and lets the user trade responsiveness for less flicker. It is intentionally predictable rather than pretending to be full optical flow. Motion-aware vertex tracking can be added later without changing the mesh contract.

In-browser video uses the browser's hardware decoder, reads frames from a hidden video element, and renders at a bounded cadence. Media never leaves the device. For older browsers or automated workflows, the app.nz endpoint accepts a video, uses FFmpeg to decode up to 12 seconds at 720p, runs the same stable Go session, and streams the frames into a VP9 WebM encoder.

Canvas or Three.js—use the cheaper renderer

Three.js is useful when the output is actually 3D. It is unnecessary overhead for hundreds of flat filled triangles, especially on every video frame. The default path therefore uses Canvas 2D; the 3D lift button dynamically loads Three.js, converts the indexed triangles into a coloured BufferGeometry, and adds a small procedural depth field and pointer parallax.

This split makes the common path fast and keeps the richer preview available. Go/WASM does the CPU-side mesh work in either case. SVG export is assembled from that same mesh and stays resolution-independent.

The API and CLI

# Local deterministic SVG
polygonalize -in photo.jpg -out photo-low-poly.svg \
  -triangles 12000 -primitive hexagon -edge-bias 0.76 -seed 42

# Serverless SVG
curl -F [email protected] -F triangles=12000 -F primitive=diamond \
  https://polygon.app.nz/api/polygonalize/image > photo.svg

# Stable server-rendered video fallback
curl -F [email protected] -F triangles=2000 -F primitive=circle -F stability=.9 \
  https://polygon.app.nz/api/polygonalize/video > clip-low-poly.webm

There is also POST /api/mesh for tools that want triangle JSON and their own renderer. Uploads are bounded; video work has duration and resolution ceilings; temporary media is removed after the response. The public browser experience does not require an account or credits.

Open source and hosted on app.nz

The repository contains the Go engine, WASM bridge, CLI, server, web interface, MIT license, CI, Dockerfile, and appnz.yaml. Every push to main can build and deploy the server runtime to polygon.app.nz. The same project is public at github.com/lee101/polygonalize and mirrored into app.nz's GitHub-compatible hosting, so the app is also a small demonstration of the hosting stack it runs on.

Try an image or clip in Polygonalize, fork the source, or use the Go package as the start of a different renderer.