Benchmark
Winding road — deep-perspective landscape and ambient loop
- Revision 1
- 14 criteria
- 2 steps
A valley landscape with a road winding into the distance, drawn to a fixed layout, then a seamless ambient loop with a wagon travelling the road toward the viewer. 4 models got the same prompt. These are the top three.
1 more model in the full ranking below.
Full ranking
Rows are models, strongest first; columns are plugin versions, newest first. Each cell is the best run's score and its craft rating.
| Model | v0.7.0 | v0.5.0 | v0.4.0 |
|---|---|---|---|
| claude-opus-5-5 | | | |
| claude-sonnet-5-5 | — | | — |
| gpt-6.1-sol | — | | — |
| gpt-6-luna | — | | — |
| Criterion | claude-opus-5-5 v0.7.0 | claude-opus-5-5 v0.5.0 | claude-opus-5-5 v0.4.0 | claude-sonnet-5-5 v0.5.0 | gpt-6.1-sol v0.5.0 | gpt-6-luna v0.5.0 |
|---|---|---|---|---|---|---|
| validate passes with no unaddressed errors (warnings may stand if the report names and justifies them). | Pass validate on step1.aseprite: 0 errors, 1 outline warning on the river bank edge (dithered bank), not a silhouette. | Pass | Pass One warning: river pixels touch the canvas edge where the water enters and leaves; named and kept. | Pass validate clean. | Pass validate clean. | Fail Not verified: validate did not complete (Aseprite stalled during the run). |
| Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get"). | Pass palette op get returns exactly the 16 PICO-8 entries; a pixel scan of every exported layer finds no off-palette or semi-transparent pixel. | Pass | Pass | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. |
| In the preview the road can be traced by eye from the bottom edge to the castle gate, and the castle, windmill, farmhouse, river and bridge are each recognisable without the ascii grid. | Pass Road, castle with a keep and tall tower, farmhouse, windmill and river all read; the bridge is a small grey arch at about (32-46,66-72), thin but readable. | Pass | Pass Every landmark reads; the far road and the switchbacks are thin and only just traceable. | Pass Road, bridge, castle, windmill, farmhouse and river are recognisable. | Pass Road traceable to the castle; castle, windmill, farmhouse and river recognisable. | Fail No bridge; the road cannot be traced to the castle gate. |
| The brief is followed: layers exactly sky/mountains/castle/hills/river/road/props/windmill-blades/foreground with each element on its layer (sprite_info, look with layer=…), horizon at y=34 ±1, sun low on the right with light from the right, river from the left edge toward the bottom-left, windmill with 4 blades. | Pass Nine layers with the exact names in the stated order; sky ends at y=34 and hills start at y=35 (within 1); sun rim spans x141-147, y17-23 on the right; river enters at the left edge y43-45 and leaves at the bottom, x8-21; windmill has 4 blades. | Pass | Pass Horizon at y 34–35, sun at the upper right, river in from the left edge and out at the bottom-left. | Pass All nine layers; sun right; river from the left; 4-blade windmill. | Pass All nine layers present; sun low right; river from the left; 4-blade windmill. | Pass All nine layers present; sun right; windmill with 4 blades. |
| The road enters the bottom edge about 60 px wide, never widens with distance, crosses the river on the bridge, is hidden behind a hill for a stretch and reappears smaller, and climbs at least 3 switchbacks to a 1 px end at the castle gate. | Pass 61 px wide at the bottom (x50-110); hidden behind a hill between y47 and y38; 4+ switchbacks up the mountain; ends at 1 px at (112,21). | Pass | Pass x 50–110 at the bottom, bridge at x 37–47, hidden past the crest at about (40–47, 41) and back at (61, 45), four switchbacks, 1 px end at the gate (110, 20). The hidden stretch is left undrawn rather than covered by the hill layer. | Fail No visible switchbacks up to the castle gate. | Fail No bridge over the river is visible and the hidden stretch is not verified. | Fail No bridge, no hidden stretch, no switchbacks reaching the gate. |
| Perspective holds: fence posts or milestones shrink and bunch with distance, surface detail thins with distance, the far mountains and castle stay light, cool and low-contrast with no black, and #000000 appears only in the foreground and nearest midground (look op="ascii" by region). | Pass Posts shrink and bunch along the near road; mountains and castle use only lavender, blue, dark blue, peach and white; #000000 appears in hills only at (140-147,63-76), a near right-hand tree, and in props and foreground from row 78 down. | Fail | Pass Five posts shrink from 11 to 4 px and bunch with distance; #000000 only in the foreground grass. | Fail Mid-ground hills are flat saturated greens with little falloff. | Fail Mid-ground hills are flat saturated greens with little falloff. | Fail Flat hills and few posts; no depth falloff. |
| frame op="list" shows exactly 24 frames, every one 100 ms, and tag op="list" shows a forward tag named `loop` covering all of them. | Pass frame op list: 24 frames of 100 ms; tag op list: loop, frames 1-24, forward. | Pass | Pass 24 frames at 100 ms, forward tag loop 1–24. | Pass 24 × 100 ms, forward tag loop 1–24. | Pass 24 × 100 ms, tag loop 1–24. | Pass 24 × 100 ms, forward tag loop 1–24. |
| Played on repeat the GIF has no hitch: look op="diff" from frame 24 to frame 1 changes about as much, and in the same places, as frame 1 to frame 2 — only the next step of each cycle, plus the wagon's absence. | Pass Pixels changed between consecutive frames (all layers except wagon) are 168-314 per step and 169 for frame 24 to 1; wagon is absent on frames 1, 2 and 24; river repeats every 12, windmill every 6, foreground every 12, smoke is within 2-12 px of itself 8 frames later. | Pass | Pass Composite diff 24→1 changes 172 px, inside the 178–515 px range of the other steps; the wagon is absent on both. | Pass No visible hitch at the wrap; wagon absent on 1 and 24. | Pass No visible hitch at the wrap; wagon absent on 1 and 24. | Fail Not verified. |
| sky, mountains, castle, hills, road and props are identical on every frame — their cels are linked (cel op="list") or look op="diff" with each layer reports nothing changed. | Pass sky, mountains, castle, hills, road and props: one distinct image across all 24 frames (hash-identical exported cels). | Pass | Pass sky, mountains, castle, hills, road and props cels are linked on all 24 frames. | Fail Not verified. | Fail Not verified with per-layer diff. | Fail Not verified. |
| Onion skin on `windmill-blades` shows 4 blades turning clockwise by an even step around a fixed hub, with the same blade length and thickness at every angle, repeating every 6 frames. | Pass Blade orientation advances 12-17 degrees per frame clockwise around a fixed hub at about (24,26), 6 distinct images, period 6; reach 10.0-10.8 px at every angle. | Pass | Pass windmill-blades shows 6 distinct images repeating every 6 frames around a fixed hub. | Fail Not verified. | Fail Not verified with onion skin. | Fail Not verified. |
| Ripples visibly travel downstream toward the viewer and the pattern repeats with a period that divides 24; the banks do not move. | Pass 12 distinct river images, period 12; glints move on average 0.48 px/frame down-screen (toward the viewer); no #1d2b53 bank pixel changes against frame 1. | Pass | Pass The river layer repeats every 8 frames; banks untouched. | Fail Not verified. | Fail Not verified. | Fail The model reports the river is not animated. |
| On the `smoke` layer puffs leave the chimney, rise, drift left, grow and break into dithered wisps before vanishing, with a new puff every 8 frames. | Pass Lavender puffs leave the chimney at about y=29, rise to y=9-12 and drift left, with dithered scatter at the top; 23 distinct images but each frame matches the one 8 frames later within 0-12 px, so a new puff starts every 8 frames. Judged from rendered frames. | Pass | Pass | Fail Not verified. | Fail Not verified; smoke barely visible. | Fail The model reports the smoke layer is empty. |
| The wagon is hidden behind the hill on frame 1, then stays on the road surface through every curve and over the bridge, grows steadily from about 4 px to at least 14 px wide, moves further per frame as it nears, and is fully off the bottom edge by frame 24 (onion skin on `wagon`, the reported positions). | Pass Hidden on frames 1-2; 4 px wide on frame 3 at (64-67,42-45), stays on the road, over the bridge area on frames 16-22, 15 px wide on frame 23, gone on frame 24; step grows from 3.0 to 11.5 px per frame. | Pass | Pass Hidden on frame 1, only the canopy tip above the crest on frame 2, 4 px wide on frame 4, 9 px on the bridge, 13 px on frame 20, gone by frame 24. It switches to a side view while it drives across the bridge. | Pass Emerges from behind the hill, grows along the road and leaves through the bottom edge. | Fail The wagon appears and grows toward the bottom edge, but no bridge crossing is visible. | Fail The wagon is visible on only two frames and jumps between them. |
| validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries. | Pass validate on step2.aseprite: clean, 0 errors, 0 warnings; palette is the 16 PICO-8 entries. | Pass | Pass | Pass validate clean; palette 16 entries. | Pass Clean with style notes; 16 entries. | Fail Not verified: validate did not complete (Aseprite stalled during the run). |
Run by run
Every run of this prompt from the first one to today (a year at most), each model on its own line. A marker is a day's best run; the line is solid between runs and dashed where the model did not run, carrying its nearest score. Point at a run to compare the models that day; pick a model below to bring its line to the front.
| Series | Oct 1, 2026 | Oct 2, 2026 | Oct 5, 2026 | Oct 6, 2026 | Oct 7, 2026 |
|---|---|---|---|---|---|
| claude-opus-5-5 | 50 | 56 | — | 66 | 61 |
| claude-sonnet-5-5 | — | — | 44 | — | — |
| gpt-6.1-sol | — | — | 42 | — | — |
| gpt-6-luna | — | — | 23 | — | — |
Every run
Compare runs9 runs of this prompt, highest score first. Runs on an older revision are marked unranked and come last.
9 pieces
How it was tested
Every model gets this prompt word for word, on the same canvas and palette, and draws it in a live Aseprite window. Each criterion is a yes-or-no check on the files it hands back.
A valley landscape with a road winding into the distance, drawn to a fixed layout, then a seamless ambient loop with a wagon travelling the road toward the viewer. Tests perspective (diminishing size, occlusion, atmospheric depth), composition at scene scale, and several independent cycles that must all close into one seamless loop.
- validate
- palette
- read
- spec
- road
- depth
- frames
- seamless
- static
- windmill
- river
- smoke
- wagon
- Canvas
- 160×96 RGB
- Palette
- pico8
- Steps
- 2
- Revision
- 1
The steps, verbatim
Step 1:The landscape
The document is already open: 160×96 pixels, RGB, with the PICO-8 palette loaded (16 colours). Use only those 16 colours — do not add, widen or replace the palette. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates below are 0-based pixels, x to the right, y down. Draw ONE frame: a green valley in late afternoon, seen from a low hill, with a dirt road that starts right under the viewer's feet and winds all the way to a castle on a distant mountain. The road and its perspective are the subject of the picture. A later step will animate this scene, so build it on the layers listed below. Layout, back to front - Horizon line at y=34. The sun sits low on the right at about (144, 20), so light comes from the right and shadows fall to the left. - Sky, y 0–34: blue (#29adff) at the top fading to peach (#ffccaa) near the horizon, using horizontal bands and ordered dithering, not noise. Two or three flat clouds with lit right edges. The sun is a small disc (#fff1e8 core, #ffec27 rim). - Far mountains along the horizon: low-contrast silhouettes in lavender (#83769c) and blue (#29adff), no black. - A castle on the highest far peak, right of centre around x 108–120: a keep and one tall tower, drawn in lavender and dark blue (#1d2b53), lit on the right. - Midground: rolling green hills (#008751, #00e436). A river (#29adff with #fff1e8 glints, #1d2b53 banks) enters from the left edge around y=44 and winds toward the viewer, leaving through the bottom-left part of the canvas. - On a hill on the left, a farmhouse with a chimney and, beside it, a windmill with exactly 4 blades. - Foreground, bottom of the canvas: tall grass tufts and a few wildflowers (#ff77a8, #ffec27) in both bottom corners, and a wooden signpost leaning beside the near road. The darkest darks (#000000) appear only in the foreground and nearest midground. The road - It enters at the bottom edge about 60 px wide (roughly x 50–110) and narrows continuously with distance: no stretch is ever wider than a nearer stretch, and it ends 1 px wide at the castle gate. - Its route, from near to far: a broad S-curve sweeping left; across the river on an arched stone bridge (#c2c3c7, #5f574f); a bend to the right where it disappears behind a hill for a stretch; it reappears smaller on the far slope; then at least 3 switchbacks climbing the mountain to the castle gate. - Surface: packed dirt (#ffccaa lit, #ab5236 shadow) with darker edges, two wheel ruts and pebbles in the near stretch. The detail gets finer and sparser with distance until the far road is a single line. - Along the near stretch, a row of fence posts or milestones on one side that shrink and bunch closer together as they recede. Perspective and depth - Everything diminishes toward the horizon: road width, posts, trees, buildings. - Atmospheric perspective: the further away, the lighter, cooler and lower in contrast. The far mountains and castle use only lavender, blue, dark blue, peach and white. - Depth is shown by overlap: the hill must visibly hide part of the road. Layers, exactly these names, bottom to top: `sky`, `mountains`, `castle`, `hills`, `river`, `road`, `props` (bridge, farmhouse, windmill tower, signpost, fence posts), `windmill-blades` (only the 4 blades and their hub), `foreground`. Order them however the overlaps need, but keep the names and keep each element on its layer. Before you finish: look at the result (use look op="ascii" with a region to inspect details, since the canvas is larger than 64×64), run validate, and fix what it reports. In your final report, describe the road's route with approximate coordinates of the bridge, the hidden stretch and the castle gate, and any decision you made that the brief did not specify.
new sessionStep 2:The loop
The open document holds a valley landscape with a road winding to a castle (canvas 160×96, PICO-8 palette, layers `sky`, `mountains`, `castle`, `hills`, `river`, `road`, `props`, `windmill-blades`, `foreground`). Keep the palette — use only its 16 colours — and keep the picture exactly as it is wherever this brief does not ask for motion. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates are 0-based pixels, x to the right, y down. Animate an ambient, seamlessly looping scene. Exactly 24 frames, every frame 100 ms, one tag named `loop` covering every frame, direction forward. Playing frame 24 straight into frame 1 must show no jump in any element. The existing frame is frame 1. What moves - Windmill: the 4 blades turn clockwise around a fixed hub, exactly 15° per frame, so the blades redraw through 6 distinct angles and repeat 4 times over the loop. Blade length and thickness stay the same at every angle. Only `windmill-blades` changes for this. - River: ripples and glints flow downstream, toward the viewer, along the river's course. The ripple pattern repeats every 8 or 12 frames so it closes on frame 24. Only `river` changes for this; the banks do not move. - Chimney smoke: on a new layer named `smoke`, puffs rise from the farmhouse chimney, drift to the left with the wind, grow and break up into dithered wisps, then vanish. A new puff starts every 8 frames, so three puffs are always at different heights and the cycle closes on frame 24. - Foreground grass: the tufts in `foreground` sway gently to the left and back in a 12-frame cycle (2 cycles per loop); the signpost does not move. - Wagon: on a new layer named `wagon`, a small covered wagon (cream canvas top #fff1e8, #ab5236 body, two dark wheels, no horse) travels the road toward the viewer. On frame 1 it is completely hidden behind the hill where the road disappears. It emerges, follows the road's curves and stays on the road surface, crosses the bridge, and drives out through the bottom edge, fully off canvas by frame 24. It grows steadily as it approaches — about 4 px wide when it first appears, at least 14 px wide at the bottom — and covers more ground per frame the nearer it is, as perspective demands. What stays still: `sky` (clouds included), `mountains`, `castle`, `hills`, `road` and `props` are identical on every frame. Link their cels. Before you finish: check the whole loop as a filmstrip, check the windmill and the wagon with onion skin, and compare frame 24 against frame 1 for each moving element. Run validate and fix what it reports. In your final report, give each cycle's length in frames and the wagon's width and position on frames 4, 12 and 20.
Setup rules
- Step 1 starts from a new document created with sprite_manage op="new" width=160 height=96 colorMode="rgb", then palette op="preset" preset="pico8" replace=true. The operator does this before handing over the step text.
- Hand the model exactly the step text — no reference image, no extra steering beyond the server's own instructions and skills.
- One session per step. Step 2 runs in a fresh session against a copy of the step-1 result.
- If the model offers to generate a concept image, a pixel-only run answers "No reference, continue from the text." A run that does use one is declared in `references` and ranked on its own row. Either answer is recorded as an intervention.
- Do not intervene mid-run unless the model asks a genuine clarifying question, and record it under the step's interventions.
14 criteria
1 Step 1: The landscape
- validate passes with no unaddressed errors (warnings may stand if the report names and justifies them).
- Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get").
- In the preview the road can be traced by eye from the bottom edge to the castle gate, and the castle, windmill, farmhouse, river and bridge are each recognisable without the ascii grid.
- The brief is followed: layers exactly sky/mountains/castle/hills/river/road/props/windmill-blades/foreground with each element on its layer (sprite_info, look with layer=…), horizon at y=34 ±1, sun low on the right with light from the right, river from the left edge toward the bottom-left, windmill with 4 blades.
- The road enters the bottom edge about 60 px wide, never widens with distance, crosses the river on the bridge, is hidden behind a hill for a stretch and reappears smaller, and climbs at least 3 switchbacks to a 1 px end at the castle gate.
- Perspective holds: fence posts or milestones shrink and bunch with distance, surface detail thins with distance, the far mountains and castle stay light, cool and low-contrast with no black, and #000000 appears only in the foreground and nearest midground (look op="ascii" by region).
2 Step 2: The loop
- frame op="list" shows exactly 24 frames, every one 100 ms, and tag op="list" shows a forward tag named `loop` covering all of them.
- Played on repeat the GIF has no hitch: look op="diff" from frame 24 to frame 1 changes about as much, and in the same places, as frame 1 to frame 2 — only the next step of each cycle, plus the wagon's absence.
- sky, mountains, castle, hills, road and props are identical on every frame — their cels are linked (cel op="list") or look op="diff" with each layer reports nothing changed.
- Onion skin on `windmill-blades` shows 4 blades turning clockwise by an even step around a fixed hub, with the same blade length and thickness at every angle, repeating every 6 frames.
- Ripples visibly travel downstream toward the viewer and the pattern repeats with a period that divides 24; the banks do not move.
- On the `smoke` layer puffs leave the chimney, rise, drift left, grow and break into dithered wisps before vanishing, with a new puff every 8 frames.
- The wagon is hidden behind the hill on frame 1, then stays on the road surface through every curve and over the bridge, grows steadily from about 4 px to at least 14 px wide, moves further per frame as it nears, and is fully off the bottom edge by frame 24 (onion skin on `wagon`, the reported positions).
- validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries.