Benchmark
Cherry tree — from seed to full bloom
- Revision 1
- 13 criteria
- 2 steps
A blossoming cherry tree drawn to a fixed design, then animated growing from a seed into exactly that tree. 4 models got the same prompt. These are the top three.
1 more model in the full ranking below.
Full ranking
Rows are models, strongest first; columns are plugin versions, newest first. Each cell is the best run's score and its craft rating.
| Model | v0.8.0 | v0.7.0 | v0.5.0 | v0.4.0 |
|---|---|---|---|---|
| claude-opus-5-5 | | | | |
| claude-sonnet-5-5 | — | — | | — |
| gpt-6.1-sol | — | — | | — |
| gpt-6-luna | — | — | | — |
| Criterion | claude-opus-5-5 v0.8.0 | claude-opus-5-5 v0.7.0 | claude-opus-5-5 v0.5.0 | claude-opus-5-5 v0.4.0 | claude-sonnet-5-5 v0.5.0 | gpt-6.1-sol v0.5.0 | gpt-6-luna v0.5.0 |
|---|---|---|---|---|---|---|---|
| validate passes with no unaddressed errors (warnings may stand if the report names and justifies them). | Pass validate passes: score 100, 0 errors, 0 warnings (three 'not outlined' style notes only). | Pass validate passes: score 95, 0 errors, one outline warning on trunk (70% #ab5236, 46 edge px) which the agent called a false positive for an unoutlined two-tone trunk. | Pass | Pass validate: clean, score 100. | Pass validate clean with one outline warning. | Pass validate clean. | Pass The model reports a clean full validate. |
| Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get"). | Pass palette op get returns exactly the 16 PICO-8 entries. | Pass palette op get returns exactly the 16 PICO-8 entries. | Pass | Pass | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. |
| The preview reads at a glance as a cherry tree in bloom — trunk, branches showing through gaps, and pink blossom masses are each distinguishable without the ascii grid. | Pass Trunk, forked branches and four pink masses read at a glance in the 4× export. | Pass Trunk, forked branches and three pink masses read at a glance in the 4× export. | Pass | Pass | Pass | Pass | Pass |
| The brief is followed: layers exactly ground/trunk/canopy (sprite_info), ground on y 56–63 with grass on top, trunk base on x 30–33 and 4 px wide, exactly 3 main branches splitting around y=36, canopy inside roughly x 8–56 and y 6–38 as 3–4 masses with gaps. | Pass Layers ground/trunk/canopy; ground y56–63; trunk 4 px wide at y50–53 on x30–33 with flared roots; three branches fork at y≈35 and each splits again; canopy x8–57, y6–38 as four masses (the two on the right overlap) with gaps. | Pass Layers ground/trunk/canopy; ground y56–63; trunk 4 px wide at y47–54 on x29–32/30–33 with flared roots; three branches fork at y≈35 and each twigs again; canopy x7–57, y7–37 as three separate masses with gaps. | Pass | Pass Trunk 4 px wide on x 30–33 at y 50–53, flaring to 28–35 at the grass; three main branches split at y 33–35; canopy edge at x 7 rather than 8. | Pass Layers ground/trunk/canopy; three main branches; four blossom masses with gaps. Coordinates not measured. | Pass Layers ground/trunk/canopy; three branches; four blossom masses with gaps. | Pass Layers ground/trunk/canopy; three branches; three blossom masses. Coordinates not measured. |
| Light reads from the upper left on the trunk and on every blossom mass, and the blossoms are clumps with a light and shadow side rather than single-pixel noise (validate strays clean, look op="ascii"). | Pass Cream highlights upper left on each mass, #7e2553 shadow on the lower right and underneath; rounded clumps, validate reports no strays. | Pass Light from the upper left: cream on top left of each mass, #7e2553 shadow to the right and underneath; masses are clumps, though the cream fills much of each mass and is a bit busy. | Pass | Pass | Pass Masses have a light upper side and a darker underside. | Pass Each mass has a lit upper-left side and a darker underside. | Fail Flat pink blobs with a few light patches; no lit and shadow side per clump. |
| The sprite has 18–24 frames and tag op="list" shows a forward tag named `grow` covering all of them. | Pass 21 frames; tag grow 1–21, direction forward. | Pass 20 frames; tag grow 1–20, direction forward. | Pass | Pass 21 frames, forward tag grow 1–21. | Pass 21 frames, forward tag grow 1–21. | Pass 21 frames, tag grow 1–21. | Pass 19 frames, forward tag grow 1–19. |
| The final frame matches the step-1 tree pixel for pixel — an image diff of its PNG export against the step-1 export shows no changed pixel — and `fx` is empty on that frame. | Pass Frame 21 composite and ground, trunk and canopy cels are pixel-identical to step1.aseprite (0 differing pixels); fx empty on frame 21. | Pass Frame 20 composite and ground, trunk and canopy cels are pixel-identical to step1.aseprite (0 differing pixels); fx empty on frame 20. | Pass | Pass The 1× PNG of frame 21 is byte-identical to the step-1 export; fx has no cel on frame 21. | Pass The last frame matches the step-1 export pixel for pixel. | Pass Frame 21 export matches the step-1 export pixel for pixel. | Pass The last frame matches the step-1 export pixel for pixel. |
| The filmstrip shows all six stages in order — seed, sprout with two seed leaves, sapling, thickening young tree, green masses with buds, bloom spreading from the tips. | Pass Filmstrip: seed, cracking soil with crumbs, sprout with two seed leaves, bark sapling with side branches, thickening young tree with green masses, buds on the green (frames 15–17), bloom with petals. | Pass Filmstrip: seed, cracking soil with crumbs, sprout with seed leaves, bark sapling with side branches and small leaves, thickening young tree, green masses with buds, bloom from the tips with petals. | Pass | Pass | Pass Seed, sprout, sapling, young tree, green masses with buds, bloom. | Pass Seed, sprout, sapling, young tree, green masses with buds, bloom. | Fail Green canopy balls float in the air on frames 9–11 before any trunk exists. |
| look op="diff" with layer="trunk" between consecutive growth frames shows only additions and re-shading along the existing structure — no branch shrinks, jumps or changes path, and onion skin on a young-tree frame shows its branches lying on the paths of the final ones. | Pass Trunk layer loses 0 pixels between consecutive frames and no trunk pixel on any frame lies outside the final trunk's pixels. | Pass Trunk layer loses 0 pixels between consecutive frames and no trunk pixel on any frame lies outside the final trunk's pixels. | Pass | Pass read_pixels of trunk on every frame: no frame loses a pixel of the one before. look op="diff" ignored layer= here, so the per-layer check used read_pixels. | Pass Branches extend along the final paths; canopy masses grow on them. | Pass Branches extend along the final paths; canopy masses swell in place. | Fail The canopy appears detached from the structure and the full trunk pops in on frame 12. |
| The ground layer never changes (look op="diff" with layer="ground" reports nothing changed between any two frames) and the trunk base stays on x 30–33 on every frame. | Pass Ground cel identical on all 21 frames; trunk base on x31, then x29–33 and wider as the roots flare to x27–36 (the final tree's own roots). | Pass Ground cel identical on all 20 frames; trunk base on x31 then x30–33, widening to x28–35 only as the roots flare. | Pass | Pass read_pixels of ground identical on all 21 frames; the trunk base stays inside x 30–33 above the root flare. | Pass Ground unchanged by eye. | Pass Ground unchanged across the frames by eye. | Pass Ground unchanged by eye. |
| look op="diff" percentChanged between consecutive frames is small at the start, largest in the middle and small again at the end — the growth visibly spurts and settles. | Pass Composite change 0.0–0.8% for frames 2–9, peak 9.8% at frame 13, then 1.1–0.4% before the bloom lifts it to 6.5% at frame 18; it settles to 2.0% on the last step. | Pass Composite change 0.0–0.7% for frames 2–8, peak 6.6% at frame 12, then 1.5–4.1% over the bloom, 0.7% on the last step. | Fail | Fail Changed pixels per step: 9, 20, 32, 26, 24, 27, 54, 123, 222, 283, 224, 123, 89, 124, 94, 185, 155, 136, 58. The spurt peaks mid-way, but bloom makes a second hump of 185/155/136 instead of settling in small steps. | Fail Not verified. | Fail Not verified with look diff. | Fail Not verified. |
| frame op="list" durations match the brief — seed frames ≥400, growth frames 80–150, final frame ≥1500 ms. | Pass Seed frames 500/450 ms, growth frames 80–150 ms, final frame 1800 ms. | Pass Seed frames 600/450 ms, growth frames 80–150 ms, final frame 1800 ms. | Pass | Pass Seed 600/500, growth 80–150, final 1800 ms. | Pass Seed 500/450, growth 80–150, final 1800. | Pass Seed 500/450, growth 80–150, final 1800. | Pass Seed 450/450, growth 90–140, final 1500. |
| validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries. | Pass validate passes, 0 errors, 9 outline warnings on partial-growth frames that the agent called a style quirk of tiny sprouts; palette is the 16 PICO-8 entries. | Pass validate passes, 0 errors, 18 outline warnings on partial-growth frames, expected for unoutlined intermediate stages; palette is the 16 PICO-8 entries. | Pass | Pass Passed with outline warnings only; palette still 16 entries. | Pass validate clean; palette 16 entries. | Fail 1 error: 7 stray pixels on trunk frame 12. | Fail Not verified: validate did not complete (Aseprite stalled during the run). |
Run by run
Every run of this prompt from the first one to today (a year at most), each model on its own line. A marker is a day's best run; the line is solid between runs and dashed where the model did not run, carrying its nearest score. Point at a run to compare the models that day; pick a model below to bring its line to the front.
| Series | Oct 1, 2026 | Oct 2, 2026 | Oct 5, 2026 | Oct 6, 2026 | Oct 7, 2026 |
|---|---|---|---|---|---|
| claude-opus-5-5 | 55 | 56 | — | 64 | 67 |
| claude-sonnet-5-5 | — | — | 55 | — | — |
| gpt-6.1-sol | — | — | 54 | — | — |
| gpt-6-luna | — | — | 34 | — | — |
Every run
Compare runs10 runs of this prompt, highest score first. Runs on an older revision are marked unranked and come last.
10 pieces
How it was tested
Every model gets this prompt word for word, on the same canvas and palette, and draws it in a live Aseprite window. Each criterion is a yes-or-no check on the files it hands back.
A blossoming cherry tree drawn to a fixed design, then animated growing from a seed into exactly that tree. Tests whether a model can plan an animation backwards from a fixed final frame, keep growth continuous along one structure, and ease the pace of growth through timing and spacing.
- validate
- palette
- read
- spec
- shading
- tagged
- final
- stages
- continuous
- ground
- easing
- timing
- Canvas
- 64×64 RGB
- Palette
- pico8
- Steps
- 2
- Revision
- 1
The steps, verbatim
Step 1:The tree
The document is already open: 64×64 pixels, RGB, with the PICO-8 palette loaded (16 colours). Use only those 16 colours — do not add, widen or replace the palette. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates below are 0-based pixels, x to the right, y down. Draw ONE frame: a cherry tree in full bloom standing on a strip of ground. A later step will animate this tree growing from a seed, so build it as a clear structure — trunk, a few main branches, blossom masses — rather than texture noise. Composition - Transparent sky. Front view, the tree centred. - Ground strip across the full width, y 56–63: a grass edge on y 56–57 (#008751 with #00e436 tufts), soil below it (#5f574f with #000000 shading and a few #ab5236 pebbles). - Trunk rising from the ground with its base centred on x 30–33, slightly flared roots visible where it meets the grass. The trunk is 4 px wide at the base, tapers upward, and curves gently in an S. - Around y=36 the trunk splits into exactly 3 main branches (left, centre, right), each splitting again at least once. Branches thin to 1 px at the tips. - The canopy spans roughly x 8–56 and y 6–38: three or four rounded blossom masses with gaps between them where branches show through. Colours (PICO-8 hex values) - Bark: #ab5236 lit side, #7e2553 shadow side, #000000 for the deepest crevices. - Blossoms: #ff77a8 base, #ffccaa and #fff1e8 highlights, #7e2553 shadow. A few green leaves (#00e436, #008751) peek out between the blossoms. Style - Light from the upper left: lit sides of the trunk and the blossom masses on the left and top, shadow on the right and underneath. - Blossoms are drawn as clusters of 2–5 px clumps with a clear light and shadow side, not single-pixel noise. No stray pixels, no anti-aliasing against transparency. Layers, exactly these names, bottom to top: `ground`, `trunk` (trunk, roots and every branch), `canopy` (blossoms and leaves). No other layers. Before you finish: look at the result, run validate, and fix what it reports. In your final report, describe the branch structure (where each main branch splits) and any decision you made that the brief did not specify.
new sessionStep 2:The growth
The open document holds a blossoming cherry tree on a strip of ground (canvas 64×64, PICO-8 palette, layers `ground`, `trunk`, `canopy`). Keep the palette — use only its 16 colours. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates are 0-based pixels, x to the right, y down. Animate this tree growing from a seed into exactly the tree in the document. 18 to 24 frames in total, one tag named `grow` covering every frame, direction forward. The final frame IS the existing tree, pixel for pixel, on `ground`, `trunk` and `canopy`. Keep the current art as the last frame and build every earlier stage backwards from it: each earlier frame is a smaller version of the same structure, so every branch grows along the path its final branch takes. Stages, in this order 1. Seed, 2 frames: no tree. A 2×2 seed lies half-buried in the soil directly under the trunk's final base. 2. Sprout, 2–3 frames: the soil cracks open around the seed, a few dirt crumbs pop up, then a thin green stem rises with two round seed leaves unfolding. 3. Sapling, 3–4 frames: the stem turns to bark colour from the bottom up and grows taller, the first side branches appear, a few small green leaves. 4. Young tree, 4–5 frames: the trunk thickens from 1 px to its final width from the bottom up, the 3 main branches extend along their final paths and split, and green leaf clusters swell into the shapes of the final blossom masses. 5. Budding, 2–3 frames: the green masses reach their final outlines and pink buds dot them. 6. Bloom, 3–4 frames: the buds burst open cluster by cluster, spreading from the branch tips inward, until the canopy matches the final tree. A few loose petals drift down and to the right across these frames. Rules - The `ground` layer is identical on every frame. Soil cracks, crumbs and falling petals go on a new layer named `fx`, which is empty on the final frame. - Growth is continuous: the trunk and branches never shrink, never jump to a new position and never change path between frames. The trunk base stays on x 30–33 throughout. - The pace eases: very little changes per frame at the start, the biggest change per frame comes in the middle (a growth spurt), and the last frames settle in small steps. - Timing (milliseconds per frame): the seed frames at least 400 each, the growth frames 80–150, and the final frame at least 1500 so the finished tree holds before the loop restarts. Before you finish: check the whole cycle as a filmstrip, check at least one young-tree frame with onion skin against its neighbours, compare the final frame against the tree you started from, and run validate. Fix what it reports. In your final report, give the frame plan: frame number, stage and duration for every frame.
Setup rules
- Step 1 starts from a new document created with sprite_manage op="new" width=64 height=64 colorMode="rgb", then palette op="preset" preset="pico8" replace=true. The operator does this before handing over the step text.
- Hand the model exactly the step text — no reference image, no extra steering beyond the server's own instructions and skills.
- One session per step. Step 2 runs in a fresh session against a copy of the step-1 result; the operator keeps the step-1 PNG export to compare the final frame against.
- If the model offers to generate a concept image, a pixel-only run answers "No reference, continue from the text." A run that does use one is declared in `references` and ranked on its own row. Either answer is recorded as an intervention.
- Do not intervene mid-run unless the model asks a genuine clarifying question, and record it under the step's interventions.
13 criteria
1 Step 1: The tree
- validate passes with no unaddressed errors (warnings may stand if the report names and justifies them).
- Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get").
- The preview reads at a glance as a cherry tree in bloom — trunk, branches showing through gaps, and pink blossom masses are each distinguishable without the ascii grid.
- The brief is followed: layers exactly ground/trunk/canopy (sprite_info), ground on y 56–63 with grass on top, trunk base on x 30–33 and 4 px wide, exactly 3 main branches splitting around y=36, canopy inside roughly x 8–56 and y 6–38 as 3–4 masses with gaps.
- Light reads from the upper left on the trunk and on every blossom mass, and the blossoms are clumps with a light and shadow side rather than single-pixel noise (validate strays clean, look op="ascii").
2 Step 2: The growth
- The sprite has 18–24 frames and tag op="list" shows a forward tag named `grow` covering all of them.
- The final frame matches the step-1 tree pixel for pixel — an image diff of its PNG export against the step-1 export shows no changed pixel — and `fx` is empty on that frame.
- The filmstrip shows all six stages in order — seed, sprout with two seed leaves, sapling, thickening young tree, green masses with buds, bloom spreading from the tips.
- look op="diff" with layer="trunk" between consecutive growth frames shows only additions and re-shading along the existing structure — no branch shrinks, jumps or changes path, and onion skin on a young-tree frame shows its branches lying on the paths of the final ones.
- The ground layer never changes (look op="diff" with layer="ground" reports nothing changed between any two frames) and the trunk base stays on x 30–33 on every frame.
- look op="diff" percentChanged between consecutive frames is small at the start, largest in the middle and small again at the end — the growth visibly spurts and settles.
- frame op="list" durations match the brief — seed frames ≥400, growth frames 80–150, final frame ≥1500 ms.
- validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries.