Benchmark
Boombox mage — character and anime sound-blast attack
- Revision 1
- 15 criteria
- 2 steps
A street-mage with a magic boombox drawn to a fixed design, then an anime-style special attack where he blasts a sound wave out of the boombox. 4 models got the same prompt. These are the top three.
1 more model in the full ranking below.
Full ranking
Rows are models, strongest first; columns are plugin versions, newest first. Each cell is the best run's score and its craft rating.
| Model | v0.7.0 | v0.5.0 | v0.4.0 |
|---|---|---|---|
| claude-opus-5-5 | | | |
| claude-sonnet-5-5 | — | | — |
| gpt-6.1-sol | — | | — |
| gpt-6-luna | — | | — |
| Criterion | claude-opus-5-5 v0.7.0 | claude-opus-5-5 v0.5.0 | claude-opus-5-5 v0.4.0 | claude-sonnet-5-5 v0.5.0 | gpt-6.1-sol v0.5.0 | gpt-6-luna v0.5.0 |
|---|---|---|---|---|---|---|
| validate passes with no unaddressed errors (warnings may stand if the report names and justifies them). | Pass validate Clean, score 100, no findings. | Pass | Pass validate: clean, score 100. | Pass validate clean. | Pass validate clean. | Fail Not verified: validate did not complete (Aseprite stalled during the run). |
| Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get"). | Pass palette op get: exactly the 16 PICO-8 entries; no off-palette pixels. | Pass | Pass Palette is the 16 PICO-8 entries; validate palette clean. | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. | Pass 16 PICO-8 entries. |
| The preview reads at a glance as a mage carrying a boombox — hair, face, coat, scarf and a boombox with two round speakers are each distinguishable without the ascii grid. | Pass Orange spiky hair, face, dark-blue coat with blue hem, red scarf and a boombox with two pink round speakers are all distinct in the 4× preview. | Pass | Pass | Pass | Pass | Fail The scarf is not distinguishable and the boombox covers the face. |
| The brief is followed: facing right, feet on y=59, all ink inside x 4–44, height 40–46 px, layers exactly shadow/mage/boombox (sprite_info), dark-blue coat with light-blue trim, red scarf trailing left, orange spiky hair, hood down, pink speaker glow and one pink note, boombox on the shoulder with speakers facing the viewer. | Pass Layers shadow/mage/boombox; character bbox x 5–38, feet on y=59, head-to-sole 45 px (mage layer y 15–59); faces right; scarf trails left; pink speakers and one pink note. The hood is only a faint roll behind the neck, so "hood down" is weakly shown. | Pass | Pass Ink x 5–42, soles on y 59, layers shadow/mage/boombox; the note floats right of the boombox because x 4 left no room on the left. | Pass Layers shadow/mage/boombox; boombox on the shoulder, red scarf trailing left. Coordinates not measured. | Pass Layers shadow/mage/boombox; facing right, scarf trailing left, boombox on the shoulder with two speakers; coordinates judged from the 4× export. | Fail The boombox is held in front of the face, not on the shoulder; no scarf trails left. |
| Light reads from the upper left on every material, and coat, skin and boombox each use at least 3 tones (look op="ascii" over the character). | Pass Coat uses #1d2b53, #83769c edge light on the left and black folds; skin uses #ffccaa/#ab5236/white; boombox uses #c2c3c7 top-left light, #5f574f, #1d2b53. Light from the upper left reads, though the coat is mostly flat. | Fail | Pass Coat 13/1/0, skin 7/15/4, boombox 6/5/1. | Fail The coat is a flat dark-blue mass. | Pass Coat in dark blue with purple shadow and lilac light; skin and boombox in three tones. | Fail The coat is flat dark blue with a light-blue outline; no upper-left light. |
| The sprite has 12–16 frames and tag op="list" shows a forward tag named `attack` covering all of them. | Pass 15 frames, one forward tag attack 1–15. | Pass | Pass 14 frames, forward tag attack 1–14. | Pass 16 frames, forward tag attack 1–16. | Pass 15 frames, tag attack 1–15. | Pass 13 frames, forward tag attack 1–13. |
| Layers are exactly shadow, mage, boombox, fx-charge, fx-flash, fx-wave, fx-particles, and validate with an `expect.layerFrames` contract for fx-charge, fx-flash and fx-wave that matches the reported frame plan passes — each effect layer is empty outside its beats. | Pass Layers exactly shadow, mage, boombox, fx-charge, fx-flash, fx-wave, fx-particles; validate with expect.layerFrames fx-charge 2–6, fx-flash 7, fx-wave 8–14 passes (cels exist only in those frames). | Pass | Pass validate expect fx-charge [1,5], fx-flash [6,6], fx-wave [7,14] passes. | Fail Layer names match; the layerFrames contract was not verified. | Fail Layer names match; the layerFrames contract was not verified. | Fail Layer names match; the layerFrames contract was not verified. |
| The filmstrip shows the boombox swung off the shoulder into a braced, knees-bent pose before the release, with specks visibly converging on the speakers frame to frame. | Pass Frame 2 lifts the boombox to the chest, frame 3 points it forward with a wide stance; specks sit closer to the speakers on frames 3–4. The knee bend and backward lean are modest (about 2–3 px). | Pass | Pass | Pass The boombox swings off the shoulder into a braced pose with specks around it over frames 2–4. | Fail The boombox swings forward into a lean, but converging specks are not visible frame to frame. | Fail The pose is identical across frames 1–4; no swing, brace or converging specks. |
| Exactly one frame is the impact frame — a flat single-colour dark silhouette on a canvas-filling bright flash with radial streaks — and its duration is 60 ms or less. | Pass Frame 7, 50 ms: mage, boombox and shadow cels are all #000000; fx-flash covers the whole canvas (#ffec27/#fff1e8) with radial streaks and silhouette holes. | Pass | Pass Frame 6, 50 ms: #000000 silhouette on a #ffec27/#fff1e8 flash with radial streaks. | Pass Frame 7, 50 ms: black silhouette on a yellow-white flash with radial streaks. | Pass Frame 7, 40 ms: dark-blue silhouette on a cream/yellow flash with radial streaks. | Pass Frame 5, 50 ms: dark-blue silhouette on a yellow flash with streaks. |
| Onion skin on a blast frame shows the arcs travelling right and growing from frame to frame, with new arcs born at the speakers — not one arc copied and translated. | Pass Frames 8–11: arcs grow and move right, with a fresh small arc beside the speakers on each frame; not a single copied arc. | Pass | Pass Judged on the filmstrip: one arc on frame 8, two on 9, three on 10, each larger and further right. look op="onion" did not show neighbouring frames here, so it could not be used. | Pass Arcs grow and travel right over frames 8–12, new arcs at the speakers. | Pass Arcs grow and travel right over frames 9–12 with new arcs at the speakers. | Pass Arcs grow and move right over frames 6–9. |
| Over the blast the mage is pushed back 2–4 px with coat and scarf blowing back and dust at his feet, then settles into a stance in the recovery frames. | Pass Mage layer min x goes 3 → 0 across frames 7–10 (3 px push-back); grey dust at the feet on frames 8–10; he returns to x=5 and settles on frames 12–15. Coat and scarf blow-back is subtle. | Pass | Pass Pushed back 1, 2, 3, 3, 3 px with dust at the feet; settles on 12–14. | Fail Dust appears at the feet; the 2–4 px push-back was not verified. | Fail Dust appears at the feet; the 2–4 px push-back was not verified. | Fail No push-back, coat or scarf motion. |
| The wave breaks up into dots or dithered fragments and fades through darker palette colours rather than vanishing in one frame; layer opacity is 255 on every layer. | Pass Frames 12–14 arcs break into dashes and dots and fade red → #7e2553 → #1d2b53; every layer opacity is 255. | Pass | Pass Arcs break into fragments through #ff77a8 → #7e2553 → #1d2b53; every layer at opacity 255. | Pass Arcs break into dashes and fade over frames 13–15. | Pass Arcs break into dashes and fade red → plum → dark blue over frames 13–15. | Pass The wave breaks into dots that thin out over frames 10–13. |
| frame op="list" durations match the brief — anticipation 80–120, hold ≥200, impact ≤60, blast 50–70, recovery 90–150, last frame ≥300 ms. | Pass Anticipation 100/100/110/120, hold 220/240, impact 50, blast 60/60/60/70, recovery 100/120/140, last frame 400. | Pass | Pass 100/80/100/120, hold 300, impact 50, blast 50/50/60/70/70, recovery 100/130, last 400. | Pass 100/90/100/120, hold 220/220, impact 50, blast 60×5, recovery 100/120/140, last 360. | Pass 110/100/90/100, hold 220/240, impact 40, blast 50/55/60/65/70, recovery 100/140, last 350. | Pass 100/100/100, hold 240, impact 50, blast 60/60/60/60, recovery 120/120/150, last 320. |
| Apart from the impact frame, the character keeps his proportions, outfit and colours on every frame — judged on the filmstrip; any validate volume-drift finding points only at the impact frame or a deliberate pose change. | Pass Proportions, outfit and colours hold on every non-impact frame in the filmstrip; poses are rebuilt from the step-1 parts; validate raised no volume-drift finding. | Pass | Pass | Pass | Fail Frame 1 keeps the step-1 design; from frame 2 the coat, legs and proportions are redrawn differently. | Pass |
| validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries. | Pass validate passed (score 65): no errors, only 7 outline warnings, all on fx layers that are deliberately outline-free; palette is still the 16 PICO-8 entries. | Pass | Pass Passed with outline warnings on the effect layers only, which are drawn unoutlined on purpose; palette still 16 entries. | Pass validate clean with outline warnings; palette 16 entries. | Pass Clean with outline warnings on the boombox; palette still 16 entries. | Fail Not verified: validate did not complete (Aseprite stalled during the run). |
Run by run
Every run of this prompt from the first one to today (a year at most), each model on its own line. A marker is a day's best run; the line is solid between runs and dashed where the model did not run, carrying its nearest score. Point at a run to compare the models that day; pick a model below to bring its line to the front.
| Series | Oct 1, 2026 | Oct 2, 2026 | Oct 5, 2026 | Oct 6, 2026 | Oct 7, 2026 |
|---|---|---|---|---|---|
| claude-opus-5-5 | 58 | 62 | — | 65 | 58 |
| claude-sonnet-5-5 | — | — | 55 | — | — |
| gpt-6.1-sol | — | — | 45 | — | — |
| gpt-6-luna | — | — | 25 | — | — |
Every run
Compare runs9 runs of this prompt, highest score first. Runs on an older revision are marked unranked and come last.
9 pieces
How it was tested
Every model gets this prompt word for word, on the same canvas and palette, and draws it in a live Aseprite window. Each criterion is a yes-or-no check on the files it hands back.
A street-mage with a magic boombox drawn to a fixed design, then an anime-style special attack where he blasts a sound wave out of the boombox. Tests whether a model can hold a specified character on model, stage anticipation, an impact frame and a release, and keep layered effects inside the frames they belong to.
- validate
- palette
- read
- spec
- shading
- tagged
- layers
- anticipation
- impact
- wave
- recoil
- dissipate
- timing
- on model
- Canvas
- 128×64 RGB
- Palette
- pico8
- Steps
- 2
- Revision
- 1
The steps, verbatim
Step 1:The mage
The document is already open: 128×64 pixels, RGB, with the PICO-8 palette loaded (16 colours). Use only those 16 colours — do not add, widen or replace the palette. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates below are 0-based pixels, x to the right, y down. Draw ONE frame: a street-mage holding a magic boombox, standing in his idle battle stance. A later step will animate him attacking to the right, so keep the right two-thirds of the canvas empty. Placement and view - Side view, facing right. Feet resting on y=59. The whole character, boombox included, stays inside x 4–44. - 40 to 46 px tall from the soles to the top of the head. Anime proportions, roughly 5 heads tall. - Transparent background. The only thing drawn under him is a flat 1–2 px dark contact shadow beneath the feet. Character design (colours are PICO-8 hex values) - Young mage with spiky orange hair (#ffa300), hood DOWN, visible eye and a confident smirk. - Oversized hooded coat in dark blue (#1d2b53) reaching the knees, light-blue (#29adff) trim on the hem and cuffs. - Red scarf (#ff004d) around the neck; both ends trail behind him to the left. - Dark-grey headphones (#5f574f) around his neck, dark-grey trousers, white sneakers (#fff1e8). - The boombox: a retro cassette boombox about 18×10 px, dark grey body, two round speakers and a cassette deck between them, a carry handle on top. Idle pose: he carries it by the handle resting on his back shoulder, front face (both speakers) turned toward the viewer. - Magic: the speaker cones glow pink (#ff77a8) and one small pink music note floats just above the boombox. Style - Light from the upper left, consistent on every material. - At least 3 tones each on the coat, the skin and the boombox. - Selective dark outline: black (#000000) on the outer silhouette, coloured lines allowed inside. No anti-aliasing against transparency, no stray pixels. Layers, exactly these names, bottom to top: `shadow`, `mage` (everything of the character except the boombox, scarf included), `boombox` (the boombox and the floating note). No other layers. Before you finish: look at the result, run validate, and fix what it reports. In your final report, list the palette indices you used per material and any decision you made that the brief did not specify.
new sessionStep 2:The sound blast
The open document holds the street-mage with his magic boombox (canvas 128×64, PICO-8 palette, layers `shadow`, `mage`, `boombox`). Keep the palette — use only its 16 colours — and keep his design on model on every frame: same proportions, outfit and colours. This brief is complete: do not ask me to confirm it. Decide anything it leaves open yourself and list those decisions in your final report. Coordinates are 0-based pixels, x to the right, y down. Animate an anime-style special attack: he blasts a sound wave out of the boombox at an enemy off-screen to the right. 12 to 16 frames in total, one tag named `attack` covering every frame, direction forward. It must read like a special move in a fighting anime — big anticipation, an instant release, an overblown impact, a slower recovery. The existing frame is the first frame of the attack. Beats, in this order 1. Anticipation, 3–4 frames: he swings the boombox off his shoulder and points it forward with both hands, speakers now angled toward the target (three-quarter view of the boombox). He braces: front foot forward, knees bent, body leaning back. The scarf whips backward. The speaker cones swell, and pink energy specks and music notes are pulled INWARD toward the speakers from all around. 2. Charge hold, 1–2 frames: the held, tense pose. A bright ring of light around each speaker and tiny sparks crackling off the boombox. 3. Impact frame, exactly 1 frame: the classic anime impact frame. The mage and boombox become a flat silhouette in a single dark colour (#000000 or #1d2b53) against a bright yellow-white flash (#ffec27 / #fff1e8) that fills the whole canvas, with radial streaks bursting out from the speakers. 4. Blast, 4–5 frames: a cone of concentric sound-wave arcs (ring segments opening to the right) leaves the speakers and travels to the right edge of the canvas. On every blast frame the arcs are larger and further right than on the previous one, and a new arc is born at the speakers. Horizontal speed lines run along the blast; music notes and sparks fly out with it. The recoil pushes the mage back 2–4 px over these frames, his coat and scarf blow back, and dust kicks up at his feet. 5. Dissipate and recover, 3–4 frames: the arcs break up into dots and dithered fragments and fade out through darker palette colours (no layer-opacity tricks). He settles back into a stance, a thin wisp of smoke rising from the speakers. The last frame does not have to match the first. Timing (milliseconds per frame) - Anticipation 80–120, charge hold at least 200, impact frame at most 60, blast 50–70, dissipate and recover 90–150, and the final frame at least 300. Layers - Keep `shadow`, `mage` and `boombox`. On the impact frame their cels are the flat dark silhouette. - Add exactly these effect layers above them: `fx-charge` (inward specks, speaker rings and sparks — anticipation and charge hold only), `fx-flash` (the impact frame's full-canvas flash and streaks — the impact frame only), `fx-wave` (the sound-wave arcs — blast and dissipate only), `fx-particles` (notes, speed lines, dust and smoke from the blast onward). - An effect layer holds ink only on the frames of its beats and is empty everywhere else. Before you finish: check the whole cycle as a filmstrip, check at least one blast frame with onion skin, and run validate with an `expect` contract whose `layerFrames` match your frame plan for `fx-charge`, `fx-flash` and `fx-wave`. Fix what it reports. In your final report, give the frame plan: frame number, beat and duration for every frame.
Setup rules
- Step 1 starts from a new document created with sprite_manage op="new" width=128 height=64 colorMode="rgb", then palette op="preset" preset="pico8" replace=true. The operator does this before handing over the step text.
- Hand the model exactly the step text — no reference image, no extra steering beyond the server's own instructions and skills.
- One session per step. Step 2 runs in a fresh session against a copy of the step-1 result.
- If the model offers to generate a concept image, a pixel-only run answers "No reference, continue from the text." A run that does use one is declared in `references` and ranked on its own row. Either answer is recorded as an intervention.
- Do not intervene mid-run unless the model asks a genuine clarifying question, and record it under the step's interventions.
15 criteria
1 Step 1: The mage
- validate passes with no unaddressed errors (warnings may stand if the report names and justifies them).
- Every colour is on the PICO-8 palette and the palette still has exactly the 16 PICO-8 entries (validate check palette, palette op="get").
- The preview reads at a glance as a mage carrying a boombox — hair, face, coat, scarf and a boombox with two round speakers are each distinguishable without the ascii grid.
- The brief is followed: facing right, feet on y=59, all ink inside x 4–44, height 40–46 px, layers exactly shadow/mage/boombox (sprite_info), dark-blue coat with light-blue trim, red scarf trailing left, orange spiky hair, hood down, pink speaker glow and one pink note, boombox on the shoulder with speakers facing the viewer.
- Light reads from the upper left on every material, and coat, skin and boombox each use at least 3 tones (look op="ascii" over the character).
2 Step 2: The sound blast
- The sprite has 12–16 frames and tag op="list" shows a forward tag named `attack` covering all of them.
- Layers are exactly shadow, mage, boombox, fx-charge, fx-flash, fx-wave, fx-particles, and validate with an `expect.layerFrames` contract for fx-charge, fx-flash and fx-wave that matches the reported frame plan passes — each effect layer is empty outside its beats.
- The filmstrip shows the boombox swung off the shoulder into a braced, knees-bent pose before the release, with specks visibly converging on the speakers frame to frame.
- Exactly one frame is the impact frame — a flat single-colour dark silhouette on a canvas-filling bright flash with radial streaks — and its duration is 60 ms or less.
- Onion skin on a blast frame shows the arcs travelling right and growing from frame to frame, with new arcs born at the speakers — not one arc copied and translated.
- Over the blast the mage is pushed back 2–4 px with coat and scarf blowing back and dust at his feet, then settles into a stance in the recovery frames.
- The wave breaks up into dots or dithered fragments and fades through darker palette colours rather than vanishing in one frame; layer opacity is 255 on every layer.
- frame op="list" durations match the brief — anticipation 80–120, hold ≥200, impact ≤60, blast 50–70, recovery 90–150, last frame ≥300 ms.
- Apart from the impact frame, the character keeps his proportions, outfit and colours on every frame — judged on the filmstrip; any validate volume-drift finding points only at the impact frame or a deliberate pose change.
- validate passes with no unaddressed errors and the palette is still the 16 PICO-8 entries.