GPT-5.6 led this drawing test on cost and quality

A new drawing arena benchmark gave four vision models the same blank canvas and a colored-pencil toolset, then asked them either to recreate famous paintings or draw from text prompts. Across 28 drawings total, the results were uneven: GPT-5.6 Sol emerged as the strongest overall performer, Gemini 3.6 Flash posted the best raw similarity scores on the two target-image tests, Claude Fable 5 was by far the most expensive, and Grok 4.5 struggled.

The setup came from tryai.dev, which says the tests are intentionally open-ended rather than strict capability benchmarks. Each model could adjust color, tip width, and pressure, draw batches of strokes, smudge, erase, and inspect its work with view_canvas. The harness is open source at github.com/hershalb/canvas-arena.

The four models tested were GPT-5.6 Sol, Claude Fable 5, Grok 4.5, and Gemini 3.6 Flash. They were run against two scored target reproductions—the Mona Lisa and Van Gogh’s Starry Night—plus five prompt-only drawings: an elderly fisherman, a sunset over the ocean, a red rose in a glass vase, a sleeping tabby cat, and a cabin interior with a lit fireplace.

Cost, speed, and tool use

Averages and totals across all seven drawings per model showed sharp differences:

• GPT-5.6 Sol: 72 average score, 96.2 min, 3.4M tokens, $7.74, 6.0, 116
• Claude Fable 5: 75 average score, 412.5 min, 14.6M tokens, $160.58, 15.6, 207
• Grok 4.5: 79 average score, 94.8 min, 34.0M tokens, $9.21, 4.3, 354
• Gemini 3.6 Flash: 77 average score, 136.9 min, 27.7M tokens, $12.87, 22.6, 190

The models also used the same tools in very different ways. GPT-5.6 Sol never called set_color, set_brush, or set_pressure, instead setting those inline during draw calls. Grok 4.5 did the opposite: 65% of its 1,349 tool calls were those setup functions, pushing it to an average of 99 steps per drawing. Claude Fable 5 leaned heavily on smudge and frequent reviews, while Gemini 3.6 Flash was the most aggressive reviewer, with roughly 23 self-checks per drawing.

Recommended reading

Google Drops Gemini Sampling Controls in New Flash Models

The models often edited past their best result

On the two target-image tasks, the site scored runs using SSIM and RMSE. Gemini 3.6 Flash posted the highest raw SSIM on both targets, including a peak 0.449 on the Mona Lisa, but the benchmark authors caution that SSIM tracks structural similarity, not whether a human would think the drawing looks better.

A bigger pattern: more reviewing did not lead to better final results. According to the test, all eight target runs finished below the best score reached mid-run. Examples included:

• GPT-5.6 Sol, Mona Lisa: peaked at 0.352 SSIM, finished at 0.325
• GPT-5.6 Sol, Starry Night: peaked at 0.179, finished at 0.130
• Gemini 3.6 Flash, Mona Lisa: peaked at 0.449, finished at 0.337

The authors' qualitative verdict was more decisive than the raw metric leaderboard. They called GPT-5.6 Sol the “runaway leader,” saying its Starry Night and rose were their favorites of the batch and that it generally beat the others on detail. Claude Fable 5 was judged second on quality, but at roughly 20x the cost of GPT-5.6 Sol. Grok 4.5, in their words, was “basically garbage here,” while Gemini 3.6 Flash was seen as clearly better than Grok but not a fair like-for-like comparison with frontier-tier models.

The result is less a clean benchmark than a useful stress test for long, tool-using visual tasks—and one that makes the cost of repeated self-correction hard to ignore.