Design

Models can now build an interface they consider finished. A designer looking at it sees the same purple gradient and centered hero on every brief, and type nobody chose on purpose. Taste is hard to grade, so most design evals settle for a crowd vote on a screenshot. We're looking for evals that pin down one part of taste well enough to score, and show which models have it.

See How is it judged? for what judges look for.

Ideas for an eval

These are only examples to start from. The best ideas will come from what you already know, or from more research into what your field lacks.

• A taste benchmark that runs the same prompts through different design skills and tools, Lovable and open-source ones included, and has people rank the results blind. Closest so far: UI-Bench.
• A design-system benchmark that hands the model a set of tokens and components, then counts every color, size and radius it invents instead. See open-design-system-bench.
• A style-fidelity benchmark that gives a reference site and asks for a new page in its style, scored on how closely the result follows the reference. See RefCraft.
• A critique benchmark that hides faults in good pages, scored on how many a model's review finds and how often it flags a page with nothing wrong. See ScreenProof.

Starting points

Evals that already exist in this field. Learn what they measure, then measure what they miss.

• Visual Aesthetic BenchmarkBake AI and University of Washington

  Pick the best and worst of a matched set of images; experts agree 68.9% of the time, and the best model manages 26.5%.

• Human Creativity BenchmarkContra Labs

  Professional creatives judge landing pages, apps, ads and video in pairs; no model leads every phase in any domain.

• AIMACGAAD Foundation and ServiceNow

  Top models build pages in 28 categories, then a browser audit counts the accessibility failures each one leaves behind.

• Design Arena: WebsitesDesign Arena

  People vote blind between two sites built from the same prompt; the clearest public signal of web taste.

• Design Arena: SlidesDesign Arena

  Agents build a deck and people vote on its design; the gaps between models are wide and easy to see.

• Design Arena: SVGDesign Arena

  Blind votes on vector graphics from the same prompt: logos, icons and illustration.

• Vibe Code BenchVals AI

  Build a whole web app from a plain-English spec, tested by clicking through it; no model passes every test.

• UI/UX Image ArenaArtificial Analysis

  Blind votes on interface mockups from image models, where type and layout still give them away.