# OpenUI Generative UI Benchmark > Agent-readable companion to the visual benchmark page. Version v1; model board updated 2026-08-25. Canonical page: https://www.openui.com/benchmarks ## Scope and definitions - 46 briefs, four generations per brief, a 16,384-token output ceiling, temperature 0.7 where supported, and minimal or no reasoning. - The six-model format comparison contains 1,104 runs per format. - Valid means: Parses, has a root, resolves every reference, has no orphaned or invented components, uses valid props, is not truncated, and meets the component-count floor. - Render success means: The format's own shipped renderer produced a non-blank screen. - Self-hosted cost is unknown and represented as null, not zero. - This is a first-party benchmark maintained by OpenUI. - Every row is scored by one build of the shipped OpenUI parser, lang-core 0.2.16. Raw outputs and per-run verdicts are committed for independent rescoring. ## Evidence status - Format comparison: published raw results, raw outputs, harness, and briefs are linked below. - OpenUI model board: summary measurements are published here, but row-level raw generations and pricing evidence are not yet linked. The format-comparison raw-results folder does not substantiate the model board. ## Format summary | Format | Valid | Render rate | Blank screens | System prompt tokens | Mean output tokens | | --- | ---: | ---: | ---: | ---: | ---: | | OpenUI | 96.9% | 99.9% | 1 | 5,031 | 1,362 | | A2UI | 95.6% | 96.6% | 37 | 12,610 | 2,823 | | json-render | 82.8% | 99.6% | 4 | 7,651 | 3,258 | ## OpenUI model board Models retain explicit family metadata, but family members are not connected in the chart. The chart draws one Pareto frontier. Models below 70% structural validity are unselected by default so the vertical scale can focus on the range worth comparing; both axes rescale to fit whatever is selected. | Model | Provider | Family | Valid | Cost per task | Cost type | Pareto frontier | Shown by default | | --- | --- | --- | ---: | ---: | --- | --- | --- | | Grok 4.6 | xAI | Grok 4.6 | 99.5% | $0.0185 | provider-list-price | yes | yes | | GPT-5.6 Sol | OpenAI | GPT-5.6 | 99.5% | $0.0476 | provider-list-price | no | yes | | Claude Opus 4.8 | Anthropic | Claude | 98.9% | $0.0493 | provider-list-price | no | yes | | Gemini 3.7 Flash | Google | Gemini Flash | 98.9% | $0.0100 | provider-list-price | yes | yes | | Claude Sonnet 5 | Anthropic | Claude | 98.4% | $0.0202 | provider-list-price | no | yes | | GPT-5.6 Terra | OpenAI | GPT-5.6 | 98.4% | $0.0241 | provider-list-price | no | yes | | Kimi K3 | Moonshot | Kimi K3 | 96.2% | $0.0333 | provider-list-price | no | yes | | Claude Opus 5 | Anthropic | Claude | 96.2% | $0.0774 | provider-list-price | no | yes | | Muse Spark 1.2 | Meta | Muse Spark 1.2 | 96.2% | $0.0157 | provider-list-price | no | yes | | Claude Sonnet 4.6 | Anthropic | Claude | 92.9% | $0.0450 | provider-list-price | no | yes | | Qwen3.8 2.4T | Alibaba | Qwen3.8 | 91.8% | $0.0170 | provider-list-price | no | yes | | GLM-5.3 | Zhipu | GLM-5.3 | 90.8% | $0.0120 | provider-list-price | no | yes | | Inkling Small | Thinking Machines | Inkling | 88.6% | $0.0033 | provider-list-price | yes | yes | | DeepSeek V4 Flash | DeepSeek | DeepSeek V4 | 85.8% | $0.0004 | provider-list-price | yes | yes | | DeepSeek V4 Pro | DeepSeek | DeepSeek V4 | 84.2% | $0.0089 | provider-list-price | no | yes | | GPT-5.6 Luna | OpenAI | GPT-5.6 | 83.7% | $0.0022 | provider-list-price | no | yes | | Qwen3.8 27B | Alibaba | Qwen3.8 | 78.8% | $0.0054 | provider-list-price | no | yes | | Gemini 3.6 Flash | Google | Gemini Flash | 78.8% | $0.0091 | provider-list-price | no | yes | | Gemini 3.5 Flash-Lite | Google | Gemini Flash | 78.3% | $0.0041 | provider-list-price | no | yes | | Inkling | Thinking Machines | Inkling | 73.4% | $0.0080 | provider-list-price | no | yes | | Qwen3.6 27B | Alibaba | Qwen3.6 | 68.5% | $0.0063 | provider-list-price | no | no | | Qwen3.6 35B-A3B | Alibaba | Qwen3.6 | 61.4% | $0.0017 | provider-list-price | no | no | | Gemma 4 31B | Google | Gemma 4 | 46.7% | $0.0009 | provider-list-price | no | no | | Phi-4 | Microsoft | Phi-4 | 44.0% | $0.0004 | provider-list-price | no | no | | Gemma 4 26B-A4B | Google | Gemma 4 | 29.9% | $0.0007 | provider-list-price | no | no | | Ministral 8B | Mistral | Ministral 8B | 27.2% | $0.0009 | provider-list-price | no | no | | Granite 4.1 8B | IBM | Granite 4.1 8B | 14.7% | $0.0002 | provider-list-price | yes | no | | LFM 2.5 2.6B | Liquid | LFM 2.5 2.6B | 3.3% | $0 / free | free | yes | no | | DiffusionGemma 26B-A4B | Google | DiffusionGemma 26B-A4B | 13.0% | not comparable | self-hosted | no | no | | Ling 3.0 Tiny | InclusionAI | Ling 3.0 Tiny | 9.8% | not comparable | self-hosted | no | no | ## Direct answers ### How large is the benchmark? 46 interface briefs are used. The format comparison covers 6 models and 3 formats for 3,312 scored runs; the OpenUI model board covers 30 models. ### What counts as a valid generation? A generation must parse, have a root, resolve every reference, contain no orphaned or invented components, use valid props, avoid truncation, and meet the component-count floor. ### Which models have the highest OpenUI validity score? Grok 4.6 and GPT-5.6 Sol tie for the highest measured validity score at 99.5%. Cost determines which of these is Pareto-efficient; a high score alone does not imply frontier membership. ### How did OpenUI perform in the format comparison? OpenUI achieved 96.9% structural validity and 99.9% render success across 1,104 runs, with 1 blank screen. ### Which format used the fewest system-prompt tokens? OpenUI used the fewest system-prompt tokens at 5,031. ### Does a zero or missing cost mean a self-hosted model is free? No. Self-hosted models have no comparable API list price, so their cost is null. A numeric zero is reserved for a source reporting zero marginal API price. ### What does Pareto frontier mean in this benchmark? A priced model is on the frontier when no other comparably priced model is at least as valid while being strictly better on cost or validity. Frontier membership is a cost-quality trade-off, not a general model ranking. ### What do the two tabs at the top of the benchmark show? Model comparison shows OpenUI structural validity and cost across ${MODEL_BOARD_SIZE} models as provider-coloured dots with one Pareto frontier. Models scoring below 70% structural validity are deselected by default but remain in the data; both axes rescale to whatever is selected, so selecting a lower-scoring model extends the vertical scale down to reach it. Format comparison shows structural validity and cost across OpenUI, A2UI, and json-render for six models. Exact values for both views are present in the page's server-rendered chart data tables and machine-readable distributions. ## Machine-readable distributions - Focused language/model benchmark: https://www.openui.com/benchmarks/language - Focused framework benchmark: https://www.openui.com/benchmarks/framework - JSON Schema: https://www.openui.com/benchmarks/data.schema.json - JSON: https://www.openui.com/benchmarks/data.json - CSV: https://www.openui.com/benchmarks/data.csv - Methodology: https://www.openui.com/benchmarks/methodology - Raw results: https://github.com/thesysdev/generative-ui-bench/tree/main/results - Harness: https://github.com/thesysdev/generative-ui-bench - Briefs: https://github.com/thesysdev/generative-ui-bench/tree/main/briefs/briefs.ts