What this benchmark answers
This is a controlled comparison of three generative UI formats: OpenUI, Google A2UI, and Vercel json-render. It covers 6 models, 46 briefs, and four generations per brief: 1,104 runs per format and 3,312 scored runs in total.
Structural validity and render success are separate. A generation can put something on screen while still failing the complete structural checks.
Format summary
| Format | Structural validity | Render success | Blank screens | Runs |
|---|---|---|---|---|
| OpenUI | 96.9% | 99.9% | 1 | 1,104 |
| A2UI | 95.6% | 96.6% | 37 | 1,104 |
| json-render | 82.8% | 99.6% | 4 | 1,104 |
Results by model and format
| Model | Format | Valid runs | Validity | Render success | Cost / 46 screens |
|---|---|---|---|---|---|
| GPT-5.6 Sol | OpenUI | 183/184 | 99.5% | 100.0% | $3.08 |
| GPT-5.6 Sol | A2UI | 177/184 | 96.2% | 97.3% | $6.84 |
| GPT-5.6 Sol | json-render | 152/184 | 82.6% | 100.0% | $6.30 |
| Claude Opus 4.8 | OpenUI | 182/184 | 98.9% | 100.0% | $2.27 |
| Claude Opus 4.8 | A2UI | 183/184 | 99.5% | 100.0% | $5.85 |
| Claude Opus 4.8 | json-render | 161/184 | 87.5% | 100.0% | $4.88 |
| Kimi K3 | OpenUI | 177/184 | 96.2% | 100.0% | $1.53 |
| Kimi K3 | A2UI | 176/184 | 95.7% | 97.3% | $3.16 |
| Kimi K3 | json-render | 131/184 | 71.2% | 100.0% | $2.91 |
| Gemini 3.7 Flash | OpenUI | 182/184 | 98.9% | 100.0% | $0.42 |
| Gemini 3.7 Flash | A2UI | 173/184 | 94.0% | 96.7% | $0.95 |
| Gemini 3.7 Flash | json-render | 171/184 | 92.9% | 100.0% | $0.85 |
| Qwen3.8 2.4T | OpenUI | 169/184 | 91.8% | 99.5% | $0.81 |
| Qwen3.8 2.4T | A2UI | 168/184 | 91.3% | 91.3% | $1.90 |
| Qwen3.8 2.4T | json-render | 149/184 | 81.0% | 97.8% | $1.46 |
| Muse Spark 1.2 | OpenUI | 177/184 | 96.2% | 100.0% | $0.71 |
| Muse Spark 1.2 | A2UI | 178/184 | 96.7% | 97.3% | $1.38 |
| Muse Spark 1.2 | json-render | 150/184 | 81.5% | 100.0% | $1.34 |
Controlled condition
- One shared catalog of 70 equivalent components.
- Each format uses the prompt generated by its own SDK.
- The same two worked examples are carried by all three prompts.
- Four generations per brief and a 16,384-token output ceiling.
- Temperature 0.7 where supported, with minimal or no reasoning.
- Each SDK’s shipped validation plus one identical shared completeness layer and component-count floor.
Every row is scored by one build of the shipped OpenUI parser, lang-core 0.2.16. See the complete methodology and reproduction steps.