46 real screens, 6 models, 3 formats: 1,104 scored runs each
OpenUIBenchmarks
Generative UIBenchmark
Reliability vs cost
One line per format across all six models at list prices. Up and to the right is better: higher structural validity, less money.
99.9% render rate
Only 1 of 1,104 screens rendered blank.
96.5% valid
Parsed, rooted, and passed every structural check.
1.8 to 2.6× cheaper
Half the tokens: lower cost, faster streaming.
88% repaired in flight
A sanitizer model patches only the broken lines, via incremental editing.
Blank screens vs. renders
Did the generation actually render?
1,104 runs per format across 6 models.
Counted as each SDK’s own renderer produced them.
OpenUI had 1 blank screen in 1,104 runs, compared with 35 for A2UI and 4 for json-render.
Render rate
Runs that rendered against runs that came back blank, out of 184 per model.
Structural validity vs. render success
Was the output valid, and did anything render?
Valid: every part it refers to exists, and every setting is a real one. Render success: something appeared at all. 1,104 runs per format.
OpenUI leads on both; A2UI and json-render each trade one off against the other.
Structural validity and render success
Share of 1,104 runs per format; longer is better. Scales start at 75% and 95%, not zero. Arrows compare with OpenUI.
What counts as valid. Parses, has a root, every reference resolves, nothing orphaned or invented, no missing or out-of-range props, not truncated, and at least as many components as the brief has requirements. That last one is a count floor, not a check that each requirement was addressed: this measures structure, not coverage.
Structural validity by model
How often the component graph holds together
Passes only if nothing is left dangling, nothing is invented, and every setting is valid. 184 runs per model, per format.
OpenUI leads overall: 96.5% vs 95.7% for A2UI. The lead changes by model, with two ties.
Structural validity by model
Runs whose component graph parses, resolves and validates, judged by each format's own SDK.
| Model | OpenUI | A2UI | json-render |
|---|---|---|---|
| SolOpenAI | 99.5 | 96.2 | 82.6 |
| Claude Opus 4.8Anthropic | 98.9 | 99.5 | 87.5 |
| Kimi K3Moonshot | 96.7 | 95.7 | 71.2 |
| Gemini 3.6 FlashGoogle | 95.1 | 95.1 | 77.2 |
| Qwen3.8 2.4TAlibaba | 91.8 | 91.3 | 81.0 |
| Muse Spark 1.2Meta | 96.7 | 96.7 | 81.5 |
| Average | 96.5 | 95.7 | 80.2 |
Token consumption and cost
Tokens and dollars for the same screens
46 screens,
priced at provider list prices.
OpenUI uses fewer tokens and costs 1.8–2.6× less across every priced model.
Token consumption
System prompt + output for one screen
Cost of one benchmark pass
The same 46 screens at list prices
| Model | OpenUI | A2UI | json-render |
|---|---|---|---|
| Gemini 3.6 FlashGoogle | $0.42 | $0.95↑2.3× | $0.85↑2.0× |
| Muse Spark 1.2Meta | $0.71 | $1.38↑1.9× | $1.34↑1.9× |
| Qwen3.8 2.4TAlibaba | $0.81 | $1.90↑2.3× | $1.46↑1.8× |
| Kimi K3Moonshot | $1.53 | $3.16↑2.1× | $2.91↑1.9× |
| Claude Opus 4.8Anthropic | $2.27 | $5.85↑2.6× | $4.88↑2.1× |
| SolOpenAI | $3.08 | $6.84↑2.2× | $6.30↑2.0× |
Streaming speed
How long a screen takes to render
Mean output per screen,
decoded at 50 tokens per second.
A screen streams in about half the time, because there is about half as much to write.
Time to stream a screen
Mean output per screen, decoded at 50 tokens per second.
Structural validity by screen complexity
Validity as requirements increase
46 briefs across 5 complexity bands,
about 9 per band.
As screens get harder, OpenUI and A2UI stay above 90%; json-render falls to 72%.
Structural validity by screen complexity
The more a brief asks for, the less every format delivers. OpenUI and A2UI track each other closely at every size; json-render falls fastest and furthest.
Production repair
What gets fixed before users see it
Based on real production data:
OpenUI Cloud traffic, not benchmark runs.
Only 0.9% of generations reach a user broken. Of the ones that fail validation, 88% are repaired via incremental editing.
Repair funnel
What happens to a broken generation before it can reach a user.