Measured performance
Benchmarks
The model matters. So does how it is served. Explore measured quality, cost and time across providers and precisions.
The model matters. So does how it is served. Explore measured quality, cost and time across providers and precisions.
All benchmarks · Equal-weight average · Higher scores are better
Recorded benchmark charges · USD / task · upper left is better
Full response time · seconds · upper left is better
audnai: penclaw-GLM-5.3-abliterated
5/5 benchmarks · Provisional · 49.5% mean dataset coverage · Includes: GPQA Diamond, IFBench, MMLU-Pro, MATH-500, AIME 2025
69.4% · $0.00164Recorded / task · 267.3s
| Model / serving variant | Score | Cost / task | Time | Tested |
|---|---|---|---|---|
| 69.4% | $0.00164Recorded | 267.3s | 5/5 benchmarksProvisional | |
| 64.9% | $0.000115Recorded | 12.9s | 5/5 benchmarksComplete | |
| 87.9% | $0.000218Recorded | 82s | 5/5 benchmarksProvisional | |
| 88.6% | $0.000579Recorded | 176.1s | 5/5 benchmarksProvisional | |
| 78.7% | $0.000349Recorded | 136.9s | 5/5 benchmarksProvisional | |
| 83.9% | $0.000688Recorded | 188.9s | 5/5 benchmarksProvisional | |
| 78% | $0.000412Recorded | 127s | 5/5 benchmarksProvisional | |
| 68.8% | $0.00107Recorded | 34s | 5/5 benchmarksProvisional | |
| 84.6% | $0.00423Recorded | 163.4s | 5/5 benchmarksProvisional | |
| 80.7% | $0.000148Recorded | 108.4s | 5/5 benchmarksProvisional | |
| 80.2% | $0.000303Recorded | 113.5s | 5/5 benchmarksProvisional | |
| 72.4% | $0.000425Recorded | 74.6s | 5/5 benchmarksProvisional | |
| 74.3% | $0.00014Recorded | 120.5s | 5/5 benchmarksProvisional | |
| 81.8% | — | 83.1s | 5/5 benchmarksProvisional | |
| 89.3% | $0.00108Recorded | 63.1s | 5/5 benchmarksProvisional |
Each benchmark has equal weight. Available provisional scores contribute; missing scores are omitted. Different benchmark mixes and tested samples can change the comparison. Cost and time average the same contributing benchmarks and remain unknown if any contribution is unmeasured.
Explore individual benchmarks
Benchmarks compare models as served, including their provider output limits. Individual views expose task evidence and request settings.
Benchmark scores and the evidence behind them. Each line represents one serving variant.
Measured accuracy · higher is better
Completed tasks / full dataset · farther means more evidence
Both scales run from 0 to 100%. Score spokes use the most represented protocol among the selected variants for each benchmark. Provider output limits are included as serving characteristics. Missing measurements leave gaps. Hollow score markers are provisional and can use different question samples. Dashed, unfilled outlines connect available provisional scores; gaps remain open. Dataset coverage measures tested tasks, not confidence or quality. Polygon area is not an overall rating.
Each column keeps its own metric. Brightness indicates a better score within that benchmark; provisional cells show the tested sample. Unscored cells distinguish collection progress, blocked runs and unavailable evaluators.
| Serving variant | GPQA DiamondAccuracy | IFBenchPrompt-level loose accuracy | MMLU-ProAccuracy | MATH-500Accuracy | AIME 2025Accuracy |
|---|---|---|---|---|---|
| audnai: penclaw-GLM-5.3-abliteratedConsensus Protocol · INT4 | |||||
| MiniMax: MiniMax M3Consensus Protocol · INT4 | |||||
| google: gemma-4-31B-itConsensus Protocol · INT8 | |||||
| XiaomiMiMo: MiMo-V2.6-Flash-RLConsensus Protocol · FP8 | |||||
| MiniMaxAI: MiniMax-M2.7Consensus Protocol · INT4 | |||||
| Z.ai: GLM 5.3 FlashImpCC · INT4 | |||||
| Z.ai: GLM 5.3 FlashConsensus Protocol · INT4 | |||||
| cebeuq: DeepSeek-V4-Flash-0731-abliteratedConsensus Protocol · FP8 | |||||
| Qwen: Qwen3.8 Flash Next Abliterated (DealignAI)ImpCC · NVFP4 | |||||
| deepseek-ai: DeepSeek-V4.1-FlashConsensus Protocol · FP4 | |||||
| openbmb: MiniCPM5-2BImpCC · BF16 | |||||
| Xiaomi: MiMo V2.6 Flash RL Abliterated (DealignAI)ImpCC · MXFP4 | |||||
| openai: gpt-oss-20bConsensus Protocol · MXFP4 | |||||
| Qwen: Qwen3.8 27BConsensus Protocol · FP8 | |||||
| Qwen: Qwen3.8 Flash NextImpCC · INT4 | |||||
| MiniMax: MiniMax M3ImpCC · INT4 |
All benchmarks averages available scores equally within this inference type. Provisional scores contribute before a dataset is complete. Missing benchmarks are omitted and listed with each average; benchmark coverage is separate from the tested share of each dataset.
Only currently served configurations and measurements from the last 30 days qualify. Provisional scores appear after five scored questions and show the exact dataset coverage. These measurements describe a provider’s serving configuration, including its quantization.