| Post Reply | RetroFed Forum Forum Index » LocalLLaMA |
| Author | Message | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
humanspiral
lemmy.ca
humanspiral@lemmy.ca Joined: 2024-10-23 |
AI Intelligence and Benchmarking Cost (Feb 2026)As per the Artificial Analysis Intelligence Index v4.0 (February 2026), the scoring ceiling is set by Claude Opus 4.6 (max) at 53. Adjusted Score FormulaThe "Adjusted Score" follows a quadratic penalty formula:
This creates a steeper penalty for performance gaps compared to a linear scale. Model Comparison Table
*Benchmark costs for proprietary models are based on Artificial Analysis evaluation token counts (typically 12M–88M depending on verbosity) multiplied by current API rates. Key Insights
Visual Summary
Source: Artificial Analysis Intelligence Index v4.0, February 2026 google AI mode made analysis, GLM 5 formatted and added cute graph. this combines the intelligence score and cost to run the intelligence benchmark from https://artificialanalysis.ai/?endpoints=openai_gpt-5-2-codex%2Cazure_kimi-k2-thinking%2Camazon-bedrock_qwen3-coder-480b-a35b-instruct%2Camazon-bedrock_qwen3-coder-30b-a3b-instruct%2Ctogetherai_minimax-m2-5_fp4%2Ctogetherai_glm-5_fp4%2Ctogetherai_qwen3-next-80b-a3b-reasoning%2Cgoogle_gemini-3-pro_ai-studio%2Cgoogle_glm-4-7%2Cmoonshot-ai_kimi-k2-thinking_turbo%2Cnovita_glm-5_fp8 look at intelligence vs cost graph for further insight. You can add much smaller models for comparison to LLMs you might run locally. The adjusted intelligence/cost metric is a useful heuristic for "how much would you pay extra to get top score". Choosing non-open models requires a much higher penalty than 2x the difference/comparison to highest score. Quantized versions don't seem to score lower. This site provides good base info to make your own model of "score deficit", model size, tps as a combined score relative to tokens/cost to get a benchmark score. I was originally researching how grok 4.2 approach would inflate costs vs performance, but it is not yet benchmarked. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 0 Original | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Post Reply |
Back to top |