Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.
-
Updated
Feb 11, 2026 - Python
#
ai-benchmark
Here are 39 public repositories matching this topic...
An agent benchmark with tasks in a simulated software company.
-
Updated
Nov 17, 2025 - Python
Frontier Models playing the board game Diplomacy.
-
Updated
Feb 11, 2026 - Python
Ranking LLMs on agentic tasks
-
Updated
Nov 18, 2025 - Jupyter Notebook
🤖 A curated list of resources for testing AI agents - frameworks, methodologies, benchmarks, tools, and best practices for ensuring reliable, safe, and effective autonomous AI systems
testing qa benchmark machine-learning evaluation chaos artificial-intelligence chaos-monkey testing-tools awesome-list quality-assurance ai-safety ai-agents chaos-engineering llm llm-evaluation agentic-ai ai-benchmark agent-evaluation
-
Updated
May 28, 2025
Benchmark for evaluating AI epistemic reliability - testing how well LLMs handle uncertainty, avoid hallucinations, and acknowledge what they don't know.
python nlp benchmark machine-learning ai evaluation collaborate ai-safety llm llm-evaluation hallucination-detection ai-benchmark llm-benchmark
-
Updated
Jan 2, 2026 - Python
GTA (Guess The Algorithm) Benchmark - A tool for testing AI reasoning capabilities
python docker flask benchmark machine-learning puzzle reverse-engineering educational pattern-recognition ctf binary-analysis algorithm-analysis computational-thinking algorithmic-reasoning ai-benchmark
-
Updated
Jan 12, 2025 - Python
MindTrial: Evaluate and compare AI language models (LLMs) on text-based tasks with optional file/image attachments and tool use. Supports multiple providers (OpenAI, Google, Anthropic, DeepSeek, Mistral AI, xAI, Alibaba, Moonshot AI, OpenRouter), custom tasks in YAML, and HTML/CSV reports.
opensource openai xai artificial-intelligence-projects anthropic ai-tool openrouter qwen deepseek mistral-ai llm-evaluation-framework google-gemini-ai llm-benchmarking moonshot-ai language-models-ai llm-comparison ai-benchmark ai-evaluation-tools grok-ai ai-model-comparison
-
Updated
Feb 28, 2026 - Go
This repository contains the results and code for the MLPerf™ Storage v2.0 benchmark.
-
Updated
Aug 4, 2025 - Python
The first comprehensive benchmark for evaluating AI coding agents on Salesforce development tasks. Tests Apex, LWC, Flows, and more.
-
Updated
Jan 27, 2026 - Python
A curated list of evaluation tools, benchmark datasets, leaderboards, frameworks, and resources for assessing model performance.
-
Updated
Jan 16, 2026 - Python
From Benchmarks to Architecture — We tested 30+ AI APIs, designed routing from the data, then Anthropic published the same patterns. 3-part series.
multilingual api-gateway mcp fallback gemini openai api-testing mistral ai-agents ai-api groq llm cerebras deepseek tavily ai-benchmark api-benchmark llm-benchmark provider-routing language-aware-routing
-
Updated
Feb 24, 2026
La Perf is a framework for AI performance benchmarking — covering LLMs, VLMs, embeddings, with power-metrics collection.
cuda mps mlx nvidia-gpu apple-silicon ollama lmstudio ai-performance ai-benchmark ml-benchmark open-source-benchmark
-
Updated
Dec 1, 2025 - Python
Python Performance Tester & More...
speedtest performance-analysis specs geekbench performance-test computer-specs gpu-testing python-performance python-performance-test pypc-tests mitchell-shibilski-unkel pypc gpu-test ai-benchmark
-
Updated
Feb 15, 2026 - Python
TrustyAI's LMEval provider for Llama Stack
-
Updated
Feb 23, 2026 - Python
PlayBench is a platform that evaluates AI models by having them compete in various games and creative tasks. Unlike traditional benchmarks that focus on text generation quality or factual knowledge, PlayBench tests models on skills like strategic thinking, pattern recognition, and creative problem-solving.
-
Updated
Mar 3, 2026 - Blade
A technical guide and live-tracking repository for the world's top AI models, specialized by coding, reasoning, and multimodal performance.
-
Updated
Feb 26, 2026
The ultimate benchmark for AI coding agents. Give an AI $500, an empty vending machine, and 90 days — race Claude, Codex, Gemini head-to-head with a live dashboard.
python flask simulation websocket gemini codex vending-machine ai-agents claude race-mode llm-evaluation ai-benchmark
-
Updated
Feb 27, 2026 - Python
AccX Clocking Kiosk — offline-first tablet app for time tracking and fire safety. Humans: run .harness/harness.sh (pw: changeme)
-
Updated
Feb 8, 2026 - Shell
Benchmark local LLM models: speed, quality, and hardware fitness scoring. CLI, MCP server, and IDE plugins.
-
Updated
Mar 3, 2026 - TypeScript
Improve this page
Add a description, image, and links to the ai-benchmark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the ai-benchmark topic, visit your repo's landing page and select "manage topics."