Summary
NVIDIA's enterprise benchmarking platform that runs your LLMs through 100+ evaluation tasks from 18+ harnesses including MMLU, HumanEval, and GSM8K. Works with any OpenAI-compatible endpoint and handles execution across local Docker, Slurm HPC clusters, or cloud platforms. The containerized approach means reproducible results, and you get built-in exports to MLflow and Weights & Biases. If you're running evals on a single machine with simpler needs, lm-evaluation-harness is lighter weight. But if you're benchmarking at scale across infrastructure or need that full harness coverage in one tool, this delivers the industrial-grade setup.
Install to Claude Code
npx -y skills add orchestra-research/ai-research-skills --skill nemo-evaluator-sdk --agent claude-codeInstalls into .claude/skills of the current project.
Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
On Capafy, your Skill runs online 24/7 as an agent product, and you get paid every time someone uses it.
Start earning →
Files
SKILL.md
Select a file.
Featured
Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →On Capafy, your Skill runs online 24/7 as an agent product, and you get paid every time someone uses it.
Start earning →
