Summary
This server gives Claude direct access to FiftyOne's computer vision dataset platform through 45+ tools spanning three operational modes. You get SDK tools for querying datasets and running aggregations anywhere, APP tools for controlling the live FiftyOne browser interface (set views, open panels, select samples), and SESSION tools for launching the app from a terminal. The real power is in the operator system: it exposes FiftyOne's entire plugin ecosystem, letting the AI discover and execute any installed operator, from brain embeddings to annotation integrations. Each tool carries a risk level tag so you can auto-approve safe reads while prompting for destructive operations. Install it as a Claude Desktop server for terminal workflows or drop it into the FiftyOne App itself as an agent panel for real-time UI control.
Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
On Capafy, your Skill runs online 24/7 as an agent product, and you get paid every time someone uses it.
Start earning →
FiftyOne MCP Server
Control FiftyOne datasets through AI assistants using the Model Context Protocol
Documentation · FiftyOne Skills · FiftyOne Plugins · Discord
What is the FiftyOne MCP Server?
Enable Agents to explore datasets, execute operators, and control the FiftyOne App through natural language. This server exposes 45+ MCP tools across data operations, App UI control, and the full operator/plugin ecosystem.
"List all my datasets"
"Load quickstart dataset and show summary"
"Find similar images in my dataset"
The server starts with 50 built-in operators. Install plugins to expand functionality - the AI can discover and install plugins automatically when needed (brain, zoo, annotation, evaluation, and more).
Available Tools
| Category | Tools | Description |
|---|---|---|
| 📊 Dataset Management | 3 | List, load, and summarize datasets |
| 🎯 App Operations | 29 | Control the App UI (views, panels, selection, ...) |
| ⚡ Operator System | 3 | Discover and execute any FiftyOne operator |
| 🔄 Pipelines | 2 | Run pipelines and manage delegated operations |
| 🔌 Plugin Management | 5 | Discover, install, and manage plugins |
| 🖥️ Session | 1 | Launch the FiftyOne App server |
| 📈 Aggregations | 8 | Count, distinct, bounds, mean, histogram, ... |
| 🧬 Samples | 5 | Add, tag, untag, and set values on samples |
| 🗂️ Schema | 2 | Inspect and modify dataset field schemas |
| 🎨 App Config | 6 | Color scheme, sidebar groups, active fields |
Tool modes
45+ tools organized by runtime mode:
- SDK: Data operations that work everywhere (datasets, aggregations, schema, samples, operators, plugins). No App connection needed.
- APP: Controls the FiftyOne App UI in real time (set_view, open_panel, notify, select_samples, reload, and 25+ more). Requires a connected browser via
ctx.ops. - SESSION: Bootstrap tools for starting a local App server (launch_app). Used from terminal environments.
Choosing your tools
Which tools are available depends on how you integrate the server:
| Integration | Modes | Use case |
|---|---|---|
| FiftyOne App plugin | app + sdk | Agent panel inside the App (full UI control + data operations) |
| Terminal / CLI | session + sdk | Headless agent (launch the App, query data, execute operators) |
Tool risk levels
Every tool is tagged with a risk level that your agent can use for auto-approval decisions:
LOWSafe to auto-execute without prompting (read-only queries, UI state changes)OPERATORWraps a FiftyOne operator whose own severity should be checked before executing
Quick Start
Step 1: Install the MCP Server
pip install fiftyone-mcp-server
⚠️ Important: Make sure to use the same Python environment where you installed the MCP server when configuring your AI tool. If you installed it in a virtual environment or conda environment, you must activate that environment or specify the full path to the executable.
Step 2: Configure Your AI Tool
Claude Code (Recommended)
claude mcp add fiftyone -- fiftyone-mcp
Claude Desktop
Edit ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"fiftyone": {
"command": "fiftyone-mcp"
}
}
}
Cursor
Add to ~/.cursor/mcp.json:
{
"mcpServers": {
"fiftyone": {
"command": "fiftyone-mcp"
}
}
}
VSCode
Add to .vscode/mcp.json:
{
"servers": {
"fiftyone": {
"command": "fiftyone-mcp"
}
}
}
ChatGPT Desktop
Edit ~/Library/Application Support/ChatGPT/config.json:
{
"mcpServers": {
"fiftyone": {
"command": "fiftyone-mcp"
}
}
}
uvx (No Install Needed)
If you have uv installed:
{
"mcpServers": {
"fiftyone": {
"command": "uvx",
"args": ["fiftyone-mcp-server"]
}
}
}
This downloads and runs the latest version automatically.
Step 3: Use It
"List all my datasets"
"Load quickstart dataset and show summary"
"Open the map panel and show me the embeddings"
"Select samples with confidence above 0.9"
"What plugins are available? Install the brain plugin"
"Find near-duplicate images in my dataset"
Claude will automatically discover operators and execute the appropriate tools.
Contributing
We welcome contributions! Here's how to set up a local development environment:
-
Clone the repository
git clone https://github.com/voxel51/fiftyone-mcp-server.git cd fiftyone-mcp-server -
Install dependencies
poetry install -
Run the server locally
poetry run fiftyone-mcp -
Test your changes
poetry run pytest poetry run black -l 79 src/ npx @modelcontextprotocol/inspector poetry run fiftyone-mcp -
Submit a Pull Request
Resources
| Resource | Description |
|---|---|
| FiftyOne Docs | Official documentation |
| FiftyOne Skills | Expert workflows for AI assistants |
| FiftyOne Plugins | Official plugin collection |
| Model Context Protocol | MCP specification |
| PyPI Package | MCP server on PyPI |
| Discord Community | Get help and share ideas |
Community
Join the FiftyOne community to get help, share your ideas, and connect with other users:
- Discord: FiftyOne Community
- GitHub Issues: Report bugs or request features
Copyright 2017-2026, Voxel51, Inc. · Apache 2.0 License
Featured
Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →On Capafy, your Skill runs online 24/7 as an agent product, and you get paid every time someone uses it.
Start earning →