Building AI agents that can seamlessly interact with external systems is a complex challenge. At Blaxel, we've been working on creating a platform that fast-tracks the developing of AI agents for developers. Along the way, we've encountered numerous technical hurdles with Model Context Protocol (MCP) servers, particularly around connection stability and scalability in cloud environments.

This article details our journey from using Server-Sent Events (SSE) to WebSockets, and the significant improvements we've seen as a result.

Understanding MCP: the foundation of modern AI agents

Before diving into our implementation challenges, let's briefly review what MCP is and why it matters. The Model Context Protocol (MCP) is an open standard developed by Anthropic that enables AI assistants to connect with external systems where data lives - including content repositories, business tools, and development environments.

MCP consists of three main components:

1. MCP Servers: These act as bridges connecting APIs, databases, or code to AI models, exposing data sources as tools. At Blaxel, we wanted to provide prebuilt and custom MCP servers for users.
2. MCP Clients: These use the protocol to interact with MCP servers and can be developed using SDKs in Python or TypeScript.
3. MCP Hosts: These systems manage communication between servers and clients, ensuring smooth data exchange.

The beauty of MCP is that tools provided by an MCP server can be accessed via any MCP host, allowing developers to connect AI agents to new tools without custom integration code. This standardization is what makes MCP so powerful for building extensible AI systems.

Our initial approach: HTTP handlers on Cloudflare

When we first started integrating MCP servers into Blaxel, we took what seemed like the most straightforward approach: implementing HTTP handlers on Cloudflare (which was already part of our stack) and mixing MCP with traditional APIs. This worked... sort of.

The problem? Adding a new MCP server was tedious and time-consuming. Each integration required hours of work, which is unsustainable when you're trying to build a platform batteries-included that needs to scale to support thousands of tools and millions of running agents. Furthermore, HTTP isn’t a standard transport in MCP, so there was no official support for it.

Standardizing MCP integration

To address the tedium of adding new MCP servers, we began looking for standardized ways to register them. Two key resources were kept as a result of our search:

1. Smithery: A registry of MCP servers that provides a standardized packaging format
2. MCP Hub: Our own open-source catalog of MCP servers designed to accelerate integration.

This infrastructure helped us organize our growing collection of MCP servers, but we still faced significant technical challenges with the underlying communication protocol.