MCP Architecture: Hosts, Clients, and Servers

Scenario

You're building an AI app that needs to search Jira, query Postgres, and read from GitHub. You create one 'MCP Client' object in your code and try to point it at all three servers. The protocol instantly throws routing errors.

A developer tries to route traffic to three different MCP servers using a single instantiated MCP Client object and encounters protocol routing errors. What architectural rule did they break?

Mental model

The Host is the airport, the Client is a single boarding gate, and the Server is the specific plane.

In standard web development, the user-facing app is the 'client'. MCP splits this. The user-facing app is the Host. The Host spawns multiple internal Clients. Each Client is a dedicated, isolated pipe that connects to exactly one Server. You cannot share one Client across multiple Servers. The Server itself is dumb: it just exposes tools and data. It doesn't run an LLM, and it doesn't care if the Host is using Claude or GPT-4.

Deep dive

MCP uses a strict three-tier architecture: Hosts, Clients, and Servers. The Host is your actual application—the thing the user sees. It holds the LLM API keys, manages the chat interface, and orchestrates the overall flow. Claude Desktop and Cursor are examples of Hosts.

The Host is the top-level application that manages the LLM and the user interface.

When the Host needs to connect to an MCP Server, it doesn't do it directly. Instead, it spawns an internal MCP Client. This Client's only job is to negotiate the strict JSON-RPC 2.0 lifecycle with one Server: initialization (where capability negotiation happens), operation (making tool/resource requests), and shutdown. The relationship is strictly 1:1.

An MCP Client maintains a strict 1:1 connection with a single MCP Server through a three-phase lifecycle.

This 1:1 rule prevents routing chaos. If a Host needs to talk to a Jira server, a GitHub server, and a Postgres server, the Host MUST instantiate three separate MCP Clients. Attempting to multiplex traffic to multiple servers through a single Client object violates the protocol and causes immediate errors.

One Host spawns many Clients; each Client connects to exactly one Server.

The MCP Server sits at the end of the line. It is completely model-agnostic. It does not run LLM inference, it does not hold OpenAI API keys, and it doesn't process natural language. It merely exposes structured data and functions, waiting to respond to the JSON-RPC requests issued by the Host's model.

Servers do not run LLMs; they just expose functions and data to the Host.

This separation of concerns means you can swap the LLM in the Host at any time without touching the Servers. The Servers remain lightweight, stateless tool-providers, while the Host retains all the intelligence and orchestration.

Separating Host intelligence from Server capability makes the system model-agnostic.

Code examples

The broken mental model: 1-to-Many Client multiplexing

# WRONG: Attempting to share one client session across multiple servers
async with stdio_client(server1_params) as (read1, write1):
    async with ClientSession(read1, write1) as session:
        await session.initialize()
        # Trying to route a second server through the same session object
        await session.connect_to(server2_params)  # Pseudo-code: this does not exist!

# The protocol enforces 1:1 isolation. A single session cannot route to multiple servers.

Because a ClientSession manages a specific JSON-RPC lifecycle and capability state, pointing it at a second server is architecturally impossible. (API shape verified against MCP docs)

The correct architecture: Host spawns multiple Clients

# CORRECT: The Host application orchestrates multiple isolated sessions
async def connect_to_servers():
    # Instantiate separate, isolated clients for each capability
    async with stdio_client(jira_params) as (read_j, write_j):
        async with ClientSession(read_j, write_j) as jira_session:
            await jira_session.initialize()
            
            async with stdio_client(github_params) as (read_g, write_g):
                async with ClientSession(read_g, write_g) as github_session:
                    await github_session.initialize()
                    # The Host LLM routes to the correct session

This maps exactly to the real python-sdk. The Host manages the aggregate capabilities, but the connections remain strictly isolated within their own contexts. (API shape verified against MCP docs)

Common mistakes

• Confusing the Host with the Client: Because standard web architecture calls the frontend the 'client', developers assume the application using the LLM is the MCP Client. Reality: The application is the Host. The Host spawns multiple internal protocol Clients, each maintaining an isolated 1:1 connection with a Server.
• Assuming MCP Servers execute LLM inference: Because they are often called 'AI Servers', developers attempt to pass LLM API keys to the server, expecting it to process prompts. Reality: MCP Servers are completely model-agnostic. They do not run inference; they solely expose functions and data, responding to JSON-RPC requests issued by the Host's model.

Glossary

MCP Host

The top-level application (like Cursor or Claude Desktop) that initiates the process, holds the LLM, and manages the user interface.

MCP Client

An internal protocol object spawned by the Host that maintains a strict 1:1 JSON-RPC connection to exactly one MCP Server.

MCP Server

A lightweight, model-agnostic program that exposes data or tools over JSON-RPC, completely unaware of which LLM the Host is using.

Recall questions

• What is the structural difference between an MCP Host and an MCP Client?
• Can one client instance connect to multiple servers simultaneously?
• A developer attempts to pass an OpenAI API key to an MCP SQLite server, expecting the server to generate a summary of the database. Why won't this work?
• A developer tries to route traffic to three different MCP servers using a single instantiated MCP Client object and encounters protocol routing errors. What architectural rule did they break?

Questions & answers

What's the high-level architecture of MCP?

MCP uses a three-tier architecture consisting of Hosts, Clients, and Servers. The Host (like an AI IDE) manages the user and the LLM. It spawns internal Clients. Each Client establishes a strict 1:1 connection with a single Server, which exposes specific data or tools without knowing anything about the LLM being used.

Approach: Break it down into the three tiers. Emphasize that the Host orchestrates, the Client connects, and the Server provides.

What is the difference between an MCP server and an MCP client?

An MCP Client is a protocol object that initiates and manages a JSON-RPC session. An MCP Server sits at the other end of that connection, exposing capabilities (tools, resources). Crucially, one Client connects to exactly one Server, and the Server never executes LLM inference itself.

Approach: Highlight the 1:1 connection rule and clarify that the Server is a 'dumb' capability provider, while the Client is the active connection manager.

Continue learning

Previous: The M×N integration problem and protocol necessity

Next: MCP Primitives: Tools, Resources, and Prompts

Return to AI Engineering Roadmap