What is ContextForge Gateway?

Welcome to the ContextForge Gateway book.

contextforge-gateway-rs is the Rust dataplane for ContextForge: a single MCP entry point that sits in front of many backend MCP servers and makes them look like one. If you have used an API gateway or reverse proxy for HTTP APIs, this is the same idea for the Model Context Protocol (MCP).

Concretely, the gateway accepts MCP streamable HTTP traffic from a client, authenticates the caller, loads that caller’s runtime configuration, opens MCP client sessions to the backend MCP servers that configuration allows, and presents those backends to the client as one merged MCP server.

Throughout this book, downstream means the client side of the gateway and upstream means the backend side.

Blue arrows show request traffic. Green arrows show backend responses returning to the gateway and the merged MCP response going back to the client.

Key idea: downstream clients interact with one logical MCP server. The gateway decides which configured backend sessions are involved.

The gateway is not the management application. It does not own the UI, IAM lifecycle, tenant administration, durable metrics storage, or long-lived configuration editing. Those concerns stay in the external ContextForge control plane. This repository owns the hot request path and the runtime pieces needed to make that path fast, observable, and enforceable.

Most operators should not think about backend MCP servers directly. They should think about one client-facing MCP endpoint:

/contextforge-rs/servers/{virtual_host_id}/mcp

Behind that endpoint, the gateway has three main boundaries:

• the downstream boundary, where clients connect over streamable HTTP and present their bearer token and MCP session id
• the configuration boundary, where the gateway turns the JWT subject into a runtime UserConfig
• the upstream boundary, where the gateway opens or reuses MCP client sessions to the configured backend servers

Most bugs in this service come from confusing those boundaries. A downstream request is not allowed to pick arbitrary upstream backends. Redis is not the routing model; it is the current config-store transport. Backend sessions are not durable cluster state; they are local process state today.

Key terms

These words appear on almost every page. It is worth pinning them down once.

Mental model

The gateway is easiest to understand as one logical MCP server assembled from a set of configured backend MCP servers.

On initialize, the gateway validates the caller, resolves the requested virtual host, and creates upstream MCP client sessions for that virtual host’s backends. On list calls, it fans out to those backends and merges the result. On targeted calls, it uses an explicit tool alias, the only configured backend, or a multi-backend prefix to route to exactly one backend.

For a stateful MCP call to work, five facts must line up:

JWT subject
  -> user config
  -> virtual host id
  -> downstream MCP session id
  -> backend session map entries

If any part is missing, the gateway should fail at the layer that owns that fact: auth, config lookup, virtual host resolution, session lookup, routing, or upstream transport.

Operational consequence: stateful MCP traffic needs session affinity until backend session state moves out of the gateway process.

Non-goals

This repository should not grow control-plane behavior by accident. It should not become the admin UI, tenant management API, policy authoring system, credential store, or durable observability backend.

It also should not expose backend topology as more than the MCP routing contract requires. Backend map keys are visible when multi-backend identifiers need prefixes, but clients should still experience one gateway endpoint and one logical MCP server.

Where to go next

• To run the gateway yourself, start with Getting Started.
• To understand how requests move through it, read Architecture.
• For the exact client-facing protocol behavior, see MCP Behavior.

Getting Started

This section is for the first working run of the gateway: start the local dependencies, launch the Rust dataplane, seed runtime config, and send real MCP traffic through it.

🚀 Goal: get from a clean checkout to one client-facing MCP endpoint at /contextforge-rs/servers/{virtual_host_id}/mcp.

What “started” means

A useful local gateway run has these pieces:

If one of those is missing, the gateway should fail at the boundary that owns that fact: authentication, config lookup, virtual host resolution, session lookup, routing, or upstream transport.

Run the Gateway Locally

This page walks through a local run that proves the dataplane can authenticate a caller, load runtime config from Redis, initialize backend MCP sessions, and return merged MCP results to a downstream client.

The local flow uses the repository’s Docker Compose stack for Redis and two sample backend MCP servers:

🧪 The token and user-config endpoints below are local bootstrap helpers. They are compiled with contextforge-gateway-rs-lib/with_tools. In a real deployment, the external ContextForge control plane mints tokens and writes config to Redis.

Run the numbered steps in order, in the same terminal. Later steps reuse shell variables such as ${TOKEN} and ${SESSION_ID} that earlier steps set, so a fresh shell will not have them.

Prerequisites

• Rust toolchain matching the workspace rust-version.
• Docker Compose.
• Free local ports: 6379, 16379, 5555, 5556, and 8001. The Compose stack maps both plain (6379) and TLS (16379) Redis ports.
• Test keys under assets/: jwt.key and jwt.key.pub.

1. Start Redis and Backend MCP Servers

docker compose -f docker/docker-compose-local.yaml up -d

The sample backends default to a CPU limit of 8 and a reservation of 4, which Docker rejects on hosts with fewer CPUs (range of CPUs is from 0.01 to ...). On smaller machines, override the sizing knobs:

GATEWAY_CPU_LIMIT=2 GATEWAY_CPU_RESERVATION=0.5 \
  docker compose -f docker/docker-compose-local.yaml up -d

Check that the local dependencies are running:

docker compose -f docker/docker-compose-local.yaml ps redis gateway-one gateway-two

The backends listen on the host so the gateway can reach them at http://127.0.0.1:5555/mcp and http://127.0.0.1:5556/mcp.

2. Start the Gateway

For the local bootstrap flow, run the binary with the with_tools dependency feature so the admin token and config endpoints are available:

cargo run -p contextforge-gateway-rs \
  --features contextforge-gateway-rs-lib/with_tools \
  --bin contextforge-gateway-rs -- \
  --address 127.0.0.1:8001 \
  --redis-address 127.0.0.1 \
  --redis-port 6379 \
  --redis-mode plain-text \
  --token-verification-public-key assets/jwt.key.pub \
  --token-verification-private-key assets/jwt.key \
  --upstream-connection-mode plain-text-or-tls \
  --number-of-cpus 4

Keep this process running. The gateway exposes MCP traffic under:

http://127.0.0.1:8001/contextforge-rs/servers/{virtual_host_id}/mcp

The local command uses --upstream-connection-mode plain-text-or-tls because the sample backend URLs are plain HTTP. Without that option, the default upstream client is HTTPS-only.

3. Mint a Test Token

In another terminal, request a JWT for the test subject:

TOKEN=$(curl --silent --show-error \
  --url http://127.0.0.1:8001/contextforge-rs/admin/tokens/admin@example.com)

printf '%s\n' "${TOKEN}"

The token’s sub claim is admin@example.com. The gateway uses that subject as the Redis user-config key.

4. Write User Config

Seed Redis with one virtual host and two backend MCP servers:

curl --silent --show-error --request POST \
  --url http://127.0.0.1:8001/contextforge-rs/admin/userconfigs/admin@example.com \
  --header 'content-type: application/json' \
  --data '{
    "virtual_hosts": {
      "c0ffee00f001f00lf00ldeadbeefdead": {
        "backends": {
          "gateway-one": {
            "name": "gateway-one",
            "url": "http://127.0.0.1:5555/mcp",
            "transport": "STREAMABLEHTTP",
            "passthrough_headers": [],
            "allowed_tool_names": [],
            "allowed_resource_names": [],
            "allowed_prompt_names": []
          },
          "gateway-two": {
            "name": "gateway-two",
            "url": "http://127.0.0.1:5556/mcp",
            "transport": "STREAMABLEHTTP",
            "passthrough_headers": [],
            "allowed_tool_names": [],
            "allowed_resource_names": [],
            "allowed_prompt_names": []
          }
        }
      }
    }
  }'

The important relationship is:

JWT subject admin@example.com
  -> Redis user config
  -> virtual host c0ffee00f001f00lf00ldeadbeefdead
  -> backend MCP URLs

5. Initialize an MCP Session

Open a streamable HTTP MCP session and save the returned mcp-session-id header:

INIT_HEADERS=$(mktemp)

curl --silent --show-error \
  --dump-header "${INIT_HEADERS}" \
  --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \
  --header "authorization: Bearer ${TOKEN}" \
  --header 'content-type: application/json' \
  --header 'accept: application/json, text/event-stream' \
  --data '{
    "jsonrpc": "2.0",
    "id": 0,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-11-25",
      "capabilities": {},
      "clientInfo": { "name": "curl", "version": "0.1.0" }
    }
  }'

SESSION_ID=$(awk 'tolower($1) == "mcp-session-id:" { gsub("\r", "", $2); print $2 }' "${INIT_HEADERS}")
printf '%s\n' "${SESSION_ID}"

During initialize, the gateway opens upstream MCP client sessions to the configured backends and stores them under the downstream session id.

Send the MCP initialized notification:

curl --silent --show-error \
  --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \
  --header "authorization: Bearer ${TOKEN}" \
  --header "mcp-session-id: ${SESSION_ID}" \
  --header 'mcp-protocol-version: 2025-11-25' \
  --header 'content-type: application/json' \
  --header 'accept: application/json, text/event-stream' \
  --data '{"jsonrpc":"2.0","method":"notifications/initialized"}'

6. List Tools

curl --silent --show-error \
  --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \
  --header "authorization: Bearer ${TOKEN}" \
  --header "mcp-session-id: ${SESSION_ID}" \
  --header 'mcp-protocol-version: 2025-11-25' \
  --header 'content-type: application/json' \
  --header 'accept: application/json, text/event-stream' \
  --data '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

The response should contain tool names prefixed with their backend name, such as gateway-one-increment. That prefix is the routing contract for later tools/call requests.

7. Call a Tool

curl --silent --show-error \
  --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \
  --header "authorization: Bearer ${TOKEN}" \
  --header "mcp-session-id: ${SESSION_ID}" \
  --header 'mcp-protocol-version: 2025-11-25' \
  --header 'content-type: application/json' \
  --header 'accept: application/json, text/event-stream' \
  --data '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/call",
    "params": {
      "name": "gateway-one-increment",
      "arguments": {}
    }
  }'

The gateway strips gateway-one-, forwards increment to the gateway-one backend session, and returns the backend result to the client.

8. End the Session

curl --silent --show-error --request DELETE \
  --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \
  --header "authorization: Bearer ${TOKEN}" \
  --header "mcp-session-id: ${SESSION_ID}"

The gateway removes local session state and backend transports for that user and MCP session id.

Troubleshooting

Tear Down

Stop the gateway with Ctrl-C, then stop local dependencies:

docker compose -f docker/docker-compose-local.yaml down

Configuration Reference

This page is the reference for every gateway setting. The binary parses its configuration with clap, so each option has both a CLI flag and an environment variable, and both feed the same Config struct. When a setting is supplied as a flag and an environment variable at the same time, the command-line flag wins.

For the always-current list, ask the binary directly:

cargo run -p contextforge-gateway-rs --bin contextforge-gateway-rs -- --help

The sections below group the options by concern: listeners, JWT, Redis, upstream transport, runtime, telemetry, and logging.

Minimum Useful Configuration

At startup, the CLI requires Redis location and Redis connection mode:

--redis-address
--redis-port
--redis-mode

To serve useful traffic, the process also needs:

If no token verification key or secret is configured, authenticated MCP requests cannot pass the claims layer.

Listener Options

--address and --tls-address may both be configured, but they must not use the same socket address.

JWT Options

The claims layer validates issuer, audience, and expiration:

The JWT sub claim selects the Redis user config key. The path virtual host id then selects one virtual host inside that config.

Redis Options

Local Compose exposes plain Redis on 127.0.0.1:6379, so local runs normally use:

--redis-address 127.0.0.1 \
--redis-port 6379 \
--redis-mode plain-text

Runtime config values are MessagePack encoded. Redis is the current transport for config, not the routing model itself.

Upstream MCP Transport Options

Connection modes:

If an upstream backend URL is http://... and the mode is omitted, calls fail before reaching that backend because the reqwest client is HTTPS-only.

Runtime Options

When runtime plugins are enabled, plugin config is read from Redis key ContextForgeGatewayRuntimePluginConfig. That key is a control-plane trust boundary because it decides which registered hooks run.

Telemetry Options

Default endpoints:

RUST_TRACE_LOG controls which spans reach the OTLP trace layer. The HTTP trace layer emits debug-level spans, so local trace verification usually needs:

RUST_TRACE_LOG=debug

Logging Options

Common Flag Sets

Local HTTP Gateway And Plain Redis

--address 127.0.0.1:8001 \
--redis-address 127.0.0.1 \
--redis-port 6379 \
--redis-mode plain-text \
--token-verification-public-key assets/jwt.key.pub \
--upstream-connection-mode plain-text-or-tls

Downstream TLS Listener

--tls-address 0.0.0.0:8443 \
--server-certificate assets/contextforgeCA/contextforge-server.cert.pem \
--server-private-key assets/contextforgeCA/contextforge-server.key.pem

You may combine this with --address to expose both HTTP and HTTPS listeners.

HMAC Token Verification

--token-verification-secret "${CONTEXTFORGE_GATEWAY_RS_TOKEN_SECRET}"

Use this only when downstream JWTs are signed with an HS* algorithm. RSA tokens need --token-verification-public-key.

Redis TLS

--redis-address 127.0.0.1 \
--redis-port 16379 \
--redis-mode tls \
--redis-tls-trust-bundle assets/contextforgeCA/contextforge.intermediate.ca-chain.cert.pem

Upstream mTLS

--upstream-connection-mode mtls-only \
--upstream-trust-bundle assets/contextforgeCA/contextforge.intermediate.ca-chain.cert.pem \
--upstream-certificate assets/contextforgeCA/contextforge-client.cert.pem \
--upstream-private-key assets/contextforgeCA/contextforge-client.key.pem

The certificate and key paths must point to PEM files accepted by reqwest.

Startup Validation

The gateway fails fast for these invalid combinations:

Architecture

This section explains how the gateway is put together and why the main boundaries exist.

🧭 Read this when changing the hot path. The architecture pages keep request handling, config lookup, backend sessions, transports, and control-plane boundaries explicit.

The pages are ordered for a first read: start at the top for the big picture, then work down into each boundary. If you are changing one area, jump straight to its page.

System Shape

🧭 Architecture lens: this page explains what the gateway is, what it is not, and which code owns each boundary.

The ContextForge Gateway is the Rust dataplane process for MCP traffic. It is not a second ContextForge application. It accepts downstream streamable HTTP MCP requests, builds request context, loads runtime config, opens or reuses backend MCP client sessions, and returns one merged MCP server view to the caller.

That shape matters because this repository sits on the traffic path. Its design should bias toward predictable request behavior, explicit state ownership, and small hot-path dependencies.

Three-Layer Model

The gateway is easiest to reason about as three layers:

In text form, the same model is:

ContextForge control plane
  -> writes runtime config and owns management workflows

contextforge-gateway-rs process
  -> authenticates, loads config, routes, fans out, applies hooks, emits signals

backend MCP servers
  -> own the actual tools, resources, and prompts

Boundary rule: Redis is the current config-store transport, not the architecture. Routing code should depend on UserConfigStore, not Redis commands.

Control Plane Boundary

The external ContextForge control plane owns the slow-changing product surface:

The Rust dataplane owns the hot path:

The front door can route only MCP traffic to this process while leaving other ContextForge traffic on existing paths:

/contextforge-rs/servers/{virtual_host_id}/mcp

From a client point of view, that endpoint behaves like one ContextForge MCP server. Internally, it is a focused proxy, fanout, policy, and merge dataplane.

Process Assembly

Startup begins in crates/contextforge-gateway-rs/src/main.rs. The binary crate does process-level assembly:

install rustls crypto provider
  -> Config::parse()
  -> logging::init_tracing_logging(&config)
  -> runtime::Runtime::from(&config)
  -> optional CpexRuntimeRegistry
  -> Gateway::builder()
       .with_config(config)
       .with_user_config_store_type(UserConfigStoreType::Redis)
       .with_session_manager(LocalSessionManager::default())
       .with_plugin_runtime(...)
       .build()
  -> runtime.execute(gateway, plugin_registry)

runtime::Runtime changes executor shape, not gateway semantics. The default path is one multi-thread Tokio runtime. The binary crate also sets tikv_jemallocator as the global allocator and owns file logging, console logging, trace export, and metrics export setup.

Gateway Assembly

crates/contextforge-gateway-rs-lib/src/lib.rs builds the dataplane stack in Gateway::run_gateway:

RedisUserConfigStore
LocalUserSessionStore
BackendTransports
reqwest::Client for backend MCP calls
RMCP StreamableHttpService<McpService>
Axum middleware stack
TCP and/or TLS listeners

The resulting route tree is:

/contextforge-rs
  /servers/{virtual_host_name}/mcp
  /admin/...          local bootstrap helpers when with_tools is enabled
  /health            local bootstrap helper when with_tools is enabled

The route segment is named virtual_host_name in Axum, but the gateway extracts the actual path value as a VirtualHostId. Routing code should treat it as a virtual host id.

Workspace Layers

The repository is a Cargo workspace with narrow responsibilities:

Keep those boundaries stable. Adding dataplane behavior to the binary crate makes the process shell harder to test and reuse. Adding control-plane behavior to the library crate makes the hot path harder to reason about.

Pipeline Shape

The target shape is a bidirectional AI traffic pipeline: authentication, rate limiting, routing and protocol selection, request mutation, optional retrieval, and request guardrails on the way upstream; response guardrails, response mutation, and telemetry on the way back. Upstreams may eventually be MCP, A2A, or model providers. Preserve the ordering as pieces land: auth and config before backend selection, request plugins before upstream calls, response plugins before returning, telemetry around both sides.

The current code implements the MCP subset of that pipeline:

downstream request
  -> virtual host extraction
  -> JWT validation
  -> session extraction
  -> user config lookup
  -> MCP handler validation
  -> request plugin hooks
  -> backend MCP call

upstream response
  -> response plugin hooks
  -> merge, namespace, or pass through
  -> metrics, tracing, and logging
  -> downstream response

The Axum layer registration order is important because Tower layers execute from the outside in. The stack is built so the request reaches handlers with:

After those extensions exist, virtual_host_config_layer rejects requests with 404 when the path’s virtual host id is not present in the loaded UserConfig, so MCP handlers only see resolvable virtual hosts.

initialize is the special call. RMCP provides a downstream session id before the gateway has a Mcp-session-id header. The gateway fans out to configured backends, opens upstream MCP client sessions, and stores those running services under:

principal + backend_name + downstream_session_id

Later calls use the Mcp-session-id header to find those stored backend services.

State Ownership

The architecture keeps request state, session state, runtime config, and process state in separate ownership scopes. The current ownership model is:

Session rule: backend MCP services are local process state today. Load-balanced deployments need sticky routing, external session state, or a reinitialize-after-failover story.

Redis-backed user session storage exists in code, but the default gateway assembly wires LocalUserSessionStore. Do not assume session state is durable or shared across gateway nodes.

Module Boundaries

The library crate has deliberately narrow internal module roles:

Concrete Redis commands stay behind UserConfigStore. Downstream listener transport stays in transports/. MCP method behavior stays in gateway/. Request extension extraction stays in layers/.

Reusable Shell

The code is MCP-first today, but the repository is intentionally not named or structured as only an MCP proxy. Authentication, configuration ingestion, TLS handling, plugin execution, telemetry, runtime shape, and session strategy are gateway-shell concerns.

MCP-specific behavior should remain isolated to the current MCP modules so future A2A or model-provider routing can reuse the shell instead of growing a parallel stack.

Why These Architecture Pages Exist

The architecture section is split into subpages because each page tracks a boundary that already exists in modules or runtime ownership:

This split should make later pages easier to fill in without turning the architecture chapter into one long mixed-concern essay.

Request Flow

🎯 Flow invariant: Axum builds request context before RMCP handlers route MCP methods. MCP handlers should read typed extensions, not parse headers, paths, or Redis keys directly.

Graph Legend

The graph uses color only to separate paths:

This page follows a normal streamable HTTP MCP request through the code. The shape below is based on the current main.rs, runtime.rs, Gateway::run_gateway, the request layers, and McpService. To watch the same flow with real requests, follow Run the Gateway Locally alongside this page.

Startup Path

Startup begins in crates/contextforge-gateway-rs/src/main.rs:

install rustls crypto provider
  -> Config::parse()
  -> logging::init_tracing_logging(&config)
  -> Runtime::from(&config)
  -> optional CpexRuntimeRegistry
  -> Gateway::builder()
       .with_config(config)
       .with_user_config_store_type(UserConfigStoreType::Redis)
       .with_session_manager(LocalSessionManager::default())
       .with_plugin_runtime(...)
       .build()
  -> runtime.execute(gateway, plugin_registry)

runtime.execute either runs one multi-thread Tokio runtime or starts multiple current-thread runtimes. In both modes it initializes the optional CPEX runtime and then calls gateway.run_gateway().

HTTP Stack Order

Gateway::run_gateway builds the service stack in crates/contextforge-gateway-rs-lib/src/lib.rs. Tower layers execute from the outside in, so a normal MCP request reaches the handler in this order:

TCP/TLS listener
  -> HttpMetricsLayer
  -> TraceLayer
  -> /contextforge-rs nested router
  -> CORS layer
  -> virtual_host_id_layer
  -> claims_layer
  -> session_id_layer
  -> user_config_store_layer
  -> virtual_host_config_layer
  -> /servers/{virtual_host_name}/mcp RMCP service

The inner Axum route is:

/servers/{virtual_host_name}/mcp

The public route is nested under:

/contextforge-rs/servers/{virtual_host_name}/mcp

The route segment is named virtual_host_name, but the layer stores the value as VirtualHostId.

Flow Checkpoints

Middleware Context

The request layers insert the context used later by RMCP handlers:

For DELETE, session_id_layer also has response-side behavior. It lets RMCP handle the request first. If the RMCP response succeeds and a session id exists, it removes the local user session mapping and removes backend transports for principal + session_id.

Initialize Flow

initialize does not require the downstream Mcp-session-id header. RMCP creates a DownstreamSessionId and places it in the request context.

McpService::initialize runs this sequence:

1. InitializeCallValidator reads DownstreamSessionId, UserConfig, VirtualHostId, and ContextForgeClaims.
2. It resolves user_config.virtual_hosts[virtual_host_id].
3. It reads the local user session mapping for claims.sub + downstream_session_id.
4. For every backend in the selected virtual host, it concurrently builds a StreamableHttpClientTransport with the configured backend URL and serves a GatewayBackendClient over that transport.
5. It collects backend capabilities and running RMCP client services. The capabilities are stored with backend transport state for future routing, but they do not shape the downstream initialize response yet.
6. It writes the local user session mapping.
7. It stores each backend service in BackendTransports keyed by principal, backend name, and downstream session id.
8. It returns InitializeResult with the gateway’s current fixed capability set: completions, prompts, resources, and tools enabled.

Backend initialization is concurrent through futures::future::join_all.

Authorized MCP Calls

Routed MCP calls after initialization use AuthorizedCallValidator. It requires:

SessionId
UserConfig
VirtualHostId
ContextForgeClaims

The validator resolves the same virtual host from the authenticated user’s config, then SessionManager locates backend services for:

principal + backend_name + session_id

Current routed method families:

GatewayBackendClient handles backend progress notifications for call_tool. RMCP assigns a new progress token to each backend request, so the gateway maps that backend token to the corresponding downstream token. Request enqueue and mapping publication are serialized against progress lookup so an immediate backend notification cannot overtake registration. When a notification matches an in-flight backend token, the gateway restores the downstream token, optionally runs the stream-event post hook, and forwards the notification to the downstream client.

call_backend_tool also watches the downstream cancellation token. If the downstream call is cancelled before the backend responds, the gateway sends a cancel request to the backend handle.

Local MCP Methods

Some MCP methods are currently local to the gateway implementation rather than backend-routed:

This path still passes through the same HTTP middleware, but it does not use backend fanout or identifier routing.

Response Path

Backend responses return to McpService first. call_tool may run response plugin hooks before returning. List calls merge backend output, preserving single-backend identifiers and namespacing multi-backend identifiers as needed.

The HTTP response then unwinds through virtual_host_config_layer, user_config_store_layer, session_id_layer, claims_layer, virtual_host_id_layer, CORS, trace, and metrics. On successful DELETE, session_id_layer performs local session and backend transport cleanup during this unwind.

Concurrency And Runtime Model

🧵 Execution lens: the gateway is async Rust on Tokio with jemalloc as the global allocator. This page explains the two executor shapes, what state is shared under which locks, and where work fans out.

Executor Shapes

--single-runtime selects between two models:

In multi-runtime mode, the first thread initializes the optional CPEX plugin runtime before the others start; the current-thread builders are tuned with a global queue interval of 1024 and 4 I/O events per tick.

⚠️ Multi-runtime consequence: each runtime thread builds its own BackendTransports map and user-session store inside run_gateway. Backend session state is therefore per-runtime-thread, and SO_REUSEPORT gives no connection affinity — later requests in a streamable HTTP session arrive on new connections and can land on a thread that does not own the session. Treat single-runtime mode as the only mode that supports stateful MCP sessions today; this is the in-process version of the load-balancing constraint.

Shared State And Locks

The design rule: locks guard maps of handles, not I/O. Backend calls, Redis reads, and plugin hooks all run outside any gateway lock.

Fanout And Cancellation

• initialize opens one backend transport per configured backend concurrently (futures::future::join_all); a failed backend degrades that backend only.
• List methods fan out to all connected backends concurrently and merge.
• Targeted calls resolve exactly one backend service handle.
• call_tool watches the downstream cancellation token and forwards a cancel to the backend if the client gives up first; backend progress notifications are forwarded downstream while the call is in flight.

Listener Behavior

The TCP listener binds with reuseaddr, reuseport, and keepalive, listens with a backlog of 1024, and serves Axum with graceful shutdown on ctrl_c. The TLS listener accepts by hand through Rustls and serves the same router via Hyper.

Allocator

The binary sets tikv_jemallocator as the global allocator, which holds up better than the system allocator under the many small, short-lived allocations of per-request JSON and header processing.

Authentication And User Config Lookup

🔐 Boundary: authentication proves who is calling. User config lookup decides which virtual hosts and backends that caller can reach.

This page follows the identity boundary in the request path. The gateway does not let an MCP method choose arbitrary backend URLs. It validates the bearer token, loads the caller’s UserConfig, and only then lets MCP validators select a virtual host from that config.

Request Order

Authentication and config lookup happen before McpService handles the MCP method:

The order matters: user_config_store_layer needs ContextForgeClaims, and MCP validators need both UserConfig and VirtualHostId.

Token Validation

claims_layer reads Authorization: Bearer ... and decodes the JWT with the algorithm declared in the JWT header.

The decoded value is stored as ContextForgeClaims. The fields currently important to routing are:

A concrete decoded payload for the local admin@example.com subject looks like this (timestamps shown as example Unix seconds). Of everything here, MCP routing depends only on sub today. The optional fields are included for illustration:

{
  "iss": "mcpgateway",
  "aud": "mcpgateway-api",
  "sub": "admin@example.com",
  "exp": 1717180800,
  "iat": 1717177200,
  "jti": "example-token",
  "token_use": "api",
  "teams": ["team_awesome"],
  "user": {
    "email": "admin@example.com",
    "full_name": "API Token User",
    "is_admin": true,
    "auth_provider": "api_token"
  },
  "scopes": {
    "server_id": "my_id",
    "permissions": ["tools.read", "servers.use"],
    "ip_restrictions": ["192.169.1.0/24"],
    "time_restrictions": null
  }
}

User Config Key

user_config_store_layer turns the subject into a typed key:

ContextForgeClaims.sub
  -> User::new(subject)
  -> UserConfigStore::get_config(&user)

The Redis adapter serializes that User key with MessagePack. The key includes both the key type and the subject, so user config data is not just stored under the raw subject string.

Cache And Redis Lookup

RedisUserConfigStore checks an in-process LRU cache before going to Redis:

The cache is an implementation detail of RedisUserConfigStore. Routing code depends on UserConfigStore, not Redis commands.

Failure Behavior

Failures before RMCP method handling are HTTP responses:

A valid user config can still fail a request: virtual_host_config_layer returns 404 before MCP method handling when the config does not contain the path’s VirtualHostId.

What This Boundary Does Not Do

Authentication and user config lookup do not route to a backend by themselves. They only establish:

caller identity
  + caller UserConfig
  + requested VirtualHostId

McpService still has to validate the MCP call, resolve the virtual host, and choose either the initialize path, routed backend path, or local method path.

Security Model And Trust Boundaries

🔒 Trust rule: the gateway trusts its process config and the control-plane-authored data in Redis. It does not trust downstream callers beyond a validated JWT, and it reaches backends only through configured URLs.

Trust Boundaries

Identity And Authorization

Authentication is bearer-JWT only:

• Accepted algorithms are RS256/RS384/RS512 (public key configured) or HS256/HS384/HS512 (shared secret configured); anything else is rejected.
• iss must be mcpgateway, aud must be mcpgateway-api, and exp is validated. There is no revocation list: a leaked token is valid until it expires.
• Authorization is config existence. The sub claim selects the caller’s UserConfig; the path selects one virtual host inside it. A caller can never reach a backend that is not in their own config, and unknown virtual hosts return 404 before MCP handling.
• jti, token_use, iat, teams, user, and scopes are carried but not yet enforced; token_use, iat, teams, and scopes are optional, as is user.full_name. Fine-grained permissions are future policy work.

What Compromise Means

Transport Security

CORS is currently wide open (any origin, method, and header). The API is bearer-token based and cookie-free, so cross-site request forgery does not apply, but expect this to tighten as policy work lands.

Local Bootstrap Helpers

The contextforge-gateway-rs-lib/with_tools feature compiles in /contextforge-rs/admin/tokens/{user}, /contextforge-rs/admin/userconfigs/{user}, and /contextforge-rs/health. These routes are registered outside the authentication middleware, so token minting and config writes are unauthenticated by design — they exist only for local bootstrap. Production builds must not enable this feature: in a real deployment the control plane mints tokens and writes config.

Secrets Handling

• The HMAC secret is held as a SecretString; key and certificate material is read from disk paths at startup.
• Log hygiene is a standing rule: never log tokens, authorization headers, secrets, Redis key/value bytes, full UserConfig documents, or backend credentials.

Runtime Configuration

🗂️ Config boundary: process config tells the gateway how to run. Runtime user config tells each request where it may route. Plugin config controls the optional CPEX hook runtime.

The gateway consumes configuration from three places. Keeping them separate is important because they change at different times and are used by different parts of the dataplane.

Config Surfaces

The control plane owns durable authoring. This repo owns reading those values and applying them on the request path.

UserConfig Shape

The API crate defines the shared runtime routing model:

UserConfig
  virtual_hosts: HashMap<String, VirtualHost>

VirtualHost
  backends: HashMap<String, BackendMCPGateway>

BackendMCPGateway
  name: String
  url: Url
  transport: Transport
  passthrough_headers: Vec<String>
  allowed_tool_names: Vec<String>
  allowed_resource_names: Vec<String>
  allowed_prompt_names: Vec<String>

Transport currently declares:

STREAMABLEHTTP
SSE
STDIO

The Rust types above are easier to picture as JSON. For a complete, working UserConfig document, see the seed request body in Run the Gateway Locally.

What The MCP Dataplane Uses Today

The struct is already wider than the current MCP routing code. That is useful, but the distinction should stay explicit:

The current route selection is:

JWT subject
  -> User::new(subject)
  -> UserConfig
  -> path VirtualHostId
  -> VirtualHost
  -> backend map key
  -> BackendMCPGateway.url

Expected config growth beyond the current fields:

• route selection across multiple MCP endpoints
• principal/virtual-host filters for tools, resources, and prompts
• backend auth/TLS material references
• request/response header pass/add/remove rules
• plugin/CPEX hook settings
• pagination/SSE behavior where protocol handling needs config
• future A2A and LLM routing/provider settings

Redis Storage

RedisUserConfigStore stores user routing config as MessagePack:

The in-process cache is an implementation detail:

Routing code should stay behind the UserConfigStore trait. That keeps Redis, MessagePack, and cache behavior out of MCP method handling.

Plugin Runtime Config

When runtime_plugins_enabled is true, startup builds a CpexRuntimeRegistry::with_redis_config(...). The registry reads a separate runtime plugin document:

Redis key: ContextForgeGatewayRuntimePluginConfig

RuntimePluginConfigDocument
  version: 1
  cpex: CpexConfig

The plugin config loader accepts JSON bytes or MessagePack bytes. It rejects documents with the wrong version, missing cpex config, unsupported CPEX features, or an unavailable config store.

Current supported plugin scope is deliberately narrow:

The registry has a watcher interval of 10 minutes. On valid reloads it swaps in the new runtime. On invalid reloads it marks the runtime failed so new plugin calls return an internal MCP error until a valid config is applied.

Process Config

Process Config is not per-user routing state. It is parsed once and controls the gateway shell:

The process config decides whether the gateway can start and what dependencies it can reach. It does not decide which backend a specific MCP caller may use; that remains in UserConfig.

Selection Invariant

Backend selection should only happen after these facts exist:

authenticated subject
  + loaded UserConfig
  + path VirtualHostId
  + MCP session context

That invariant is why runtime config access is in middleware and MCP validators, not hidden inside individual backend calls.

Control-Plane Integration

🤝 Provisional: no formal contract with the control plane has been stipulated yet. This page is a snapshot of the current de facto integration surface with IBM/mcp-context-forge as implemented today. Any row may change while the project is early; when a proper contract is agreed, this page should track it.

Current Integration Surface

These are the values both sides currently rely on:

Changing any of these is a cross-repo change: the dataplane, the control-plane publisher, and the integration harness all need updating together.

Config Publishing

The control plane owns durable config and publishes runtime snapshots to Redis. With DATAPLANE_PUBLISHER=true, it rewrites the dataplane’s UserConfig keys on an interval — every 60 seconds by default, configurable in newer control-plane images.

Config staleness on the dataplane is bounded by two knobs:

worst-case staleness = publisher interval + user config cache expiry

The dataplane’s in-process cache defaults to 60 seconds (--user-config-cache-expiry-seconds; 0 disables it and reads Redis on every request). Functional test setups shorten the publisher interval and disable the cache; production keeps both at 60.

Schema Generation

contextforge-gateway-rs-apis is the single source of truth for the shared config shapes. It generates the JSON Schemas the control plane can validate against:

cargo run -p contextforge-gateway-rs-apis

This writes schemas/user.json and schemas/user_config.json. Regenerate and commit them whenever UserConfig, VirtualHost, BackendMCPGateway, or the User key type changes.

Front-Door Split

Only MCP dataplane traffic comes to this process. The repository’s reference docker/nginx.conf proxies location ^~ /contextforge-rs to the gateway; all UI, management API, and other ContextForge traffic stays on the existing control-plane paths.

Verifying The Integration

The cf-integration harness tests exactly this surface: it runs the stock upstream control-plane stack with the nginx split and the dataplane publisher enabled, then drives probe, live-test, and load lanes through the public route. When the integration surface changes, its lanes are what prove both sides still agree. See Testing for the commands.

Backend Connections And Transports

🚚 Transport boundary: downstream listener traffic, upstream backend traffic, and config-store traffic are separate concerns. Keep them separate even when they all use TCP underneath.

The gateway has three transport classes on the hot path. They are built in different modules, configured from different fields, and serve different architecture roles.

Transport Classes

The current MCP dataplane only uses streamable HTTP for backend MCP traffic. BackendMCPGateway.transport already has STREAMABLEHTTP, SSE, and STDIO, but upstream routing does not branch on that field yet.

Downstream Listeners

Gateway::run_gateway builds one Axum router and can expose it through TCP, TLS, or both.

TLS listener setup has two important constraints:

The downstream TLS listener currently uses with_no_client_auth(). Client identity is established by the gateway’s bearer JWT layer, not by downstream mTLS.

Upstream Backend Client

The upstream HTTP client is built once at gateway startup:

Config
  -> reqwest::Client::try_from(&config)
  -> clone per backend initialize task
  -> StreamableHttpClientTransport::with_client(...)

The process-level upstream mode controls whether backend URLs may use plain HTTP, HTTPS, or HTTPS with client identity:

If upstream_trust_bundle is configured, the PEM bundle is merged into the client’s TLS trust roots. For mTLS modes, the upstream certificate and private key are read from disk and combined into a reqwest::Identity.

Backend MCP Transport

During initialize, the selected virtual host fans out to every configured backend:

VirtualHost.backends
  -> for each backend URL
  -> build StreamableHttpClientTransportConfig
  -> serve GatewayBackendClient over StreamableHttpClientTransport
  -> store running service in BackendTransports

For HTTPS backend URLs, the gateway also sets a custom Host header from the backend URL host and optional port. HTTP backend URLs do not get this custom header in the current code.

Backend connection failures are not fatal to the whole initialize call. The gateway stores the backend entry with no running service, so list calls can continue with available backends and routed calls to that backend can fail locally.

Config-Store Transport

Redis is the current config-store transport. It is used for user config and, when runtime plugins are enabled, plugin runtime config.

The Redis user config adapter stores:

MessagePack(User::new(claims.sub)) -> MessagePack(UserConfig)

The Redis connection manager is configured with 1,000 retries. The adapter keeps an in-process LRU cache in front of Redis, but routing code should only depend on the UserConfigStore trait.

What Should Move To Runtime Config

Transport security is mostly process config today. That keeps startup simple, but it is not the final shape for every backend-specific decision.

The boundary to preserve is simple: listener code should not know Redis schema, MCP routing code should not know Redis command details, and backend transport creation should stay behind a small, explicit upstream boundary.

Session Ownership

🧷 Session invariant: backend MCP services are local process state keyed by authenticated principal, backend namespace, and downstream MCP session id.

The current gateway keeps backend MCP client services in memory. That choice is simple and fast, but it defines how initialized MCP sessions can be routed in a deployment.

Owned State

Several pieces of state participate in one MCP session. They do not all have the same owner.

RedisUserSessionStore exists, but Gateway::run_gateway wires LocalUserSessionStore today. Even if the user session mapping moved to Redis, the live RMCP backend services in BackendTransports would still be local unless that architecture changes too.

Initialize Creates Backend Services

initialize is the ownership creation point:

InitializeCallValidator
  -> selected VirtualHost
  -> claims.sub
  -> RMCP DownstreamSessionId
  -> one upstream client service per backend
  -> BackendTransports entries

The backend transport key is:

BackendTransportKey
  principal: claims.sub
  backend_name: backend map key
  session_id: downstream session id

The backend map key matters because one downstream MCP session can fan out to many backend MCP sessions. The principal matters because two callers could present the same downstream session id value.

Borrowing Backend Services

SessionManager is a request-scoped view over the selected virtual host, session id, principal, and shared backend transport map.

borrow_transports() does not move services out of the map in the current code. It clones shared service handles, so there is no return step after a request completes.

List Calls And Routed Calls

Backend service lookup splits into two patterns:

An initialized backend entry can still contain no running service if backend initialize failed. List calls skip unavailable backends. Routed calls to that backend return a routing error.

If routed lookup ever detects duplicate backend matches, the session is treated as invalid and cleanup_backends removes the local backend entries for that principal and session.

Delete Cleanup

session_id_layer owns response-side cleanup for downstream DELETE:

DELETE with Mcp-session-id
  -> let RMCP handle the request
  -> if RMCP response is successful
  -> remove LocalUserSessionStore entry
  -> remove BackendTransports entries for principal + session id

Cleanup is intentionally tied to a successful downstream delete response. If RMCP rejects the delete, the layer returns that response and leaves local session state untouched.

Load-Balancing Consequence

Random load balancing is not safe for stateful MCP sessions today. After initialize, later requests with the same Mcp-session-id must reach the process that owns the backend running services.

Known deployment options are:

Until backend service ownership changes, design request handling as if a stateful MCP session belongs to one gateway process. The same constraint appears inside one host in multi-runtime mode, where each runtime thread owns its own backend transports; see Concurrency And Runtime Model.

Architectural Choices

🧱 Design rule: these choices are not permanent, but changing one should be a deliberate architecture decision with code, tests, and migration notes.

This page records the choices that should stay visible as the gateway evolves. They describe the shape of the current Rust dataplane, not just preferences.

Choice Matrix

Dataplane, Not Control Plane

The gateway should enforce decisions already made elsewhere:

control plane authors config and policy
  -> Redis/config transport exposes runtime data
  -> Rust dataplane enforces it on MCP traffic

New features should start with one question:

Is this hot-path enforcement, or is this a management workflow?

If it is a workflow, it probably belongs outside this repo. The Rust gateway should enforce the result of management decisions, not become the place where those decisions are authored.

Config Access Is Abstracted

Redis is the current storage and transport adapter. It is not the routing model. The routing model is:

UserConfig
  -> VirtualHost
  -> BackendMCPGateway

That is why request code should stay behind UserConfigStore. Redis key encoding, MessagePack, cache expiry, and retry settings belong in the adapter, not in MCP method handling.

Backend Names Are Conditionally Public

Backend map keys are visible in the MCP namespace when multiple backends need disambiguation:

backend map key: gateway-one
backend tool: increment
gateway tool: gateway-one-increment

For a single backend, upstream identifiers pass through unchanged. Explicit tool_name_aliases published by the control plane take precedence in either case. Otherwise the map key, not BackendMCPGateway.name, is the namespace used by multi-backend routing.

Session State Is Local Today

Backend services are not just data. They are live RMCP running services stored under:

principal + backend_name + downstream_session_id

That makes the current state model fast and direct, but not horizontally portable. Do not design request handling as if every node can serve every stateful MCP session until backend service ownership has an external owner or can be rebuilt safely.

Merged MCP Semantics Define The Contract

The downstream client should reason about one MCP server:

client
  -> ContextForge Gateway
  -> merged tools/resources/prompts

Backend identity appears only when namespacing is needed, and clients should not need to know transport details, Redis storage, fanout mechanics, or plugin runtime internals.

This choice leaves room for filtering, policy, and route changes without turning every backend topology change into a client integration change.

Plugin Boundaries Stay Explicit

Plugins can inspect or mutate payloads. That is more powerful than a header filter, so the hook points need to stay obvious.

Current CPEX support is intentionally narrow:

Avoid adding ad hoc plugin calls in the middle of routing code. If a new hook is needed, define its ownership, failure behavior, timeout behavior, cancellation behavior, streaming behavior, and telemetry attribution.

Transport Security Is Split

Downstream TLS is listener-level process config. Upstream backend security is also process config today, but some of it is naturally backend-specific.

Keep this split visible:

The gateway should not bury transport security decisions inside MCP method handlers. They should remain either startup assembly or explicit backend transport construction.

MCP-First, Not MCP-Only

The current code implements MCP behavior, but the shell is broader:

auth
config lookup
transport setup
plugin runtime
telemetry
session strategy

Keep protocol-neutral concerns reusable. Future A2A or model-provider routing should be able to reuse the gateway shell without copying the MCP routing stack.

When A Choice Changes

Changing one of these choices should update more than one file.

These pages are part of that safety net: they make architecture drift visible before it becomes accidental API behavior.

MCP Behavior

This section describes what the gateway exposes as an MCP server and how it maps downstream MCP calls onto configured backend MCP servers.

📋 Use this section for protocol behavior. It covers the public MCP surface, backend fanout, namespacing, targeted routing, pagination, and streaming gaps.

MCP Method Reference

📋 Reference lens: this page lists what each MCP method does at the gateway today, from the client’s point of view. For how identifiers are preserved, aliased, or namespaced, see MCP Routing Semantics.

Gateway methods fall into three groups: initialize creates backend sessions, routed methods use them, and ping remains local to the gateway process.

Initialize

Routed List Methods

list_tools, list_resources, list_prompts, and list_resource_templates share one fanout path:

Routed Targeted Methods

Targeted calls resolve exactly one backend using an explicit tool alias, a single-backend pass-through, or a multi-backend prefix:

Routed failures are JSON-RPC errors: an identifier that cannot select a backend or an unavailable backend returns an internal error, and duplicate backend matches invalidate the session; see Failure Modes.

Local Methods

This method passes through the same HTTP middleware but does not touch backends:

Session Delete

A downstream DELETE with Mcp-session-id is handled by RMCP first. On a successful response, session_id_layer removes the local user session mapping and the BackendTransports entries for that principal and session id. See Session Ownership for the cleanup rules.

MCP Routing Semantics

The gateway presents multiple backend MCP servers as one downstream MCP server. It fans out list operations, preserves identifiers when only one backend is configured, and adds a backend namespace when multiple backends need to be disambiguated. Explicit tool aliases published by the control plane take precedence over both forms.

Backend Prefixes

Backend map keys are part of the public namespace only for multi-backend virtual hosts without an explicit tool alias:

backend tool "increment" from backend "gateway-one"
  -> "gateway-one-increment"

backend resource "counter" from backend "gateway-one"
  -> "gateway-one-counter"

backend prompt "summarize" from backend "research"
  -> "research-summarize"

For a single-backend virtual host, those identifiers remain increment, counter, and summarize. Resource URIs and resource-template URI templates follow the same rule. This keeps a transparent gateway from changing the MCP contract when no collision is possible.

For tools, BackendMCPGateway.tool_name_aliases maps an exact downstream alias to the upstream original name. An alias is advertised and routed exactly as published, including case, dots, and underscores. When no alias exists, single-backend hosts preserve the original tool name and multi-backend hosts fall back to {backend-map-key}-{tool-name}.

When a prefix is required, it is a routing contract rather than a display detail. Changing a backend map key changes downstream identifiers for that multi-backend virtual host.

List Operations

List operations fan out:

list_tools               -> all connected backends -> merged sorted tools
list_resources           -> all connected backends -> merged sorted resources
list_prompts             -> all connected backends -> merged sorted prompts
list_resource_templates  -> all connected backends -> merged sorted templates

For a single backend, each successful result is returned with its identifier unchanged, except for explicit tool aliases. For multiple backends, resources, prompts, and tools without aliases are rewritten with their backend map-key prefix. Resource-template names and URI templates are both prefixed. Failed or unavailable backends are logged and omitted from the current merged list result.

Routed Operations

Calls that target one object use the inverse rule. A single-backend host selects its only backend and forwards the identifier unchanged. A multi-backend host splits the prefixed identifier:

gateway-one-increment
  -> backend_name = gateway-one
  -> upstream tool name = increment

read_resource, subscribe, unsubscribe, get_prompt, and complete share this conditional routing helper. For complete, the routed value is the prompt name or resource URI inside its ref. Resource-update notifications apply the same rule on the response path, so their URI matches the URI originally exposed to the downstream client.

call_tool first resolves an exact control-plane alias, then applies the same single-versus-multi-backend fallback. Backend names can themselves contain - (as in gateway-one), so the splitter does not cut on the first -. Instead it walks the configured backend names, takes the first one the prefixed name starts with, and then requires a - immediately after that name. That is why gateway-one-increment resolves to backend gateway-one and tool increment, while a malformed name such as gateway-oneincrement is rejected.

After selecting a route, the gateway resolves exactly one connected backend service for the principal and downstream session. Missing backends fail the call. Duplicate matches are treated as invalid session state and trigger backend cleanup.

Known Gaps

Pagination is not complete. list_tools, list_resources, list_prompts, and list_resource_templates currently perform one backend call and return a merged response with no downstream cursor. Full parity needs to gather all backend pages or define a merged cursor strategy.

Streaming/SSE behavior is also still a tracked design area. The target is to stream downstream as backend chunks arrive while preserving plugin behavior, backpressure, cancellation, and telemetry attribution.

Operations

This section covers runtime behavior after the gateway is serving traffic: plugins, diagnostics, failure modes, tests, load tests, and deployment constraints.

🧪 Use this section when operating or verifying the gateway. It focuses on plugin hooks, observability, failure boundaries, load testing, and deployment assumptions.

Plugins And Policy

Plugins are a policy boundary, not just middleware. They can inspect and mutate payloads, so the gateway keeps the supported hook surface narrow and explicit.

Runtime Enablement

Runtime plugins are disabled by default. When enabled, the binary creates a CPEX runtime registry and the runtime initializes it before serving traffic.

Plugin configuration is loaded from Redis at:

ContextForgeGatewayRuntimePluginConfig

The runtime registry builds an initialized immutable plugin manager from that configuration. Reloading swaps the manager instead of mutating a live one.

Supported Hooks

The supported surface is deliberately narrow:

cmf.tool_pre_invoke
cmf.tool_post_invoke

The gateway rejects route-based plugin selection, plugin directories, global policies/defaults, non-tool hooks, and plugin conditions. Those features need clear behavior for streaming, failures, timeouts, backpressure, context propagation, and observability before they belong on the hot path.

Tool Call Behavior

For call_tool, the pre hook runs after backend routing has selected the backend and stripped the public prefix. The hook sees the backend name, routed tool name, and arguments. It can:

• leave arguments unchanged
• replace arguments
• deny the call

After the upstream backend returns, the post hook can:

• leave the result unchanged
• rewrite the result payload
• deny the response

Hook state is carried across the upstream call so pre and post hooks can share CPEX context for the same logical tool call.

Boundary Rules

Plugin execution must not poison shared gateway state. A plugin denial becomes an MCP error. Soft plugin errors are logged. Unsupported plugin configuration fails validation before the runtime is accepted.

Future hook expansion should define behavior for streaming/SSE, cancellation, timeouts, backpressure, and telemetry before adding new hook points.

Telemetry And Diagnostics

📈 Observability lens: the gateway emits three signals — logs, request traces, and HTTP metrics. This page explains how each is produced, how to verify them locally, and where to look when a request misbehaves.

Signal Overview

All export settings are process config; see the telemetry section of the Configuration Reference.

Logging

Console and file logging are always on. The file appender writes contextforge-gateway-rs.log (configurable with --log-name) in the current working directory and rotates hourly by default (--log-rotation).

Three environment filters control verbosity independently:

Dataplane log messages use a stable method_name - event text field = value shape, so boundary-specific prefixes are grep-friendly: claims_layer, user_config_store_layer, virtual_host_config_layer, AuthorizedCallValidator::validate, initialize:, call_tool, and so on.

Traces

TraceLayer emits its request spans at DEBUG level.

⚠️ RUST_TRACE_LOG=debug is required for trace export. The default span filter (info) drops the HTTP spans before they reach the OTLP exporter, so nothing arrives at the trace backend.

Enable export with --enable-open-telemetry true plus the OTLP protocol, endpoint, and header flags. Each handled HTTP request produces one span with method, route, status, and latency.

Metrics

--enable-otel-metrics true turns on the HTTP server instruments: request duration histogram, active request gauge, and request/response body size histograms, all labeled with method, status code, and service.name.

Metrics are pushed by a PeriodicReader every 30 seconds, so allow about 35 seconds after the first request before the first data point appears downstream.

Local Verification Stack

A complete local pipeline ships under docker/ as overlays on top of the local run stack:

1. Start the stack

docker compose \
  -f docker/docker-compose-local.yaml \
  -f docker/docker-compose-langfuse.yaml \
  -f docker/docker-compose-otel-collector.yaml \
  up -d

Re-run with ps instead of up -d and wait until every container is healthy.

2. Run the gateway with export enabled

RUST_TRACE_LOG=debug \
cargo run --release --bin contextforge-gateway-rs -- \
  --address 0.0.0.0:8001 \
  --redis-port 6379 --redis-address 127.0.0.1 --redis-mode=plain-text \
  --token-verification-public-key assets/jwt.key.pub \
  --number-of-cpus 4 \
  --upstream-connection-mode=plain-text-or-tls \
  --enable-open-telemetry true \
  --enable-otel-metrics true \
  --otlp-protocol http-protobuf \
  --otlp-endpoint  http://127.0.0.1:3100/api/public/otel/v1/traces \
  --otlp-headers   "Authorization=Basic cGstbGYtY29udGV4dGZvcmdlOnNrLWxmLWNvbnRleHRmb3JnZQ==" \
  --otlp-metrics-endpoint http://127.0.0.1:4318/v1/metrics \
  --otlp-service-name contextforge-gateway-rs

The Authorization header is Basic auth for the seeded Langfuse project keys (pk-lf-contextforge:sk-lf-contextforge, base64-encoded).

3. Generate traffic

for i in {1..10}; do
  curl -s -o /dev/null -w "%{http_code}\n" \
    http://127.0.0.1:8001/contextforge-rs/admin/tokens/admin@example.com
done

Any response counts: each request is traced and recorded as a metric sample.

4. Inspect the data

• Langfuse: open http://localhost:3100 and check the project — one span per request with method, route, status, and latency.
• Prometheus: open http://localhost:9090; Status → Targets should show otel-collector:8889 as UP, then try the starter queries below.
• Collector stdout: docker logs otel-collector --tail 200 shows raw OTLP trace and metric dumps.

The data path is:

gateway :8001
  -> OTLP/HTTP traces  -> Langfuse :3100 (span UI)
  -> OTLP/HTTP metrics -> OTel Collector :4318
                            -> stdout dump (docker logs)
                            -> Prometheus exposition :8889 -> Prometheus :9090

Tear the stack down with the same three -f files and down.

Prometheus Starter Queries

Debugging Checklist

Work down the boundaries in request order; each failure has an owning signal. Failure Modes lists the exact responses per boundary.

Known Gaps

Tracked upstream, not yet implemented here:

• W3C trace-context propagation across gateway hops (mcp-context-forge#4723).
• MCP-semantic spans with tool names and JSON-RPC method attributes (mcp-context-forge#4722).

Failure Modes

🚧 Boundary rule: every failure should come from the layer that owns the missing fact. Identity and config failures are HTTP responses before MCP handling; routing and backend failures are JSON-RPC errors.

HTTP Layer Failures

These happen in middleware, before any MCP method runs:

MCP Validation Failures

InitializeCallValidator and AuthorizedCallValidator re-check the request context before method handling. These are defense-in-depth errors: in a healthy stack the middleware has already established the context.

Routing Failures

Backend Session Failures

Plugin Failures

Config Store Failures

For a symptom-first version of this table, see the troubleshooting section in Run the Gateway Locally.

Testing

🧪 Verification rings: workspace checks prove the code compiles and unit behavior holds, in-repo integration tests prove MCP routing against mock backends, and the cf-integration harness proves the whole control-plane-to-dataplane path end to end. Load and benchmark runs have their own page: Performance.

Workspace Validation

CI runs these on every change; run them locally before pushing:

cargo fmt --all --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo nextest run --locked --workspace

Use cargo test when nextest is unavailable. For book changes, also run mdbook build docs/book and mdbook test docs/book.

In-Repo Integration Tests

crates/contextforge-gateway-rs-lib/tests/ exercises the gateway against in-process mock MCP backends (shared helpers live in tests/support/):

These run in cargo nextest run with no Docker dependencies.

Full-Stack Integration Harness

cf-integration wires the external ContextForge control plane (cf-controlplane) to this dataplane the way production intends: the stock upstream Compose stack, plus exactly two intentional differences — nginx routes only /servers/{virtual_host_id}/mcp to the dataplane (as /contextforge-rs/servers/{virtual_host_id}/mcp), and the control plane runs with DATAPLANE_PUBLISHER=true so virtual server configs reach the dataplane through Redis. Because the stack otherwise matches upstream, test failures measure dataplane behavior, not stack drift.

Quick Start

scripts/cf-integration.sh up

This checks out the control plane under .integration/mcp-context-forge, pulls the published dataplane image, and starts the combined stack plus a local MCP counter backend. The admin UI is at http://localhost:8080/admin (admin@example.com / changeme). A Fast Time backend is auto-registered as a fixed virtual server, so the commands below work with no manual UI step; backends added through the UI are published to the dataplane by the control-plane publisher.

Route Probe

scripts/cf-integration.sh probe

Verifies the public nginx-to-dataplane route end to end: a 401 negative check, initialize, session reuse, tools/list, and tools/call.

Full Test Runs

Individual lanes are live-mcp, live-rbac, live-protocol, and live-all. live-mcp is the green lane: the full MCP protocol end-to-end suite passes against this harness. Remaining failures in the other lanes measure known dataplane feature gaps; the harness reports/ directory keeps the current classification.

Control-Plane Baseline

To separate dataplane regressions from upstream behavior, the harness can run the stock control-plane-only stack (no dataplane, no nginx split, no publisher) with the same commands:

scripts/cf-integration.sh down                    # frees the shared host ports
scripts/cf-integration.sh controlplane-test-all   # up + live core + locust

Individual steps are controlplane-up, controlplane-live-core, controlplane-live-all, controlplane-locust, and controlplane-down. The baseline load run is covered in Performance.

Key Settings

See the harness README for the full command and override list.

Performance

⚡ Two load paths: contextforge-load-test measures the Rust dataplane alone, and the cf-integration harness measures the full nginx-to-control-plane-to-dataplane stack with Locust. Use the first to profile gateway changes and the second to measure what users would see.

Dataplane-Only Load Testing

crates/contextforge-load-test is a Goose-based traffic driver that speaks the full streamable HTTP MCP flow against a running gateway. Start the local stack with seeded user config first, then:

cargo run --release --bin contextforge-load-test -- \
  --host 'http://127.0.0.1:8001' \
  -u 120 -r 40 --run-time 120s \
  --report-file report.html

-u is concurrent users, -r is the spawn rate per second, and --report-file writes an HTML report. This measures the Rust dataplane alone, without a control plane or front door in the path. Curated run reports live in the repository’s reports/ directory.

Full-Stack Load With Locust

The cf-integration harness runs Locust against the public nginx route with a streamable-HTTP-aware locustfile:

Tune with environment variables:

LOCUST_USERS=20 LOCUST_SPAWN_RATE=5 LOCUST_RUN_TIME=2m \
  scripts/cf-integration.sh locust

MCP_VIRTUAL_SERVER_ID targets a UI-created virtual server instead of the auto-registered Fast Time one, and MCP_TOOL_NAMES picks the tools to call. Locust HTML/CSV output lands under .integration/mcp-context-forge/reports/; curated run reports live in the harness reports/ directory.

Headless Versus Locust Web UI

The harness runs Locust headless by default (LOCUST_MODE=headless): a timed run that writes the HTML and CSV reports and prints only the summary. Setting LOCUST_MODE to any other value (for example web) switches the Locust service to interactive mode: a master with the web UI on port 8089 and a class picker (LOCUST_EXPECT_WORKERS controls the expected worker count). The one-off locust command does not publish container ports, so for the web UI start the Locust service through the stack’s testing Compose profile — the upstream stack maps 8089:8089 — then open http://localhost:8089 and drive the run from the browser.

Benchmark Settings

The harness tunes config propagation for functional runs, not throughput:

Restore both to 60 before measuring throughput, or the fast publish loop and per-request Redis reads distort the numbers.

Control-Plane Baseline Load

To compare against the stack without the dataplane in the path:

scripts/cf-integration.sh down                # frees the shared host ports
scripts/cf-integration.sh controlplane-locust

The baseline run defaults to the non-UI Locust class subset (health, Fast Time, Fast Test, version/meta). CONTROLPLANE_LOCUST_CLASSES=all adds the admin/UI/mutating surfaces. LOCUST_USERS, LOCUST_SPAWN_RATE, and LOCUST_RUN_TIME apply here too.

Deployment Notes

🏗️ Deployment lens: the gateway is one stateless-config, stateful-session process behind a front door. Everything here follows from that: route only MCP traffic to it, keep Redis close and trusted, and give stateful sessions affinity.

Front-Door Routing

The reference docker/nginx.conf shows the intended split:

• location ^~ /contextforge-rs proxies to the gateway upstream.
• Everything else — UI, management APIs, other ContextForge traffic — stays on the existing control-plane paths.
• The reference listener uses backlog=4096 reuseport and configures upstream retries (error timeout http_502/503/504, 2 tries, 10 s window). For MCP POST bodies this effectively retries only connection-stage failures: nginx does not re-send non-idempotent requests once they reached an upstream, and MCP calls are not idempotent.

There is no production health endpoint today: /health is a with_tools bootstrap helper served at /contextforge-rs/health, and production builds compile it out. Use TCP-level checks or the exported metrics for liveness until a real health endpoint exists. (The reference nginx config’s location = /health predates this and does not match the gateway’s route.)

The cf-integration harness runs the same split with the stock upstream control-plane stack and rewrites public /servers/{id}/mcp to /contextforge-rs/servers/{id}/mcp.

TLS Choices

Session Affinity And Failover

Backend MCP sessions are local process state (Session Ownership):

• More than one replica requires sticky routing by Mcp-session-id; the reference nginx config does not provide this, so today’s safe shapes are a single replica or a front door that adds stickiness.
• On restart or failover, sessions are gone; clients must re-run initialize. Design clients to treat a session-not-found error as “reinitialize”, not “retry”.
• A Redis-backed user session store exists in code, but live backend services would still be process-local; a remote session story is future work.
• Inside one host, the same constraint applies to --single-runtime false; see Concurrency And Runtime Model.

Redis Availability

Redis is required at startup and on every uncached config lookup. The connection manager retries heavily (1,000 retries) rather than failing fast, and the in-process cache (default 60 s) rides out short blips for warm subjects. A cold subject during a Redis outage fails at config lookup; the current Redis adapter reports failed GET calls as missing data, so the layer returns 400 until Redis returns.

Images And Sizing

• CI builds docker/Dockerfile (a rust:1.96.1 builder stage) on every push to main and pushes ghcr.io/<owner>/contextforge-gateway-rs:<version>, where the tag is the Cargo package version. There is no latest tag — pin the version.
• The reference Compose stack runs the gateway with raised limits worth copying to real deployments: nofile 65535 and TCP tuning (tcp_fin_timeout=15, widened local port range) for high connection churn.
• Size CPU with --number-of-cpus (defaults to host CPU count) and keep the default single multi-thread runtime for stateful traffic. Memory scales with active sessions (live backend services) and the config caches (up to 50,000 entries each).

Deployment Checklist

1. Front door routes only /contextforge-rs here.
2. JWT verification key or secret in place and rotated with the control plane’s signing key.
3. Redis reachable, TLS/mTLS across trust zones, write access restricted to the control plane; DATAPLANE_PUBLISHER enabled on the control plane.
4. Upstream connection mode matches the backend URL schemes.
5. One replica per Mcp-session-id (single replica or sticky routing).
6. with_tools disabled in the production build.
7. Telemetry export pointed at the collector; see Telemetry And Diagnostics.

Project

This section covers how to work on the repository and the book itself.

🛠️ Use this section for repo workflow. It keeps contribution and publishing guidance separate from runtime architecture.

Contributing To The Gateway

🛠️ Contribution rule: put behavior in the crate that owns it, keep the client-visible contracts stable, and update the matching book page in the same change.

Where Changes Belong

Inside the library crate, keep the module boundaries from System Shape: config validation in common.rs, extension extraction in layers/, MCP behavior in gateway/, listeners in transports/, Redis details behind UserConfigStore.

Changing MCP Routing

The backend prefix namespace is a client-visible contract (MCP Routing Semantics). Any change to it must update the merge logic, the split logic, and the tests in the same PR — and the Control-Plane Integration if the client-facing surface moves.

The project is still early, with no external users: prefer the right architecture over preserving unstable APIs or compatibility surfaces.

Adding Plugin Hooks

New hook points need defined behavior for failure, timeout, cancellation, streaming, and telemetry attribution before they land on the hot path — see Plugins And Policy. Avoid ad hoc plugin calls in the middle of routing code.

Branch And Pull Request Workflow

Name contribution branches user/<user>/<pr-branch>, where <user> is the contributor’s GitHub username and <pr-branch> is a short, kebab-case summary of the change. For example: user/alice/fix-session-cleanup.

Open pull requests as drafts while work or validation is still in progress. Mark a pull request ready for review only after its implementation, documentation, tests, and required checks are complete.

Validation

The pre-commit hooks run these local gates:

cargo fmt --all --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo deny check advisories licenses
cargo nextest run --locked --workspace
cargo build --locked --workspace
cargo bench --locked --workspace --no-run

CI runs the same gates and additionally runs cargo shear --check-test-targets --deny-warnings --locked.

Expectations by change type:

For end-to-end confidence against the real control plane, run the cf-integration lanes.

Keep The Book True

Every page in this book states verifiable behavior. When a change makes a page wrong — a flag, a status code, a lock, a boundary — fix the page in the same PR. Stale architecture docs are worse than none.

Publishing This Book

📚 Publishing path: mdBook renders docs/book/src/ into static HTML, and the GitHub Pages workflow deploys that HTML from main.

Local Build And Preview

The Pages workflow uses mdBook 0.5.3; use the same version locally:

cargo install mdbook --version 0.5.3 --locked

Build and preview:

mdbook build docs/book
mdbook serve docs/book --hostname 127.0.0.1 --port 3000 --open

The generated HTML lands in docs/book/book/, which is ignored by git. Never edit files there; edit docs/book/src/ instead.

Validation Before Pushing

mdbook build docs/book
mdbook test docs/book
git diff --check

mdbook test runs Rust code blocks as tests and confirms that every chapter in SUMMARY.md parses.

Workflow Behavior

.github/workflows/pages.yml runs when a change touches docs/book/** or the workflow file itself:

Builds are cancelled when a newer run starts on the same ref; deploys are serialized and never cancelled mid-flight.

Repository Settings

Publishing requires the repository’s GitHub Pages source to be set to GitHub Actions. Without that setting, the deploy job cannot publish the uploaded artifact.

Adding Or Renaming Chapters

1. Add or rename the Markdown file under docs/book/src/.
2. Update docs/book/src/SUMMARY.md; its order is the reader’s path through the book.
3. Run the validation commands above.

Draft chapters use the > Status: draft. To be implemented. marker followed by a ## To implement list, so unfinished pages stay visible and navigable.