Architecture

This page describes the current Rust implementation, including temporary backend fan-out and session behavior. The tentative configuration-driven end state is in ContextForge 2.0 Target Architecture and Roadmap.

Middleware Stack Order

Tower layers execute outside-in. A request reaches MCP handlers with these extensions already set:

TCP/TLS listener
  -> HttpMetricsLayer
  -> TraceLayer
  -> /contextforge-rs nested router
  -> mcp_origin_layer          → validates Origin                 (403 when invalid/disallowed)
  -> CORS layer
  -> mcp_header_limits_layer   → MCP standard header budgets      (431 when exceeded)
  -> virtual_host_id_layer       → inserts VirtualHostId           (400 on path mismatch)
  -> claims_layer                → inserts ContextForgeClaims      (401 on bad/missing JWT)
  -> session_id_layer            → inserts SessionId if present
  -> user_config_store_layer     → inserts UserConfig              (400 no config, 500 store error)
  -> virtual_host_config_layer   → rejects unknown vhost           (404 "Server not found")
  -> /servers/{virtual_host_name}/mcp RMCP service → validates Host, then dispatches MCP

DNS-rebinding validation is split by behavior. mcp_origin_layer rejects any present Origin that is malformed or not allowlisted; requests without Origin continue. RMCP validates the optional Host allowlist at the MCP service boundary. See Security. mcp_header_limits_layer rejects excessive MCP standard headers before JWT validation, config lookup, session creation, backend fanout, or RMCP body parsing.

MCP handlers read typed extensions and never parse paths or Redis keys directly. tools/call reads the downstream header map from RMCP’s request-context Parts extension for parameter-header validation.

Pipeline Shape

downstream request
  -> Origin validation → MCP header limits → virtual host extraction → JWT validation → session extraction
  -> user config lookup → RMCP request validation → MCP handler validation
  -> request plugin hooks
  -> backend MCP call (concurrent via join_all for initialize/list)

upstream response
  -> response plugin hooks → merge/namespace/passthrough
  -> metrics, tracing, logging → downstream response

flowchart TD
    bin["binary\nCLI · logging · runtime"]
    lib["lib\nrouting · middleware\nsessions · transports"]
    apis["apis\nUserConfig · VirtualHost\nBackendMCPGateway"]
    cpex["cpex\nCPEX hook factories"]
    bin --> lib
    lib --> apis
    lib --> cpex

Hot-path pipeline (each stage must complete before the next):

flowchart TD
    D(["downstream request"])
    A["virtual host · JWT\nsession extract"]
    C["user config lookup\nRMCP · MCP validate"]
    P1["request plugins\ntool_pre_invoke"]
    B["backend MCP call\njoin_all for init/list"]
    P2["response plugins\ntool_post_invoke"]
    M["merge · namespace\npassthrough"]
    T["metrics · tracing · logging"]
    U(["downstream response"])
    D --> A --> C --> P1 --> B --> P2 --> M --> T --> U

RMCP enforces its configured request-body cap and validates modern standard headers before dispatch. The tools/call handler then resolves the request’s backend and original tool name. When UserConfig contains that tool’s input schema, it validates recognized Mcp-Param-* headers against the request body; it does not call backend tools/list. Without a published schema, parameter headers are unrecognized and forwarded without local validation. Published annotations are validated for MCP token, uniqueness, primitive type, and properties-only reachability constraints. Nested annotations read the exact argument path. Present non-null values require a matching header; absent or null values require no header. Parameter headers are forwarded unchanged; request plugins run afterward, so a plugin that changes an annotated argument also owns any resulting upstream mismatch.

Order is invariant: auth/config before backend selection; request plugins before upstream; response plugins before returning.

Module Boundaries (contextforge-data-plane-lib)

State Ownership

Session rule: backend MCP services are local process state. Sticky routing required for load-balanced deployments.

Executor Shapes

In multi-runtime mode, the first thread initializes the optional CPEX plugin runtime before the others start; the current-thread builders are tuned with a global queue interval of 1024 and 4 I/O events per tick.

Multi-runtime consequence: each runtime thread builds its own BackendTransports map and user-session store. Backend session state is per-runtime-thread, and SO_REUSEPORT gives no connection affinity — later requests in a streamable HTTP session can land on a thread that does not own the session. Treat single-runtime as the only mode supporting stateful MCP sessions today.

Lock Design

Design rule: locks guard maps of handles, not I/O. Backend calls, Redis reads, and plugin hooks all run outside any gateway lock.

Listener Behavior

The TCP listener binds with reuseaddr, reuseport, and keepalive, listens with a backlog of 1024, and serves Axum with graceful shutdown on ctrl_c. The TLS listener accepts by hand through Rustls and serves the same router via Hyper.

Allocator

The binary sets tikv_jemallocator as the global allocator. jemalloc holds up better than the system allocator under the many small, short-lived allocations of per-request JSON and header processing.

Fanout And Cancellation

• initialize opens one backend transport per configured backend concurrently (futures::future::join_all); a failed backend degrades that backend only.
• List methods fan out to all connected backends concurrently and merge.
• Targeted calls (except call_tool) resolve exactly one backend service handle from BackendTransports.
• Targeted tool, prompt, and resource calls run configured pre/post plugin hooks after backend routing. call_tool creates a fresh per-request backend connection via connect_backend_for_request, then explicitly closes it before returning.
• call_tool watches the downstream cancellation token and forwards a cancel to the backend if the client gives up first; backend progress notifications are forwarded downstream while the call is in flight.

Resource reads carry a concrete, request-owned hook state across backend I/O. It pins the runtime selected before the read, or records that no post hook was configured. Post processing consumes that state without type erasure, downcasts, or a second registry lookup.

Startup And Response Flow

Startup sequence (main.rs → Gateway::run_gateway):

install rustls crypto provider
  -> Config::parse()
  -> logging::init_tracing_logging(&config)
  -> Runtime::from(&config)          ← sets executor shape
  -> optional CpexRuntimeRegistry
  -> Gateway::builder()
       .with_config(config)
       .with_user_config_store_type(UserConfigStoreType::Redis)
       .with_session_manager(LocalSessionManager::default())
       .with_plugin_runtime(...)
       .build()
  -> runtime.execute(gateway, plugin_registry)

Response unwind order (Tower layers execute outside-in, so unwind is inside-out):

backend response
  -> response plugin hooks (tool, prompt, and resource calls)
  -> merge / namespace / pass through
  -> virtual_host_config_layer response side
  -> user_config_store_layer response side
  -> session_id_layer response side  ← on DELETE success: remove session + backend transports
  -> claims_layer response side
  -> virtual_host_id_layer response side
  -> CORS, mcp_origin_layer, TraceLayer, HttpMetricsLayer
  -> downstream response

Flow checkpoints — each must exist before the next dependency runs:

MCP-First, Not MCP-Only

The current code implements MCP behavior, but the gateway shell is broader:

auth → config lookup → transport setup → plugin runtime → telemetry → session strategy

Keep protocol-neutral concerns (auth, config ingestion, TLS handling, plugin execution, telemetry, runtime shape, session strategy) reusable. Future A2A or model-provider routing should reuse the gateway shell without copying the MCP routing stack. MCP-specific behavior must remain isolated to the current MCP modules.

Transport Security Split

Transport security is split across two owners; keep this visible:

Do not bury transport security decisions inside MCP method handlers. They belong in startup assembly or explicit backend transport construction.

Plugin Hook Expansion Requirements

Current supported hooks cover tool, prompt, and resource pre/post lifecycles. Before adding any new hook point, define all of the following:

Avoid ad hoc plugin calls in routing code. New hook points belong at explicit, documented pipeline positions.

Architecture-Change Follow-Through Matrix

Changing a load-bearing choice requires updating more than one file: