Session affinity

Design

Session affinity uses a current-assignment model: the session_affinity table stores a single row per session — the most recent provider assignment. There is no history of prior assignments; that role is served by the routing_events audit log.

The X-Session-Id header carries the session identifier (generated by the client, e.g. an ACP server). The switchboard reads it on every request and queries the session database.

SessionManager trait

#[async_trait]
trait SessionManager: Send + Sync {
    async fn lookup(&self, session_id: &str) -> Result<Option<SessionAffinity>, SessionError>;
    async fn assign(&self, session_id: &str, provider: &str, model: &str, surface: &str) -> Result<(), SessionError>;
    async fn update_tokens(&self, session_id: &str, input: u64, output: u64) -> Result<(), SessionError>;
    async fn increment_switch(&self, session_id: &str, new_provider: &str) -> Result<(), SessionError>;
    async fn insert_routing_event(&self, event: RoutingEvent) -> Result<(), SessionError>;
}

Two implementations exist:

• MemorySessionManager — HashMap-backed, used in tests.
• SqliteSessionManager — sqlx-backed with migrations, used in production. Both pass the same parameterized test suite.

SQLite schema

CREATE TABLE session_affinity (
    session_id          TEXT PRIMARY KEY,
    provider_identity   TEXT NOT NULL,
    model_name          TEXT NOT NULL,
    api_surface         TEXT NOT NULL DEFAULT 'openai',
    assigned_at         INTEGER NOT NULL,
    last_used_at        INTEGER NOT NULL,
    total_input_tokens  INTEGER DEFAULT 0,
    total_output_tokens INTEGER DEFAULT 0,
    total_requests      INTEGER DEFAULT 0,
    switch_count        INTEGER DEFAULT 0,
    is_active           INTEGER DEFAULT 1
);

CREATE INDEX idx_affinity_last_used ON session_affinity(last_used_at);
CREATE INDEX idx_affinity_provider ON session_affinity(provider_identity);

CREATE TABLE routing_events (
    id                  INTEGER PRIMARY KEY AUTOINCREMENT,
    session_id          TEXT,
    request_id          TEXT NOT NULL,
    model_name          TEXT NOT NULL,
    provider_identity   TEXT NOT NULL,
    billing_model       TEXT NOT NULL,
    decision_reason     TEXT NOT NULL,
    input_tokens        INTEGER,
    output_tokens       INTEGER,
    response_status     INTEGER,
    latency_ms          INTEGER,
    created_at          INTEGER NOT NULL
);

CREATE INDEX idx_routing_session ON routing_events(session_id);

Affinity mechanics

1. First request (no affinity): The router selects the best provider via ranking, then upserts a session_affinity row. switch_count starts at 0.

2. Subsequent requests (affinity hit): The router reads the existing row. If the assigned provider is still healthy (not degraded, has a valid credential), it skips ranking and uses it directly. This is the affinity fast path and is why session affinity matters — it preserves the upstream provider's KV cache for the conversation context.

3. Provider degrades (affinity break): When the assigned provider is filtered out, the router falls through to candidate ranking, selects a new provider, calls increment_switch() (updates provider_identity and increments switch_count), and logs a routing event with reason "fallback".

4. No provider available: The session row is left unchanged (the old provider remains recorded). A future request may succeed when the provider recovers.

Persistence rationale

Session-to-provider mappings must survive restarts for two reasons:

1. KV cache preservation: Switching providers mid-session destroys the conversation context cached by the first provider. Both OpenAI and Anthropic charge less for cached input tokens. If the switchboard restarts and forgets the provider assignment, it may re-route to a different provider, incurring a full context re-send cost that can be 30-50% of the session's token budget.

2. Credential pooling stability: With multiple API keys for the same provider (configured as separate providers), switching keys forces a context cache miss because the cache is keyed by the API key/project. Persisting the assignment ensures the same key is used for the same session across restarts.

Database file

The default path follows platform conventions via agentkit-path:

Platform	Path
Linux	~/.local/state/agentkit/switchboard/sessions.db
macOS	~/Library/Application Support/AgentKit/switchboard/sessions.db
Windows	~/AppData/LocalLow/AgentKit/switchboard/sessions.db

Overridable via --session-db flag or session_db_path in config.

On corruption, the file is renamed to .sessions.db.corrupted and a fresh database is created. Write failures are logged but do not block the request — the session assignment is already committed in the routing decision.