How It Works

This page explains Sesame's internal architecture and how sessions are executed.

Overview

Sesame is a monorepo application with a Hono backend API and TanStack Router frontend, orchestrating AI coding agents in sandboxed environments.

Monorepo Structure

Sesame uses Turborepo for monorepo management with the following structure:

Session Execution Flow

Step-by-Step

1. User creates a session with a prompt and selects a repository
2. API validates the request and creates a session record
3. SandboxProvider creates a workspace:
   • Creates a temp directory for the session
   • Clones the repository using the user's GitHub PAT
   • Creates a new branch for the session
4. Mise bootstraps the environment:
   • Installs runtime versions from .nvmrc, .tool-versions, etc.
   • Installs dependencies via mise prepare (npm, pip, etc.)
5. Agent executor runs the selected agent:
   • Injects credentials (API keys or subscription tokens)
   • Spawns the agent CLI via mise exec for correct versions
   • Each provider's execute() is an async generator yielding typed StreamEvent variants
   • The stream handler accumulates events into Part objects, persists to the messages and message_parts tables
6. Output streams to the UI via Server-Sent Events:
   • SessionEvent objects are broadcast using the type field as the SSE event name
   • On reconnect, all messages and parts are replayed from the database
   • Clients de-duplicate by part ID
7. On completion:
   • Commits changes to the branch
   • Pushes to the remote repository
   • Optionally creates a pull request
8. Sandbox is cleaned up

Directory Structure

Sandbox Directory Structure

Each session runs in complete isolation. The sandbox contains:

• A fresh clone of the repository
• mise installation with project-specific tool versions
• Agent credential files (if using subscriptions)
• Environment variables for API keys

Messages + Parts Model

All real-time data flows through a Messages + Parts model. Messages represent user or assistant turns, and parts represent individual content units within a message (text, tool calls, reasoning, etc.). This is exposed to clients via SessionEvent objects over SSE.

Event Pipeline

Each provider's execute() method is an async generator that parses raw agent output into typed StreamEvent variants via a pure parseLine() function, then yields them. The session executor consumes events with a for await loop and passes each to the stream handler closure.

SessionEvent

SSE events are discriminated by a type field, which is also used as the SSE event name:

Part Types

Parts are discriminated by a type field:

StreamEvent Protocol

Each agent provider's execute() is an async generator yielding StreamEvent variants — a typed discriminated union defined in packages/agents/src/types/agent.ts:

Each provider has a pure parseLine() function in its events.ts that converts raw agent output lines into StreamEvent[]. The createStreamHandler() closure in apps/server/src/services/session-stream-handler.ts consumes these events, accumulates stateful parts (text, tool lifecycle, reasoning), persists to DB, and broadcasts via SSE.

SSE Replay

When a client connects or reconnects, the stream route:

1. Reads all messages and message_parts from the database for the session
2. Sends message.updated and message.part.updated events for each
3. Clients de-duplicate by part ID to avoid showing duplicate content

Dynamic Port Forwarding

Sesame automatically detects and exposes ports when dev servers start, allowing you to preview running applications directly in the UI.

How It Works

Framework Detection

The port detection system recognizes output patterns from common frameworks:

Provider Implementations

Local Provider: Ports are directly accessible on localhost - no forwarding needed.

Docker Provider: Uses a pre-allocated port pool with socat proxies inside the container:

The Docker provider:

• Pre-allocates a pool of host ports at container startup (default: 10, configurable via SANDBOX_PORT_POOL_SIZE)
• Maps each pool slot to an internal proxy port (39000+)
• Uses socat inside the container for dynamic forwarding without container restart
• Reclaims pool slots when ports are unexposed

Manual Port Exposure

If automatic detection fails, users can manually expose ports via the Preview pane dropdown menu. This is useful for:

• Services that don't output their port in a detectable format
• Non-standard ports or secondary services
• Monorepos running multiple dev servers

Mise Integration

Sesame uses mise for automatic runtime version and dependency management. This ensures agents work with the exact tool versions specified in the project.

How It Works

Runtime Version Detection

Mise automatically reads version specifications from common config files:

Dependency Installation

The mise prepare command auto-detects and runs the appropriate install command:

Sandbox Isolation

• Docker sandboxes (with sesame-sandbox image):
  • mise pre-installed at /usr/local/bin/mise
  • Common runtimes pre-cached at /opt/mise/ (Node 20/22/24, Python 3.12/3.13/3.14, Go, Rust, Ruby, Bun)
  • Agent CLIs pre-installed (claude, codex, copilot, gemini, opencode, amp)
  • Per-sandbox mise config symlinks to system shims for instant availability
• Local sandboxes: mise is downloaded per-sandbox to {workDir}/.mise/bin/mise
• Version is pinned in packages/sandbox/src/mise/bootstrap.ts for reproducibility

Database Schema

Configuration System

Settings are resolved in priority order:

The admin UI at /admin/settings allows editing config file values. Settings locked by environment variables display a lock icon and cannot be changed.

Security Model

OS-Level Sandbox Security

Sesame uses @anthropic-ai/sandbox-runtime to provide OS-level sandboxing on supported platforms (macOS and Linux). This provides defense-in-depth beyond simple process isolation:

Filesystem Restrictions:

• Write access: Limited to project directory and /tmp
• Read denied: Sensitive paths like ~/.ssh, ~/.aws, /etc/passwd
• Write denied: System directories, credential files

Network Restrictions:

• Allowed domains: Agent-specific APIs (e.g., api.anthropic.com for Claude), declared per-provider via networkDomains property
• Global allowed: Configurable via admin settings
• Denied domains: Configurable blocklist

Violation Monitoring:

• Real-time detection of sandbox violations
• Streamed to UI via Server-Sent Events
• Stored in session record

Sandbox Configuration

Security can be configured at multiple levels:

Directory Isolation

• Each session runs in a separate directory
• Agent processes are spawned with restricted environment
• Cleanup runs after session completion

Secrets Management

• API keys encrypted at rest (AES-256-GCM)
• GitHub PATs encrypted with ENCRYPTION_KEY
• Credentials injected via environment variables, not files

Authentication

• Default: no auth required (single-user app)
• Optional HTTP Basic Auth via AUTH_PASSWORD env var

Error Handling

All API errors follow the RFC 7807 Problem Details format:

{
  "type": "urn:sesame:error:not-found",
  "title": "Not Found",
  "status": 404,
  "detail": "Session not found"
}

How It Works

Route handlers throw AppError instances (defined in apps/server/src/lib/errors.ts) with factory functions like badRequest(), notFound(), unauthorized(), etc. The global error middleware catches these and returns properly formatted RFC 7807 JSON responses.

Error Types

Agent Models API

Agent model lists (which models are available for each agent) are fetched from a centralized public API at api.sesame.works. This allows model lists to be updated without releasing a new version of Sesame. The frontend fetches through the Sesame server's proxy endpoints rather than contacting api.sesame.works directly.

How It Works

1. The useAgentModels hook fetches models from /api/external/models (a server-side proxy) via TanStack Query
2. The server proxies the request to the configured sesameApi.baseUrl (default: https://api.sesame.works)
3. Results are cached and deduplicated across components using Jotai atoms
4. If the API is unavailable, hardcoded fallback models are used
5. Models include metadata like provider, display name, and which model is the default
6. Self-hosters can point SESAME_API_URL to their own API instance

API Response Format

{
  "updatedAt": "2025-01-15T00:00:00Z",
  "agents": {
    "claude": {
      "models": [
        { "id": "claude-sonnet-4-5", "name": "Claude Sonnet 4.5", "provider": "anthropic", "default": true },
        { "id": "claude-opus-4-5", "name": "Claude Opus 4.5", "provider": "anthropic" }
      ]
    },
    "codex": { "models": [...] },
    "copilot": { "models": [...] },
    "gemini": { "models": [...] },
    "amp": { "models": [...] },
    "opencode": { "models": [...] }
  }
}

Tech Stack

Development Workflow

# Install dependencies
bun install

# Start both apps (via Turborepo)
bun run dev

# Frontend: http://localhost:13530 (proxies /api to backend)
# Backend: http://localhost:13531

In development, the Vite dev server proxies /api requests to the Hono backend on port 13531.

Deployment

When built, the Hono server serves both the API and the pre-built static frontend files:

The Docker image bundles everything into a single container that serves both frontend and API from port 13531.