How It Works

This page explains Sesame's internal architecture and how tasks are executed.

Overview

Sesame is a monorepo application with a Hono backend API and TanStack Router frontend, orchestrating AI coding agents in sandboxed environments.

Monorepo Structure

Sesame uses Turborepo for monorepo management with the following structure:

Task Execution Flow

Step-by-Step

1. User creates a task with a prompt and selects a repository
2. API validates the request and creates a task record
3. SandboxProvider creates a workspace:
   • Creates a temp directory under TASK_DIR_BASE
   • Clones the repository using the user's GitHub PAT
   • Creates a new branch for the task
4. Mise bootstraps the environment:
   • Installs runtime versions from .nvmrc, .tool-versions, etc.
   • Installs dependencies via mise prepare (npm, pip, etc.)
5. Agent executor runs the selected agent:
   • Injects credentials (API keys or subscription tokens)
   • Spawns the agent CLI via mise exec for correct versions
   • Raw output is converted to UniversalEvent by a per-provider converter
   • Events are persisted to the task_events table (append-only, one INSERT per event)
6. Output streams to the UI via Server-Sent Events:
   • Each event has a monotonic sequence ID (seq) used as the SSE id: field
   • Clients can reconnect with Last-Event-ID to replay missed events
   • Events are buffered in-memory for fast replay, with DB fallback
7. On completion:
   • Commits changes to the branch
   • Pushes to the remote repository
   • Optionally creates a pull request
8. Sandbox is cleaned up (unless "Keep Alive" is enabled)

Directory Structure

Sandbox Directory Structure

Each task runs in complete isolation. The sandbox contains:

• A fresh clone of the repository
• mise installation with project-specific tool versions
• Agent credential files (if using subscriptions)
• Environment variables for API keys

Event System

All real-time data flows through a unified UniversalEvent schema. This provides a single, consistent event format regardless of which AI agent produced the output.

Event Pipeline

UniversalEvent

Every event has these core fields:

Additional fields are populated based on kind (e.g., textDelta/fullText for messages, toolCall for tool calls, permissionId/decision for permissions).

Per-Provider Converters

Each agent has a converter that extends BaseConverter in packages/agents/src/converters/. The base class provides:

• Text accumulation: Tracks textDelta → fullText across events
• Tool call tracking: Auto-closes open tool calls on flush
• Sequence ID generation: Via callback to EventSequenceManager

Subclasses override convert() for provider-specific normalization. Most agents (Claude, Copilot) use the default logic. Codex and Gemini have custom handling for their streaming formats.

SSE Replay

When a client reconnects, the stream route checks for a Last-Event-ID header:

1. In-memory buffer hit: Events after the given seq are replayed from the ring buffer (fast path)
2. DB fallback: If the buffer has been evicted, events are loaded from task_events (slow path)
3. Fresh connection: Current task status + all persisted events are sent

Append-Only Event Storage

Events are stored in the task_events table with a unique (taskId, seq) index. This replaces the previous tasks.logs JSON column, eliminating the read-modify-write pattern that caused contention under concurrent writes.

Dynamic Port Forwarding

Sesame automatically detects and exposes ports when dev servers start, allowing you to preview running applications directly in the UI.

How It Works

Framework Detection

The port detection system recognizes output patterns from common frameworks:

Provider Implementations

Local Provider: Ports are directly accessible on localhost - no forwarding needed.

Docker Provider: Uses a pre-allocated port pool with socat proxies inside the container:

The Docker provider:

• Pre-allocates a pool of host ports at container startup (default: 10, configurable via SANDBOX_PORT_POOL_SIZE)
• Maps each pool slot to an internal proxy port (39000+)
• Uses socat inside the container for dynamic forwarding without container restart
• Reclaims pool slots when ports are unexposed

Manual Port Exposure

If automatic detection fails, users can manually expose ports via the Preview pane dropdown menu. This is useful for:

• Services that don't output their port in a detectable format
• Non-standard ports or secondary services
• Monorepos running multiple dev servers

Mise Integration

Sesame uses mise for automatic runtime version and dependency management. This ensures agents work with the exact tool versions specified in the project.

How It Works

Runtime Version Detection

Mise automatically reads version specifications from common config files:

Dependency Installation

The mise prepare command auto-detects and runs the appropriate install command:

Sandbox Isolation

• Docker sandboxes (with sesame-sandbox image):
  • mise pre-installed at /usr/local/bin/mise
  • Common runtimes pre-cached at /opt/mise/ (Node 20/22/24, Python 3.12/3.13/3.14, Go, Rust, Ruby, Bun)
  • Agent CLIs pre-installed (claude, codex, copilot, gemini, opencode)
  • Per-sandbox mise config symlinks to system shims for instant availability
• Local sandboxes: mise is downloaded per-sandbox to {workDir}/.mise/bin/mise
• Version is pinned in packages/sandbox/src/mise/bootstrap.ts for reproducibility

Database Schema

Configuration System

Settings are resolved in priority order:

The admin UI at /admin/settings allows editing config file values. Settings locked by environment variables display a lock icon and cannot be changed.

Security Model

OS-Level Sandbox Security

Sesame uses @anthropic-ai/sandbox-runtime to provide OS-level sandboxing on supported platforms (macOS and Linux). This provides defense-in-depth beyond simple process isolation:

Filesystem Restrictions:

• Write access: Limited to project directory and /tmp
• Read denied: Sensitive paths like ~/.ssh, ~/.aws, /etc/passwd
• Write denied: System directories, credential files

Network Restrictions:

• Allowed domains: Agent-specific APIs (e.g., api.anthropic.com for Claude)
• Global allowed: Configurable via admin settings
• Denied domains: Configurable blocklist

Violation Monitoring:

• Real-time detection of sandbox violations
• Streamed to UI via Server-Sent Events
• Stored in task record for audit

Sandbox Configuration

Security can be configured at multiple levels:

Directory Isolation

• Each task runs in a separate directory under TASK_DIR_BASE
• Agent processes are spawned with restricted environment
• Cleanup runs after task completion (unless "Keep Alive" enabled)

Secrets Management

• API keys encrypted at rest (AES-256)
• Session tokens signed with BETTER_AUTH_SECRET
• GitHub PATs encrypted with ENCRYPTION_KEY
• Credentials injected via environment variables, not files

Authentication

• Sessions managed by better-auth
• OIDC support for enterprise SSO
• Role-based access (user, admin)
• Admin routes protected with requireAdmin middleware

Error Handling

All API errors follow the RFC 7807 Problem Details format:

{
  "type": "urn:sesame:error:not-found",
  "title": "Not Found",
  "status": 404,
  "detail": "Task not found"
}

How It Works

Route handlers throw AppError instances (defined in apps/server/src/lib/errors.ts) with factory functions like badRequest(), notFound(), unauthorized(), etc. The global error middleware catches these and returns properly formatted RFC 7807 JSON responses.

Error Types

Agent Models API

Agent model lists (which models are available for each agent) are fetched from a centralized public API at api.sesame.works/models. This allows model lists to be updated without releasing a new version of Sesame.

How It Works

1. The useAgentModels hook fetches models from api.sesame.works/models via TanStack Query
2. Results are cached and deduplicated across components using Jotai atoms
3. If the API is unavailable, hardcoded fallback models are used
4. Models include metadata like provider, display name, and which model is the default

API Response Format

{
  "updatedAt": "2025-01-15T00:00:00Z",
  "agents": {
    "claude": {
      "models": [
        { "id": "claude-sonnet-4-5", "name": "Claude Sonnet 4.5", "provider": "anthropic", "default": true },
        { "id": "claude-opus-4-5", "name": "Claude Opus 4.5", "provider": "anthropic" }
      ]
    },
    "codex": { "models": [...] },
    "copilot": { "models": [...] },
    "gemini": { "models": [...] },
    "opencode": { "source": "models.dev", "models": [...] }
  }
}

Tech Stack

Development Workflow

# Install dependencies
bun install

# Start both apps (via Turborepo)
bun run dev

# Frontend: http://localhost:13530 (proxies /api to backend)
# Backend: http://localhost:13531

In development, the Vite dev server proxies /api requests to the Hono backend on port 13531.

Deployment

When built, the Hono server serves both the API and the pre-built static frontend files:

The Docker image bundles everything into a single container that serves both frontend and API from port 13531.