Agents

This guide explains how to add support for a new AI coding agent to Sesame.

Overview

Each agent in Sesame is defined by:

Provider - Metadata, spawn configuration, and credential injection
Converter - Normalizes provider-specific output into UniversalEvent objects
Schema entry - Agent type registered in shared schemas

File Structure

Each agent has a provider under providers/ and a converter under converters/:

base.ts # Base agent provider class

index.ts # Provider exports

claude/ # Claude Code agent

codex/ # OpenAI Codex agent

copilot/ # GitHub Copilot agent

gemini/ # Google Gemini agent

opencode/ # OpenCode multi-provider agent

mock/ # Mock agent (testing)

base.ts # BaseConverter (text accumulation, tool tracking)

base.test.ts # Converter tests

index.ts # createConverter() factory

claude.ts # Claude → UniversalEvent

codex.ts # Codex → UniversalEvent

copilot.ts # Copilot → UniversalEvent

gemini.ts # Gemini → UniversalEvent

opencode.ts # OpenCode → UniversalEvent

types # Shared type definitions

registry.ts # Agent registry

index.ts # Package exports

Step 1: Create Agent Provider

Create a new directory for your agent in packages/agents/src/providers/:

packages/agents/src/providers/my-agent/provider.ts

import { BaseAgentProvider } from "../base";
import type { AgentExecutionContext, AgentMetadata } from "../../types";

export class MyAgentProvider extends BaseAgentProvider {
  readonly id = "my-agent";
  readonly name = "My Agent";
  readonly provider = "My Company";

  getMetadata(): AgentMetadata {
    return {
      id: this.id,
      name: this.name,
      provider: this.provider,
      description: "AI coding agent from My Company",
      authMethods: ["api", "oauth"], // or just ["api"]
      envVars: ["MY_AGENT_API_KEY"],
    };
  }

  async checkAuth(context: AgentExecutionContext): Promise<boolean> {
    // Check if credentials are available
    return !!(context.apiKeys?.MY_AGENT_API_KEY || context.credentials);
  }

  async install(context: AgentExecutionContext): Promise<void> {
    // Install CLI if needed
    await context.sandbox.exec("npm", ["install", "-g", "my-agent-cli"]);
  }

  async injectCredentials(context: AgentExecutionContext): Promise<Record<string, string>> {
    // Return environment variables for the agent
    const env: Record<string, string> = {};

    if (context.apiKeys?.MY_AGENT_API_KEY) {
      env.MY_AGENT_API_KEY = context.apiKeys.MY_AGENT_API_KEY;
    }

    // Or write credential files if needed
    if (context.credentials?.accessToken) {
      const credPath = `${context.workDir}/.my-agent/auth.json`;
      await context.sandbox.writeFile(credPath, JSON.stringify({
        token: context.credentials.accessToken,
      }));
    }

    return env;
  }

  async execute(context: AgentExecutionContext): Promise<void> {
    const env = await this.injectCredentials(context);

    // Run the agent CLI
    await context.sandbox.spawn("my-agent", ["run", context.prompt], {
      env,
      onStdout: (data) => context.onOutput?.(data),
      onStderr: (data) => context.onOutput?.(data),
    });
  }
}

Step 2: Create Index Export

Create a barrel export in your agent's directory:

packages/agents/src/providers/my-agent/index.ts

export { MyAgentProvider } from "./provider";

Step 3: Register the Agent

Add your agent to the registry in packages/agents/src/registry.ts:

import { MyAgentProvider } from "./providers/my-agent";

export const agentRegistry = {
  // ... existing agents
  "my-agent": new MyAgentProvider(),
};

And export from packages/agents/src/providers/index.ts:

export { MyAgentProvider } from "./my-agent";

Step 4: Create Event Converter

Each agent needs a converter that normalizes its raw StreamEvent output into UniversalEvent objects. Create a converter in packages/agents/src/converters/:

packages/agents/src/converters/my-agent.ts

import { BaseConverter } from "./base";

/**
 * My Agent Converter
 *
 * If your agent emits standard StreamEvent types (content, tool_use,
 * tool_result, thinking, permission_request), the base class handles
 * everything. Override convert() only for custom event formats.
 */
export class MyAgentConverter extends BaseConverter {
  // Uses default conversion logic.
  // Override convert() if your agent has non-standard output:
  //
  // convert(event: StreamEvent): UniversalEvent[] {
  //   if (event.type === "my-custom-type") {
  //     return [createMessageEvent(this.getNextSeq(), {
  //       textDelta: event.text,
  //       fullText: this.accumulatedText + event.text,
  //       messageId: this.messageId,
  //     })];
  //   }
  //   return this.convertDefault(event);
  // }
}

import { MyAgentConverter } from "./my-agent";

export function createConverter(agentType: AgentType, options: ConverterOptions): BaseConverter {
  switch (agentType) {
    // ... existing agents
    case "my-agent":
      return new MyAgentConverter(options);
  }
}

The BaseConverter provides:

Text accumulation: Tracks textDelta → fullText across events
Tool call tracking: Maintains a map of open tool calls, auto-closes on flush()
Sequence ID generation: Via the getNextSeq() callback from EventSequenceManager

Step 5: Add to Schema

Update the agent enum in packages/shared/src/schemas/task.ts:

export const agentEnum = z.enum([
  "claude-code",
  "codex",
  "copilot",
  "gemini",
  "opencode",
  "my-agent", // Add your agent
]);

Step 6: Add UI Support

Agent Selector

Add your agent to the CODING_AGENTS array in apps/web/src/lib/models.ts:

apps/web/src/lib/models.ts

export const CODING_AGENTS = [
  { value: "claude", label: "Claude" },
  { value: "codex", label: "Codex" },
  { value: "copilot", label: "Copilot" },
  { value: "gemini", label: "Gemini" },
  { value: "opencode", label: "OpenCode" },
  { value: "my-agent", label: "My Agent" }, // Add your agent
] as const;

Add an icon component in apps/web/src/components/logos/:

apps/web/src/components/logos/my-agent.tsx

export function MyAgent({ className }: { className?: string }) {
  return <svg className={className}>...</svg>;
}

Agent model lists are fetched from the public API at api.sesame.works/models. To add fallback models for your agent (used when the API is unavailable), update FALLBACK_AGENT_MODELS in apps/web/src/lib/models.ts:

apps/web/src/lib/models.ts

export const FALLBACK_AGENT_MODELS: Record<AgentType, AgentModel[]> = {
  // ... existing agents
  "my-agent": [
    { id: "my-model-pro", name: "My Model Pro", provider: "my-company", default: true },
    { id: "my-model-lite", name: "My Model Lite", provider: "my-company" },
  ],
};

For your agent's models to appear in production, they need to be added to the public API at api.sesame.works. Contact the Sesame maintainers or submit a PR to add your agent's models.

Agent Provider Interface

interface AgentProvider {
  // Identity
  readonly id: string;
  readonly name: string;
  readonly provider: string;

  // Metadata
  getMetadata(): AgentMetadata;

  // Lifecycle
  checkAuth(context: AgentExecutionContext): Promise<boolean>;
  install(context: AgentExecutionContext): Promise<void>;
  injectCredentials(context: AgentExecutionContext): Promise<Record<string, string>>;
  execute(context: AgentExecutionContext): Promise<void>;
}

interface AgentMetadata {
  id: string;
  name: string;
  provider: string;
  description?: string;
  authMethods: ("api" | "oauth")[];
  envVars?: string[];
}

interface AgentExecutionContext {
  taskId: string;
  prompt: string;
  workDir: string;
  sandbox: SandboxProvider;
  apiKeys?: Record<string, string>;
  credentials?: AgentCredentials;
  onOutput?: (data: string) => void;
  onProgress?: (progress: number) => void;
}

Testing Your Agent

Manual testing:

# Test CLI works
my-agent --version

# Test with Sesame
bun run dev
# Create a task with your agent

Unit tests:

packages/agents/src/providers/my-agent/__tests__/provider.test.ts

import { MyAgentProvider } from "../provider";

describe("MyAgentProvider", () => {
  const provider = new MyAgentProvider();

  it("has valid metadata", () => {
    const metadata = provider.getMetadata();
    expect(metadata.id).toBe("my-agent");
    expect(metadata.authMethods).toContain("api");
  });

  it("checks auth correctly", async () => {
    const context = {
      apiKeys: { MY_AGENT_API_KEY: "test-key" },
    };
    expect(await provider.checkAuth(context as any)).toBe(true);
  });
});

Common Patterns

API Key Authentication

For agents that only use API keys:

class MyAgentProvider extends BaseAgentProvider {
  getMetadata(): AgentMetadata {
    return {
      id: "my-agent",
      name: "My Agent",
      provider: "My Company",
      authMethods: ["api"],
      envVars: ["MY_AGENT_API_KEY"],
    };
  }

  async injectCredentials(context: AgentExecutionContext): Promise<Record<string, string>> {
    return {
      MY_AGENT_API_KEY: context.apiKeys?.MY_AGENT_API_KEY ?? "",
    };
  }
}

OAuth/Subscription Authentication

For agents with OAuth or subscription-based auth:

class MyAgentProvider extends BaseAgentProvider {
  getMetadata(): AgentMetadata {
    return {
      id: "my-agent",
      name: "My Agent",
      provider: "My Company",
      authMethods: ["api", "oauth"],
    };
  }

  async injectCredentials(context: AgentExecutionContext): Promise<Record<string, string>> {
    // Write credential file that agent reads
    if (context.credentials?.accessToken) {
      const authPath = `${context.workDir}/.my-agent/auth.json`;
      await context.sandbox.writeFile(authPath, JSON.stringify({
        access_token: context.credentials.accessToken,
        refresh_token: context.credentials.refreshToken,
        expires_at: context.credentials.expiresAt,
      }));
    }

    return {};
  }
}

Structured Output Parsing

For agents with JSON or structured output, override the execute method:

async execute(context: AgentExecutionContext): Promise<void> {
  await context.sandbox.spawn("my-agent", ["--format", "json", context.prompt], {
    onStdout: (line) => {
      try {
        const data = JSON.parse(line);
        if (data.type === "progress") {
          context.onProgress?.(data.percent);
        } else if (data.type === "output") {
          context.onOutput?.(data.content);
        }
      } catch {
        // Plain text fallback
        context.onOutput?.(line);
      }
    },
  });
}

Mock Agent

Sesame includes a mock agent provider for testing the full streaming pipeline without real agent CLIs or API keys. It produces deterministic event sequences via predefined scenarios (simple text, tool calls, thinking, permissions, errors, multi-turn).

To use it: select "Mock Agent" when creating a task, and set the prompt to scenario:<name> (e.g., scenario:toolCall). See packages/agents/src/providers/mock/scenarios.ts for available scenarios.