Factory definition syntax

A factory definition is a versioned tree of YAML and Markdown files rooted at factory.yaml. It can define agents, automations, runners, model routers, benchmarks, scorers, skills, and webhooks. Keys are case-sensitive.

For working definitions you can copy, see warp-factory-examples.

Where the definition lives

Choose where to host the definition when you create a factory:

• Warp-managed (default) - Edit the definition in the Warp Factories web app. Warp validates, commits, and applies changes; its repository stays hidden.
• GitHub - On a paid plan, keep the definition in your own repository. The web app links to the files in a read-only view. Merges to the production branch (main by default) update the factory.

Both modes share the file format. You can link a Warp-managed factory to GitHub later.

Work items, runs, and metrics live in the web app, not in the definition.

Directory structure

Paths identify resources: agents/reviewer/agent.md defines an agent named reviewer. Some resources also declare a name or display label in their fields, as described below.

factory.yaml
agents/
  foreman/
    agent.md
    skills/
      incident-triage/
        SKILL.md
  reviewer/
    agent.md
automations/
  labeled-issue/
    automation.md
runners/
  linux-build.yaml
routers/
  by-task.yaml
benchmarks/
  pull-request-review/
    suite.yaml
    tasks/
      broken-doc-link.yaml
scorers/
  tests-run/
    scorer.md
skills/
  repository-conventions/
    SKILL.md
webhooks/
  internal-ci.yaml

See the example factory definition, the minimal 01-single-repo-quickstart, or the full 02-sdlc-issue-to-pr.

Validate a definition

Warp validates syntax, required fields, and resource access before applying a definition. A Warp-managed factory validates on save; a GitHub-backed factory validates in pull requests to its production branch. Warp applies the definition as one unit and keeps the last valid definition if validation fails.

Pull request checks

The warp/factory-config check validates definition changes against your team and factory. For check results, branch protection, and troubleshooting, see factory-definition pull request checks.

Validate locally or in CI

The warp-factory-examples repository includes validate_factory_files.py. From a checkout of that repository, run the script against your definition’s root directory. It requires Python 3 and a network connection to Warp’s validation service, but no Warp login or existing factory.

python3 scripts/validate_factory_files.py FACTORY_ROOT

Replace FACTORY_ROOT with the directory containing factory.yaml. The script checks syntax and cross-file references and reports errors by file and line. Team- and factory-dependent checks, such as whether a secret or environment exists, run in the GitHub check and when the definition is applied.

For a CI job built on the script, see the example repository’s validation workflow.

Validate with a coding agent

• Warp Agent - Ask the Warp Agent to change or check the definition. Its factory-files skill validates before opening a pull request. For example: “Add a nightly dependency-audit automation and validate the definition.”
• Factory MCP - Connect another coding agent to Factory MCP for schema and validation tools.
• Other agents - Have the agent run the validator script.

JSON Schema

Warp publishes unauthenticated JSON Schemas for editor completion and validation. They describe each field and reject unknown keys.

• https://app.warp.dev/api/v1/factory-files/schemas - Lists supported versions and the current version.
• https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1 - Returns all v1alpha1 documents, keyed by document name.
• https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1/<document> - Returns one document for use as a schema reference.

Schema document	Applies to
factory.schema.json	factory.yaml
agent.schema.json	Agent Markdown frontmatter
automation.schema.json	Automation Markdown frontmatter
scorer.schema.json	Scorer Markdown frontmatter
runner.schema.json	Runner YAML files
router.schema.json	Model-router YAML files
webhook.schema.json	Webhook YAML files
benchmark_suite.schema.json	Benchmark suite YAML files
benchmark_suite_task.schema.json	Benchmark task YAML files
common.schema.json	Shared field definitions

For completion and inline validation in the VS Code YAML extension, add the document URL on the first line:

factory.yaml

# yaml-language-server: $schema=https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1/factory.schema.json
schemaVersion: v1alpha1
name: payments-factory

This YAML-extension configuration doesn’t validate Markdown frontmatter. Use one of the definition validators for Markdown files.

factory.yaml

Required resource. The root document names the factory, scopes it to repositories, and sets execution defaults.

factory.yaml

schemaVersion: v1alpha1
name: payments-factory
repositories:
  - owner: acme
    name: payments-service
agentDefaults:
  model: auto

Field	Required	Default	Description
schemaVersion	Yes	None	Set to v1alpha1.
name	Yes	None	The factory’s name.
description	No	Not specified	What the factory is for.

alias

Optional. The handle used to @-mention the foreman on platforms such as Slack and Linear. The factory dashboard labels it “Foreman name”. It accepts up to 60 letters, numbers, spaces, ., _, and -, and is case-insensitively unique in the workspace.

credentialStrategy

Optional. Whose credentials runs use: EXECUTOR for the identity executing the run, or CREATOR for the user who created it. CREATOR requires a paid plan. Omitting the key preserves the stored setting; the factory default is EXECUTOR. Agents can override the strategy by role.

automations

Optional. Factory-wide automation settings. automations.githubOnboardingNotices defaults to true. Set it to false to suppress GitHub account-connection and team-membership notices. Users still need a connected GitHub account and active membership in the factory’s Warp team to trigger its GitHub automations.

factory.yaml

automations:
  githubOnboardingNotices: false

codeForges

Optional. The enabled code hosts: GITHUB, GITLAB, and AZURE_DEVOPS. Omit the key to preserve them. With multiple hosts, set codeForge on every repository. An empty codeForges list requires empty repositories and configures no code host.

repositories

Required. The factory’s repositories. The list can be empty. Each entry requires owner and name; optional codeForge selects GITHUB, GITLAB, or AZURE_DEVOPS. With multiple codeForges, every repository requires codeForge.

Optional deferred defaults to false. Set it to true to attach a repository without preparing it for every run. Warp prepares it when an agent or automation lists it in required_repos, or a run adds it.

codeForges:
  - GITHUB
  - GITLAB
repositories:
  - codeForge: GITHUB
    owner: acme
    name: payments-service
  - codeForge: GITLAB
    owner: acme/platform
    name: deployment-config

benchmarkRepoSubstitutions

Optional. Repository substitutions for benchmark starting states. Each entry requires source and target objects, each with codeForge, owner, and name. Both codeForge values must be GITHUB; the source must belong to the factory, and the target must not.

secrets

Optional. Names of managed secrets granted to every agent, in addition to secrets declared by an agent.

mcpServers

Optional. MCP servers granted to every agent, keyed by the name shown to the agent. Each entry selects exactly one transport:

• warpId - A Warp-managed MCP server ID.
• command - A stdio command. Optional args is a list of strings, and optional env maps names to string values.
• url - An absolute HTTP or HTTPS URL. Optional headers maps header names to string values.

Don’t put credentials in env or headers. Reference a managed secret as {{SECRET_NAME}}, or use warpId for a managed MCP installation.

mcpServers:
  sentry:
    warpId: SENTRY_MCP_SERVER_ID

cloudProviders

Optional. Cloud-provider identity federation for agent runs:

Field under cloudProviders	Required	Description
gcp.projectNumber	For Google Cloud	Project number. Quote it so YAML keeps it as a string.
gcp.workloadIdentityFederationPoolId	For Google Cloud	Workload identity federation pool ID.
gcp.workloadIdentityFederationProviderId	For Google Cloud	Workload identity federation provider ID.
gcp.serviceAccountEmail	No	Service account to use.
aws.roleArn	For AWS	Role ARN for agent runs.

cloudProviders:
  aws:
    roleArn: arn:aws:iam::123456789012:role/warp-factory

integrations

Optional. The integration providers attached to the factory: slack, microsoft-teams, linear, and jira. Each entry requires type. Declare at most one issue tracker because linear and jira are mutually exclusive. Code-forge access comes from repositories, not this list.

Omit integrations to preserve the currently attached providers. Set it to an empty list to detach every provider.

integrations:
  - type: slack
  - type: linear

For Slack and Microsoft Teams, slack.autoRespondToThreadReplies and microsoft-teams.autoRespondToThreadReplies control whether eligible plain replies in an existing factory thread can continue work without another mention. The Slack setting defaults to true. The Microsoft Teams setting defaults to true when the shared per-factory reply setting is enabled.

integrations:
  - type: slack
    slack:
      autoRespondToThreadReplies: false

For issue discovery and routing, use automation filters such as team_ids and project_keys.

Compatibility fields

Field	Behavior
integrations[].linear.teamIds, integrations[].jira.projectKeys	Accepted but ignored for issue discovery and routing; omitted when Warp rewrites the definition.
providers	Optional legacy alias for cloudProviders, with the same keys. Use cloudProviders in new files; Warp writes that key when updating older definitions.

scorerDefaults

Optional. Sets runner, secrets, and mcpServers for file-defined scorers. A scorer can override each value. The mcpServers map uses the factory-level transport syntax.

selfImprovement

Optional. Configures scheduled self-improvement for a GitHub-backed factory:

• failedRunThreshold - Distinct scored failures per agent required before a scheduled self-improvement run. Use a value from 1 through 50, or omit it for the server default.
• reviewerType - The pool used to request one reviewer: admins (the default), team, custom, or none.
• reviewerEmails - Required when reviewerType is custom and invalid otherwise. Values must be active members of the factory’s team.

agentDefaults

Required. Execution defaults inherited by every agent. Declare exactly one of model or harness. Agents can override these defaults.

agentDefaults:
  model: auto
  runner: linux-build
  environmentId: PAYMENTS_ENVIRONMENT_ID

Field under agentDefaults	Required	When omitted	Description
runner	No	No runner selected here	Name of a runner defined under runners/. See runner OS requirements before choosing a factory default.
environmentId	No	Warp manages the workspace from the factory’s repositories	ID of an existing environment.
secrets	No	No defaults declared here	Managed secrets inherited by agents without their own list. An agent-level list replaces this list; factory-wide secrets still apply.
mcpServers	No	No defaults declared here	MCP servers inherited by agents without their own map. An agent-level map replaces this map; factory-wide servers still apply. Uses the mcpServers syntax.

agentDefaults.model

Conditionally required. Set exactly one of model or harness. The model_id used for runs. See model choice for agents. model is shorthand for the Warp Agent harness:

model: auto

is equivalent to:

harness:
  type: oz
  model: auto

model and harness are mutually exclusive everywhere they appear.

agentDefaults.harness

Conditionally required. Set exactly one of model or harness. The run harness and model. The object requires type and model. type accepts oz, claude, codex, or claude-code (an alias for claude). Prefer the canonical harness identifier used by the CLI and the Warp Platform API. See supported harnesses for availability and behavior.

harness:
  type: codex
  model: gpt-5.3-codex
  reasoningLevel: high
  auth:
    source: managedSecret
    secretName: CODEX_API_KEY

Optional auth requires source:

auth.source	auth.secretName	Requirement
managedSecret	Required	Name of the managed authentication secret.
workerEnvironment	Not allowed	Requires a self-hosted workerHost; uses authentication from that worker’s environment.

The oz harness supplies its own credentials and rejects auth and reasoningLevel. Third-party harnesses accept optional reasoningLevel.

For per-agent harnesses and managed-secret authentication, see 03-multi-harness. To use the dashboard instead, see configuring a third-party harness.

agentDefaults.workerHost

Optional. Where runs execute: warp for Warp-hosted compute, or the ID of a connected self-hosted worker. Omit the key for the workspace default. At the agent or automation level, use null or an empty string to clear an inherited host. Configure the backend on the worker; the definition selects only the worker and runner.

agentDefaults.computerUseModel

Optional. The model_id used by Computer Use. Omit it for automatic selection.

agentDefaults:
  model: auto
  computerUseModel: claude-5-sonnet-high

Use computer-use-agent-auto for automatic selection. A model suffix selects its effort level; thinking enables thinking, and xhigh-fast selects fast mode.

Model	Supported values
Auto	computer-use-agent-auto
Claude Sonnet 5.5	claude-5-5-sonnet-low, claude-5-5-sonnet-medium, claude-5-5-sonnet-high, claude-5-5-sonnet-xhigh, claude-5-5-sonnet-max
Claude Opus 5.5	claude-5-5-opus-low, claude-5-5-opus-medium, claude-5-5-opus-high, claude-5-5-opus-xhigh, claude-5-5-opus-xhigh-fast, claude-5-5-opus-max
Claude Sonnet 5	claude-5-sonnet-low, claude-5-sonnet-medium, claude-5-sonnet-high, claude-5-sonnet-xhigh, claude-5-sonnet-max
Claude Opus 5	claude-5-opus-low, claude-5-opus-medium, claude-5-opus-high, claude-5-opus-xhigh, claude-5-opus-xhigh-fast, claude-5-opus-max
Claude Fable 5.1	claude-5-1-fable-low, claude-5-1-fable-medium, claude-5-1-fable-high, claude-5-1-fable-xhigh, claude-5-1-fable-max
Claude Fable 5	claude-5-fable-low, claude-5-fable-medium, claude-5-fable-high, claude-5-fable-xhigh, claude-5-fable-max
Claude Opus 4.8	claude-4-8-opus-low, claude-4-8-opus-medium, claude-4-8-opus-high, claude-4-8-opus-xhigh, claude-4-8-opus-xhigh-fast, claude-4-8-opus-max
Claude Opus 4.7	claude-4-7-opus-high, claude-4-7-opus-xhigh, claude-4-7-opus-max
Claude Opus 4.6	claude-4-6-opus-high, claude-4-6-opus-max
Claude Sonnet 4.6	claude-4-6-sonnet-high, claude-4-6-sonnet-max
Claude Opus 4.5	claude-4-5-opus, claude-4-5-opus-thinking
Claude Sonnet 4.5	claude-4-5-sonnet, claude-4-5-sonnet-thinking
Claude Haiku 5.5	claude-5-5-haiku-low, claude-5-5-haiku-medium, claude-5-5-haiku-high, claude-5-5-haiku-xhigh, claude-5-5-haiku-max
Claude Haiku 4.5	claude-4-5-haiku

The value must be available to your plan and workspace. See model choice for agents for availability and data retention.

Agents and automations can override computerUseModel. An omitted or null agent value inherits agentDefaults.computerUseModel; an automation inherits the selected agent’s effective value. The setting applies only to Computer Use on the Warp Agent harness (type: oz). Warp preserves but ignores it otherwise. It can appear with model or harness because it doesn’t select the main model.

agents/<name>/agent.md

Required resource. Every definition includes at least one agent file. Each agent has one file. The directory names the agent, the YAML frontmatter configures it, and the Markdown body contains its instructions.

agents/reviewer/agent.md

---
description: Reviews factory-produced pull requests
agentType: REVIEW
---

Review each pull request against the repository's standards. Request
changes when tests are missing; never approve your own edits.

All frontmatter keys are optional.

Field	Description
description	What the agent does.
agentType	The agent’s role.
credentialStrategy	Overrides the factory-level credentialStrategy. Omit it to keep the agent’s current setting.
spawnableBy	Agents allowed to start this agent. Omit it to allow only the foreman, use an empty list to allow none, or list exact agent names.
model, harness	Override the corresponding agentDefaults. Set at most one.
runner, environmentId, secrets, mcpServers, workerHost, computerUseModel	Override the corresponding agentDefaults, subject to each field’s inheritance rules.
idleTimeoutMinutes	Keeps a completed session available for follow-up for 1 through 60 minutes. Omit it or use null to inherit. Without an inherited or run-level value, Warp uses 10 minutes, or 60 minutes for an orchestrated child run.
required_repos	Repositories prepared for every run by this agent. Each entry requires owner and name; use codeForge to distinguish the same repository across hosts.

An agent’s harness object is a sparse override of agentDefaults.harness. Set auth to null to clear inherited authentication. Agent-level mcpServers entries use the same transport keys as the factory-level mcpServers map.

agentType

Optional. The agent’s role: CUSTOM (default), FOREMAN (alias MAIN), TRIAGE, SPEC, IMPLEMENT, REVIEW, or VERIFY. Every factory has one foreman, which serves as its entry point and the default automation target. See About factory agents for role behavior.

automations/<name>/automation.md

Optional resource. Each automation has one file. The directory names the automation, the frontmatter defines its triggers and execution settings, and the Markdown body contains the starting prompt.

automations/labeled-issue/automation.md

---
agent: foreman
triggers:
  - provider: github
    event: issue_labeled
    filter:
      repos: [acme/payments-service]
      labels: [factory-ready]
---

Review the labeled issue and decide the next required stage. Return
unresolved product questions to a human.

Field	Required	Default	Description
enabled	No	true	Turns the automation on or off.
agent	No	Foreman	The agent that handles the automation’s runs.

triggers

Required. One or more events that start runs. Each trigger declares provider and event, and can include filter or, for scheduled runs, schedule.

The providers and their events:

Provider	Events
azure_devops (pull requests and pushes)	pull_request_closed, pull_request_commented, pull_request_created, pull_request_mentioned, pull_request_merged, pull_request_updated, push
azure_devops (work items)	work_item_assigned, work_item_created, work_item_labeled, work_item_mentioned
factory	work_item_stage_changed
github (issues)	issue_assigned, issue_commented, issue_created, issue_labeled, issue_mentioned
github (pull requests)	pull_request_assigned, pull_request_closed, pull_request_commented, pull_request_dequeued, pull_request_enqueued, pull_request_labeled, pull_request_mentioned, pull_request_merged, pull_request_opened, pull_request_ready, pull_request_reopened, pull_request_synchronized
github (reviews)	pull_request_review_requested, pull_request_review_submitted
github (code and CI)	check_run_rerequested, check_suite_completed, check_suite_rerequested, push, workflow_run_completed
gitlab	bot_mentioned, merge_request, push
jira	agent_session_created, issue_created, issue_labeled, status_changed
linear	agent_session_created, comment_created, issue_assigned, issue_created, issue_labeled, issue_state_changed
schedule	cron_fired
slack	app_mention, member_joined_channel, message_dm, message_im, message_mpim, message_posted, reaction_added
teams	app_mention, message_posted
webhook	received

Slack, Microsoft Teams, Linear, and Jira triggers require the matching integration to be connected. Code-forge triggers use the factory’s repositories and provider connection; see the GitHub, GitLab, and Azure DevOps integration guides. webhook triggers listen to custom webhooks declared under webhooks/.

triggers[].filter

Conditionally required. Required for webhook / received; optional for other triggers. Narrows which events start runs. Available keys depend on the provider and event. Keys combine with AND; within an inclusion list, any value can match. Omitting an optional list filter adds no restriction, but provider defaults still apply. For example, GitHub’s skip_bots defaults to all.

A webhook trigger for received requires webhook_ids: a one-item list with one webhook UID. It supports in only and doesn’t accept a file name. To listen to multiple webhooks, add one trigger per UID. The optional payload pattern matches against the delivery’s JSON body. See webhook payload filters for the pattern syntax.

Filter keys by provider:

• Factory: stages.
• Azure DevOps: assignees, authors, base_branches, branches, labels, mentioned, repos, source_repos, and work_item_types.
• GitHub: assignees, authors, baseBranches, base_branches, branches, conclusions, keywords, labels, mentioned, paths, prNumbers, pr_numbers, repos, review_states, reviewer_teams, reviewers, skip_bots, and workflows.
• GitLab: actions, base_branches, branches, mentioned, and repos.
• Jira: keywords, labels, project_keys, and status_ids.
• Linear: assignee_ids, creator_ids, issue_ids, issues, keywords, labels, mentioned_user_ids, project_ids, projects, state_ids, states, team_ids, and teams.
• Schedule: schedule_ids is server-managed and cannot be set in a factory definition.
• Slack: channel_ids, channels, emojis, itemUsers, item_user_ids, keywords, user_ids, and users.
• Microsoft Teams: channel_ids, keywords, team_ids, and user_ids.
• Webhook: payload and webhook_ids.

Most canonical keys accept matcher objects: in includes values and not_in excludes them. Webhook payload also accepts exists. Microsoft Teams team_ids and channel_ids support only in.

For Microsoft Teams, team_ids contains exactly one Microsoft Graph team UUID, and channel_ids contains at least one channel ID.

baseBranches and prNumbers alias base_branches and pr_numbers. The name-based aliases teams, projects, states, issues, channels, users, and itemUsers take plain lists; Warp resolves their values to their _ids counterparts. An alias and its canonical key are mutually exclusive.

triggers:
  - provider: webhook
    event: received
    filter:
      webhook_ids: [WEBHOOK_UID]
      payload:
        status: [failed]

triggers[].schedule

Conditionally required. Required for schedule / cron_fired and invalid for other triggers. Defines an inline UTC schedule for a schedule / cron_fired trigger. The object requires cron, which accepts a five-field expression or a descriptor such as @daily or @every 1h. Use optional name to distinguish multiple schedules; at most one can omit it. Changing only cron updates a schedule, while changing name replaces it.

triggers:
  - provider: schedule
    event: cron_fired
    schedule:
      name: weekday-mornings
      cron: "0 9 * * 1-5"

Execution overrides

Optional. An automation can declare displayName, plus model or harness, runner, environmentId, secrets, mcpServers, workerHost, computerUseModel, idleTimeoutMinutes, and required_repos. Use null to clear displayName. Execution settings override the target agent for runs from this automation.

Automation required_repos entries extend the target agent’s list. Each item requires required_repos[].owner and required_repos[].name; use required_repos[].codeForge to distinguish the same repository across hosts.

The harness object uses the same sparse override as an agent. Automation-level mcpServers use the factory-level transport syntax. Omit idleTimeoutMinutes, or set it to null, to inherit the target agent’s value.

runners/<name>.yaml

Optional resource. Runner files define compute. Agents and automations select a file by name with runner. See cloud agent runners for runtime behavior and 02-sdlc-issue-to-pr for Linux and macOS examples.

runners/linux-build.yaml

description: Linux runner for payments builds and tests
setupCommands:
  - corepack enable
instanceShape:
  vcpus: 4
  memoryGb: 8
platform:
  os: linux
  arch: x86_64
  linux:
    dockerImage: ubuntu:22.04

Field	Required	When omitted	Description
description	No	Not specified	A description of the runner.
setupCommands	No	No commands declared here	Shell commands that run in order while the sandbox is prepared.
failureSessionRetentionMinutes	No	Uses the environment setting	Keeps a failed session open for inspection for 1 through 60 minutes.

instanceShape

Optional. The compute size. Set both vcpus and memoryGb, or omit instanceShape for the workspace default.

Operating system	Supported sizes
Linux and Windows	Powers of two for both vCPUs and memory in GB.
macOS	4 vCPUs and 7 GB; 6 vCPUs and 14 GB; 8 vCPUs and 14 GB; 12 vCPUs and 28 GB; or 12 vCPUs and 56 GB.

platform

Required. The operating system and architecture:

platform.os	platform.arch	OS-specific settings
linux (default)	x86_64 (default) or aarch64	Requires platform.linux.dockerImage.
macos	aarch64	platform.mac.version accepts "14", "15", "26", or "27". Quote the version; omitting platform.mac uses "26".
windows	x86_64	No OS-specific image setting.

For a private Linux image, set linux.registryCredentialSecretName to the name of a managed Docker registry (docker_registry) or AWS ECR (aws_ecr_credential) credential. The credential’s registry host must match the host in linux.dockerImage.

routers/<name>.yaml

Optional resource. Router files define factory-owned custom model routers. Use the file name anywhere the definition accepts a model.

Field	Requirement	Description
name	Optional	Display label.
type	Required	complexity or prompt.
default	Required	Concrete model used when no route matches.
routing	Optional	Type-specific routing rules.

routers/by-task.yaml

name: By task
type: prompt
default: claude-4-6-sonnet-high
routing:
  - description: Routine documentation changes
    model: claude-4-5-haiku
  - description: Complex implementation or debugging
    model: claude-4-8-opus-high

The optional routing field takes a different shape for each type:

• Complexity - For type: complexity, routing.easy, routing.medium, and routing.hard optionally map complexity levels to concrete models.
• Prompt - For type: prompt, routing is an ordered list of objects, each requiring description and model.

Targets must be concrete supported models, not Auto models or other routers. See custom model routers for routing behavior.

Factory members who can access the router can also select it from model catalogs in the factory dashboard, including agent settings and benchmark configurations.

benchmarks/<suite-slug>/suite.yaml

Optional resource. Benchmark suite files define trials for one agent, with reusable configurations and an ordered task list. The directory slug is the suite’s stable identity; changing name doesn’t move the file. See benchmarks for how to run a suite.

benchmarks/pull-request-review/suite.yaml

name: Pull request review
description: Compare configurations for the review agent.
agent: reviewer
configurations:
  - name: Baseline
    agents:
      - agent: reviewer
        model: auto
  - name: Candidate
    agents:
      - agent: reviewer
        model: auto-efficient
tasks:
  - broken-doc-link

Field	Required	Default	Description
name	Yes	None	Suite display name; must be unique in the factory.
description	No	Not specified	What the suite measures.
agent	Yes	None	The factory agent used for every task in the suite.

configurations

Optional. One to six reusable configurations. Warp uses these presets when a benchmark run doesn’t supply configurations. Each configuration requires a unique name.

The optional agents list pins factory agents to models and harnesses for the configuration’s trials:

Field in configurations[].agents[]	Required	Description
agent	Yes	The factory agent’s name. List each agent once per configuration.
model or harness	Exactly one	Model selection. A harness requires type and model; type accepts oz, claude, claude-code, or codex. Optional auth follows the harness authentication rules.

Omitted agents use their settings at launch time.

Deprecated metadata

Optional configuration role is accepted for compatibility. Warp ignores it and omits it when rewriting the suite.

tasks

Optional. An ordered list of unique task slugs. Each slug matches one file under benchmarks/<suite-slug>/tasks/, and every task file appears once. A suite without tasks can be saved but not run.

benchmarks/<suite-slug>/tasks/<task-slug>.yaml

Optional resource. Each task file belongs to its parent suite. The file name without .yaml is the stable task slug.

benchmarks/pull-request-review/tasks/broken-doc-link.yaml

title: Fix a broken documentation link
prompt: Find the broken internal documentation link and update it.
successCriteria: The destination resolves and the link text names the destination.
startingRepoRefs:
  - github.com:acme/payments-docs@0123456789abcdef0123456789abcdef01234567

Field	Required	Default	Description
title	Yes	None	Task display name; doesn’t need to match the task slug.
prompt	Yes	None	Instructions sent to the agent.
successCriteria	Yes	None	Correctness Scorer’s evaluation criteria.
sourceRunId	No	Not specified	ID of the prior run that produced the task; can refer to a deleted run.
tags	No	Not specified	Ordered list of task labels.
linearSeed	No	Not specified	JSON-compatible starting state for the trial’s isolated Linear workspace.

startingRepoRefs

Optional. The GitHub or GitLab repositories and commits used as the task’s starting state. Each entry accepts github.com:OWNER/REPO@COMMIT_SHA, gitlab.com:OWNER/REPO@COMMIT_SHA, or an object with codeForge (GITHUB or GITLAB), owner, repo, and ref. All four object fields are required. COMMIT_SHA and ref must be full 40-character commit SHAs, not branches or tags. Omit the key to use the agent’s checkout defaults.

scorers/<name>/scorer.md

Optional resource. Scorer files classify sampled runs against a rubric. The directory is a stable slug; name identifies the scorer. Frontmatter defines the labels and scoring settings, and the Markdown body holds the rubric. See Configuring Scorers for how scores are used.

scorers/tests-run/scorer.md

---
name: tests-run
description: Checks whether implementation runs include test evidence.
agents:
  - reviewer
labels:
  - value: tests_run
    description: The transcript contains a test command and its result.
    score: 1
  - value: tests_skipped
    score: 0
passingScore: 1
samplingRate: 25
model: claude-4-5-haiku
---
Evaluate whether the agent ran the relevant tests before finishing. Return
exactly one declared label.

Field	Required	Default	Description
name	Yes	None	Scorer name. Changing it doesn’t move the directory.
description	No	Not specified	What the scorer checks.
output	No	Not specified	Output form; classification is the supported value.
passingScore	Yes	None	Passing threshold, from 0 through 1.
samplingRate	No	25	Percentage of eligible runs to score, from 0 through 100. Rounds to two decimal places; 0 disables automatic scoring.
model	Yes	None	Model that judges runs.
selfImprovement	No	false	When true, failing scores can feed self-improvement, which proposes definition changes in pull requests.

agents

Required. The agents whose runs the scorer evaluates. A string names one agent and scores only that run. To include child runs as evidence, use an object with required name and optional includeDescendants: true.

agents:
  - name: foreman
    includeDescendants: true

Scorer execution settings

Optional. runner, secrets, and mcpServers override the factory’s scorerDefaults for this scorer. Scorer-level mcpServers use the factory-level transport syntax. Set shared values under scorerDefaults in factory.yaml.

labels

Required. One to 20 classifications. Each label requires a unique value and a score from 0 through 1; description is optional. At least one label must meet passingScore, and at least one must fall below it.

webhooks/<name>.yaml

Optional resource. Webhook files define authenticated URLs for JSON POST requests. Automation webhook triggers subscribe to them; the file name identifies the webhook. secretName references an existing managed secret, not secret material.

webhooks/internal-ci.yaml

authMode: token
secretName: INTERNAL_CI_WEBHOOK_SECRET
deliveryIdHeader: X-CI-Run-Id
enabled: true

Field	Required	Default	Description
enabled	No	true	Whether the webhook accepts deliveries. Set to false to stop deliveries without deleting it, or to create it before the provider issues a signing secret. See Setting up a Vercel webhook.

You can’t change authMode or signatureScheme on an existing webhook. Delete and recreate it, or create the replacement under a different name.

authMode

Optional. How deliveries authenticate: token (default, using an Authorization: Bearer header), url_token (secret in the URL path), or signature (the provider’s signature scheme). See webhook authentication modes.

signatureScheme

Conditionally required. Required for authMode: signature and invalid otherwise. The provider signature to verify: github, pagerduty, sentry, standard_webhooks, stripe, or vercel.

secretName

Required. The managed secret containing the bearer token, URL token, or provider signing secret. A url_token secret becomes part of the ingress URL and accepts only letters, digits, -, ., _, and ~.

deliveryIdHeader

Optional. The sender’s delivery ID header, used to deduplicate retries. It accepts up to 64 characters and must be a valid HTTP header name. Credential-bearing headers such as Authorization, Cookie, and provider signature headers are rejected.

On a Warp-managed factory, creating a webhook in the dashboard writes this file for you and stores the secret under the name you enter in the “Secret name” field.

Skills

Optional resource. A skill is a directory containing SKILL.md, not a YAML key. Put shared skills under skills/ and agent-specific skills under agents/<name>/skills/. See Factory skills for placement and Skills for agents for the file format.

Example factory definition

This definition includes one repository, a foreman, an automation triggered by a GitHub label, and a Linux runner. Replace PAYMENTS_ENVIRONMENT_ID and SENTRY_MCP_SERVER_ID with existing environment and MCP server IDs. Create a managed secret named SENTRY_AUTH_TOKEN, or replace that name with an existing secret.

For more complete definitions, see warp-factory-examples.

factory.yaml

schemaVersion: v1alpha1
name: payments-factory
description: Processes approved work for the payments service
alias: payments
repositories:
  - owner: acme
    name: payments-service
agentDefaults:
  model: auto
  runner: linux-build
  environmentId: PAYMENTS_ENVIRONMENT_ID

agents/foreman/agent.md

---
description: Routes approved payments work through the factory
agentType: FOREMAN
secrets:
  - SENTRY_AUTH_TOKEN
mcpServers:
  sentry:
    warpId: SENTRY_MCP_SERVER_ID
---

Own each work item from intake through human handoff.

Confirm the request is ready before dispatching implementation. Require
repository validation and independent review before marking work complete.

automations/labeled-issue/automation.md

---
enabled: true
agent: foreman
triggers:
  - provider: github
    event: issue_labeled
    filter:
      repos: [acme/payments-service]
      labels: [factory-ready]
---

Review the labeled issue and decide the next required stage. Preserve the
issue's acceptance criteria and return unresolved product questions to a human.

runners/linux-build.yaml

description: Linux runner for payments builds and tests
setupCommands:
  - corepack enable
instanceShape:
  vcpus: 4
  memoryGb: 8
platform:
  os: linux
  arch: x86_64
  linux:
    dockerImage: ubuntu:22.04

Routing to a self-hosted worker

Set agentDefaults.workerHost to a connected worker’s ID:

factory.yaml

agentDefaults:
  model: auto
  runner: linux-build
  workerHost: SELF_HOSTED_WORKER_ID

Replace SELF_HOSTED_WORKER_ID with the worker’s ID and pair it with a runner whose platform matches the worker. See the Self-hosting quickstart or copy 07-self-hosted-worker.

Related pages

• Factory MCP for coding agents - Read schemas and validate definitions through an agent
• Factory-definition pull request checks - Review validation results and protect merges
• Factory dashboard - Edit a Warp-managed definition
• warp-factory-examples - Copy complete definitions and validation workflows