Guard rails overview¶

Guard rails are safety boundaries that stop AI agents doing things they should not. Osmia has six core layers, shown below. If one fails, the others can still catch the problem. Two more layers adjust behaviour while agents run: real-time coaching (layer 7) and adaptive watchdog calibration (layer 8).

The six core layers¶

graph LR
    A["📥 Incoming Ticket"] --> B["1. Controller<br/>Validation"]
    B --> C["2. Engine<br/>Hooks"]
    C --> D["3. Repository<br/>Rules"]
    D --> E["4. Task<br/>Profiles"]
    E --> F["5. Quality<br/>Gate"]
    F --> G["6. Progress<br/>Watchdog"]
    G --> H["✅ Safe Output"]

    style B fill:#7c4dff,color:#fff
    style C fill:#7c4dff,color:#fff
    style D fill:#7c4dff,color:#fff
    style E fill:#7c4dff,color:#fff
    style F fill:#7c4dff,color:#fff
    style G fill:#7c4dff,color:#fff

Layer 1: Controller validation¶

When: before any agent starts working. What: the controller checks the ticket against configurable rules before creating a job.

Check	What it prevents
Allowed repositories	Agent working on repos it shouldn't touch
Allowed task types	Agent performing task categories that aren't approved
Concurrent job limit	Too many agents running at once (resource exhaustion)
Blocked file patterns	Agent modifying sensitive files (injected into engine config)

A ticket that fails validation is rejected immediately and marked as failed, with the reason.

Layer 2: Engine hooks¶

When: during agent execution, before every tool call. What: scripts that intercept and block dangerous operations in real time.

Only Claude Code supports hook-based guard rails

Other engines (Codex, Aider, OpenCode, Cline) use prompt-based rules, which are advisory and not enforced.

Blocked operations include:

Destructive commands: rm -rf /, sudo, eval, chmod 777
Remote code execution: curl | bash, wget | sh
Destructive SCM operations: git push --force to main/master
Writes to sensitive files: .env, *.pem, *.key

If a hook blocks an operation, the agent sees the rejection and adjusts its approach.

Layer 3: Repository rules¶

When: read by the agent from the target repository during execution. What: a guardrails.md or CLAUDE.md file in the target repository that tells the agent what it must and must not do.

Advisory, not enforced by the controller

The controller does not currently inject these files into the agent prompt. Claude Code reads CLAUDE.md automatically when it starts work; other engines do not. Prompt-builder injection for all engines is on the roadmap.

Example guardrails.md (for Claude Code via CLAUDE.md):

## Never Do
- Never modify CI/CD pipeline configuration files
- Never change database migration files
- Never alter authentication or authorisation logic

## Always Do
- Always run the full test suite before creating a PR
- Always add tests for new functionality

These rules are repo-specific and maintained by the repository owners.

Layer 4: Task profiles¶

When: applied when the controller selects the engine configuration. What: different task types get different cost and duration budgets.

guardrails:
  task_profiles:
    documentation:
      allowed_file_patterns: ["*.md", "docs/**"]
      max_cost_per_job: 10.0
    bug_fix:
      blocked_file_patterns: ["**/migrations/**", "**/auth/**"]
      max_cost_per_job: 50.0

Config schema only: file pattern enforcement not yet wired

max_cost_per_job and max_job_duration_minutes overrides per task type are read from config. However, allowed_file_patterns and blocked_file_patterns at the profile level are not yet enforced at runtime. Only the global blocked_file_patterns is injected into engine hooks. Per-profile enforcement is on the roadmap.

Layer 5: Quality gate¶

Not built yet

The quality_gate settings are read from the config file, but the controller does not run a review yet. Setting enabled: true has no effect.

When: after the agent finishes, before the PR is created. What: a separate review step that checks the agent's output.

The quality gate can:

Scan for accidentally committed secrets
Check for OWASP security patterns
Verify the agent followed guard rail instructions
Check dependency changes for known CVEs

On failure, it can retry with feedback, block the PR, or notify a human.

Layer 6: Progress watchdog¶

When: continuously, while the agent is running. What: monitors agent behaviour and terminates unproductive agents.

graph TD
    A["Watchdog Tick<br/>(every 60s)"] --> B{"Loop<br/>detected?"}
    B -->|Yes, 2+ ticks| C["Terminate with feedback"]
    B -->|No| D{"Thrashing?<br/>High tokens, no progress"}
    D -->|Yes, 2+ ticks| E["Warn → Terminate"]
    D -->|No| F{"Stalled?<br/>No activity"}
    F -->|Yes, 2+ ticks| G["Terminate"]
    F -->|No| H{"Cost velocity<br/>too high?"}
    H -->|Yes| I["Warn"]
    H -->|No| J["✅ Healthy"]

Detection rule	What it catches	Default threshold
Loop detection	Agent calling the same tool with the same arguments repeatedly	10 consecutive identical calls
Thrashing	High token consumption without file changes	80,000 tokens without progress
Stall	No tool calls despite heartbeat advancing	300 seconds idle
Cost velocity	Spending rate too high	$15 USD per 10 minutes
Telemetry failure	Heartbeat stopped advancing	3 stale ticks
Unanswered human	`NeedsHuman` state with no response	30 minutes

Avoiding false positives

All detection rules require the anomaly to persist for min_consecutive_ticks (default: 2) before action is taken. New TaskRuns also get a research_grace_period (default: 5 minutes) during which thrashing detection is relaxed, since agents often consume many tokens during initial code analysis.

Layer 7: Real-time agent coaching (PRM)¶

When: continuously, while the agent is running (alongside the watchdog). What: evaluates agent productivity at each tool call and intervenes with guidance before problems escalate.

Only applies to Claude Code

The PRM works on the NDJSON event stream. Claude Code is currently the only engine that produces one, so other engines are not scored by the PRM.

The Process Reward Model (PRM) reads the NDJSON event stream from Claude Code. It scores each evaluation window of tool calls on a 1-10 scale, tracks the score trajectory over time, and decides on interventions:

With the default thresholds:

Score 7 or above: the agent is productive, so continue.
Score 3 or below with a sustained decline: escalate to the watchdog for termination.
Any other score below 7 with a declining trend, oscillation or plateau: write a hint file (/workspace/.osmia-hint.md) with targeted guidance, such as "Your approach appears to be oscillating. Try committing to a single strategy."

Unlike the watchdog, which detects anomalies in raw telemetry, the PRM evaluates productivity patterns: whether the agent is making progress toward the goal, not only whether it is alive.

Layer 8: Adaptive watchdog calibration¶

When: with progress_watchdog.adaptive_calibration.enabled: true, once enough historical data has accumulated. What: replaces static watchdog thresholds with adaptive per-(repo, engine, task_type) thresholds.

A bug fix in a small Python script has very different "normal" telemetry from a TypeScript monorepo refactor. Static thresholds either miss real anomalies in simple tasks or give false positives on complex ones. Adaptive calibration learns what "normal" looks like for each context from historical TaskRun data.

Cold-start safety: a minimum of 10 completed TaskRuns for a given context is required before calibrated thresholds override static defaults.

How the layers work together¶

For example, a ticket asks the agent to "update the deployment manifest":

Controller validation checks that the repo is in allowed_repos. ✅ Pass.
Engine hooks would block writes to .github/workflows/** if configured. ✅ Blocked if needed.
The repository guardrails.md says "Never modify CI/CD pipeline configuration", so the agent should avoid it (advisory).
The task profile for the infrastructure type defines a lower cost budget. ✅ Enforced. (File pattern enforcement is on the roadmap.)
The quality gate would flag changes to deployment files. Not built yet.
The watchdog stops the agent looping endlessly while it looks for the right file. ✅ Monitored.

If the agent ignores layer 3 (prompt-based), layer 2 still catches the problem, provided the hooks are configured.