Agentic Guardrails Improvements

Date: 2026-03-06 Branch: main Commits: 82709df3 and prior Goal: Reduce task failures caused by overly conservative guardrails and loop detection. Competing coding-agent products give the LLM more room to iterate, retry, and self-correct; these changes bring our agent closer to that baseline.

These runtime guardrail changes do not grant access. The effective access profile remains the ceiling for command tools, sandboxing, filesystem, network, domain, and approval behavior; relaxed retry or loop thresholds cannot widen a denied capability or bypass a hard guardrail.

---

Root Causes Identified

---

Phase 1 — Relax Circuit Breakers

1a. Tool Failure Tracker (src/electron/agent/executor-helpers.ts)

Problem: A tool was permanently disabled after just 2 consecutive failures. A shell command failing twice (e.g., due to a typo being fixed) disabled run_command for the rest of the task.

Tool-specific getMaxInputDependentFailures thresholds:

New method added: getDisabledToolNames(): string[] — returns names of currently disabled tools, used by graceful degradation (Phase 6).

New run_command guidance in getAlternativeApproachGuidance:

• "command not found" → suggest installing the package or using full path
• "permission denied" → suggest checking permissions or using a different approach
• "non-zero exit" → clarify the command itself failed (normal during development), not the tool

---

1b. Tool Call Deduplicator (src/electron/agent/executor.ts)

Problem: The same exact tool call (e.g., re-reading a file after editing it) was blocked after just 2 occurrences within 60 seconds.

---

1c. Completion Validation (src/electron/agent/completion-checks.ts)

Problem: The validator required minimum response lengths (40–120 chars) even when tool execution had already confirmed task completion. Short, correct responses like "Done." were rejected.

Changes to evaluateDomainCompletion:

• When hadAnyToolSuccess is true: accept any non-empty response without length checks. Tool evidence is sufficient proof of completion.
• Only reject truly empty responses (no text at all) in tool-backed scenarios.

Reduced minimum lengths for non-tool-backed responses:

---

Phase 2 — Better Progress Scoring (src/electron/agent/progress-score-engine.ts)

Problem: Only file writes counted as forward progress. An agent exploring a codebase, reading files, and running searches received a progress score of zero — triggering the no-progress circuit breaker prematurely.

New signals added to ProgressScoreAssessment interface:

readOperations: number;
searchOperations: number;
toolSuccesses: number;

Updated score formula:

// Before
rawScore = stepCompleted * 1.0
         + writeMutations * 0.6
         + resolvedErrorRecoveries * 0.4
         - repeatedErrorPenalty          // (count - 1) * 0.8
         - emptyNoOpTurns * 1.0

// After
rawScore = stepCompleted * 1.0
         + writeMutations * 0.6
         + readOperations * 0.2          // NEW: read_file, list_directory, search_files, glob, find_in_file
         + searchOperations * 0.3        // NEW: web_search, web_fetch, search*
         + min(toolSuccesses, 5) * 0.1   // NEW: general tool successes, capped at 5
         + resolvedErrorRecoveries * 0.4
         - repeatedErrorPenalty          // (count - 1) * 0.4 — reduced from 0.8
         - emptyNoOpTurns * 0.3          // reduced from 1.0

Penalty reductions:

• repeatedErrorPenalty multiplier: 0.8 → 0.4 (fixing the same error is iterating, not looping)
• emptyNoOpTurns penalty: 1.0 → 0.3 (only penalize truly empty turns, not thinking/planning)
• Empty turn threshold tightened: only turns with trimmed.length < 5 count as no-ops (was any whitespace-only message)

---

Phase 3 — Increase Execution Limits

3a. Per-Step Iteration Limits (src/electron/agent/executor.ts)

---

3b. Global Guardrail Limits (src/electron/guardrails/guardrail-manager.ts)

---

3c. Step Timeout (src/electron/agent/executor-helpers.ts)

---

Phase 4 — Improve Loop Detection

4a. Loop Guardrail Configs (src/electron/agent/completion-checks.ts)

DEFAULT_LOOP_GUARDRAIL:

CODE_LOOP_GUARDRAIL:

NON_CODE_LOOP_GUARDRAIL:

---

4b. detectToolLoop Threshold (src/electron/agent/executor.ts + executor-loop-utils.ts)

Productive-cycle exemption added to detectToolLoop:

Before returning true for a detected loop, checks if the wider call window contains a mix of read-like and write-like tool categories:

const widerWindow = recentCalls.slice(-(threshold + 3));
const allCategories = new Set(widerWindow.map((c) => c.tool));
const hasReadLike = [...allCategories].some((cat) => /^(read|search|list|glob|find|get)/.test(cat));
const hasWriteLike = [...allCategories].some((cat) =>
  /^(write|edit|create|run|execute|shell|command|browser)/.test(cat),
);
if (hasReadLike && hasWriteLike) return false; // it's a fix-cycle, not a loop

This prevents read_file → edit_file → run_command → read_file → … from being classified as a degenerate loop.

---

Phase 5 — Better Context Management

5a. Compaction Constants (src/electron/agent/executor-helpers.ts)

---

5b. Smart Message Retention (src/electron/agent/context-manager.ts)

Problem: During context compaction, older messages containing error-recovery context and prior decisions about actively-worked files were removed, causing the agent to lose crucial context mid-task.

New helpers added:

• extractFilePathsFromMessages(messages) — extracts file paths from message content using regex /(?:\/[\w.@-]+){2,}(?:\.\w+)?/g
• messageReferencesActivePaths(message, activePaths) — returns true if a message mentions any of the active file paths

Enhancement to removeOlderMessagesWithMeta:

1. Extracts file paths from the last 4 messages as the "active work context"
2. When pruning older messages, reserves up to 15% of the token budget for messages that reference actively-worked files
3. Active-file messages are kept preferentially; only removed if budget is fully exhausted

---

Phase 6 — Graceful Degradation on Tool Failure

6a–6b. Tool Alternatives Injection (src/electron/agent/executor.ts)

Problem: When computeToolFailureDecision flagged a soft failure, the step loop immediately stopped — giving the agent no chance to switch approaches.

TOOL_ALTERNATIVES mapping added:

const TOOL_ALTERNATIVES: Record<string, string[]> = {
  browser_navigate: ["web_fetch", "web_search"],
  run_command: ["run_applescript", "write_file"],
  edit_file: ["write_file"],
  search_files: ["glob", "list_directory"],
  web_search: ["web_fetch", "browser_navigate"],
  web_fetch: ["web_search"],
};

Graceful degradation logic (toolAlternativesInjected flag):

• On first soft-failure stop signal (not hard failure), injects a system message listing available alternative tools for the disabled tool(s)
• toolAlternativesInjected is set to true to prevent repeated injection
• Loop continues for one more iteration, giving the LLM a chance to use alternatives
• Only stops unconditionally on hard failures or if alternatives were already injected and tools still fail

6c. run_command Failure Guidance (src/electron/agent/executor-helpers.ts)

Enhanced getAlternativeApproachGuidance with run_command-specific messages:

---

Test Updates

Four tests were updated to match the new thresholds:

---

Phase 7 — False-Positive Source Validation Guard (Post-deployment fix, 2026-03-07)

Root Cause

A build task ("Create an app about breaking news references") failed at finalization with:

Task missing source validation: release/funding claims require web_fetch sources with explicit publish dates.

The failure chain:

1. Task prompt contained "breaking news" → taskLikelyNeedsWebEvidence() returned true
2. The final agent response included "announcement/launch/release" wording — from HTML seed data the agent had just written into the app — not from real-world research claims
3. The agent had not called web_fetch (correctly — it was building an app, not fetching news)
4. requiresStrictResearchClaimValidation() returned true → finalization threw

Two complementary fixes applied in src/electron/agent/executor.ts:

Fix 7a: Build-task escape hatch in requiresStrictResearchClaimValidation

// Before
private requiresStrictResearchClaimValidation(candidate: string): boolean {
  if (!this.taskLikelyNeedsWebEvidence()) return false;
  return this.responseHasHighRiskResearchClaim(candidate);
}

// After
private requiresStrictResearchClaimValidation(candidate: string): boolean {
  if (!this.taskLikelyNeedsWebEvidence()) return false;
  // Build tasks that created files are not research tasks — the output describes
  // built artifacts, not factual claims about current events.
  const createdFiles = this.fileOperationTracker?.getCreatedFiles?.() || [];
  if (createdFiles.length > 0) return false;
  return this.responseHasHighRiskResearchClaim(candidate);
}

If the agent created any files during the task, it's a build task. Skip research claim validation.

Fix 7b: Build-intent signals in taskLikelyNeedsWebEvidence

The method triggered on broad keywords like "breaking", "news", "search" even when the task was about building a tool related to those topics, not fetching them. Added build-intent counterweight:

private taskLikelyNeedsWebEvidence(): boolean {
  const prompt = `${this.task.title}\n${this.task.prompt}`.toLowerCase();
  const researchSignals = ["news", "latest", "today", "trending", "breaking", ...];
  if (!researchSignals.some((signal) => prompt.includes(signal))) return false;
  // Build/creation tasks mentioning news-related keywords don't need web evidence.
  const buildSignals = [
    "create an app", "build an app", "make an app", "write an app",
    "create a tool", "build a tool", "create a website", "build a website",
    "implement", "develop an app", "develop a tool",
  ];
  if (buildSignals.some((signal) => prompt.includes(signal))) return false;
  return true;
}

This also prevents misleading step-context prompts (lines 16901–16909) from instructing the LLM to fetch news when it should be building something.

---

Files Modified

---

Verification

• TypeScript: npx tsc --noEmit — no errors (unrelated GuardrailSettings.tsx errors pre-existed)
• Tests: npx vitest run src/electron/agent/__tests__/ — 624/624 passed