Automation Bias Alert

Install

npx ax-depute@latest add automation-bias-alert

pnpm dlx ax-depute@latest add automation-bias-alert

yarn dlx ax-depute@latest add automation-bias-alert

bunx ax-depute@latest add automation-bias-alert

Overview

[!NOTE] AutomationBiasAlert is not a standalone UI module. It is a policy wrapper pattern designed to sit exclusively over existing approval boundaries.

Automation bias occurs when humans over-trust automated systems, leading to passive oversight or "rubber-stamping" of errors. This wrapper counteracts this by monitoring approval patterns and introducing a mandatory review pause when bypass behavior is detected.

Basic usage

The component works as a wrapper around interactive oversight elements (like ApprovalGate). It uses the useAutomationBias hook to monitor events and trigger the alert.

import { AutomationBiasAlert, useAutomationBias } from '@/depute';

function MyOversightUI() {
  const { isAlertTriggered, recordAction, dismissAlert } = useAutomationBias();

  const handleAction = (action: 'approved' | 'rejected') => {
    // Record the action to update internal bias heuristics
    recordAction(action);
    executeAction(action);
  };

  return (
    <AutomationBiasAlert 
      isActive={isAlertTriggered} 
      onAcknowledge={dismissAlert}
      actionName="this deployment"
    >
      <button onClick={() => handleAction('approved')}>Approve</button>
    </AutomationBiasAlert>
  );
}

With custom thresholds

You can tune the sensitivity of the bias detection by passing custom thresholds to the hook.

const { isAlertTriggered, recordAction, dismissAlert } = useAutomationBias({
  consecutiveApprovals: 3,   // Alert after 3 consecutive approvals
  minApprovalTimeMs: 2500,  // Require 2.5s review time
  approvalRateCeiling: 0.8  // Alert if approval rate > 80%
});

Detection Heuristics

The useAutomationBias hook monitors three specific behavioral signals:

1. Consecutive Approvals: Detecting long streaks of approvals without a single rejection or modification.
2. Approval Velocity: Measuring the time between the UI rendering and the user clicking "Approve". If it's faster than a human could realistically review the content, it triggers friction.
3. Session Ceilings: Monitoring the overall approval-to-rejection ratio. If a user approves >90% of actions over a long period, it indicates potential over-trust.

Props

AutomationBiasAlert

useAutomationBias

When to use

• Users are approving high-stakes agent actions in rapid succession — financial transactions, code deployments, medical recommendations
• You have evidence or suspicion of rubber-stamping behavior in your product (high approval rates, very short review times)
• The consequence of a false positive (approving a bad action) significantly outweighs the friction cost of a pause
• You are operating a swarm inbox or batch approval queue where velocity bias is structurally likely

When not to use

• The approval flow is for low-stakes reversible actions — introducing friction here erodes trust in the friction itself, making it ineffective when it matters
• Users have no ability to meaningfully review the content before approving (e.g., the action details aren't surfaced) — fix the information architecture first
• You are using it as a substitute for actually improving the clarity of the approval interface

Accessibility

• The friction overlay renders as a role="alertdialog" with aria-modal="true" to capture focus during the mandatory pause
• The countdown timer is announced as a live region (aria-live="polite") so screen reader users are aware of the pause duration
• The acknowledge button is focussed automatically when the overlay appears
• The alert message includes explicit text explaining why the pause was triggered — not just a visual warning icon
• Color (amber/red) is never the sole indicator of the alert state; an icon and text label always accompany it

Solution Patterns

The Interceptor Layer

AutomationBiasAlert is an interceptor layer. It should wrap components that have onApprove or onProceed callbacks:

Plan Card → [Automation Bias Alert → Approval Gate] → Run Controls

It monitors the final interaction point before an irreversible action is executed.

Progressive Friction against Approval Fatigue

In high-throughput environments (like a SwarmInbox queue), users experience "velocity bias" — they approve faster as the queue grows, actively degrading their scrutiny. You can dynamically scale the friction thresholds to combat this cognitive failure.

// Instead of static thresholds, progressively harden them based on state
const consecutiveThreshold = approvalStreak >= 5 ? 2 : 5;
const requiredPauseMs = approvalStreak >= 5 ? 5000 : 1500;

const { isAlertTriggered, recordAction, dismissAlert } = useAutomationBias({
  consecutiveApprovals: consecutiveThreshold,
  minApprovalTimeMs: requiredPauseMs,
});

If a user approves 5 items in a row without rejection, the enforced pause climbs from 1.5 seconds to 5 seconds. This explicitly breaks their automatic rhythm.

Design rationale

Why friction? When an interface is too smooth, users stop paying attention. They settle into a rhythm of clicking — "rubber-stamping" — and critical details slip past them unreviewed.

Why does familiarity increase errors? The more a user feels "in control" of an AI system, the more likely they are to blindly accept its suggestions. Research by Shalaleh Rismani, an AI safety researcher at McGill and Mila, confirms this: users who better understood an AI writing assistant trusted it more — and accepted significantly more errors as a result. More transparency doesn't always lead to better oversight; it often leads to more trust and less scrutiny.

Why a mandatory pause? AutomationBiasAlert detects when repeated approvals suggest a user has stopped actually reviewing — and introduces a pause to break the pattern. The discomfort is the point.

• Non-blocking: It doesn't prevent the user from approving, but it breaks the physical rhythm of repetitive clicking.
• Contextual: It only appears when behavioral patterns suggest the human is no longer actually reviewing the data.
• Safety-Critical: Essential for agents performing financial transactions, medical evaluations, or high-consequence code deployments.