Decision Runtime: Bounded Judgment for Agent OS

This guide covers @hazeljs/decision — when to use a decision runtime instead of free-form LLM JSON, how the native hazel-agent pipeline works, how confidence policies relate to Gatekeeper, and how Lab / history / calibration stay evaluation-only.

Package reference: Decision package.

Quick Reference

Purpose: Closed-set decisions (choices as const) with risk routing, evidence, critique, confidence, deterministic policy, Gatekeeper, durable HITL, and optional capability execution.
When to use: Operational judgments — remediation, routing, refunds, risk triage, urgency / legal gates — where inventing free-form actions is unsafe.
Key concepts: Reasoning ≠ Decision ≠ Policy ≠ Authorization ≠ Execution; createDecisionRuntime / decide; strategies fast|standard|deliberate|human-required|rules; hazel-agent default provider; Skillgate risk floor; Gatekeeper deny-by-default; Decision Lab shadow compare; history never auto-prompted.
Dependencies: @hazeljs/decision + @hazeljs/agent; production: @hazeljs/agent-gatekeeper (+ optional @hazeljs/skillgate, @hazeljs/ai, @hazeljs/audit, @hazeljs/cli).
Common mistakes: Treating confidence as permission; open-ended “decide anything” prompts; executing without Gatekeeper; feeding history into LLM context by default; using Flow projection as the decision engine.

Why not just call an LLM?

Typical agent stacks collapse everything into one step:

State → LLM prompt → free-form reasoning → JSON → parse → validate → execute tool

That is unnecessarily open-ended for operational judgments. Prefer:

STATE → EVIDENCE → BOUNDED DECISION → CONFIDENCE → CRITIQUE
  → POLICY → AUTHORIZATION → DURABLE EXECUTION

Layer	Role	Package
Reasoning	Planning, explanation, critique	`@hazeljs/ai` / agent tools
Decision	Closed-set selection + confidence	`@hazeljs/decision`
Policy	Deterministic allow / critique / review / deny	`@hazeljs/decision`
Authorization	Per-invocation allow/deny	`@hazeljs/agent-gatekeeper`
Skill curation	Which APIs become tools + risk class	`@hazeljs/skillgate`
Execution	Checkpoints, HITL, receipts	`@hazeljs/agent` (+ Agent VM on host)

Decision confidence is evidence for policy routing, not permission to mutate the world. Confidence 0.99 + Gatekeeper deny = no execution.

Progressive complexity

createDecisionRuntime() + decide() — typed closed set
Provider (hazel-agent / mock / external adapter)
Risk + confidence bands
Critic + deterministic policy
Gatekeeper + durable HITL + capability handlers
Decision DNA + Lab + history evaluation + optional calibration

Level 1 users are not forced into Level 6 concepts.

Real-world use cases

Production incident remediation

Choose retry | rollback | escalate | ignore from error rate, deploy age, and health checks. High risk → deliberate strategy (evidence, candidates, critic). High confidence may policy-allow into Gatekeeper for deployment.rollback; mid confidence routes to critique; low confidence pauses for human review.

Support ticket routing

Classify billing | technical | sales. Ambiguous candidate separation → critique or review instead of the wrong queue.

Fraud / risk triage

Choose approve | hold | reject | escalate on payment or login state. High risk still needs Gatekeeper for payment.hold.

Refund path

Choose auto_refund | partial | deny | escalate. Auto-execute only above policy threshold and when Gatekeeper allows order.refund.

Urgency / legal gates

Treat yes/no as a two-choice decision before expensive tools or customer contact. Critical risk forces human-required.

Install

npm install @hazeljs/decision @hazeljs/agent
npm install @hazeljs/agent-gatekeeper   # production execute
npm install @hazeljs/skillgate          # risk floor from skill class
npm install @hazeljs/ai @hazeljs/audit  # optional LLM stages + audit

hazel add decision
hazel decision lab

Quick start

import { createDecisionRuntime } from '@hazeljs/decision';

const decisions = createDecisionRuntime();

const result = await decisions.decide({
  objective: 'Choose the safest response to this production incident',
  state: {
    service: 'checkout',
    errorRate: 0.38,
    previousErrorRate: 0.01,
    deployment: { ageMinutes: 4 },
    failedHealthChecks: 8,
  },
  choices: ['retry', 'rollback', 'escalate', 'ignore'] as const,
  risk: 'high',
  provider: 'hazel-agent',
});

console.log(result.decision, result.confidence, result.policy.outcome);

Mental model of a run

1. Resolve definition (request + DNA registry)
2. Resolve risk (trusted config; Skillgate may raise floor)
3. Select strategy (auto from risk / choice count, or explicit)
4. Provider proposes decision inside choices[]
5. Confidence composed (heuristic / provider / ensemble / calibrated*)
6. Policy evaluates → allow | critique | review | deny | …
7. If execute && allow → Gatekeeper → capability handler
8. Checkpoint + metrics + optional history append
9. If review → HumanTask + WAITING_FOR_HUMAN

*Calibrated only when calibrate: true and a model was fitted — reporting only.

Strategies

Strategy	Typical trigger	Includes
`fast`	low risk, small choice set	judge
`standard`	medium risk	evidence, judge, confidence
`deliberate`	high risk	+ candidates, critic
`human-required`	critical / DNA force	+ HITL
`rules`	DNA / hybrid rules mode	rules path before probabilistic policy

await decisions.decide({
  ...,
  strategy: 'deliberate', // or 'auto'
});

Providers in depth

Native `hazel-agent` (default)

Pipeline stages (depending on strategy):

Evidence — project structured keys from state
Candidates — score each allowed choice
Judge — pick within the closed set (heuristic or generateObject)
Critic — challenge uncertain / high-risk proposals
Confidence — compose components; mark calibrated: false unless remapped

HazelDecisionAgent exposes the same stages as @Agent + @Tool methods for registration on AgentRuntime.

import { createGovernedDecisionRuntime } from '@hazeljs/decision';

const decisions = createGovernedDecisionRuntime({
  generateObject: aiService,
  agentRuntime,
  gatekeeper,
  auditService,
});

On LLM failure → heuristic fallback. On invalid choice → reject (never coerce to “allow”).

Mock

import { MockDecisionProvider, createDecisionRuntime } from '@hazeljs/decision';

const runtime = createDecisionRuntime();
const mock = runtime.providers.get('mock') as MockDecisionProvider;
mock.when('incident-remediation').return({
  decision: 'rollback',
  confidence: 0.92,
});

External adapters (optional)

import {
  createOpenAiDecisionProvider,
  createGeminiDecisionProvider,
  createJevDecisionProvider,
  createLocalDecisionProvider,
} from '@hazeljs/decision';

runtime.providers.register(createOpenAiDecisionProvider(aiService));
runtime.providers.register(
  createJevDecisionProvider({ decide: async (req) => jevDecide(req) })
);

App code should prefer runtime.decide({ provider: '…' }) so providers stay swappable. Do not treat an external model’s score as authorization.

Ensemble

import { createEnsembleDecisionProvider } from '@hazeljs/decision';

runtime.providers.register(
  createEnsembleDecisionProvider({
    providers: [hazel, openai],
    strategy: 'majority', // weighted | unanimous | risk-sensitive
  })
);

Votes outside choices throw. Results still pass policy + Gatekeeper.

Cost-aware auto route

import { createCostAwareRouter } from '@hazeljs/decision';

runtime.providers.setRouterHook(
  createCostAwareRouter([
    { name: 'local', costTier: 'economy' },
    { name: 'hazel-agent', costTier: 'balanced' },
    { name: 'openai', costTier: 'premium', minRisk: 'high' },
  ])
);

await runtime.decide({ ..., provider: 'auto' });

Decision DNA

Declare decisions next to tools on Agent DNA:

decisions?: {
  'incident-remediation': {
    version: '3';
    objective: string;
    choices: string[];
    risk: { level: 'high' };
    confidence?: { high?: number; medium?: number };
    evidence?: { required?: string[]; projections?: unknown };
    scoring?: Record<string, Array<{ evidenceKey: string; weight: number }>>;
    execution?: Record<string, { capability: string }>;
  };
};

runtime.registry.registerFromDna(dna);
// or
runtime.registry.register({ name: 'incident-remediation', ...definition });

Risk and capability maps are not model-writable.

Policy → Gatekeeper → execution

policy allow  ──execute:true──►  Gatekeeper.evaluate  ──allow──►  capability handler
                      │
                      └──deny──► DENIED (audited)
policy review ─────────────────► HumanTask / WAITING_FOR_HUMAN
policy deny   ─────────────────► DENIED

const decisions = createDecisionRuntime({
  gatekeeper,
  skillgate, // optional floor from skill class
  checkpoints,
  humanTasks,
  capabilityHandlers: {
    'deployment.rollback': async ({ decisionId, state }) => {
      // side effects only after Gatekeeper allow
      return { rolledBack: true, decisionId };
    },
  },
});

await decisions.decide({
  name: 'incident-remediation',
  ...,
  execute: true,
  context: { tenantId: 'acme', agentId: 'incident-agent' },
});

Missing Gatekeeper with execute: true ⇒ execution denied (fail closed).

See Skillgate and Agent Gatekeeper.

HITL resume

await decisions.resumeFromHuman({
  decisionId: result.id,
  runId: result.trace.runId!,
  action: 'approve', // reject | override
  decision: 'escalate', // required for override
  actor: 'ops-lead',
  reason: 'Customer impact still rising',
  execute: true,
});

Receipts prevent double-invoke. Overrides are audited when an audit sink is configured.

Decision Lab & Agent Office

import { createDecisionLab, createDecisionRuntime } from '@hazeljs/decision';

const lab = createDecisionLab(createDecisionRuntime());
const run = await lab.run(request);           // graph + stages
const cmp = await lab.compare(request, ['hazel-agent', 'mock']);
lab.projectFlow({ risk: 'critical' });        // Flow-shaped projection

Comparison mode sets executionForbidden: true
Office UI: /office/decisions — Run · Flow graph · History · Calibration
APIs: /decisions/run|compare|flow|history|trends|calibration|evaluate

History, trends, calibration

import {
  DecisionHistory,
  fitCalibrationFromHistory,
  createDecisionRuntime,
} from '@hazeljs/decision';

const history = new DecisionHistory();
const runtime = createDecisionRuntime({ history });

const r = await runtime.decide({ ... });
runtime.evaluate({
  decisionId: r.id,
  expectedDecision: 'rollback',
});

const trends = history.trends();
// { total, byDecision, avgConfidence, accuracy?, promptInjectionForbidden: true }

runtime.setCalibration(fitCalibrationFromHistory(history));
await runtime.decide({ ..., calibrate: true }); // remaps confidenceDetail for reporting

History is append-only for humans / Lab / ECE. It is never automatically injected into model prompts (avoids feedback loops). Calibration never authorizes.

Cache (opt-in)

import { DecisionCache, createDecisionRuntime } from '@hazeljs/decision';

const runtime = createDecisionRuntime({
  cache: new DecisionCache({ ttlMs: 30_000 }),
});

await runtime.decide({ ..., cache: true }); // requires both runtime.cache and request.cache

Cache keys fingerprint state. Hits never re-invoke capabilities. Do not enable globally by default.

Flow graph projection

import { projectDecisionFlow, buildDecisionFlowDefinition } from '@hazeljs/decision';

const graph = projectDecisionFlow({
  name: 'incident-remediation',
  risk: 'high',
  strategy: 'auto',
});
// graph.nodes / edges for Lab UI

const stub = buildDecisionFlowDefinition(graph, {
  judge: async () => ({ status: 'ok', output: { decision: 'rollback' } }),
});
// optional register with @hazeljs/flow FlowEngine

Projection mirrors stages for orchestration UIs. DecisionRuntime remains authoritative.

Decorator

import { Decision } from '@hazeljs/decision';

class IncidentOps {
  @Decision({
    name: 'incident-remediation',
    choices: ['retry', 'rollback', 'escalate', 'ignore'],
    risk: 'high',
  })
  async remediate(_state: unknown) {
    // metadata recorded for discovery; call DecisionRuntime.decide at the boundary
  }
}

Observability

Metrics (in-process): hazeljs_decisions_total, hazeljs_decision_duration, hazeljs_decision_confidence, critic / review / override / denied / execution / failures counters.

Optional span: decision.run (tracer hazeljs). Prefer not putting raw state into telemetry by default; use redacted audit events (buildAuditEvent).

Security checklist

Rule	Why
Closed `choices`	Models cannot invent new verbs
State is data	Reduces prompt-injection surface
Risk not model-writable	Prevents self-downgrade of critical paths
Gatekeeper after policy allow	Confidence ≠ permission
No Gatekeeper ⇒ no execute	Fail closed
Replay ≠ re-execute	Idempotent receipts
History not in prompts by default	Avoids feedback loops
Calibration reporting-only	Scores do not authorize

Package examples: prompt-injection.ts, high-confidence-denied.ts.

Testing

import {
  createDecisionRuntime,
  MockDecisionProvider,
} from '@hazeljs/decision';

const runtime = createDecisionRuntime();
const mock = runtime.providers.get('mock') as MockDecisionProvider;
mock.setDefault({ decision: 'escalate', confidence: 0.8 });

const result = await runtime.decide({
  objective: 'x',
  state: {},
  choices: ['retry', 'escalate'] as const,
  provider: 'mock',
});
expect(result.decision).toBe('escalate');

npm test --workspace=@hazeljs/decision

No paid model APIs required for the suite.

CLI cheat sheet

hazel decision run --state ./incident.json --risk high --provider hazel-agent
hazel decision compare --providers hazel-agent,mock
hazel decision flow --risk critical
hazel decision lab --json

How this differs from Skillgate / Gatekeeper

Package	Question it answers
`@hazeljs/decision`	Which allowed action should we choose given state?
`@hazeljs/skillgate`	Which APIs become agent skills, and what risk class?
`@hazeljs/agent-gatekeeper`	May this agent invoke this tool now, with this input?

Use all three together for production Agent OS: decide → (policy) → authorize → execute.

Intentionally out of scope

Aggressive cost-minimizing auto-route beyond createCostAwareRouter
Automatic injection of historical outcomes into prompts
Replacing Gatekeeper with confidence thresholds