Decision Runtime: Bounded Judgment for Agent OS

This guide covers @hazeljs/decision — when to use a decision runtime instead of free-form LLM JSON, how the native hazel-agent pipeline works, how confidence policies relate to Gatekeeper, and how Lab / history / calibration stay evaluation-only.

Package reference: Decision package.

Quick Reference

Purpose: Closed-set decisions (choices as const) with risk routing, evidence, critique, confidence, deterministic policy, Gatekeeper, durable HITL, and optional capability execution.
When to use: Operational judgments — remediation, routing, refunds, risk triage, urgency / legal gates — where inventing free-form actions is unsafe.
Key concepts: Reasoning ≠ Decision ≠ Policy ≠ Authorization ≠ Execution; createDecisionRuntime / decide; strategies fast|standard|deliberate|human-required|rules; hybrid rules and custom policy rules; hazel-agent default provider; Skillgate risk floor; Gatekeeper deny-by-default; Decision Lab shadow compare (runWithShadow); durable replay / getResultAsync; history never auto-prompted.
Dependencies: @hazeljs/decision + @hazeljs/agent; production: @hazeljs/agent-gatekeeper (+ optional @hazeljs/skillgate, @hazeljs/ai, @hazeljs/audit, @hazeljs/cli).
Common mistakes: Treating confidence as permission; open-ended “decide anything” prompts; executing without Gatekeeper; feeding history into LLM context by default; using Flow projection as the decision engine.

Why not just call an LLM?

Typical agent stacks collapse everything into one step:

State → LLM prompt → free-form reasoning → JSON → parse → validate → execute tool

That is unnecessarily open-ended for operational judgments. Prefer:

STATE → EVIDENCE → BOUNDED DECISION → CONFIDENCE → CRITIQUE
  → POLICY → AUTHORIZATION → DURABLE EXECUTION

Layer	Role	Package
Reasoning	Planning, explanation, critique	`@hazeljs/ai` / agent tools
Decision	Closed-set selection + confidence	`@hazeljs/decision`
Policy	Deterministic allow / critique / review / deny	`@hazeljs/decision`
Authorization	Per-invocation allow/deny	`@hazeljs/agent-gatekeeper`
Skill curation	Which APIs become tools + risk class	`@hazeljs/skillgate`
Execution	Checkpoints, HITL, receipts	`@hazeljs/agent` (+ Agent VM on host)

Decision confidence is evidence for policy routing, not permission to mutate the world. Confidence 0.99 + Gatekeeper deny = no execution.

Progressive complexity

createDecisionRuntime() + decide() — typed closed set
Provider (hazel-agent / mock / external adapter)
Risk + confidence bands
Critic + deterministic policy
Gatekeeper + durable HITL + capability handlers
Decision DNA + Lab + history evaluation + optional calibration

Level 1 users are not forced into Level 6 concepts.

Real-world use cases

Production incident remediation

Choose retry | rollback | escalate | ignore from error rate, deploy age, and health checks. High risk → deliberate strategy (evidence, candidates, critic). High confidence may policy-allow into Gatekeeper for deployment.rollback; mid confidence routes to critique; low confidence pauses for human review.

Support ticket routing

Classify billing | technical | sales. Ambiguous candidate separation → critique or review instead of the wrong queue.

Fraud / risk triage

Choose approve | hold | reject | escalate on payment or login state. High risk still needs Gatekeeper for payment.hold.

Refund path

Choose auto_refund | partial | deny | escalate. Auto-execute only above policy threshold and when Gatekeeper allows order.refund.

Urgency / legal gates

Treat yes/no as a two-choice decision before expensive tools or customer contact. Critical risk forces human-required.

Install

npm install @hazeljs/decision @hazeljs/agent
npm install @hazeljs/agent-gatekeeper   # production execute
npm install @hazeljs/skillgate          # risk floor from skill class
npm install @hazeljs/ai @hazeljs/audit  # optional LLM stages + audit

hazel add decision
hazel decision lab

Quick start

import { createDecisionRuntime } from '@hazeljs/decision';

const decisions = createDecisionRuntime();

const result = await decisions.decide({
  objective: 'Choose the safest response to this production incident',
  state: {
    service: 'checkout',
    errorRate: 0.38,
    previousErrorRate: 0.01,
    deployment: { ageMinutes: 4 },
    failedHealthChecks: 8,
  },
  choices: ['retry', 'rollback', 'escalate', 'ignore'] as const,
  risk: 'high',
  provider: 'hazel-agent',
});

console.log(result.decision, result.confidence, result.policy.outcome);

Mental model of a run

1. Resolve definition (request + DNA registry)
2. Resolve risk (trusted config; Skillgate may raise floor)
3. Select strategy (auto from risk / choice count, or explicit)
4. Provider proposes decision inside choices[]
5. Confidence composed (heuristic / provider / ensemble / calibrated*)
6. Policy evaluates → allow | critique | review | deny | …
7. If execute && allow → Gatekeeper → capability handler
8. Checkpoint + metrics + optional history append
9. If review → HumanTask + WAITING_FOR_HUMAN

*Calibrated only when calibrate: true and a model was fitted — reporting only.

Strategies

Strategy	Typical trigger	Includes
`fast`	low risk, small choice set	judge
`standard`	medium risk	evidence, judge, confidence
`deliberate`	high risk	+ candidates, critic
`human-required`	critical / DNA force	+ HITL
`rules`	`strategy: 'rules'`	DNA `metadata.rules` pick a choice; judge is skipped when a rule matches

await decisions.decide({
  ...,
  strategy: 'deliberate', // or 'auto'
});

Critical risk resolves to human-required, including when the request asks for fast, standard, or deliberate. A capability map with execution.<choice>.hitl: true forces the same strategy. strategy: 'rules' is the exception: a matching DNA rule still selects the choice, and policy runs afterward.

Rules strategy

strategy: 'rules' reads Decision DNA metadata.rules. Each rule is { whenPath, gte, then }. whenPath is a dotted path into state. then must be one of choices. The first numeric match returns that choice with heuristic confidence 1 and stage rules. Policy and Gatekeeper still run.

await decisions.decide({
  objective: 'Escalate after repeated attempts',
  state: { attempts: 3 },
  choices: ['retry', 'escalate'] as const,
  strategy: 'rules',
  definition: {
    objective: 'Escalate after repeated attempts',
    choices: ['retry', 'escalate'],
    metadata: {
      rules: [{ whenPath: 'attempts', gte: 3, then: 'escalate' }],
    },
  },
});

Hybrid policy overrides

hybridRules on the runtime force a policy action after the provider chooses. They beat confidence bands and custom policy rules. The first match wins.

const decisions = createDecisionRuntime({
  hybridRules: [
    { whenPath: 'amount', gte: 5000, then: 'review', reason: 'amount >= 5000' },
  ],
});

Providers in depth

Native `hazel-agent` (default)

Pipeline stages (depending on strategy):

Evidence — project structured keys from state
Candidates — score each allowed choice
Judge — pick within the closed set (heuristic or generateObject)
Critic — challenge uncertain / high-risk proposals
Confidence — compose components; mark calibrated: false unless remapped

HazelDecisionAgent exposes the same stages as @Agent + @Tool methods for registration on AgentRuntime.

import { createGovernedDecisionRuntime } from '@hazeljs/decision';

const decisions = createGovernedDecisionRuntime({
  generateObject: aiService,
  agentRuntime,
  gatekeeper,
  auditService,
});

On LLM failure → heuristic fallback. On invalid choice → reject (never coerce to “allow”).

Mock

import { MockDecisionProvider, createDecisionRuntime } from '@hazeljs/decision';

const runtime = createDecisionRuntime();
const mock = runtime.providers.get('mock') as MockDecisionProvider;
mock.when('incident-remediation').return({
  decision: 'rollback',
  confidence: 0.92,
});

External adapters (optional)

import {
  createOpenAiDecisionProvider,
  createGeminiDecisionProvider,
  createJevDecisionProvider,
  createLocalDecisionProvider,
} from '@hazeljs/decision';

runtime.providers.register(createOpenAiDecisionProvider(aiService));
runtime.providers.register(
  createJevDecisionProvider({ decide: async (req) => jevDecide(req) })
);

App code should prefer runtime.decide({ provider: '…' }) so providers stay swappable. Do not treat an external model’s score as authorization.

Ensemble

import { createEnsembleDecisionProvider } from '@hazeljs/decision';

runtime.providers.register(
  createEnsembleDecisionProvider({
    providers: [hazel, openai],
    strategy: 'majority', // weighted | unanimous | risk-sensitive
  })
);

Votes outside choices throw. Results still pass policy + Gatekeeper.

Cost-aware auto route

import { createCostAwareRouter } from '@hazeljs/decision';

runtime.providers.setRouterHook(
  createCostAwareRouter([
    { name: 'local', costTier: 'economy' },
    { name: 'hazel-agent', costTier: 'balanced' },
    { name: 'openai', costTier: 'premium', minRisk: 'high' },
  ])
);

await runtime.decide({ ..., provider: 'auto' });

Decision DNA

Declare decisions next to tools on Agent DNA:

decisions?: {
  'incident-remediation': {
    version: '3';
    objective: string;
    choices: string[];
    risk: { level: 'high' };
    confidence?: { high?: number; medium?: number };
    evidence?: { required?: string[]; projections?: unknown };
    scoring?: Record<string, Array<{ evidenceKey: string; weight: number }>>;
    execution?: Record<string, { capability: string }>;
  };
};

runtime.registry.registerFromDna(dna);
// or
runtime.registry.register({ name: 'incident-remediation', ...definition });

Risk and capability maps are not model-writable.

Policy → Gatekeeper → execution

policy allow  ──execute:true──►  Gatekeeper.evaluate  ──allow──►  capability handler
                      │
                      └──deny──► DENIED (audited)
policy review ─────────────────► HumanTask / WAITING_FOR_HUMAN
policy deny   ─────────────────► DENIED

const decisions = createDecisionRuntime({
  gatekeeper,
  skillgate, // optional floor from skill class
  checkpoints,
  humanTasks,
  capabilityHandlers: {
    'deployment.rollback': async ({ decisionId, state }) => {
      // side effects only after Gatekeeper allow
      return { rolledBack: true, decisionId };
    },
  },
});

await decisions.decide({
  name: 'incident-remediation',
  ...,
  execute: true,
  context: { tenantId: 'acme', agentId: 'incident-agent' },
});

Missing Gatekeeper with execute: true ⇒ execution denied (fail closed).

What each policy action does

Action	Status	Execution
`allow`	Gatekeeper path when `execute: true` and a capability is mapped	Invoked only after Gatekeeper allow
`critique`	`DECISION_FINALIZED`	Not invoked
`review`	`WAITING_FOR_HUMAN`	Not invoked until resume
`escalate`	`WAITING_FOR_HUMAN` (same HITL path as `review`)	Not invoked until resume
`deny`	`DENIED`	Not requested
`fallback`	`COMPLETED`	Not invoked

Custom rules override the default bands. Pass an array, or { rules, confidence }. DNA policy is used when the request omits rules.

await decisions.decide({
  ...,
  policy: {
    rules: [
      { when: { risk: 'critical' }, action: 'review' },
      { when: { confidence: { gte: 0.95 } }, action: 'allow' },
      { when: { confidence: { lt: 0.4 } }, action: 'deny' },
    ],
  },
});

when.confidence accepts gte, gt, lt, and lte.

Skillgate risk floor

Skill class	Floor
`read`	`low`
`write`	`medium`
`destructive`	`high`
`admin`	`critical`

The floor can only raise risk. lookupSkill matches the capability name on the Skillgate registry.

See Skillgate and Agent Gatekeeper.

HITL resume

await decisions.resumeFromHuman({
  decisionId: result.id,
  runId: result.trace.runId!,
  action: 'approve', // reject | override
  decision: 'escalate', // required for override
  actor: 'ops-lead',
  reason: 'Customer impact still rising',
  execute: true,
});

Receipts prevent double-invoke. Overrides are audited when an audit sink is configured.

Durable reads

Pass Agent checkpoint and human-task stores at construction. After a restart, resume and lookup read those checkpoints. Handlers are not called again.

decisions.registerCapability('payments.refund', async ({ decisionId }) => ({
  refunded: true,
  decisionId,
}));

const cached = decisions.getResult(result.id);
const hydrated = await decisions.getResultAsync(result.id, result.trace.runId);
const replayed = await decisions.replay(result.trace.runId!);

getResult is memory-only. getResultAsync loads the newest checkpoint for that decision id when memory is empty. replay returns a Lab run from the latest checkpoint. clearMemory() drops the in-memory index without deleting checkpoints.

Useful result fields beyond decision / confidence / policy: alternatives (other candidates), cost (modelCalls, tokens), and latency (per stage plus totalMs).

Decision Lab & Agent Office

import { createDecisionLab, createDecisionRuntime } from '@hazeljs/decision';

const lab = createDecisionLab(createDecisionRuntime());
const run = await lab.run(request);           // graph + stages
const cmp = await lab.compare(request, ['hazel-agent', 'mock']);
lab.projectFlow({ risk: 'critical' });        // Flow-shaped projection

Comparison mode sets executionForbidden: true
Shadow helpers: registerShadowProvider(runtime.providers, provider) registers a provider; runWithShadow(lab, request, ['local', 'mock']) runs the primary decision and a compare with execute: false on both
Agent Office UI (hazeljs-agent-office): /office/decisions — Run · Flow graph · History · Calibration (not Inspector)
Meridian teaching API: POST /api/decision/refund → resume (confidence ≠ permission)
Lab APIs in Agent Office: /decisions/run|compare|flow|history|trends|calibration|evaluate

History, trends, calibration

import {
  DecisionHistory,
  fitCalibrationFromHistory,
  createDecisionRuntime,
} from '@hazeljs/decision';

const history = new DecisionHistory();
const runtime = createDecisionRuntime({ history });

const r = await runtime.decide({ ... });
runtime.evaluate({
  decisionId: r.id,
  expectedDecision: 'rollback',
});

const trends = history.trends();
// { total, byDecision, byPolicy, byProvider, avgConfidence, accuracy?, promptInjectionForbidden: true }

const similar = history.similarByEvidence(['error-rate-change'], 10);
// ranked by shared evidence keys — analysis only, never prompted

runtime.setCalibration(fitCalibrationFromHistory(history));
await runtime.decide({ ..., calibrate: true }); // remaps confidenceDetail for reporting

History is append-only for humans / Lab / ECE. It is never automatically injected into model prompts (avoids feedback loops). Calibration never authorizes.

Cache (opt-in)

import { DecisionCache, createDecisionRuntime } from '@hazeljs/decision';

const runtime = createDecisionRuntime({
  cache: new DecisionCache({ ttlMs: 30_000 }),
});

await runtime.decide({ ..., cache: true }); // requires both runtime.cache and request.cache

Cache keys fingerprint state. Hits never re-invoke capabilities. Do not enable globally by default.

Flow graph projection

import { projectDecisionFlow, buildDecisionFlowDefinition } from '@hazeljs/decision';

const graph = projectDecisionFlow({
  name: 'incident-remediation',
  risk: 'high',
  strategy: 'auto',
});
// graph.nodes / edges for Lab UI

const stub = buildDecisionFlowDefinition(graph, {
  judge: async () => ({ status: 'ok', output: { decision: 'rollback' } }),
});
// optional register with @hazeljs/flow FlowEngine

Projection mirrors stages for orchestration UIs. DecisionRuntime remains authoritative.

Decorator

import { Decision } from '@hazeljs/decision';

class IncidentOps {
  @Decision({
    name: 'incident-remediation',
    choices: ['retry', 'rollback', 'escalate', 'ignore'],
    risk: 'high',
  })
  async remediate(_state: unknown) {
    // metadata recorded for discovery; call DecisionRuntime.decide at the boundary
  }
}

Observability

Metrics (in-process): hazeljs_decisions_total, hazeljs_decision_duration, hazeljs_decision_confidence, critic / review / override / denied / execution / failures counters.

Optional span: decision.run (tracer hazeljs). Prefer not putting raw state into telemetry by default; use redacted audit events (buildAuditEvent).

Errors

Catch DecisionError and branch on code. The runtime throws these:

Code	Class	When
`DECISION_VALIDATION`	`DecisionValidationError`	Empty choices, missing objective, unknown DNA, override without `decision`
`DECISION_INVALID_OUTPUT`	`DecisionInvalidOutputError`	Verb outside `choices`
`DECISION_PROVIDER`	`DecisionProviderError`	Unknown provider, or a failure that is not a heuristic fallback
`DECISION_REVIEW_REQUIRED`	`DecisionReviewRequiredError`	`metadata.throwOnReview` and the run is waiting for a human

Without throwOnReview, review returns status: 'WAITING_FOR_HUMAN'. Gatekeeper deny is status: 'DENIED' on the result. DecisionTimeoutError, DecisionPolicyError, DecisionAuthorizationError, and DecisionExecutionError are exported and are not raised by the current runtime.

Security checklist

Rule	Why
Closed `choices`	Models cannot invent new verbs
State is data	Reduces prompt-injection surface
Risk not model-writable	Prevents self-downgrade of critical paths
Gatekeeper after policy allow	Confidence ≠ permission
No Gatekeeper ⇒ no execute	Fail closed
Replay ≠ re-execute	Idempotent receipts
History not in prompts by default	Avoids feedback loops
Calibration reporting-only	Scores do not authorize

Package examples: prompt-injection.ts, high-confidence-denied.ts.

Testing

import {
  createDecisionRuntime,
  MockDecisionProvider,
} from '@hazeljs/decision';

const runtime = createDecisionRuntime();
const mock = runtime.providers.get('mock') as MockDecisionProvider;
mock.setDefault({ decision: 'escalate', confidence: 0.8 });

const result = await runtime.decide({
  objective: 'x',
  state: {},
  choices: ['retry', 'escalate'] as const,
  provider: 'mock',
});
expect(result.decision).toBe('escalate');

npm test --workspace=@hazeljs/decision

No paid model APIs required for the suite.

CLI cheat sheet

--execute on run still requires Gatekeeper allow. compare never executes.

hazel decision run \
  --name incident-remediation \
  --state ./incident.json \
  --choices retry,rollback,escalate,ignore \
  --risk high \
  --provider hazel-agent \
  --strategy auto \
  --json

hazel decision compare --providers hazel-agent,mock --state ./incident.json --risk high --json
hazel decision flow --name incident-remediation --risk critical --strategy auto --json
hazel decision lab --state ./incident.json --risk high --provider hazel-agent --json

Command	Extra flags
`run`	`--name`, `--objective`, `--state`, `--choices`, `--risk`, `--provider`, `--strategy`, `--execute`, `--json`
`compare`	`--name`, `--objective`, `--state`, `--choices`, `--risk`, `--providers`, `--json`
`flow`	`--name`, `--risk`, `--strategy`, `--json`
`lab`	`--state`, `--risk`, `--provider`, `--json`

How this differs from Skillgate / Gatekeeper

Package	Question it answers
`@hazeljs/decision`	Which allowed action should we choose given state?
`@hazeljs/skillgate`	Which APIs become agent skills, and what risk class?
`@hazeljs/agent-gatekeeper`	May this agent invoke this tool now, with this input?

Use all three together for production Agent OS: decide → (policy) → authorize → execute.

Intentionally out of scope

Aggressive cost-minimizing auto-route beyond createCostAwareRouter
Automatic injection of historical outcomes into prompts
Replacing Gatekeeper with confidence thresholds