HazelJS Self-Healing Package

@hazeljs/self-healing adds self-healing microservices to HazelJS: automatic error diagnosis, recovery strategies, config rollback, memory guard, Kubernetes pod restart, HPA boost, and incident notifications.

Quick Reference

Purpose: Diagnose repeating failures and apply recovery strategies (restart, rollback, drain, scale) instead of paging a human for every blip.
When to use: Production APIs on Kubernetes (or locally with in-memory clients), services that already use @hazeljs/resilience, or ops workflows that should open Jira / Slack / PagerDuty when healing fails.
Key concepts: @SelfHealing, @SelfHeal, @MemoryGuard, strategies (auto-restart, config-rollback, memory-cleanup, safe-mode, pod-restart, hpa-boost), AI diagnostics, graceful drain, notifiers.
Dependencies: @hazeljs/core. Optional: @hazeljs/ai, @hazeljs/ops-agent, @hazeljs/resilience.
Common patterns: Decorate the module with @SelfHealing({ strategies, kubernetes, notifications }) → wrap hot methods with @SelfHeal → recordLatency() for performance-driven HPA boost.
Common mistakes: Enabling pod-restart without a Kubernetes client; skipping drain so in-flight requests die; turning on AI diagnostics without an LLM client.

Phases

Phase	What shipped
1	Rule-based diagnosis + recovery primitives (decorators + `createHealingCoordinator`)
2	LLM diagnosis via `@hazeljs/ai`, Kubernetes `pod-restart`, Slack / PagerDuty notifiers
3	Graceful drain, `hpa-boost` + auto-restore, performance-driven scaling, Jira via `@hazeljs/ops-agent`

Pair with @hazeljs/predictive-scaling for proactive HPA (forecast) vs this package's reactive HPA boost.

Architecture

graph TD
  E["Method error / latency"] --> D["Diagnostician<br/>rules + optional LLM"]
  D --> S["Strategy"]
  S --> R["auto-restart / rollback / cleanup"]
  S --> K["pod-restart + drain"]
  S --> H["hpa-boost"]
  S --> N["Slack / PagerDuty / Jira"]
  
  style E fill:#ef4444,stroke:#f87171,stroke-width:2px,color:#fff
  style D fill:#6366f1,stroke:#818cf8,stroke-width:2px,color:#fff
  style S fill:#f59e0b,stroke:#fbbf24,stroke-width:2px,color:#fff
  style R fill:#10b981,stroke:#34d399,stroke-width:2px,color:#fff
  style K fill:#3b82f6,stroke:#60a5fa,stroke-width:2px,color:#fff
  style H fill:#8b5cf6,stroke:#a78bfa,stroke-width:2px,color:#fff
  style N fill:#ec4899,stroke:#f472b6,stroke-width:2px,color:#fff

Installation

npm install @hazeljs/self-healing @hazeljs/core

Optional integrations:

npm install @hazeljs/resilience @hazeljs/ai @hazeljs/ops-agent

Quick Start

import {
  SelfHealing,
  SelfHeal,
  MemoryGuard,
  createSlackHealingNotifier,
} from '@hazeljs/self-healing';

@SelfHealing({
  enabled: true,
  strategies: ['auto-restart', 'config-rollback', 'pod-restart'],
  aiDiagnostics: true,
  notifyOn: ['critical-healing', 'auto-rollback', 'healing-failed', 'pod-restart'],
  notifications: createSlackHealingNotifier({ channel: '#incidents' }),
  kubernetes: {
    deployment: process.env.K8S_DEPLOYMENT!,
    namespace: process.env.K8S_NAMESPACE ?? 'default',
  },
})
export class AppModule {}

@MemoryGuard({ threshold: '500MB', action: 'memory-cleanup' })
export class PaymentService {
  @SelfHeal({ onError: 'diagnose-and-fix', maxAttempts: 3, fallback: 'chargeSafe' })
  async charge(): Promise<string> {
    return 'ok';
  }

  async chargeSafe(): Promise<string> {
    return 'safe-mode';
  }
}

Strategies

Strategy	When used	Action
`auto-restart`	Dependency / timeout errors	Re-run `onModuleDestroy` + `onModuleInit`
`config-rollback`	Config errors	Restore last config snapshot
`memory-cleanup`	Memory pressure	`clearCache()` + `global.gc()` if exposed
`safe-mode`	Unrecoverable errors	Invoke named fallback method
`pod-restart`	Cluster-level failures	Drain in-flight work, then PATCH deployment `restartedAt`
`hpa-boost`	Performance / load spikes	Temporarily raise HPA `minReplicas`, restore after cooldown

Drain, HPA, Jira

import {
  createHealingCoordinator,
  createJiraHealingNotifier,
  createHealingNotifierChain,
  FetchKubernetesScalingClient,
} from '@hazeljs/self-healing';
import { createJiraTool } from '@hazeljs/ops-agent';

const healing = createHealingCoordinator({
  drain: { timeoutMs: 30000 },
  performance: {
    enabled: true,
    autoScaleOnDegradation: true,
    thresholds: { criticalLatencyMs: 2000, sampleSize: 10 },
  },
  strategies: ['hpa-boost', 'pod-restart', 'config-rollback'],
  notifications: createHealingNotifierChain([
    createJiraHealingNotifier({ jira: createJiraTool(), project: 'OPS' }),
  ]),
  kubernetes: {
    deployment: 'payments-api',
    namespace: 'prod',
    drainBeforeRestart: true,
    hpa: {
      name: 'payments-hpa',
      client: new FetchKubernetesScalingClient(),
      boostMinReplicas: 4,
      restoreAfterMs: 300000,
    },
  },
});

await healing.recordLatency('PaymentService.charge', durationMs);

AI diagnostics

createHealingCoordinator({ aiDiagnostics: true });

createHealingCoordinator({
  aiDiagnostics: createHazelAIDiagnosticsProvider(aiService, { model: 'gpt-4o-mini' }),
});

Recipes

Recipe: Self-heal a service method

import { Service } from '@hazeljs/core';
import { SelfHeal } from '@hazeljs/self-healing';

@Service()
export class InventoryService {
  @SelfHeal({ maxAttempts: 3, onError: 'diagnose-and-fix' })
  async reserve(sku: string): Promise<void> {
    await this.downstream.reserve(sku);
  }
}

Recipe: Jira healing incidents

import { createHealingCoordinator, createJiraHealingNotifier } from '@hazeljs/self-healing';
import { createJiraTool } from '@hazeljs/ops-agent';

const healing = createHealingCoordinator({
  strategies: ['pod-restart'],
  notifications: createJiraHealingNotifier({
    jira: createJiraTool(),
    project: 'OPS',
  }),
});

Blog: Production Ops Stack — Config Server, Self-Healing, and Predictive Scaling together
Predictive Scaling — forecast + proactive HPA
Resilience — circuit breaker / retry / timeout
Ops Agent — Jira / Slack tools
Config Server — Git-backed config for rollback snapshots