Self-Healing System — API Documentation

Complete reference for all REST API endpoints across the TaskApi and Analyzer services.

TaskApi Endpoints
Analyzer Endpoints
Data Models
Error Handling
Internal Communication

Base URLs

Service	Local (port-forward)	In-Cluster
TaskApi	`http://localhost:5186`	`http://taskapi`
Analyzer	`http://localhost:5297`	`http://analyzer`
Prometheus	`http://localhost:9090`	`http://prometheus:9090`
Grafana	`http://localhost:3000`	`http://grafana:3000`

TaskApi Endpoints

Tasks

`GET /api/tasks`

List all tasks, ordered by creation date (newest first).

Response: 200 OK

[
  {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "title": "Setup monitoring",
    "createdAt": "2025-01-15T10:30:00Z"
  }
]

`POST /api/tasks`

Create a new task.

Request Body:

{
  "title": "My new task"
}

Response: 201 Created

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "title": "My new task",
  "createdAt": "2025-01-15T10:30:00Z"
}

`DELETE /api/tasks/{id}`

Delete a task by ID.

Parameter	Type	Location	Description
`id`	`guid`	path	Task UUID

Response: 204 No Content

Error: 404 Not Found if task doesn't exist.

Failure Events

`GET /api/failureevents`

List failure events with optional filters.

Parameter	Type	Location	Default	Description
`resolved`	`bool?`	query	—	Filter by resolution status
`failureType`	`string?`	query	—	Filter by type (e.g. `MemoryLeakSuspected`)
`limit`	`int`	query	`50`	Max results to return

Response: 200 OK

[
  {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "failureType": "MemoryLeakSuspected",
    "severity": "Warning",
    "description": "Memory trending +25.3 MB/min (R²=0.87, n=20, 1.6min window)",
    "detectedAt": "2025-01-15T10:30:00Z",
    "resolved": false,
    "resolvedAt": null
  }
]

`POST /api/failureevents`

Create a new failure event.

Request Body:

{
  "failureType": "MemoryLeakSuspected",
  "severity": "Warning",
  "description": "Memory trending +25.3 MB/min (R²=0.87, n=20, 1.6min window)",
  "detectedAt": "2025-01-15T10:30:00Z"
}

Field	Type	Required	Default	Description
`failureType`	`string`	yes	—	Type identifier
`severity`	`string`	no	`"Info"`	`Info`, `Warning`, `Critical`
`description`	`string`	no	`""`	Human-readable description
`detectedAt`	`datetime`	no	`DateTime.UtcNow`	Detection timestamp

Response: 200 OK — Returns the created entity with generated id.

`PATCH /api/failureevents/{id}/resolve`

Mark a failure event as resolved.

Parameter	Type	Location	Description
`id`	`guid`	path	Failure event UUID

Response: 200 OK

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "failureType": "MemoryLeakSuspected",
  "severity": "Warning",
  "description": "Memory trending +25.3 MB/min (R²=0.87, n=20, 1.6min window)",
  "detectedAt": "2025-01-15T10:30:00Z",
  "resolved": true,
  "resolvedAt": "2025-01-15T10:35:00Z"
}

Error: 404 Not Found if event doesn't exist.

Recovery Actions

`GET /api/recoveryactions`

List recovery actions with optional filters.

Parameter	Type	Location	Default	Description
`failureEventId`	`guid?`	query	—	Filter by failure event
`limit`	`int`	query	`50`	Max results to return

Response: 200 OK

[
  {
    "id": "d2e4f6a8-1234-5678-9abc-def012345678",
    "failureEventId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "actionType": "RestartPod",
    "targetDeployment": "taskapi",
    "status": "Success",
    "details": "Restarted pod taskapi-xyz",
    "performedAt": "2025-01-15T10:31:00Z",
    "completedAt": "2025-01-15T10:31:05Z"
  }
]

`POST /api/recoveryactions`

Create a new recovery action.

Request Body:

{
  "failureEventId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "actionType": "RestartPod",
  "targetDeployment": "taskapi",
  "details": "Scaling up to 3 replicas"
}

Field	Type	Required	Default	Description
`failureEventId`	`guid`	yes	—	Associated failure event
`actionType`	`string`	yes	—	`RestartPod`, `ScaleUp`, etc.
`targetDeployment`	`string`	yes	—	K8s deployment name
`details`	`string?`	no	`null`	Additional details

Response: 200 OK — Returns entity with status: "Pending", generated id, and performedAt.

`PATCH /api/recoveryactions/{id}/status`

Update the status of a recovery action.

Parameter	Type	Location	Description
`id`	`guid`	path	Recovery action UUID

Request Body:

{
  "status": "Success",
  "details": "Pod restarted successfully"
}

Field	Type	Required	Description
`status`	`string`	yes	`Pending`, `InProgress`, `Success`, `Failed`
`details`	`string?`	no	Updates details (keeps existing if null)

When status is "Success" or "Failed", completedAt is automatically set.

Response: 200 OK

Error: 404 Not Found if action doesn't exist.

Metrics

`GET /api/metrics`

List metric records with optional filters.

Parameter	Type	Location	Default	Description
`metricId`	`string?`	query	—	Filter by metric identifier
`from`	`datetime?`	query	—	Start of time range
`to`	`datetime?`	query	—	End of time range
`limit`	`int`	query	`200`	Max results to return

Response: 200 OK

[
  {
    "id": "a1b2c3d4-5678-9abc-def0-123456789abc",
    "metricId": "system_runtime_working_set",
    "metricType": "memory",
    "metricValue": 128.45,
    "recordedAt": "2025-01-15T10:30:00Z"
  }
]

Error: 400 Bad Request if limit <= 0.

`POST /api/metrics`

Batch insert metric records.

Request Body: Array of metric records.

[
  {
    "metricId": "system_runtime_working_set",
    "metricType": "memory",
    "metricValue": 128.45,
    "recordedAt": "2025-01-15T10:30:00Z"
  }
]

Field	Type	Required	Description
`metricId`	`string`	yes	Metric identifier
`metricType`	`string`	yes	Type category (`memory`, `cpu`, etc.)
`metricValue`	`double`	yes	Numeric value
`recordedAt`	`datetime`	no	Timestamp (defaults to `DateTime.UtcNow`)

Response: 200 OK

{
  "inserted": 5
}

Error: 400 Bad Request if array is empty.

Health Check

`GET /health`

Application health check with memory and thread diagnostics.

Response: 200 OK

{
  "status": "OK",
  "memoryMb": 85,
  "threadCount": 24
}

Unhealthy Responses: 503 Service Unavailable

Condition	Response
Memory > 500 MB	`"Unhealthy: High memory usage {x} MB"`
Threads > 200	`"Unhealthy: Too many threads {x}"`

Also exposed via ASP.NET Health Checks middleware at the same path.

Stress Testing

`GET /stress/cpu`

Simulate CPU stress for testing anomaly detection.

Parameter	Type	Location	Default	Description
`seconds`	`int`	query	`10`	Duration in seconds

Response: 200 OK

CPU stressed for 10 seconds

Increments cpu_stress_requests_total Prometheus counter.

`GET /stress/memory`

Simulate a memory leak for testing anomaly detection.

Parameter	Type	Location	Default	Description
`mb`	`int`	query	`100`	Megabytes to leak

Response: 200 OK

Leaked 100 MB

Updates memory_allocated_mb Prometheus gauge. Memory is intentionally NOT garbage collected.

Prometheus Metrics

`GET /metrics`

Standard Prometheus metrics endpoint (exposed via prometheus-net).

Response: 200 OK with text/plain content type.

Includes:

http_request_duration_seconds — Request duration histogram
http_requests_received_total — Request counter by method/status
cpu_stress_requests_total — CPU stress call counter
memory_allocated_mb — Memory allocated via stress endpoint
dotnet_* — .NET runtime metrics
process_* — Process metrics (CPU, memory, threads)

Analyzer Endpoints

Status

`GET /status`

Get the last analysis result from the monitoring service.

Response: 200 OK

If no analysis has been performed yet:

"No verdict yet"

If an anomaly was detected:

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "failureType": "MemoryLeakSuspected",
  "severity": "Warning",
  "description": "Memory trending +25.3 MB/min (R²=0.87, n=20, 1.6min window)",
  "detectedAt": "2025-01-15T10:30:00Z",
  "resolved": false
}

Events

`GET /events`

Get the last 20 failure events detected by the analyzer.

Response: 200 OK

[
  {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "failureType": "MemoryLeakSuspected",
    "severity": "Warning",
    "description": "Memory trending +25.3 MB/min (R²=0.87, n=20, 1.6min window)",
    "detectedAt": "2025-01-15T10:30:00Z",
    "resolved": false
  }
]

Recoveries

`GET /recoveries`

Get the last 20 recovery results from the analyzer.

Response: 200 OK

[
  {
    "failureEventId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "actionType": "RestartPod",
    "status": "Success",
    "details": "Restarted pod taskapi-xyz",
    "performedAt": "2025-01-15T10:31:00Z"
  }
]

Learning Report

`GET /learning`

Get the full learning loop report — historical success/failure rates for each root cause and strategy combination.

Response: 200 OK

{
  "generatedAt": "2026-02-16T19:36:08Z",
  "totalPatterns": 2,
  "patterns": [
    {
      "rootCause": "MemoryLeakSuspected",
      "strategyName": "RestartWithBuffer",
      "successes": 3,
      "failures": 0,
      "successRate": 1.0,
      "totalAttempts": 3
    },
    {
      "rootCause": "HighCpuUsage",
      "strategyName": "ScaleUpForCpu",
      "successes": 2,
      "failures": 1,
      "successRate": 0.67,
      "totalAttempts": 3
    }
  ],
  "recentEvents": [
    {
      "timestamp": "2026-02-16T19:33:42Z",
      "rootCause": "MemoryLeakSuspected",
      "strategyName": "RestartWithBuffer",
      "success": true,
      "newSuccessRate": 1.0,
      "totalAttempts": 1
    }
  ]
}

Learning Recommendation

`GET /learning/{rootCause}`

Get the best-performing strategy recommendation for a specific root cause.

Path Parameter: rootCause — one of: MemoryLeakSuspected, HighCpuUsage, ResourceExhaustion, TrafficOverload, ApplicationError, DependencyFailure, HighErrorRate

Response: 200 OK

If data exists:

{
  "strategyName": "RestartWithBuffer",
  "successRate": 1.0,
  "totalAttempts": 3,
  "successes": 3,
  "failures": 0
}

If no data yet:

{
  "message": "No learning data yet for 'HighCpuUsage'"
}

Data Models

TaskItem

Field	Type	Description
`id`	`guid`	Unique identifier
`title`	`string`	Task title
`createdAt`	`datetime`	Creation timestamp (UTC)

FailureEvent

Field	Type	Description
`id`	`guid`	Unique identifier
`failureType`	`string`	Type (e.g. `MemoryLeakSuspected`)
`severity`	`string`	`Info`, `Warning`, `Critical`
`description`	`string`	Human-readable description
`detectedAt`	`datetime`	Detection timestamp (UTC)
`resolved`	`bool`	Resolution status
`resolvedAt`	`datetime?`	Resolution timestamp

MetricRecord

Field	Type	Description
`id`	`guid`	Unique identifier
`metricId`	`string`	Metric identifier
`metricType`	`string`	Type category
`metricValue`	`double`	Numeric value
`recordedAt`	`datetime`	Recording timestamp (UTC)

RecoveryAction

Field	Type	Description
`id`	`guid`	Unique identifier
`failureEventId`	`guid`	Associated failure event
`actionType`	`string`	`RestartPod`, `ScaleUp`
`targetDeployment`	`string`	Kubernetes deployment name
`status`	`string`	`Pending`, `InProgress`, `Success`, `Failed`
`details`	`string?`	Additional information
`performedAt`	`datetime`	Action start timestamp (UTC)
`completedAt`	`datetime?`	Completion timestamp

RecoveryResult (Analyzer internal)

Field	Type	Description
`failureEventId`	`guid`	Associated failure event
`actionType`	`string`	Type of recovery
`status`	`string`	Outcome status
`details`	`string?`	Additional information
`performedAt`	`datetime`	Action timestamp

MetricSample (Analyzer internal)

Field	Type	Description
`value`	`double`	Metric value in MB
`timestamp`	`datetime`	Sample timestamp

Anomaly Detection — Memory Leak Rule

The memory leak detector uses ordinary least-squares (OLS) linear regression over a sliding window of metric samples. It fires only when both slope and statistical confidence exceed their thresholds.

Algorithm

Given a window of n samples (x_i, y_i) where x = elapsed minutes, y = memory MB:

slope = (n·Σxy − Σx·Σy) / (n·Σx² − (Σx)²)
R²    = 1 − SS_res / SS_tot

The rule alerts when both conditions are met:

slope > 15 MB/min (memory is growing fast)
R² > 0.60 (the upward trend is statistically consistent, not noise)

Parameters

Parameter	Value	Description
`WindowSize`	20	Max samples in sliding window (~100s at 5s poll)
`MinSamplesForRegression`	10	Minimum samples before first evaluation
`SlopeMbPerMinute`	15	Slope threshold (MB/min)
`MinR2`	0.60	Minimum R² confidence
`CriticalSlopeMbPerMinute`	50	Slope for Critical severity upgrade
`HighConfidenceR2`	0.85	R² for Critical severity upgrade
`CooldownPeriod`	3 min	Minimum gap between alerts

Severity Classification

Condition	Severity
`slope > 50` AND `R² > 0.85`	`Critical`
`slope > 15` AND `R² > 0.60`	`Warning`
Otherwise	No alert

Example Description Format

Memory trending +18.7 MB/min (R²=0.74, n=20, 1.6min window)

This means: the OLS regression over 20 samples spanning 1.6 minutes computed a growth rate of 18.7 MB/min with 74% of variance explained by the linear trend.

Why R² Matters

R² ≈ 1.0 → Memory is rising in a straight line (true leak)
R² ≈ 0.5 → Noisy, some upward trend but inconsistent
R² < 0.3 → Random fluctuations, no real trend

The dual gate (slope + R²) means a single memory spike does NOT trigger an alert — only sustained, statistically significant growth does.

Error Handling

All endpoints follow standard HTTP status codes:

Status	Meaning
`200`	Success
`201`	Created (only `POST /api/tasks`)
`204`	No Content (only `DELETE /api/tasks/{id}`)
`400`	Bad Request — invalid input
`404`	Not Found — resource doesn't exist
`503`	Service Unavailable — health check failure

Error responses return a plain string or problem details JSON.

Internal Communication

The system has the following internal service communication flow:

┌──────────────┐     Prometheus Query API      ┌────────────────┐
│   Analyzer    │ ←─────────────────────────── │   Prometheus    │
│  (Port 8080)  │                               │  (Port 9090)   │
└──────┬───────┘                               └───────┬────────┘
       │                                               │
       │  POST /api/failureevents                      │ Scrape /metrics
       │  POST /api/metrics                            │ every 5s
       │  POST /api/recoveryactions                    │
       │  PATCH /api/recoveryactions/{id}/status       │
       ▼                                               ▼
┌──────────────┐                               ┌────────────────┐
│   TaskApi     │ ─── exposes /metrics ──────→ │   TaskApi Pod   │
│  (Port 8080)  │                               │  (in cluster)  │
└──────────────┘                               └────────────────┘
       │
       │  Kubernetes API
       │  (Scale, Restart, Delete pods)
       ▼
┌──────────────┐
│  Kubernetes   │
│  API Server   │
└──────────────┘

Analyzer → Prometheus

Query: GET /api/v1/query?query=process_working_set_bytes{job="taskapi"}
Interval: Every 5 seconds
Purpose: Fetch current memory usage of TaskApi pods

Analyzer → TaskApi

POST /api/metrics — Store collected metric samples
POST /api/failureevents — Persist detected anomalies
POST /api/recoveryactions — Record recovery actions taken
PATCH /api/recoveryactions/{id}/status — Update action status after execution

Analyzer → Kubernetes API

Scale: PATCH /apis/apps/v1/namespaces/default/deployments/taskapi/scale
Restart: PATCH /apis/apps/v1/namespaces/default/deployments/taskapi (annotation update)
Delete Pod: DELETE /api/v1/namespaces/default/pods/{name}
Auth: ServiceAccount analyzer-sa with RBAC role

Prometheus → TaskApi

Scrape: GET /metrics on taskapi.default.svc.cluster.local:80
Interval: Every 5 seconds

Quick Examples

Create a task

curl -X POST http://localhost:5186/api/tasks \
  -H "Content-Type: application/json" \
  -d '{"title": "Test task"}'

Trigger memory leak (for testing)

curl "http://localhost:5186/stress/memory?mb=200"

Check analyzer status

curl http://localhost:5297/status

Get failure events (unresolved only)

curl "http://localhost:5186/api/failureevents?resolved=false"

Get recovery actions for a failure

curl "http://localhost:5186/api/recoveryactions?failureEventId=<guid>"

Get metrics (last hour)

curl "http://localhost:5186/api/metrics?metricId=system_runtime_working_set&limit=100"

Health check

curl http://localhost:5186/health

FilesExpand file tree

API.md

Latest commit

History

API.md

File metadata and controls

Self-Healing System — API Documentation

Table of Contents

Base URLs

TaskApi Endpoints

Tasks

GET /api/tasks

POST /api/tasks

DELETE /api/tasks/{id}

Failure Events

GET /api/failureevents

POST /api/failureevents

PATCH /api/failureevents/{id}/resolve

Recovery Actions

GET /api/recoveryactions

POST /api/recoveryactions

PATCH /api/recoveryactions/{id}/status

Metrics

GET /api/metrics

POST /api/metrics

Health Check

GET /health

Stress Testing

GET /stress/cpu

GET /stress/memory

Prometheus Metrics

GET /metrics

Analyzer Endpoints

Status

GET /status

Events

GET /events

Recoveries

GET /recoveries

Learning Report

GET /learning

Learning Recommendation

GET /learning/{rootCause}

Data Models

TaskItem

FailureEvent

MetricRecord

RecoveryAction

RecoveryResult (Analyzer internal)

MetricSample (Analyzer internal)

Anomaly Detection — Memory Leak Rule

Algorithm

Parameters

Severity Classification

Example Description Format

Why R² Matters

Error Handling

Internal Communication

Analyzer → Prometheus

Analyzer → TaskApi

Analyzer → Kubernetes API

Prometheus → TaskApi

Quick Examples

Create a task

Trigger memory leak (for testing)

Check analyzer status

Get failure events (unresolved only)

Get recovery actions for a failure

Get metrics (last hour)

Health check

`GET /api/tasks`

`POST /api/tasks`

`DELETE /api/tasks/{id}`

`GET /api/failureevents`

`POST /api/failureevents`

`PATCH /api/failureevents/{id}/resolve`

`GET /api/recoveryactions`

`POST /api/recoveryactions`

`PATCH /api/recoveryactions/{id}/status`

`GET /api/metrics`

`POST /api/metrics`

`GET /health`

`GET /stress/cpu`

`GET /stress/memory`

`GET /metrics`

`GET /status`

`GET /events`

`GET /recoveries`

`GET /learning`

`GET /learning/{rootCause}`