Skip to content

Commit a622ff6

Browse files
ostermanclaude
osterman
and
claude
authored
feat(workflows): http step type (webhook alias) with retries (#2641)
* feat(workflows): add webhook step type with timeouts and retries Add a native `type: webhook` step for workflows and custom commands that performs an HTTP request (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS) with query-string parameters, headers, and a request body (raw or form/JSON). Requests get a per-attempt timeout and retries that compose with the existing `retry:` policy. Because extended/registry step types are not wrapped by the legacy retry.Do path, the handler applies retry itself via retry.WithPredicate with HTTP-aware classification: transport errors, 5xx, and 429 retry by default; other 4xx fail fast; retry.conditions regexes force additional cases. Success criteria are configurable via expect.status and expect.response (regex). The response body and status are captured as step value/metadata for downstream steps. Includes schema fields on WorkflowStep and Task, JSON manifest updates, docs, an example, a changelog blog post, and a roadmap milestone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(workflows): address review feedback on webhook step - Replace dynamic fmt.Errorf returns with static sentinel errors (ErrTemplateEvaluation / ErrWebhookRequestFailed) per the static-error contract. - Bound webhook response-body reads with io.LimitReader (4 MiB) to avoid memory blowups on large/error responses. - Reject relative URLs in buildWebhookRequest so they fail fast instead of being retried as transport errors. - Add OPTIONS to the Method field comment to match supported verbs. - Add mustGetWebhookHandler helper asserting registration; move ParseForm assertion out of the httptest handler goroutine. - Expand tests to cover error/helper paths, raising patch coverage above the 80% gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(workflows): rename webhook step type to http (webhook alias) The step is a general-purpose, verb-agnostic outbound HTTP client, not an inbound webhook receiver. Rename the canonical type to `http` and keep `webhook` as a working alias for the fire-a-notification use case. - Add first-class alias support to the step registry: NewBaseHandler is now variadic for aliases, Get() resolves aliases, and List/Count report only the canonical entry (no duplicate step). Aliases are exposed via an optional GetAliases() interface so the StepHandler interface (and its mock) are unchanged. - Rename webhook.go/test/e2e -> http*.go and all Webhook*/webhook* symbols to HTTP*/http*; register as "http" with "webhook" alias. - Rename WebhookExpect -> HTTPExpect (workflow.go, task.go) and the 7 ErrWebhook* sentinels -> ErrHTTPStep* in errors/errors.go. - JSON manifest: url-required conditional now matches type in [http, webhook]; descriptions updated to "http step type". - Rename example examples/workflow-webhook -> examples/http-webhooks (type: http, webhook noted as alias). - Docs/blog/roadmap: step-types reference, retry.conditions link, blog slug http-step-type, roadmap changelog slug; all note the webhook alias. - Add a contract test asserting the webhook alias resolves to the http handler and is not listed as a distinct step type. - Drive-by: extract a const for a repeated "secret %s" literal in azure_keyvault_store.go to satisfy the repo-wide lint hook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(workflows): make http transport error test deterministic --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 33f92ff commit a622ff6

18 files changed

Lines changed: 1974 additions & 9 deletions

File tree

‎errors/errors.go‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -657,6 +657,13 @@ var (
657657
ErrStepNoFilesFound = errors.New("no files found matching criteria")
658658
ErrStepFieldRequired = errors.New("required field missing for step")
659659
ErrStepTTYRequired = errors.New("interactive terminal required for step")
660+
ErrHTTPStepURLRequired = errors.New("url is required for http step")
661+
ErrHTTPStepInvalidMethod = errors.New("invalid HTTP method for http step")
662+
ErrHTTPStepBodyFormConflict = errors.New("http step cannot set both body and form")
663+
ErrHTTPStepInvalidExpectPattern = errors.New("invalid expect.response regex pattern for http step")
664+
ErrHTTPStepRequestFailed = errors.New("http request failed")
665+
ErrHTTPStepUnexpectedStatus = errors.New("http response did not match expected status")
666+
ErrHTTPStepUnexpectedResponse = errors.New("http response body did not match expected pattern")
660667
ErrWorkingDirNotFound = errors.New("working directory does not exist")
661668
ErrWorkingDirNotDirectory = errors.New("working directory path is not a directory")
662669
ErrWorkingDirAccessFailed = errors.New("failed to access working directory")

‎examples/http-webhooks/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# HTTP Step Type
2+
3+
This example demonstrates the `http` workflow step type, which performs an HTTP
4+
request with a configurable method/verb, query-string parameters, headers, and a request
5+
body (raw or form/JSON). Requests get per-attempt timeouts and retries that compose with
6+
the step's `retry:` policy.
7+
8+
> `webhook` is an accepted alias for `http` — `type: webhook` behaves identically and is
9+
> handy for the fire-a-notification use case.
10+
11+
## Workflows
12+
13+
- **`notify`** — `POST`s a JSON payload to `WEBHOOK_URL`, retrying transient failures
14+
(`5xx`, `429`, network errors) with exponential backoff, then prints the status code.
15+
- **`poll-health`** — `GET`s `HEALTH_URL` and retries until the response body matches a
16+
"healthy" pattern.
17+
18+
## Run It
19+
20+
Point the workflows at any reachable endpoint via environment variables:
21+
22+
```shell
23+
# Notify an endpoint (use your own URL, or a request-bin style service).
24+
WEBHOOK_URL=https://example.com/hook atmos workflow notify -f http
25+
26+
# Poll a health endpoint until it reports healthy.
27+
HEALTH_URL=https://example.com/healthz atmos workflow poll-health -f http
28+
```
29+
30+
## Key Fields
31+
32+
| Field | Description |
33+
|-----------------|-----------------------------------------------------------------------------|
34+
| `url` | Request URL (required, supports templates). |
35+
| `method` | HTTP verb: `GET` (default), `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`. |
36+
| `query` | Query-string parameters. |
37+
| `headers` | Request headers. |
38+
| `body` / `form` | Raw body, or key-value params (urlencoded, or JSON when `Content-Type` is JSON). Mutually exclusive. |
39+
| `expect.status` | Acceptable status codes (default: any `2xx`). |
40+
| `expect.response` | Regexes the response body must match (at least one). |
41+
| `timeout` | Per-attempt timeout (default `30s`). |
42+
| `retry` | Retry policy; transport errors / `5xx` / `429` retry by default. |
43+
44+
The response is available to later steps as `{{ .steps.<name>.value }}` (body) and
45+
`{{ .steps.<name>.metadata.status_code }}`.

‎examples/http-webhooks/atmos.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
workflows:
2+
base_path: "workflows"

‎examples/http-webhooks/workflows/http.yaml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
workflows:
2+
notify:
3+
description: |
4+
Calls an external HTTP endpoint with a JSON payload, retrying transient
5+
failures (5xx/429/network) with exponential backoff. Set WEBHOOK_URL to a
6+
reachable endpoint before running:
7+
8+
WEBHOOK_URL=https://example.com/hook atmos workflow notify -f http
9+
steps:
10+
- name: trigger
11+
# `webhook` is an accepted alias for `http`.
12+
type: http
13+
url: "{{ .env.WEBHOOK_URL }}"
14+
method: POST
15+
query:
16+
source: atmos
17+
headers:
18+
Content-Type: application/json
19+
body: '{"event":"deploy","status":"started"}'
20+
expect:
21+
status: [200, 201, 202, 204]
22+
timeout: 30s
23+
retry:
24+
max_attempts: 5
25+
backoff_strategy: exponential
26+
initial_delay: 1s
27+
max_delay: 30s
28+
29+
- name: report
30+
type: log
31+
level: info
32+
content: "Webhook returned HTTP {{ .steps.trigger.metadata.status_code }}"
33+
34+
poll-health:
35+
description: |
36+
Polls a health endpoint until it returns a healthy body, retrying until the
37+
response matches. Set HEALTH_URL before running.
38+
steps:
39+
- name: health
40+
type: http
41+
url: "{{ .env.HEALTH_URL }}"
42+
method: GET
43+
expect:
44+
status: [200]
45+
response:
46+
- /"status"\s*:\s*"(ok|healthy)"/
47+
timeout: 5s
48+
retry:
49+
max_attempts: 10
50+
backoff_strategy: constant
51+
initial_delay: 2s

‎pkg/datafetcher/schema/atmos/manifest/1.0.json‎

Lines changed: 100 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1398,6 +1398,13 @@
13981398
"max_elapsed_time": {
13991399
"type": "string",
14001400
"description": "Maximum total time for all retry attempts (e.g., '5m', '1h')"
1401+
},
1402+
"conditions": {
1403+
"type": "array",
1404+
"items": {
1405+
"type": "string"
1406+
},
1407+
"description": "Regular expressions; an error/response matching any pattern is treated as retryable"
14011408
}
14021409
},
14031410
"required": []
@@ -1426,10 +1433,101 @@
14261433
"type": "string"
14271434
},
14281435
"description": "Environment variables for the workflow step"
1436+
},
1437+
"timeout": {
1438+
"type": "string",
1439+
"description": "Step timeout duration (e.g., '30s', '1m'). For the http step type this is the per-attempt timeout."
1440+
},
1441+
"url": {
1442+
"type": "string",
1443+
"description": "Request URL for the http step type (supports templates)"
1444+
},
1445+
"method": {
1446+
"type": "string",
1447+
"enum": [
1448+
"GET",
1449+
"POST",
1450+
"PUT",
1451+
"PATCH",
1452+
"DELETE",
1453+
"HEAD",
1454+
"OPTIONS"
1455+
],
1456+
"description": "HTTP method/verb for the http step type (default: GET)"
1457+
},
1458+
"headers": {
1459+
"type": "object",
1460+
"additionalProperties": {
1461+
"type": "string"
1462+
},
1463+
"description": "HTTP request headers for the http step type (supports templates)"
1464+
},
1465+
"query": {
1466+
"type": "object",
1467+
"additionalProperties": {
1468+
"type": "string"
1469+
},
1470+
"description": "Query-string parameters for the http step type (supports templates)"
1471+
},
1472+
"body": {
1473+
"type": "string",
1474+
"description": "Raw request body for the http step type (supports templates); mutually exclusive with form"
1475+
},
1476+
"form": {
1477+
"type": "object",
1478+
"additionalProperties": {
1479+
"type": "string"
1480+
},
1481+
"description": "Form/JSON body parameters for the http step type; mutually exclusive with body"
1482+
},
1483+
"expect": {
1484+
"type": "object",
1485+
"additionalProperties": false,
1486+
"description": "Success criteria for the http step type; defaults to any 2xx response",
1487+
"properties": {
1488+
"status": {
1489+
"type": "array",
1490+
"items": {
1491+
"type": "integer"
1492+
},
1493+
"description": "Acceptable HTTP status codes"
1494+
},
1495+
"response": {
1496+
"type": "array",
1497+
"items": {
1498+
"type": "string"
1499+
},
1500+
"description": "Regular expressions; the response body must match at least one (/.../ literals or bare regex strings)"
1501+
}
1502+
}
14291503
}
14301504
},
1431-
"required": [
1432-
"command"
1505+
"allOf": [
1506+
{
1507+
"if": {
1508+
"required": [
1509+
"type"
1510+
],
1511+
"properties": {
1512+
"type": {
1513+
"enum": [
1514+
"http",
1515+
"webhook"
1516+
]
1517+
}
1518+
}
1519+
},
1520+
"then": {
1521+
"required": [
1522+
"url"
1523+
]
1524+
},
1525+
"else": {
1526+
"required": [
1527+
"command"
1528+
]
1529+
}
1530+
}
14331531
]
14341532
}
14351533
}

‎pkg/runner/step/handler_base.go‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,16 +15,19 @@ type BaseHandler struct {
1515
name string
1616
category StepCategory
1717
requiresTTY bool
18+
aliases []string
1819
}
1920

20-
// NewBaseHandler creates a new BaseHandler.
21-
func NewBaseHandler(name string, category StepCategory, requiresTTY bool) BaseHandler {
21+
// NewBaseHandler creates a new BaseHandler. Optional aliases are alternate type
22+
// names that resolve to the same handler (e.g. "webhook" as an alias for "http").
23+
func NewBaseHandler(name string, category StepCategory, requiresTTY bool, aliases ...string) BaseHandler {
2224
defer perf.Track(nil, "step.NewBaseHandler")()
2325

2426
return BaseHandler{
2527
name: name,
2628
category: category,
2729
requiresTTY: requiresTTY,
30+
aliases: aliases,
2831
}
2932
}
3033

@@ -35,6 +38,13 @@ func (h BaseHandler) GetName() string {
3538
return h.name
3639
}
3740

41+
// GetAliases returns the alternate type names that resolve to this handler.
42+
func (h BaseHandler) GetAliases() []string {
43+
defer perf.Track(nil, "step.BaseHandler.GetAliases")()
44+
45+
return h.aliases
46+
}
47+
3848
// GetCategory returns the step category.
3949
func (h BaseHandler) GetCategory() StepCategory {
4050
defer perf.Track(nil, "step.BaseHandler.GetCategory")()

0 commit comments

Comments
 (0)