Webhooks

Comparisons take constant time, and query parameters are never credentials. Other signing schemes are not checked, and a body that is a JSON array is refused, so set such a sender to post a JSON object with the secret header.

A mistake in workflows/settings.yaml, such as invalid YAML or an unknown verify, turns workflows off for that Hub, with the error in workflow health. Chat and other Hubs keep running, and the Hub's webhook addresses answer 503 until the file is fixed and the Hub restarts.

1. event_key, a template filled from fields of the body, such as {ticket.id}:{ticket.updated_at}.
2. Without it, or when a field is missing, the first delivery id header: X-GitHub-Delivery, X-Delivery-Id, X-Webhook-Id, Idempotency-Key or X-Request-Id.
3. Otherwise a hash of the body, with a warning in the log.

A repeat with the same key and body attaches to the stored event, even after it finished, and starts nothing. Put what changes with each occurrence in the body and in event_key, such as the time the change happened.

The run reads its event as hub.event, with id, key, body, safe headers (the content type, delivery ids and X-GitHub-Event), webhook, received_at and attempt. It is None for a manual or scheduled run of the same workflow, so handle that case if the workflow has other triggers.

• Retries. Hubzoid retries a failed event itself: three attempts by default, after 60 and 300 seconds (webhook_retry_delays: [60, 300] in workflows/settings.yaml, at most 9 delays). Then the event is marked failed and an alert is raised.
• Redrive. After checking what the event's earlier attempts already did, retry it with hubzoid schedule redrive <event-id> --hub my-hub. hubzoid schedule deliveries my-hub lists events and alert deliveries without their payloads. Cancelling a webhook run fails its event, which then waits for a redrive.
• Code changes. An event that never started runs on the current code, found by its webhook name. An event whose attempt had started under the old code is failed visibly for a redrive, never replayed silently on changed code.
• Concurrency. concurrency=N limits how many runs of the workflow go at once, and concurrency_key="ticket.id" runs events with the same value one at a time. max_executor_threads (32 by default) in workflows/settings.yaml bounds the Hub's worker threads. A backlog on one webhook workflow does not hold up another.
• Deadline. A webhook run has a 15-minute deadline unless timeout= or workflow_timeout says otherwise.
• Pausing. Failed webhook runs never pause the workflow and do not count toward pausing its schedule.
• At least once. A step that changes something outside the Hub can run again after a crash. Use hub.event.key as the idempotency key in the other system, or check its current state before writing.

In Settings, a webhook with failures expands to show each failed event with Copy redrive command. Settings never shows payloads, headers or digests, and has no redrive button.

Event and digest records do not expire, so a sender's late retries stay deduplicated. Include the operational store in backups.

Comparisons take constant time. In shared secret mode a ?token=<secret> query parameter is also accepted, but query strings end up in access logs and in the stored event, so send the secret in a header.

For GitHub, set WEBHOOK_INBOUND_HMAC=true. In the repository's webhook settings, use the endpoint as the payload URL, choose the application/json content type and enter the same secret as WEBHOOK_INBOUND_SECRET.

To test HMAC mode, sign the body yourself:

BODY='{"alert": "disk_full", "host": "store-2", "severity": "critical"}'
SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$WEBHOOK_INBOUND_SECRET" | sed 's/^.* //')
curl -X POST http://127.0.0.1:3080/webhooks/my-hub/alerting \
  -H "X-Signature-256: sha256=$SIG" \
  -H "Content-Type: application/json" \
  -d "$BODY"

A body that is not JSON is kept as text, never rejected.

• When the request carries a delivery id, the first of X-GitHub-Delivery, X-Delivery-Id, X-Webhook-Id, Idempotency-Key or X-Request-Id, that id is remembered and a repeat is answered 200 duplicate.
• Without an id header, a body identical to one stored in the current or previous 10-minute window counts as a repeat. Identical events far apart are both accepted.
• A delivery is marked as seen only after it is stored, so a crash while storing leaves no mark and the provider's retry is accepted.

Seen markers live in .inbound/dedup/ inside the Hub.

Files are named so that name order is arrival order, and written in one go so a task never reads half an event. Request headers are used for checking and deduplication and are not written to the file. When the event type travels in a header, give each sender its own query parameter in the address, which is stored under query.

• A webhook task is due while .json files wait in its inbox and no run of it is queued or running. The scheduler checks every 30 seconds, and a due task waits while the Hub is answering chat.
• When it queues a run, the scheduler claims every waiting file. An agent task finds them listed in every round's prompt. A run: script finds them in HUBZOID_WEBHOOK_EVENTS, one path per line, and the run's account in HUBZOID_RUN_AS.
• When the run finishes done, Hubzoid moves the claimed files into .processed/. A run that ends error or incomplete leaves them, so the task runs again. Handling is at least once, so record what you handled in the state file.
• Events that arrive during a run wait for the next run and are never dropped.
• A manual hubzoid schedule run claims nothing.

Each Hub's inbox server runs and restarts with its Hub, on a loopback port chosen automatically. A HUBZOID_INBOUND_PORT in a Hub's .env is still honoured. Set HUBZOID_HUB_SLUG only when two Hubs share a folder name. See running several Hubs.