Workflows and schedules

A Hub can do work on its own: on a clock, when a webhook arrives, or when an operator starts it. Hubzoid gives you two ways to write that work. Both run on the same durable engine inside the hub process, share one run history and one set of controls, and use the same Hub knowledge, tools and access rules as chat.

Two ways to define recurring workCopy Anchor Link

A markdown task is one file in schedule/. Its frontmatter says when it runs and which paths it may change. Its body is plain instructions that the hub's own agent carries out unattended, or it names a command to run instead.

A code workflow is a Python function in workflows/<name>/ decorated with @workflow. You write the exact steps, loops and thresholds, and call the model only where judgment helps.

Start with a markdown task when you can describe the job in a paragraph. Move to a code workflow when the steps must be exact, when a threshold or loop decides what happens, or when you call your own APIs. The packaged Watchtower sample (hubzoid init my-watchtower --template watchtower) is a complete code workflow to copy from.

One engineCopy Anchor Link

Every scheduled run executes on the hub's embedded DBOS (opens in a new tab) engine. There is no separate job server. The engine keeps its run history in a per-hub SQLite file, .hubzoid/dbos.db, or in PostgreSQL when HUBZOID_DBOS_DB or a PostgreSQL DATABASE_URL is set. Workflow state (hub.state and hub.shared_state), pauses and the access audit live in the deployment's operational database, so every bridge on a shared database sees the same pauses.

Because both kinds share the engine, they share:

• one run history, shown by hubzoid schedule status and each agent's Runs & schedules tab in the Console
• the same hubzoid schedule pause, resume and cancel commands
• the same hold while hubzoid backup copies the deployment

The engine starts with the hub process whenever there is scheduled work. Markdown tasks and scheduled evals start whenever their files exist. Code workflows fire on schedule only where the deployment is marked with HUBZOID_SCHEDULES=1, or automatically under hubzoid gateway, so a copy of the hub on a laptop does not fire alongside the server.

Runs and stepsCopy Anchor Link

A run is one execution of a task or workflow. It has an id, a status, a result and a list of steps. A step is a unit of work whose result the engine saves before the run moves on. When a process restarts, a run resumes after its last completed step.

1. Trigger fires
2. Run queued
3. Steps checkpointed
4. Result recorded
5. Shown in Console

Every run first records the account it acts as, in a step named hz_md_identity for a markdown task or hz_run_identity for a code workflow. A resumed run reuses that record, so it never switches account midway. A markdown run then has up to four steps:

1. work: the agent rounds, or the run: command
2. commit: only when the work finished done and the task declares commit: paths. It commits those paths alone and makes no commit when nothing changed
3. push: only when the task sets push: true and the commit step made a commit. It runs git pull --rebase, then git push
4. finish: records the result and archives the webhook events the run handled

A code workflow's steps are your @step functions and each call to hub.call_llm, hub.call_agent, hub.call_jev, hub.publish_artifact or hub.send_email. Code outside a step runs again when a run resumes, so keep side effects inside steps.

Run ids tell you where a run came from:

Because a scheduled id is derived from its slot, the same slot is never queued twice, even when two scheduler ticks or two processes see it due at once.

TriggersCopy Anchor Link

A markdown task has exactly one trigger: schedule: or on_webhook:, never both.

ConcurrencyCopy Anchor Link

• Markdown tasks run one at a time per hub, across processes. Scheduled evals share this queue. This keeps the git commits and pushes of different tasks from overlapping.
• A due markdown task waits while the hub is answering chat. The scheduler dispatches it on a later tick once no chat request is in flight. The gate applies at start only. A running task continues alongside new chat requests.
• A webhook task has at most one run queued or running. Events that arrive meanwhile wait for the next run.
• Code workflows run one at a time per workflow. A slot that comes due while the previous run is still going waits in the queue behind it. Different workflows run side by side.
• A hub-wide cap is optional. Set max_concurrent_workflows: N in workflows/settings.yaml to limit how many code workflows run at once. The engine reads it at start. There is no cap by default.
• Markdown tasks and code workflows use separate queues, so one of each can run at the same time.

Delivery semanticsCopy Anchor Link

Code workflows run at least onceCopy Anchor Link

• A completed step is not run again when a run resumes. Its saved result is used instead.
• A step that was running when the process stopped runs again. Side effects inside a step can therefore happen twice.
• A step is tried once unless you set @step(max_attempts=N).
• hub.call_llm has no side effects and is retried once. hub.call_agent runs tools, so it is retried only when workflows/settings.yaml sets agent_max_attempts.
• hub.publish_artifact and hub.send_email are recorded steps. A resumed run gets back the artifact it already published and never sends an accepted email again. A send interrupted mid-transfer is reported as ambiguous and never resent automatically.
• A hub.call_jev call still in flight when the process stopped is made again on resume, so it can be billed twice.
• A write to hub.state commits immediately and is not part of the checkpoint. A read, modify and write counter such as hub.state["n"] = hub.state.get("n", 0) + 1 can double count when a run is replayed.

Make every step that changes something outside the hub safe to repeat:

• Use a key the other system deduplicates on, such as a record id or the run's slot, instead of creating a new record each time.
• Check before you write: does the ticket for this alert already exist?
• Record what you finished as a per-item marker, for example hub.state[f"done:{item_id}"] = True, and skip marked items.

Markdown tasks never repeat their workCopy Anchor Link

• The work step is never repeated. If the process stops mid-work, the recovered run fails with an error that starts interrupted by a restart. Progress carries forward through the task's state file.
• A run that finished its work but stopped before commit or push resumes from that step.
• The push step is retried once. A rebase conflict fails the run and leaves the commit local for a person to resolve.
• Webhook events are archived only when a run finishes DONE. A failed or incomplete run leaves them pending, and the task runs again once that run has ended. Handling is at least once, so keep a record of handled events in the state file.

Missed runs and catch-upCopy Anchor Link

The two kinds treat time they could not run differently.

A markdown task computes its next run from its last fire (or from when it was first seen). That is why downtime produces one catch-up run instead of one run per missed slot. The count of slots the catch-up stood in for is kept for 31 days in the task's missed_log in .hubzoid/schedule-state.json. Code workflow misses are kept for 31 days in the hub's runtime health. Each agent's Runs & schedules tab shows missed code workflow runs beside the scheduler's last dispatch, and the Console's summary API (/portal/api/summary) adds both kinds into a missed-slots figure per agent for the chosen period.

When workflow code changesCopy Anchor Link

Runs belong to the code that started them. The code version is a hash of the installed Hubzoid version and every workflows/**/*.py file in the hub. The engine resumes an interrupted run only under the same version.

When the hub starts under a new version, runs that were queued or interrupted under the previous code are cancelled, so they cannot hold a one-at-a-time queue. They stay in the run list as CANCELLED. A markdown run that was queued but had not started is queued again under the new code first, with :requeued added to its id, so its slot is not lost.

Editing a markdown task file, workflows/settings.yaml, knowledge or configuration does not change the version. Code a workflow imports from outside workflows/ is not part of the hash. Upgrading Hubzoid does change it. Before you deploy a change, drain queued runs as described in Operating runs.

Edits reach runs at different times:

• The scheduler re-reads schedule/*.md on every tick, so markdown task edits apply without a restart.
• workflows/settings.yaml is read at the start of each code workflow run.
• Code workflow modules load when the hub starts. Restart the hub to pick up a code change.

How runs appear in the ConsoleCopy Anchor Link

Runs and schedules live inside each agent. An agent's Runs & schedules tab lists its code workflows and markdown tasks (named md:<task>) with their schedule, state, Runs as and next run. Open a workflow to list its runs, and open a run to see its timing, its steps and the account it ran as.

A run's result belongs to the account it acted as. The hub's managers see each run's workflow, status, timing and a failure summary. The result, step outputs and detailed errors are shown only to that account, except for a legacy service run, which acts for no person. hubzoid schedule status on the server shows everything. Existing links to the cross-agent run list from earlier releases still open it. Run controls stay on the server command line, and the Console records every pause, resume and cancel in Activity.

See Console runs for the screens and Operating runs for the commands.

Who a run acts asCopy Anchor Link

Every scheduled run acts as an ordinary Hubzoid account: a person, or an account your team creates for shared automation, such as reports@company.com. There is no separate service account type. The first match wins:

1. run_as on the declaration: @workflow(..., run_as="priya@company.com"), or run_as: in a markdown task's frontmatter.
2. HUBZOID_WORKFLOW_USER in the hub's configuration: a hub secret, else <hub>/.env.
3. HUBZOID_WORKFLOW_USER in the deployment's configuration: the deployment secret or the gateway's environment.
4. On a hub whose access is managed in the Console, the owner recorded at setup. That is the configured initial owner (HUBZOID_GATEWAY_ADMIN_EMAIL or WEBUI_ADMIN_EMAIL), recorded the first time it signs in. A local quickstart with sign-in off uses admin@localhost.

hubzoid schedule list and the Runs as column in the Console use this same resolution and say where the account came from. run_as chooses whose permissions a run uses and grants nothing. It is read only from files and operator configuration, never from an API, a tool or a model.

The account must be usable: signed in at least once, not awaiting approval, blocked or replaced, and on a Console-managed hub it must hold Use this agent. Otherwise the run fails with an error that names the fix. Hubzoid never falls back to another account. The account is checked again before every model or agent call, tool call, publish, email and connection, so a run whose account is blocked stops at its next call.

What the run touches belongs to that account:

State written before 1.0.1, in hub.state or a markdown task's .hubzoid/schedule/<task>/ folder, is kept as it was and belongs to no one, so a person's state starts empty. To carry a markdown task's progress over, copy its old state.json into the person's folder before their first run. Keep state that must survive a change of account in hub.shared_state.

Before 1.0.1, a code workflow acted as the service identity workflow:<name> and a markdown task as workflow:md:<task>. Those identities are legacy. Their grants stay, labelled Legacy service identity in the Console, but a run that acts as an account does not use them. When the account lacks a permission the old identity held, the run log and the server log name it, so you can grant it to the account. New workflow:* identities cannot be added in the Console. A legacy hub, whose access is still managed in the chat app, switches only when you set run_as or HUBZOID_WORKFLOW_USER. Until then its runs keep their service identity, state and scratch folder, with a warning, and cannot publish, send email or use personal connections. See Identity and access.

Next stepsCopy Anchor Link

Read this page as Markdown