Doctor checks

Every check hubzoid doctor runs, with its statuses and the fix for each failure. Doctor checks a Hub and the deployment around it: files, configuration layers and secrets, the agent build, schedules and workflows, database schemas, required evals, sign-in, the chat UI, the deployment key, network exposure, model credentials, backups, scheduled work, who may use the agent, the Hub's own settings and stored secrets. It only reads. It never creates a database, runs a migration or starts the workflow engine, so it is safe to run against a live deployment. It reports setting names and where they come from, never their values.

hubzoid doctor my-hub

hubzoid doctor my-hub --json
hubzoid doctor my-hub --skip-secret-fetch

Doctor loads the Hub's .env into its own process before it checks anything, the same way the Hub does when it starts. Run it with the environment the Hub runs with, for example as the service user on the server, so checks such as auth.bridge_keys see the real values. When the configuration names AWS Secrets Manager secrets, doctor reads each one once to prove it is reachable. --skip-secret-fetch lists them without calling AWS, and the other checks then use the files alone.

OutputCopy Anchor Link

Each check has a status:

The text output prints one line per check with its id in brackets, and lists the details under any warn or fail. Under config.layers it lists each configured key with its layer and source. This is a real run against the starter from hubzoid init my-hub, after its first start and one scheduled task added:

✓ AGENTS.md present (hub.agents_md)
✓ .env present (hub.env)
· 2 configured key(s): 2 from the hub (config.layers)
    BRIDGE_API_KEYS: hub (~/agents/my-hub/.env)
    MODEL: hub (~/agents/my-hub/.env)
✓ Agent builds: 'my-hub' via ClaudeRuntime (runtime.build)
✓ 1 enabled task(s) (schedule.tasks)
· hubzoid <version> (deps.versions)
✓ SQLite 3.53.4 (deps.sqlite)
✓ Schema at op_0081 (db.operational)
✓ Schema at hub_0001 (db.hub)
✓ 1 bridge key(s) set (auth.bridge_keys)
· Sign-in is off: local mode, whoever opens the web app is the hub's owner (admin@localhost). Fine on this machine. (auth.chat_signin)
· Chat UI: Hubzoid web app (ui.mode)
✓ Local mode: the web app stays on 127.0.0.1 (exposure.local_mode)
· No deployment key yet; one is created in ~/agents/my-hub/.hubzoid/secret.key when first needed. Back it up apart from the data. (deployment.key)
✓ The public port listens on 127.0.0.1; bridges stay on 127.0.0.1 (exposure.bind)
· claude-local: uses the local `claude` login (check it with `claude /status`) (model.credentials)
✓ Last backup 0.0 days ago. Scheduled backups run weekly. (backup.age)
✓ Scheduled work is not held or paused (scheduler.health)
· Only the owner (admin@localhost) may use my-hub so far. Add people in Settings, on the agent's Access tab. (access.who)

Before a Hub's first start, db.operational and db.hub say they are created at first start, and the checks that read the operational store (backup.age, scheduler.health, access.who) do not appear yet.

Exit codes: 0 when no check fails, 1 when any check fails or a required eval is pending, 2 when the Hub folder does not exist. Warnings do not change the exit code.

JSON formatCopy Anchor Link

--json prints one object. Check ids are only ever added, never renamed, so scripts and monitoring can key on them. This is the same kind of run, shortened to five checks:

{
  "format": 1,
  "hub": "~/agents/my-hub",
  "hubzoid": "<version>",
  "ok": true,
  "checks": [
    {
      "id": "hub.agents_md",
      "status": "ok",
      "summary": "AGENTS.md present",
      "detail": null
    },
    {
      "id": "config.layers",
      "status": "info",
      "summary": "2 configured key(s): 2 from the hub",
      "detail": [
        { "key": "BRIDGE_API_KEYS", "layer": "hub", "source": "~/agents/my-hub/.env", "shadows": [] },
        { "key": "MODEL", "layer": "hub", "source": "~/agents/my-hub/.env", "shadows": [] }
      ]
    },
    {
      "id": "auth.chat_signin",
      "status": "info",
      "summary": "Sign-in is off: local mode, whoever opens the web app is the hub's owner (admin@localhost). Fine on this machine.",
      "detail": { "mode": "local" }
    },
    {
      "id": "ui.mode",
      "status": "info",
      "summary": "Chat UI: Hubzoid web app",
      "detail": { "ui_mode": "hubzoid", "auth": false, "public_url": "", "allowed_origins": [], "source": "default" }
    },
    {
      "id": "deployment.key",
      "status": "info",
      "summary": "No deployment key yet; one is created in ~/agents/my-hub/.hubzoid/secret.key when first needed. Back it up apart from the data.",
      "detail": { "source": "file", "path": "~/agents/my-hub/.hubzoid/secret.key" }
    }
  ]
}

When the Hub folder is missing, the output holds a single hub.dir check with status fail.

A monitoring job can alert on failures only:

hubzoid doctor ~/agents/my-hub --json | jq -r '.checks[] | select(.status == "fail") | .id'

ChecksCopy Anchor Link

Checks run in the order below. Some appear only when the thing they check exists, as noted.

hub.agents_mdCopy Anchor Link

Whether AGENTS.md exists at the Hub root. Fail: create it, or run hubzoid init to scaffold a Hub. Nothing else works without it.

hub.envCopy Anchor Link

Whether the Hub has a .env. Info when it does not, which is normal when settings come from the process environment (a systemd unit or a container).

secrets.deployment, secrets.hub and secrets.restrictedCopy Anchor Link

One check for each AWS Secrets Manager secret the configuration names. See configuration secrets.

• Ok when the secret can be read, with the number of keys it holds. The detail names the secret, its region and where the name was set, never a value.
• Info with --skip-secret-fetch, which lists the secret without reading it.
• Fail when it cannot be read. The summary names the secret, the layer and the AWS error class. Check the name, the region and the credentials of the account that runs the Hub. The other checks then use the files alone.

secrets.namesCopy Anchor Link

Warn when a secret name is set where Hubzoid ignores it: AWS_SECRET_NAME in the .env of a Hub that runs with others, HUBZOID_HUB_SECRET_NAME anywhere but the Hub's .env, or HUBZOID_RESTRICTED_SECRET_NAME anywhere but restricted/.env. Move the name to the place the message gives.

config.layersCopy Anchor Link

Info listing each key a file or secret sets, with the layer it comes from (hub, hub secret, restricted, restricted secret or deployment secret), its source and any lower layer it overrides (shadows). Names only, never values. Use it to see which value wins.

runtime.buildCopy Anchor Link

Builds the agent exactly as the bridge would, and reports the agent's name and the runtime it chose: OpenAIAgentsRuntime, ClaudeRuntime or CodexRuntime. Fail shows the error. Common causes:

• Invalid frontmatter in AGENTS.md or a sub-agent, or a file with no body.
• A model id whose provider key is missing, for example MODEL is set to 'openrouter/...' but OPENROUTER_API_KEY is not.
• A skill file that cannot be parsed, or invalid JSON in connectors/.mcp.json.

Fix the file named in the message, then confirm with hubzoid test.

runtime.codex_loginCopy Anchor Link

Appears when the agent builds on codex-local. Ok when a supported Codex CLI (0.163.0 or 0.147.0) is installed and its file-backed login works. Fail otherwise: install one of those versions, or point HUBZOID_CODEX_BIN at one, and run codex -c 'cli_auth_credentials_store="file"' login as the account that runs the Hub. The check reads the login status only and makes no model call. See agents and models.

schedule.tasksCopy Anchor Link

Parses every file in schedule/. Ok reports the number of enabled tasks and lists all task names. Fail lists each invalid file with the reason, for example a missing trigger (schedule: or on_webhook:), both triggers at once, a cron that never matches, an empty body on an agent task, push: true without commit:, a limit that is not a positive integer, an invalid run_as or alert_to, or publish_artifacts or send_email on a run: script task. Invalid tasks are skipped at runtime while valid ones keep firing. hubzoid schedule list shows the same problems. See markdown tasks.

workflows.definitionsCopy Anchor Link

Reads the code workflows under workflows/ without running them. Ok lists each one with its schedule (or manual) and timezone. Fail lists <file>: <error> for each workflow that does not load. While a workflow module is broken, code workflows are off for that start, and markdown tasks keep running. See code workflows.

access.restrictedCopy Anchor Link

Imports restricted/*.py. Info lists the permissions the restricted tools require. Fail means a file does not import. Fix the Python error it names. See restrict tools.

identity.resolverCopy Anchor Link

Loads the Hub's own sender lookup, identity/access.py. Info confirms it is present. Fail means it raised an error while loading. identity/access.csv is no longer read: record phone numbers on people in Settings instead. See Hub files.

deps.versionsCopy Anchor Link

Info with the installed versions of hubzoid, dbos, litellm, openai-agents, claude-agent-sdk, alembic, sqlalchemy, casbin, fastmcp, pwdlib and authlib. A missing package shows null. Include this detail when you report a problem.

data.dirCopy Anchor Link

Appears when HUBZOID_DATA_DIR keeps the Hub's state apart from its folder, as on a hosted Hub whose files come from an image.

• Info names where state lives. The Hub's own files are read-only in Settings: change them in Git and redeploy.
• Warn when the Hub folder still holds an older .hubzoid/hub.db. Stop the Hub, move that folder into the data folder or remove it, then start it, so only one copy is used.
• Fail when the data folder is not a folder this process can write. Point it at a durable volume such as /data.

deps.sqliteCopy Anchor Link

Appears when the workflow engine's database is SQLite. Ok reports the SQLite version. Fail means Python 3.12 or newer is linked against SQLite older than 3.42, which the workflow engine cannot use, so scheduled tasks and workflows cannot run. Use a Python build with a newer SQLite (python.org, uv, Homebrew, Debian 13 or Ubuntu 24.04) or PostgreSQL. The Docker image already ships a newer SQLite.

db.operational and db.hubCopy Anchor Link

The schema version of the operational store (accounts, sessions, conversations, access, audit and usage) and of the Hub's own database. On a single Hub with SQLite both live in .hubzoid/hub.db.

See upgrading.

evals.quality and required evalsCopy Anchor Link

Appear when the Hub requires evals in quality/manifest.yaml (required: case ids or globs, required_tags:). One eval.<case> row per required case:

• Ok when its latest result passed for the case and the Hub as they are now.
• Fail when its latest result failed, or the case file cannot be read.
• Pending when it never ran, ran without its judge, or the case or the Hub's instructions, tools or configuration changed since. Rerun it with hubzoid eval run. Doctor is not ok until it passes.

evals.quality fails when the manifest itself is invalid or the results cannot be read. See evals.

auth.bridge_keysCopy Anchor Link

Checks BRIDGE_API_KEYS.

• Fail when it is unset or includes dev. dev is a public default, so anyone who reaches the bridge could use it. Set one or more random keys, for example from openssl rand -hex 32. hubzoid init writes a random key into a new Hub's .env.
• Warn when a key is shorter than 16 characters.

auth.chat_signinCopy Anchor Link

Checks sign-in for the chat UI the Hub runs.

Sign-in is on with HUBZOID_AUTH=true, or as the deployment recorded it:

• Info when sign-in is off. This is local mode: whoever opens the web app is the Hub's owner (admin@localhost), which is fine on your own machine. exposure.local_mode checks that the port stays local.
• Ok when sign-in is on and Hubzoid accounts exist, or a first administrator is configured.
• Warn when sign-in is on but there is no account yet and no first administrator configured. Set HUBZOID_ADMIN_EMAIL and HUBZOID_ADMIN_PASSWORD for the first start, or create one with hubzoid admin create. Also warn when the accounts cannot be read. See db.operational.

See authentication.

ui.modeCopy Anchor Link

Info naming the chat UI, the Hubzoid web app. The detail gives whether sign-in is on, the public URL and the allowed origins.

ui.retired_setting and ui.openwebui_dataCopy Anchor Link

For deployments upgraded from the older Open WebUI chat.

• ui.retired_setting names an Open WebUI setting Hubzoid no longer uses, with what to do instead. Fail when the setting stops hubzoid run, warn otherwise. Remove it.
• ui.openwebui_data finds an Open WebUI install whose people are not in Hubzoid yet. Info with sign-in off: its chats can be imported with hubzoid migrate openwebui. Fail with sign-in on: hubzoid run stops until you run the migration. Warn or fail when the Hubzoid accounts cannot be read. See upgrading.

exposure.local_modeCopy Anchor Link

Appears when sign-in is off. Ok when HUBZOID_HOST is a loopback address, so the web app stays on this machine. Fail when it is not: hubzoid run refuses to start. Turn sign-in on with HUBZOID_AUTH=true, or bind 127.0.0.1. Warn when HUBZOID_ALLOW_UNAUTHENTICATED_NETWORK=true accepts that risk: anyone who can reach the port acts as the Hub's owner.

deployment.keyCopy Anchor Link

The key that encrypts stored credentials, such as personal connection tokens and connector secrets, and signs identity assertions between Hubzoid processes. It is reported by fingerprint only and never created by doctor. It comes from HUBZOID_SECRET_KEY, or from a secret.key file next to the manifest of a deployment of several Hubs, or .hubzoid/secret.key for a single Hub.

• Ok with the source and fingerprint.
• Info when no key exists yet. One is created when first needed. Back it up apart from the data.
• Warn when the key file is readable by other accounts. Run chmod 600 on it.
• Fail when HUBZOID_SECRET_KEY is malformed or the key file cannot be read. Without the key, encrypted credentials such as personal connection tokens cannot be read.

auth.google_mergeCopy Anchor Link

Appears when GOOGLE_CLIENT_ID is set. Ok when OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true, so Google sign-in attaches to an account created in Settings or with hubzoid admin. Warn otherwise, because people with such an account cannot then sign in with Google. Set OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true.

exposure.bindCopy Anchor Link

Where the public port listens. Warn when HUBZOID_HOST is 0.0.0.0, :: or empty: put TLS in front of the port, for example with a reverse proxy. Ok otherwise. The detail confirms that bridges stay on 127.0.0.1.

tools.legacy_management_flagCopy Anchor Link

Warn when HUBZOID_MANAGEMENT_TOOLS=true. The setting is deprecated: it gives every manager the access tools without a grant. Grant Manage access from chat (access_tools) in Settings to the managers who use them, then remove the setting.

mcp.access_groupCopy Anchor Link

Warn when MCP_ACCESS_GROUP is set. It is no longer read: who may use an agent over MCP is decided by the agent's Use this agent grant in Settings. Remove the setting.

model.credentialsCopy Anchor Link

Whether the Hub's model has credentials in the environment.

• For claude-local: ok when ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN is set, info when the claude CLI is on the PATH (check its login with claude /status), fail when there is neither. On a server, set CLAUDE_CODE_OAUTH_TOKEN. The optional backup credentials CLAUDE_CODE_OAUTH_TOKEN_SECONDARY and CLAUDE_CODE_OAUTH_TOKEN_TERTIARY are not checked.
• For any other model: ok when LiteLLM finds the provider's keys, fail listing the missing variables, warn when the check itself could not run.
• For codex-local, this check reports ok without testing anything. runtime.codex_login checks the Codex CLI and its login.

See agents and models.

backup.ageCopy Anchor Link

When the last backup of this deployment completed, and how often scheduled backups run (HUBZOID_BACKUPS, weekly by default).

• Ok with the age of the last backup.
• Warn when no backup is recorded, the last one is more than 7 days old, or the last attempt failed. The message gives the error. Fix the cause, then run hubzoid backup.
• Fail when HUBZOID_BACKUPS is not weekly, daily or off.

See backup and restore.

scheduler.healthCopy Anchor Link

The state of scheduled work, checked in this order:

See operating runs.

access.whoCopy Anchor Link

Who may use this agent, from the grants in Settings. Access comes only from those grants.

• Warn when nobody may use it. Grant Use this agent in Settings (Agents, then the agent's Access), or sign in once as the configured owner.
• Info when only the owner may use it so far. Add people in Settings.
• Ok with the number of people who may use it, or when an older grant still opens it to everyone signed in.

db.readCopy Anchor Link

Fail when the operational store exists but cannot be read. The message names the error. Check the database file's permissions, or the database server.

settings.files, settings.values and probesCopy Anchor Link

The Hub's own declarations, only when it has them. settings.files fails when settings.yaml, hub.settings.yaml or hub.manifest.yaml does not parse. settings.values fails when a declared setting has no valid value or reference, and lists each problem. Each check the Hub declares in hub.manifest.yaml reports as probe.<id> with its own status and recovery. hubzoid hub doctor runs these alone. See Hub files.

secrets.store, secrets.missing, secrets.blank and secrets.overridesCopy Anchor Link

Names and status only, never a value.

• secrets.store warns when a stored secret cannot be used, because the deployment key or its destination changed (set it again with hubzoid secrets set), or when an operator source such as .env also sets it, so the stored one is unused.
• secrets.missing warns for each secret the Hub expects that nothing sets, with the hubzoid secrets set command to fix it.
• secrets.blank warns for a line such as OPENAI_API_KEY=. Give it a value or remove the line.
• secrets.overrides warns when the Hub's files set a different value than the deployment. Keep shared values in one place.

Use it in operationsCopy Anchor Link

• Run hubzoid doctor after installing, after every upgrade and after hubzoid restore.
• With several Hubs, run it for each Hub. The operational store and backup checks read the store they all share.
• Wire --json into monitoring and alert on ok: false, or on specific ids such as scheduler.health, backup.age and deployment.key.

Next stepsCopy Anchor Link