Documentation
Documentation/Execution

Workers and pool

Connect and manage workspace workers with the CLI, then configure shared execution capacity and concurrency limits.

workersCLIsetupbrowser approvalpoolconcurrencyqueueingcost
View Markdown
On this page

Workers in BernOps provide execution capacity shared by projects within their enrolled workspace. Connected workers run Playwright jobs or discovery work according to their capabilities; personal coding-agent hosts are separate.

Worker pool model

Each worker connects to /ws/worker, registers its identity, and can run one assigned job at a time. The backend tracks connected workers in memory and assigns pending runs to idle workers.

Every worker is bound to one workspace by its own credential. It cannot take another workspace's jobs. Enrollment records persist even when a worker disconnects; live connection and assignment state is tracked by the gateway.

See and manage workers

Click the worker cog in the topbar, or Workspace → Workers in the sidebar, to open the Workers page. Select a workspace if none is selected. A project does not need to be selected.

  • The page lists named workers, including connected, busy, connecting, offline, and revoked enrollments. It refreshes every ten seconds while visible.
  • Workspace members can view the inventory. Current job or discovery links appear only when the viewer can read the corresponding project.
  • Last authenticated records credential authentication, not exact connection or machine uptime history. Revoked/offline status does not prove an old process has stopped.
  • Workspace owners and admins can open Add worker for the setup command and revoke access with confirmation. Revocation can interrupt work; it does not uninstall the worker or guarantee an immediate process stop.
  • Add worker does not issue or display a credential in the browser. The CLI creates and stores the worker credential locally; browser approval binds it to the selected workspace.

The project availability tooltip remains a project-filtered capacity view and anonymizes idle workers. Use the workspace page for stable worker names and enrollment history. A failed refresh is shown as unavailable or stale, not as an empty fleet.

An empty inventory is expected before enrollment. Creating a workspace does not enroll this computer. Local demo tooling may provision its own worker; inspect the inventory rather than assuming the demo account is empty or this computer is already enrolled.

Connect a worker

Use the private bootstrap on an Ubuntu/Debian x86-64 host with UID 1000, rootful local Docker over a Unix socket, seccomp, and enforcing default AppArmor, without user-namespace remapping. The CLI checks these prerequisites; it does not install Docker or change host security. Rootless Docker, arbitrary UIDs, ARM, Windows, and macOS are outside this path. Existing Ubuntu acceptance does not qualify fresh desktop/headless OS installs or cloud production deployment.

Obtain the complete approved private bootstrap archive from your operator. Authenticate its checksum and provenance through a trusted channel before extraction or execution; no supported public download URL, curl | sh, or npm install name exists. After verified extraction, run as your regular user without sudo, replacing the example path:

/absolute/verified/bootstrap-linux-x64/install.sh
export PATH="$HOME/.local/bin:$PATH"

Then run the normal setup command on the computer that will execute workspace jobs:

bernops setup

Choose 3 (automation worker), or 4 (both) if you also want a personal coding agent. The menu defaults to 1 (platform authentication only), which installs no worker. The bootstrap bundles Node 22.19.0; normal installation needs neither NVM nor a source checkout or manual build. The operator must configure approved signed releases.

  1. Complete or reuse platform login inside the wizard. This creates a personal CLI session but does not authorize execution; no preliminary bernops auth login command is required.
  2. For a new worker, setup checks Docker, obtains and verifies the signed immutable private worker release with the SDK bundled, and runs real SDK/browser readiness checks before new worker identity creation and workspace consent. It does not build from a checkout or ask you to install the SDK or mount a tarball.
  3. Separately approve workspace enrollment in the browser. Check the computer name and fingerprint against the terminal, review the requested image and capabilities, select a workspace where you are an owner or admin, and explicitly approve running its workloads on this computer.
  4. The CLI stores a workspace-bound worker credential, installs and starts the container, and confirms a fresh Ready session, or Busy if work has already arrived. Unified setup requires both SDK/image readiness and a confirmed connection. Browser approval, a smoke test, or a running container alone is insufficient. Check the named worker on the Workers page.

The wizard reuses its saved worker name, otherwise a sanitized hostname. To choose explicitly, use bernops setup --components worker --worker-name build-worker; --worker-name belongs to unified setup, while lifecycle commands below use --name. Names use lowercase letters, digits, and hyphens, at most 48 characters. Workspace selection happens in the browser, not through a --workspace flag.

Without an explicit --target, setup uses saved user identity, then its journal, then the compiled bootstrap default: local for locally prepared bootstraps and cloud for general builds. Viewing these docs locally does not select a target. Conflicts fail rather than migrating identities. Use --target cloud for an operator-configured cloud environment, or see local development.

A worker is not a personal host

Unified setup shares platform login, but never passes that personal OAuth session to the worker container. The worker credential authorizes execution only for its enrolled workspace. Worker-only selection installs no personal host, Pi, or provider login. bernops auth login is auth-only and never enrolls a host; choose the agent in Coding agent CLI for separate host approval. Workspace jobs have network access inside a shared container, not VM isolation, so approve only workloads you trust.

Manage the installed worker

Choose the operation you need, replacing NAME with the name reported by setup:

bernops worker status --name NAME
bernops worker start --name NAME
bernops worker stop --name NAME
bernops worker restart --name NAME
bernops worker logs --name NAME

These commands use local worker state, not personal OAuth. Stop disables Docker restart before graceful draining; it does not revoke access. Default logs show a bounded supervisor snapshot, not raw job output. The advanced --unsafe-raw logs option can expose workload secrets; do not share that output. For run logs, steps, and traces, see Workers and execution.

OperationCommand
Upgrade to an operator-approved immutable imagebernops worker upgrade --name NAME --image "$APPROVED_IMAGE"
Restore the retained previous imagebernops worker rollback --name NAME
Remove the local containerbernops worker uninstall --name NAME

Set APPROVED_IMAGE to the private image digest supplied by your operator, not a mutable tag. Upgrades wait for connection readiness. Uninstall retains local state unless --purge is explicitly added; neither option revokes the remote credential or removes the original enrollment identity. Revoke unwanted access separately from the workspace's Workers page.

Headless and advanced setup

To approve from another browser without attempting to open one on the worker host:

bernops setup --components worker --no-browser

Leave the CLI polling and open each printed approval URL on another device. Compare the human code, signed-in account, and worker fingerprint before approving the relevant transaction. --no-browser suppresses launch, not consent. --headless is agent-specific and is not accepted for worker-only setup. Keep URLs and codes private; never copy credential files, private keys, or device secrets into chat or logs.

Platform credentials default to native Linux Secret Service for local and cloud, including worker-only unified setup. It requires secret-tool, gdbus, and an unlocked Secret Service in the user's D-Bus session. For a new identity only, deliberately add --credential-store file if you accept owner-only unencrypted refresh-credential storage. There is no automatic plaintext fallback, and the flag does not migrate a saved native identity. Repair locked or unavailable native storage. Access tokens stay memory-only.

Low-level alternative, not unified setup: bernops worker setup remains available for operator-controlled source/image installation. It defaults to cloud (https://api.bernops.com, https://app.bernops.com) regardless of a saved personal login or the local bootstrap default. It does not perform platform CLI authentication or store a personal OAuth session; it uses browser sign-in and separate workspace approval. bernops auth login is not its prerequisite.

bernops worker setup --target cloud --name build-worker --image "$APPROVED_IMAGE"

This low-level command can prepare an image from the CLI's trusted sibling worker source in an approved developer checkout, not the current directory's Dockerfile. A standalone bootstrap without that source needs --image or BERNOPS_WORKER_SETUP_IMAGE: an operator-approved repository digest or existing local sha256: image ID, never a mutable tag. This path does not resolve the unified signed release catalog; image trust belongs to the operator. Source/image installation is not a substitute for an SDK-capable signed release and its separately approved capabilities. Native SDK artifact mounts are not an installer option. --no-browser and explicit --target local remain available on this low-level command.

For credential creation, rotation, revocation, or manual installation, see the private checkout's agent-host/README.md, section Docker Worker Administration. Remote credential administration uses a personal CLI OAuth session; neither the low-level enrollment flow nor local worker lifecycle commands do. Local credential replacement after remote rotation is not supported by the installer.

Local development only

From the approved BernOps checkout, prepare signed local releases and activate the local-target bootstrap without restarting the stack:

./dev.sh install-cli
bernops setup

Choose 3 (automation worker) or 4 (both), not the auth-only default. Local preparation has developer build prerequisites and may use an already-installed NVM Node 22.19.0; it never installs NVM or Node. The local backend must already be running with its signed release catalog configured. For an explicit local selection instead of the menu:

bernops setup --target local --components worker

Local enrollment uses http://127.0.0.1:3410 and browser approval uses http://app.bernops-dev. Setup discovers the matching local Compose backend and Docker network automatically. Without that service, its host-gateway fallback must be reachable from Docker; a host loopback-only listener may not be reachable. Keep the same target and worker name when resuming.

Setup recovery

  • After a network interruption, rerun bernops setup with the same selections, worker name, target, and credential store. Setup preserves completed components and does not silently replace identities or move an installed worker to another workspace. Exit 0 means selected components are ready, 2 means attention, and 1 means failure.
  • After a verified local disconnect, review Reconnect this PC? if the wizard offers it. Consented relink enrolls a separate child worker with new workspace approval while preserving the old container and credential. Stop/revoke the old worker separately after checking the replacement; a network error alone does not prove revocation.
  • worker_sdk_upgrade_required: the existing worker is preserved. After the operator configures signed releases, use an unused name with bernops setup --components worker --worker-name workstation-sdk and approve its SDK capabilities in the browser. Image upgrade or credential rotation alone cannot expand existing approved capabilities.
  • component_release_trust_unconfigured or component_releases_not_configured: obtain the approved bootstrap or ask the operator to configure the signed catalog. Never bypass verification or substitute a public package/image.
  • Low-level bernops worker setup --reset-enrollment is only for a confirmed expired, denied, or cancelled request you deliberately want to archive. It is not a unified setup option or a general retry flag, and cannot reset an installed or uncertain identity.
  • For path or permission failures, check XDG_STATE_HOME and its parents. Retrying unchanged permissions will not help. Do not chmod the checkout or shared state directories, use sudo/chown, or delete identity files to bypass checks.
  • A crash can leave a setup or installer lock. Confirm the original process has exited and rerun the same operation; secure locks reclaim only verified stale owners. Ask the operator to investigate remaining locks using the relevant recovery procedure, not blind deletion. Never remove identity state or force-remove an unknown container.

What the pool is

The worker pool is a dispatch target for queued BernOps runs.

When a run is created, BernOps stores it as pending. The dispatcher promotes it to running only when an idle worker exists and the run fits within its configured scope limits.

Scope limits can be configured at:

  • Workspace level
  • Project level
  • Job level

What it is not

The worker pool is not autoscaling. Setting max_concurrent_executions to 5 does not create five workers. It only allows up to five matching runs to execute if enough worker processes are already connected.

The worker pool is not reserved capacity. A project limit of 3 means the project may use up to three concurrent execution slots, not that three workers are held idle for that project.

Workspace binding is enforced, but project/job pinning is not offered: a named worker can serve eligible work across its workspace's projects.

The operator still provisions worker processes. The private CLI path requires Linux/amd64, UID 1000, and rootful local Docker with seccomp/AppArmor. Existing Ubuntu acceptance is not certification of fresh OS installs or other deployment platforms.

Concurrency limits

Concurrency limits are maximums. They are evaluated together.

ScopeField
Workspaceworkspaces.max_concurrent_executions
Projectprojects.max_concurrent_executions
Jobjobs.max_concurrent_executions

Example:

Workspace max: 10
Project max: 3
Job max: 1

That job can run only one execution at a time. The project can run at most three total executions. The workspace can run at most ten total executions across all projects.

If a project or job limit is unset, that scope does not add an extra cap. The workspace limit still applies.

Queueing behavior

Runs follow this lifecycle:

pending -> running -> passed | failed | error

A run remains pending when no worker is connected, all workers are busy, or a scope limit is already saturated.

The dispatcher checks pending runs repeatedly. It also wakes up when workers connect or finish jobs. Older pending runs are considered first, but if an older run is blocked by its own scope limit, another run from a scope with available capacity can still start. Dispatch also requires a worker with the capability needed for the work, such as test_runner for Playwright runs or agent for discovery explorer work.

Worker selection

When several workers are idle, BernOps prefers workers that have not been assigned recently. This avoids relying on random map iteration and gives basic fair distribution across the shared pool.

This is not weighted load balancing, priority scheduling, custom worker pinning, or historical duration-aware scheduling.

Configuration

Set the workspace limit:

PUT /api/workspaces/{workspaceId}
{
  "max_concurrent_executions": 5
}

Set the project limit:

PUT /api/projects/{projectId}
{
  "max_concurrent_executions": 2
}

Set the job limit:

PUT /api/jobs/{jobId}
{
  "max_concurrent_executions": 1
}

Operational guidance

Use worker process count to control cost at the infrastructure level. Use BernOps concurrency limits to control how that capacity is consumed by workspaces, projects, and jobs.

Recommended starting points:

  • Keep workspace limits low by default.
  • Set project limits for noisy or expensive projects.
  • Set job limits to 1 when overlapping runs would be unsafe.
  • Increase actual worker count only when queue time is too high and cost is acceptable.
Cost control

BernOps will not start more workers automatically. A higher concurrency limit allows more work to run only when you have already provisioned enough workers.

Current limitations

The current model does not include autoscaling, project/job worker reservations, priority queues, or cross-backend distributed dispatch locking. Use one active scheduling gateway. Worker enrollments persist, but terminal-result replay is memory-only and does not survive worker process loss.

Capabilities include agent, test_runner, secret_env_redaction_v1, and native_bridge_v1. New unified worker setup requires a signed SDK-capable release with the exact private SDK already bundled, real SDK/browser readiness checks, separately approved capabilities, and a confirmed connection. Existing workers keep their images and credentials; setup checks SDK readiness without silently upgrading them. Legacy enrollment alone does not add SDK support. CLI/worker distribution remains private, with no public package or worker image implied by these instructions.