Job registry

Every scan, discovery, and workflow step in OASM is a job: one row that says what to run, and one worker that runs it. The registry is the pipeline in the middle — the core API writes the work, workers pull it, results come back, and the inventory is updated. Nothing in the platform starts a scan any other way. This page describes the pipeline’s mechanics. Its operational surface — browsing, re-running, cancelling — lives under Job Registry.

What is recorded

Two tables carry the whole story:

• jobs — the unit of work. One row per tool execution: its category, status, priority, the tool and asset it belongs to, the worker that took it, the retry count, and a pointer to the raw result. Indexed for the two queries that matter — the claim (status, priority, createdAt) and the per-worker, per-run, per-asset and per-category lookups the console and dashboard make.
• job_histories — the run. One row per workflow run or discovery, holding the jobs it produced, the step states, the workflow and project it came from, and whether it finished. A run detail view is a projection over this row plus its jobs.

Findings and artifacts hang off the run, not off the job: HTTP responses, discovered URLs, and ports cascade from job_histories; vulnerabilities keep their run reference as provenance but survive the run’s deletion. That asymmetry is why retention can prune jobs without erasing what they found.

The states

Only the terminal four are prunable. A pending or in-progress row is live work no matter how old it is.

Claiming

Workers pull; the platform never pushes a job to a worker. A worker advertises its free slot count and the claim returns at most that many jobs — never more than the server-side cap, because a worker controls how many slots it fills but not how much of the queue one call may drain. The claim is one transaction: select eligible rows in dispatch order (priority, then oldest first), take a row lock so two racing workers cannot receive the same job, then mark the whole batch in_progress in a single update. Empty result means “nothing runnable” and the worker idles. What is eligible is decided in the same query, not after it — one ineligible job at the head of the queue used to starve everything behind it:

• Tool set — a built-in-tool worker only sees jobs for tools it carries; connector (image-based) jobs are only eligible for workers running in container mode.
• Workspace — a workspace-scoped worker never sees another tenant’s work.
• Internal network — a worker bound to an internal network only gets targets on that network.
• Runnable shape — a job needs a command (built-in CLI) or a connector image, or it can never be executed.

A newly written job nudges the workers of its workspace over their alive stream, so pickup is immediate rather than one poll away. The hint is fire-and-forget: the jobs are already durable, and a dropped hint costs one poll interval.

Results

Results do not travel back inline. The worker reports over a category-specific RPC (subdomains, HTTP probe, ports, vulnerabilities, screenshot, URL discovery), and the core API stages the payload as JSON in the job-results object-storage bucket, then enqueues the reference on the job-result queue. Two consequences:

• a large scan result never has to fit in one gRPC message or one DB write;
• processing is retryable on its own (three attempts, exponential backoff) without re-running the scan.

The processor then does the merge: parse the payload for the category, hand it to the data adapter (which is what turns raw tool output into assets, services, ports, and findings — see the data pipeline), mark the job completed, and give the job to the workflow engine so it can dispatch whichever steps now have their inputs satisfied. On completed, the staged file is deleted. On final failure the failure is recorded once — the retry count is incremented, an error log is written with the partial payload the tool had produced, dependent steps are skipped, and the run is closed rather than left hanging. Every terminal write is conditional on the job still being in_progress. A job cancelled or re-run while its result was in flight keeps the newer outcome and the late result is discarded, instead of a cancel quietly turning into a completion.

When something goes wrong

The registry is built to never leave a job owned by a dead worker:

• Worker disconnects — its in-progress jobs are released back to pending immediately, so another worker can take them.
• Worker goes silent — a 60-second heartbeat sweep removes it and releases its jobs the same way.
• Stuck or orphaned rows — a recurring sweep requeues in-progress jobs whose worker no longer exists, and failed jobs that still have retry budget left and a real asset. A job with no asset can never be claimed, so it is deliberately left alone instead of being rewritten every sweep. The persistent retry budget is 4 attempts.
• Run stopped advancing — a two-minute sweep finds workflow runs that are unfinished but have no pending or in-progress job left, and asks the engine to advance them again. Advancing is idempotent, so healthy runs are untouched. Lookback is bounded to 7 days and 50 runs per sweep.

Re-run creates a new attempt rather than mutating history; cancellation is terminal and discards the in-flight result.

Retention

Housekeeping is bounded on purpose:

• Jobs — terminal rows older than 180 days are pruned in batches, daily, behind a lock so one replica does it. Error logs cascade away with them.
• Runs — job_histories rows are kept. Findings reference a run, so deleting old runs would delete scan results. The price is that a pruned run keeps a thin row; a run whose jobs are all gone drops out of the registry list.
• Staged results — normally deleted right after processing. A 30-day sweep removes the orphans left by a lost queue entry or a process that died mid-processing. Files with no readable age are never deleted.

Events

The registry does not notify anyone directly — it publishes and lets the event bridge fan out. The job domain covers the lifecycle: Notifications, dashboard statistics, ticketing and webhooks all read those events; none of them polls the jobs table.

Related