> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oasm.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Job registry

> How work flows through OASM — from the row that records a job to the claim by a worker, the staged result, the inventory merge, and the run that advances

Every scan, discovery, and workflow step in OASM is a **job**: one row that says what to run, and one worker that runs it. The registry is the pipeline in the middle — the core API writes the work, workers pull it, results come back, and the inventory is updated. Nothing in the platform starts a scan any other way.

This page describes the pipeline's mechanics. Its operational surface — browsing, re-running, cancelling — lives under [Job Registry](https://docs.oasm.io/jobs.md).

```mermaid theme={null}
flowchart TD
    T["Trigger<br/>request · schedule · target event · re-run"] --> Create["Admit<br/>row in jobs, PENDING"]
    Create --> Hint["jobs-available hint<br/>pushed over the worker's alive stream"]
    Hint --> Claim["Claim<br/>up to the worker's free slots"]
    Claim -->|"one transaction · row lock · priority then oldest"| Run["IN_PROGRESS<br/>workerId + pickJobAt"]
    Run --> Spawn["Worker resolves the connector image<br/>and spawns the container"]
    Spawn --> Result["Result RPC for the job's category"]
    Result --> Stage["Stage the result JSON<br/>job-results bucket"]
    Stage --> Queue[("BullMQ queue: job-result")]
    Queue --> Merge["Data adapter merges<br/>into the inventory"]
    Merge --> Done["Conditional write<br/>COMPLETED / FAILED"]
    Done --> Advance["Workflow engine advances the run"]
    Done --> Bus["job.* events<br/>event bridge"]
    Run -.->|"worker dies"| Requeue["released back to PENDING<br/>or swept by the stuck-job reset"]
    Requeue -.-> Hint
```

## What is recorded

Two tables carry the whole story:

* **`jobs`** — the unit of work. One row per tool execution: its category, status, priority, the tool and asset it belongs to, the worker that took it, the retry count, and a pointer to the raw result. Indexed for the two queries that matter — the claim (`status, priority, createdAt`) and the per-worker, per-run, per-asset and per-category lookups the console and dashboard make.
* **`job_histories`** — the run. One row per workflow run or discovery, holding the jobs it produced, the step states, the workflow and project it came from, and whether it finished. A run detail view is a projection over this row plus its jobs.

Findings and artifacts hang off the run, not off the job: HTTP responses, discovered URLs, and ports cascade from `job_histories`; vulnerabilities keep their run reference as provenance but survive the run's deletion. That asymmetry is why retention can prune jobs without erasing what they found.

## The states

| Status | Meaning |
| - | - |
| `pending` | Written, waiting for a worker. Consumes no compute. |
| `in_progress` | Claimed by a worker; `workerId` and `pickJobAt` are set. |
| `completed` | Result merged; `completedAt` is set. |
| `failed` | Tool error, after the retry budget is spent. Carries an error log with the partial payload. |
| `cancelled` | Withdrawn before completion. Results are discarded. |
| `skipped` | Bypassed as a consequence of the run (an upstream step failed, so its dependents never ran). |

Only the terminal four are prunable. A pending or in-progress row is live work no matter how old it is.

## Claiming

Workers **pull**; the platform never pushes a job to a worker. A worker advertises its free slot count and the claim returns at most that many jobs — never more than the server-side cap, because a worker controls how many slots it fills but not how much of the queue one call may drain.

The claim is one transaction: select eligible rows in dispatch order (priority, then oldest first), take a row lock so two racing workers cannot receive the same job, then mark the whole batch `in_progress` in a single update. Empty result means "nothing runnable" and the worker idles.

What is *eligible* is decided in the same query, not after it — one ineligible job at the head of the queue used to starve everything behind it:

* **Tool set** — a built-in-tool worker only sees jobs for tools it carries; connector (image-based) jobs are only eligible for workers running in container mode.
* **Workspace** — a workspace-scoped worker never sees another tenant's work.
* **Internal network** — a worker bound to an internal network only gets targets on that network.
* **Runnable shape** — a job needs a command (built-in CLI) or a connector image, or it can never be executed.

A newly written job nudges the workers of its workspace over their alive stream, so pickup is immediate rather than one poll away. The hint is fire-and-forget: the jobs are already durable, and a dropped hint costs one poll interval.

## Results

Results do **not** travel back inline. The worker reports over a category-specific RPC (subdomains, HTTP probe, ports, vulnerabilities, screenshot, URL discovery), and the core API stages the payload as JSON in the `job-results` object-storage bucket, then enqueues the reference on the `job-result` queue. Two consequences:

* a large scan result never has to fit in one gRPC message or one DB write;
* processing is retryable on its own (three attempts, exponential backoff) without re-running the scan.

The processor then does the merge: parse the payload for the category, hand it to the data adapter (which is what turns raw tool output into assets, services, ports, and findings — see [the data pipeline](https://docs.oasm.io/architecture/overview.md#the-data-pipeline)), mark the job `completed`, and give the job to the workflow engine so it can dispatch whichever steps now have their inputs satisfied. On `completed`, the staged file is deleted. On final failure the failure is recorded once — the retry count is incremented, an error log is written with the partial payload the tool had produced, dependent steps are skipped, and the run is closed rather than left hanging.

Every terminal write is **conditional** on the job still being `in_progress`. A job cancelled or re-run while its result was in flight keeps the newer outcome and the late result is discarded, instead of a cancel quietly turning into a completion.

> [!NOTE]
>
> An empty result is a result. A built-in tool that exits cleanly with no output — no open ports, a name that does not resolve — completes the job with nothing to merge, rather than failing the step and skipping everything downstream.

## When something goes wrong

The registry is built to never leave a job owned by a dead worker:

* **Worker disconnects** — its in-progress jobs are released back to `pending` immediately, so another worker can take them.
* **Worker goes silent** — a 60-second heartbeat sweep removes it and releases its jobs the same way.
* **Stuck or orphaned rows** — a recurring sweep requeues in-progress jobs whose worker no longer exists, and failed jobs that still have retry budget left and a real asset. A job with no asset can never be claimed, so it is deliberately left alone instead of being rewritten every sweep. The persistent retry budget is **4 attempts**.
* **Run stopped advancing** — a two-minute sweep finds workflow runs that are unfinished but have no pending or in-progress job left, and asks the engine to advance them again. Advancing is idempotent, so healthy runs are untouched. Lookback is bounded to 7 days and 50 runs per sweep.

Re-run creates a new attempt rather than mutating history; cancellation is terminal and discards the in-flight result.

## Retention

Housekeeping is bounded on purpose:

* **Jobs** — terminal rows older than 180 days are pruned in batches, daily, behind a lock so one replica does it. Error logs cascade away with them.
* **Runs** — `job_histories` rows are **kept**. Findings reference a run, so deleting old runs would delete scan results. The price is that a pruned run keeps a thin row; a run whose jobs are all gone drops out of the registry list.
* **Staged results** — normally deleted right after processing. A 30-day sweep removes the orphans left by a lost queue entry or a process that died mid-processing. Files with no readable age are never deleted.

## Events

The registry does not notify anyone directly — it publishes and lets the [event bridge](https://docs.oasm.io/architecture/event-bridge.md) fan out. The `job` domain covers the lifecycle:

| Event | Fired when |
| - | - |
| `job.created` | The row is written, `pending` |
| `job.started` | A worker claims it |
| `job.completed` / `job.failed` | Terminal outcome applied |
| `job.skipped` | The run bypassed it |
| `job.cancelled` | An operator stopped it |
| `job.retried` / `job.released` | Requeued for a new attempt, or released by a lost worker |
| `job.reset_stuck` | Recovered by the stuck-job sweep |
| `job.result.cleaned` | A staged result file was removed by the sweep |

Notifications, dashboard statistics, ticketing and webhooks all read those events; none of them polls the `jobs` table.

## Related

> [card] [Job Registry](https://docs.oasm.io/jobs.md)
>
> The operational surface over this pipeline

> [card] [Worker](https://docs.oasm.io/worker.md)
>
> The execution layer that claims and runs this work

> [card] [Projects](https://docs.oasm.io/project.md)
>
> The scopes and schedules that originate work

> [card] [Workflows](https://docs.oasm.io/workflows.md)
>
> The step graph a run advances through

> [card] [Event bridge](https://docs.oasm.io/architecture/event-bridge.md)
>
> Where the job lifecycle events go

> [card] [Architecture overview](https://docs.oasm.io/architecture/overview.md)
>
> The five tiers this pipeline sits in


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.