> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oasm.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Developer Guide

> Set up the OASM development environment, run services from source, and manage the database, code generation, and environment variables

This guide covers developing OASM from source: cloning the monorepo, initializing the environment, running the Console, Core API, and workers locally, managing database migrations, and the environment variables each service expects. For a production deployment, see [Deployment](/deployment).

## Prerequisites

Before you begin, ensure you have the following installed:

* **Task (taskfile)** — [Installation Guide](https://taskfile.dev/#/installation)
* **Node.js v22+** — [Installation Guide](https://nodejs.org/en/download/package-manager)
* **Go 1.26+** — [Installation Guide](https://go.dev/doc/install)
* **PostgreSQL v17+** (with pgvector extension)
* **Docker & Docker Compose** (recommended for database and full stack)

## Clone the repository

```bash theme={null}
git clone https://github.com/oasm-platform/open-asm.git
cd open-asm
```

The repository is a monorepo with **pnpm workspaces** combining the Node-based services — `console` and `core-api` — alongside the Go `worker`.

## Project Structure

```
open-asm/
├── core-api/           # NestJS API server
│   ├── src/            # Source code
│   ├── example.env     # Environment template
│   └── package.json
├── console/            # React web interface
│   ├── src/            # React components
│   ├── public/         # Static assets
│   └── package.json
├── worker/             # Go-based scanning workers
│   ├── cmd/            # CLI and App entry points
│   ├── internal/       # Business logic
│   ├── scripts/        # Install scripts (install.ps1, install.sh)
│   ├── go.mod          # Go module definition
│   └── .example.env    # Environment template
├── grpc-client/        # Generated gRPC stubs (Go + TypeScript)
├── .open-api/          # Auto-generated API docs
├── docker-compose.yml  # Container orchestration
├── taskfile.yml        # Task automation
└── README.md           # Documentation
```

* **`console/`** — React 19 + TanStack Router + Vite.
* **`core-api/`** — NestJS 11 (REST on port `6276`, gRPC on port `16276`).
* **`worker/`** — Go scanning workers (CLI and app entry points).

## Initialize Developer Environment

To set up your local development environment, run the following command:

```bash theme={null}
task init
```

This command will:

* Copy example environment files (`.env`) for `core-api`, `console`, and `worker`
* Install project dependencies using `npm` (managed by the task for each workspace)
* Install Go dependencies for the worker
* Install worker security tools (nuclei, subfinder, httpx, naabu, dnsx) into `worker/oasm-tools/`

After running `task init`, you can start all services using `task dev` or run them individually as described below.

## Running Services

### All Services with Task

To start the API and Console development servers simultaneously:

```bash theme={null}
task dev
```

This starts:

* Core API at `http://localhost:6276`
* Console at `http://localhost:5173` (Vite dev server)

### Core API

```bash theme={null}
task api:dev
```

Or directly:

```bash theme={null}
cd core-api && npm run start:dev
```

The API runs on port `6276` (base path `http://localhost:6276/api`) with the gRPC server on port `16276`.

### Console (Web Interface)

```bash theme={null}
task console:dev
```

Or directly:

```bash theme={null}
cd console && npm run dev
```

### Workers

To run workers locally in CLI mode:

```bash theme={null}
task worker:dev
```

With custom parameters:

```bash theme={null}
task worker:dev replicas=3 maxJobs=10 apiKey=<your-api-key> network=<target-network>
```

To run workers in app mode (env-driven):

```bash theme={null}
task worker:dev-app
```

See [Worker](/worker) for the environment variables workers consume.

## Database Setup

`task init` does **not** automatically start PostgreSQL. You can either:

1. Use Docker Compose to start PostgreSQL and Redis (both are required for local development):
   ```bash theme={null}
   docker compose up postgres redis -d
   ```
2. Use your own PostgreSQL instance and update `core-api/.env` accordingly.

The database uses PostgreSQL 17 with the pgvector extension for vector operations. For the full stack (including workers, geo-ip, and rustfs), run `task docker-compose` instead — see [Using Docker Compose](#using-docker-compose).

## Database Migration

Database migrations are managed using TypeORM. The migration scripts are defined in `core-api/taskfile.yml` and can be executed using the task commands.

### Run All Pending Migrations

This command executes all pending database migrations:

```bash theme={null}
task migration:run
```

This will:

* Connect to the PostgreSQL database
* Check for pending migrations in the `migrations` table
* Run all new migrations that haven't been applied yet

### Generate a New Migration

To generate a new migration with a custom name:

```bash theme={null}
task migration:generate MIGRATION_NAME=YourMigrationName
```

For example:

```bash theme={null}
task migration:generate MIGRATION_NAME=AddUserTable
```

This will create a new migration file in `core-api/src/database/migrations/`.

### Revert the Last Migration

To rollback the most recently executed migration:

```bash theme={null}
task migration:revert
```

**Note:** This will only revert one migration at a time. Repeat if needed.

### Migration with Docker Compose

If you prefer to run migrations using Docker (useful when not running PostgreSQL locally):

```bash theme={null}
docker compose up migration
```

This starts the PostgreSQL container (if not running), runs the migration service, applies all pending migrations, removes the migration container, and starts `core-api` once migrations complete. To keep the container for debugging:

```bash theme={null}
docker compose run --rm migration
```

The `--rm` flag ensures the container is removed after it stops.

## Development Conventions

### Code Style

* **Core API (NestJS):** Uses ESLint and Prettier.
  ```bash theme={null}
  task api:lint
  ```
* **Console (React):** Uses ESLint and Prettier.
  ```bash theme={null}
  task console:lint
  ```
* **Workers (Go):** Uses `go fmt` and `go vet`.
  ```bash theme={null}
  task worker:format
  task worker:lint
  ```

### Testing

* **Core API:** Uses Jest for testing.
  ```bash theme={null}
  task api:test           # Unit tests
  task api:test:one SPEC=path/to/spec.ts   # Run a single test file
  cd core-api && npm run test:watch    # Watch mode
  cd core-api && npm run test:e2e      # End-to-end tests
  ```

* **Console:** Uses Vitest for unit tests and Playwright for e2e tests.
  ```bash theme={null}
  task console:test       # Unit tests
  cd console && npm run e2e           # E2E tests
  ```

* **Workers:** Uses Go testing.
  ```bash theme={null}
  task worker:test
  ```

### Build

```bash theme={null}
task api:build      # Build the Core API
task build          # Build all services (API + Console + Worker)
```

### API Client Generation

After making changes to the API contract, regenerate the console API client:

```bash theme={null}
task gen-api
```

This uses orval to generate TanStack Query hooks from the OpenAPI spec. The source spec lives at `.open-api` in `core-api` (the docs site mirrors it at `api-reference/openapi.json`).

### gRPC Stub Generation

After modifying proto files, regenerate gRPC stubs:

```bash theme={null}
task proto
```

This generates Go and TypeScript stubs into `grpc-client/`.

## Environment variables

Each service reads its configuration from its own `.env` file, copied from the example templates by `task init`. The table below lists the variables used in local development with their defaults.

| Service | Variable | Purpose | Default |
| - | - | - | - |
| `core-api` | `POSTGRES_HOST` | Host of the PostgreSQL service | — |
| `core-api` | `REDIS_URL` | Redis connection string (contains the DB password) | `redis://:password@redis:6379/0` |
| `core-api` | `GEO_IP_URL` | Geo IP service endpoint for asset locations | `geo-ip:4360` |
| `core-api` | `OASM_CLOUD_APIKEY` | OASM Cloud API key | — |
| `core-api` | `RUSTFS_ENDPOINT` | Object storage endpoint for screenshots and artifacts | — |
| `worker` | `WORKER_API_KEY` | Workspace API key used to authenticate with core-api | `change_me` |
| `worker` | `WORKER_GRPC_HOST` | gRPC host of the Core API | `core-api` |
| `worker` | `WORKER_GRPC_PORT` | gRPC port of the Core API | `16276` |
| `worker` | `WORKER_MAX_CONCURRENCY` | Max concurrent jobs per worker | `10` |
| `worker` | `WORKER_TOOL_PATH` | Directory where scanning tools are installed | `/app/oasm-tools` |
| `console` | `VITE_API_URL` | Base URL the Console uses to reach the Core API (build/dev default) | `http://localhost:6276` |

> [!CAUTION]
>
> Never commit `.env` files — they are gitignored. The worker's `WORKER_API_KEY` must match the workspace key the Core API expects; a mismatch prevents workers from registering, see [Worker](/worker).

## Using Docker Compose

To run the entire stack using Docker Compose:

```bash theme={null}
task docker-compose
```

This starts:

* Console (port 3000)
* Core API (port 6276, gRPC port 16276)
* 3 Worker instances
* PostgreSQL with pgvector (port 5432)
* Redis (port 6379)
* Geo-IP proxy (port 4360)
* Rustfs S3 storage (port 9000)

## Local CI Testing

Before pushing changes, you can run GitHub Actions workflows locally using [act](https://github.com/nektos/act) to catch issues early.

### Prerequisites

* Docker Desktop must be installed and running
* Install act:
  ```bash theme={null}
  # Linux/macOS
  curl -fsSL https://raw.githubusercontent.com/nektos/act/master/install.sh | bash

  # Windows (Git Bash)
  curl -fsSL https://raw.githubusercontent.com/nektos/act/master/install.sh | bash
  ```

### Usage

```bash theme={null}
# List available workflows
bash .github/scripts/test-local.sh

# Run a specific workflow
bash .github/scripts/test-local.sh check-lint
bash .github/scripts/test-local.sh worker-ci

# Validate all workflows (dry-run)
bash .github/scripts/test-local.sh --all

# Custom act binary path
ACT_BIN=act bash .github/scripts/test-local.sh check-lint
```

### Local Test Equivalents

| CI Workflow | Local Command |
| - | - |
| `check-lint.yml` | `task lint` |
| `check-test.yml` | `task api:test` |
| `check-build.yml` | `task build` (requires Docker) |
| `frontend-tests.yml` | `cd console && npm run test:run` |
| `worker-ci.yml` | `task worker:lint && task worker:check` |

### Notes

* Workflows using `docker/build-push-action` with multi-platform builds (`build-release.yml`, `build-nightly.yml`) need QEMU and native CI runners — they cannot run locally.
* `dorny/paths-filter` may not detect file changes correctly in shallow clones. Use `--full-history` or test specific jobs.
* Docker layer caching (`type=gha`) is not available locally, but builds will still work.

## Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository.
2. Create a feature branch: `git checkout -b feature/amazing-feature`.
3. Make your changes and commit them following [Conventional Commits](https://www.conventionalcommits.org/):
   ```bash theme={null}
   git commit -m 'feat(scope): add amazing feature'
   ```
4. Test CI workflows locally: `bash .github/scripts/test-local.sh <workflow>`
5. Push to the branch: `git push origin feature/amazing-feature`.
6. Open a Pull Request.

Please ensure your code adheres to the project's coding standards and passes all tests before submitting a PR.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.