ADR-0006: Engine job contracts and worker execution

Status: Proposed. Implementation available for review.

Date: 2026-09-21

Follow-up: ADR-0007: Domain worker and flow structure supersedes the target domain layout, workflow interface, export strategy, and single-queue rule in this proposal. The implementation described below remains the migration baseline.

Context

Producers and workers need shared job names, payloads, and results. The current contracts mix API schemas and worker messages. A new worker structure needs clear ownership without requiring all current callers to migrate together.

Decision

apps/engine/packages/jobs provides @gigflow/engine-jobs. Its JobClient submits jobs from a queue name, job name, and payload. Typed enqueue functions validate payloads before submission. There is no job definition registry. The client has no domain-specific queue types or methods. new JobClient() validates Redis settings from the environment with T3 Env. It takes no connection or dependency argument. It opens resources on first use and closes them explicitly.

src/schemas/<domain>/ contains payload schemas and queue/job name constants. The vacancy example has embed.ts and finalize.ts. src/enqueue.ts composes these exports with a caller-owned JobClient. Nested schema barrels are a user-approved exception to the global export rule. src/types.ts defines inferred public types. The package exposes /client, /enqueue, /schemas, and /types, without a package-root export.

The generic client supports BullMQ flow submission. Workflow owners define their graph and failure policies. The client does not embed a vacancy pipeline. The new package does not import old worker contracts or enqueue functions. Domain vacancy content keeps its existing schema owner.

packages/worker provides WorkerClient and BaseProcessor. Processors declare input and result schemas. The base class validates both boundaries. The embed result schema can also be the next processor's input schema.

The vacancy embed app demonstrates env.ts, queue.ts, and processor classes. Each worker owns one queue with one or more named processors. Each processor uses its own input and result schemas. Other apps and packages retain their current APIs. This change adds no compatibility adapters to the new packages.

Consequences

The producer and worker share schema modules and name constants. New job types do not require a change to the generic client. Applications must close owned clients on shutdown. Schema-only imports do not require environment settings or Redis.

Static parent data takes precedence when combined with a child result. Invalid processor results fail before BullMQ records success. Raw flow payloads are validated by workers. Typed enqueue functions also validate input. The generic client accepts unknown payloads and does not validate domain contracts.

The runtime drains workers before closing queues. Startup failures trigger cleanup. Process signal handling is explicit and can close multiple clients. The original runtime remains available for apps that have not migrated.

Validation

Focused checks cover schema validation, generic submissions, environment errors, startup cleanup, shutdown order, readiness, and the vacancy embed consumer. Real Redis checks use an isolated test instance and controlled processors. No external embedding provider or application database is required.