ADR-0001: LinkedIn SDK requests

Status: Proposed

Date: 2026-09-10

Context

Engine owns LinkedIn search and fetch workers. Consumers need to start work and retrieve results without depending on queue names, Redis access, or provider implementations. Fetching can take minutes and incurs provider charges.

Decision

Expose engine.linkedin.search.start/get and engine.linkedin.fetch.start/get. Each endpoint has its own file under apps/engine/apps/api/src/trpc/routers/linkedin/search or linkedin/fetch, with router assembly in _app.ts, following the neighboring endpoint folders. Public schemas live in apps/engine/packages/schema/src/api/linkedin.ts and the SDK resource lives in packages/engine-sdk/src/resources/linkedin.ts.

Starts return an opaque ID. Gets expose typed queued, running, completed, and failed states. Dedicated endpoints accept authenticated internal services. IDs and optional idempotency keys are scoped to the authenticated key ID and action. Reusing a key with different normalized input returns a conflict.

Reuse BullMQ's atomic job-ID deduplication and persisted results. Preserve the 45-day terminal retention policy and remove count-based cleanup for both LinkedIn queues, including jobs started by administrative triggers. This keeps busy queues from evicting pollable results early. Queue storage is retained request state, not permanent product data.

Consequences

Engine centralizes provider access, ranking, normalization, and retries. Gigflow keeps user authorization, profile selection, import approval, and product writes. The dashboard polls its own API; that API calls the SDK with server credentials.

Polling avoids holding HTTP requests open during long provider runs. Applications choose their polling interval and persist results they need long term. Automatic cleanup, administrative deletion, or Redis data loss ends result availability and deduplication. A separate request ledger would be needed if consumers require durability beyond queue storage. Retaining records by age rather than count also increases Redis memory use on busy queues.

This decision adds the Engine contract. Replacing Gigflow's unfinished import workflow is a separate integration change.