Background Job Workflow With Retries

Design a background job for work that must not run inside a web request. JOB Name: [JOB_NAME] Trigger: [USER ACTION | CRON | WEBHOOK] What it does: [STEPS] External calls: [EMAIL | PAYMENT | AI | HTTP] Idempotency key: [WHAT MAKES A DUPLICATE THE SAME JOB] Acceptable latency: [SECONDS | MINUTES] What happens if it fails for good: [USER-VISIBLE STATE] Stack preference: [INNGEST | TRIGGER.DEV | BULLMQ | CLOUDFLARE QUEUES | "pick and justify"] Database: [POSTGRES] DELIVER 1) Why this is a job and not a request: the timeout and retry reason in two sentences. 2) Payload schema (TypeScript). Small. No entire ORM objects. 3) Step breakdown. Each step is independently retryable. State which steps are safe to retry and which need an idempotency guard (sending email, charging, writing a file). 4) Retry policy: attempts, backoff, which errors are permanent (validation, 4xx) vs transient (429, 5xx, timeout). Show the classification function. 5) Concurrency: what must not run twice for the same user or tenant, and the lock or idempotency row. 6) Observability: the log fields and the metric that pages someone (failure rate, age of oldest queued job). 7) A dead-letter path: where the payload goes, who is notified, and how a human replays one job without replaying the batch. 8) Local dev: how to run one job from a test with a fake external call. Do not put the only copy of the user's intent in the queue. Persist the business row first, then enqueue an id.