Skip to main content

Python worker

The zeridion-flare package ships a built-in worker that runs your background jobs end-to-end: it registers with Zeridion, long-polls for work, dispatches each job to your handler, sends liveness heartbeats with progress, honors server-side cancellation, reports the outcome, and drains in-flight jobs on shutdown.

The worker is an opt-in import so thin-client usage stays lean:

from zeridion_flare.worker import FlareWorker, JobContext

Requires: Python 3.10+ · the same httpx dependency as the client (no extra mandatory packages). Typed payloads via Pydantic are an optional extra.

A complete worker​

from dataclasses import dataclass
from zeridion_flare.worker import FlareWorker, JobContext

worker = FlareWorker()  # reads FLARE_API_KEY from the environment

@dataclass
class NewUserEvent:
    email: str
    name: str

@worker.job("SendWelcomeEmail", queue="email", max_attempts=5, timeout=60)
def send_welcome(payload: NewUserEvent, ctx: JobContext) -> None:
    # At-least-once delivery → make handlers idempotent.
    send_email(payload.email)
    ctx.report_progress(1.0)

@worker.recurring("NightlyCleanup", cron="0 3 * * *", queue="maintenance", timezone="UTC")
def nightly(ctx: JobContext) -> None:
    purge_expired()

if __name__ == "__main__":
    worker.run()  # blocks; SIGTERM / SIGINT → graceful drain

Run it like any Python program:

export FLARE_API_KEY=zf_live_sk_...
python worker.py

How it runs​

The worker is threaded: jobs run in a thread pool, and because Python releases the GIL on I/O, I/O-bound handlers get real concurrency. Each cycle:

  1. Register — on startup, the worker announces its id, queues, job types, and any recurring schedules. Registration is best-effort; if it fails the worker keeps polling.
  2. Poll — it long-polls for jobs, advertising how many free slots it has so the server never hands it more work than it can run.
  3. Dispatch — each returned job is decoded, its payload is bound to your handler's type, and the handler runs on a pool thread.
  4. Heartbeat — while a job runs, the worker sends periodic heartbeats carrying the latest reported progress. If the server responds asking to cancel, the job's cancellation handle is tripped.
  5. Acknowledge — when the handler returns, the worker reports success; if it raises (including on cancellation or timeout), it reports failure. The server decides whether to retry or dead-letter — the worker never makes that call itself.
  6. Drain — on SIGTERM / SIGINT the worker stops polling, waits for in-flight jobs to finish (up to the shutdown grace window), and exits.

Worker identity​

Each worker gets a unique id of the form wrk_<host>_<pid>_<random>. The host segment is sanitized to the allowed character set and the random suffix is drawn from a cryptographic source, so two workers — even replicas on the same hostname — never collide.

At-least-once & idempotency​

The final acknowledgement is best-effort. If a worker finishes a job but crashes before the ack lands, the server redelivers it. Write handlers to be idempotent — for example, check whether the work for ctx.job_id was already done before repeating a side effect.

Cancellation is cooperative​

There is no safe way to forcibly kill a running Python thread, so cancellation (on timeout, server cancel, or shutdown) is cooperative: your handler must notice it. Check ctx.is_cancelled() in loops, sleep on the cancellation handle instead of time.sleep, and pass it to long-running calls. A CPU-bound handler that never yields cannot be interrupted and is bounded only by the server's own reclaim window — see Job context.

Next​

Last updated on