Skip to main content

Graceful shutdown

w.Run(ctx) blocks until either the context you pass is cancelled or the process receives SIGINT / SIGTERM. On either signal the worker shuts down gracefully:

  1. It stops polling for new jobs.
  2. It waits for in-flight jobs to finish.
  3. It returns from Run.
// Run wires SIGINT/SIGTERM itself, then drains.
if err := w.Run(context.Background()); err != nil {
    log.Fatal(err)
}

To drive shutdown from your own context instead of (or in addition to) signals, cancel the context you pass to Run:

ctx, cancel := context.WithCancel(context.Background())
go func() { <-someStopSignal; cancel() }()
_ = w.Run(ctx) // returns after the drain

Draining​

By default the drain is unbounded: Run waits for every in-flight handler to return. Because cancellation is cooperative, a handler that honours its context finishes promptly when shutdown begins (its context is cancelled), and the drain completes quickly.

Set Options.ShutdownTimeout to bound the wait:

w := worker.NewWorker(client, worker.Options{
    ShutdownTimeout: 25 * time.Second,
})

If the timeout elapses before every job finishes, Run returns anyway and logs a warning. This is useful under an orchestrator that will hard-kill the process after its own grace period — set ShutdownTimeout a little under that window so the drain is the thing that ends the process, not the kill.

Interrupted jobs are re-delivered​

A job that is still running when the drain ends (because it ignored cancellation, or the bounded timeout elapsed) is not reported as finished. Zeridion Flare reclaims it after its liveness window and re-delivers it to another worker. This is the safe at-least-once behaviour — and the reason handlers must be idempotent.

To make a clean shutdown reliable:

  • honour ctx in handlers so they stop promptly when the drain starts,
  • keep handlers idempotent so a re-delivered job is harmless,
  • size ShutdownTimeout (when set) to fit inside your orchestrator's grace period.

See also​

Last updated on