Heartbeats, progress & cancellation
While a job runs, the worker sends periodic heartbeats to prove the job is still alive, carry its progress, and pick up a cancellation directive. This is fully automatic — you do not call the heartbeat yourself.
Cadence
For each in-flight job the worker:
- sends one heartbeat immediately on claim, then
- sends a heartbeat every
max(10s, timeout / 3)until the job finishes.
The immediate first beat matters for long jobs: it puts the job into the more forgiving liveness window right away, so a slow job is not reclaimed prematurely.
If a heartbeat call fails (a transient network error, say), the worker ignores it and continues — the next heartbeat is the retry. A failed heartbeat never stops the job.
Why this affects your timeouts
The server reclaims a job whose heartbeats go silent. The worker heartbeats automatically as long as the goroutine running your handler can make progress. A handler that fully blocks the program (rather than waiting on I/O or honouring its context) does not prevent heartbeats — they run on a separate goroutine — but a job that never honours cancellation cannot be stopped early. Keep handlers context-aware so cancellation and timeouts take effect promptly.
Progress
Report progress from the handler with jc.ReportProgress(f):
jc.ReportProgress(0.0)
// ... first half ...
jc.ReportProgress(0.5)
// ... second half ...
jc.ReportProgress(1.0)
The most recently reported valid value is attached to the next heartbeat and
shown on the dashboard. Values outside [0, 1] and non-finite values are
ignored, so progress is only reported once you set a valid value. The latest
valid value wins.
Cancellation
A heartbeat can come back with a cancel directive — for example, when a job is cancelled from the dashboard or API. When that happens the worker cancels the handler's context:
func(ctx context.Context, p Payload, jc worker.JobContext) error {
select {
case <-ctx.Done():
return ctx.Err() // server asked to cancel (or timeout/shutdown)
case res := <-work:
return finish(res)
}
}
Cancellation is cooperative. Once the handler returns, the worker reports the job as failed (a cancelled job is reported as failed, not as a separate state) and stops heartbeating it.