Webhook Delivery and Retry Reference
Webhooks are authenticated HTTP event deliveries with at-least-once behavior. Endpoints must accept duplicates, delayed delivery, retries, and out-of-order events without duplicating business effects.
Endpoint contract
Use a stable HTTPS endpoint with a valid certificate, bounded request size, and a documented content type. Record a provider delivery ID, event ID, event type, timestamp, and local request ID without logging secrets or sensitive payload data. Restrict methods and media types and return concise statuses within the provider's acknowledgement deadline.
Signature verification
Verify the signature over the exact raw request bytes before transforming JSON or form data. Follow the provider's field parsing, timestamp, encoding, and algorithm rules and compare authentication values in constant time. Support secret rotation with a controlled overlap. Do not put the signing secret in source code, URLs, browser code, or logs.
Replay limits
A valid signature proves origin and integrity under the key, not freshness. Enforce the documented timestamp tolerance and reject implausibly old deliveries, allowing only the required clock skew. Timestamp checks do not replace event deduplication because the same event can be retried legitimately within the allowed window.
Durable acknowledgement
After verification, atomically record the event or publish it to a durable queue, then return the provider's success status promptly. Do not acknowledge before durability, or a process crash can lose the event. Do not run remote calls, emails, or expensive business logic before responding unless they are essential to safe acceptance.
Idempotent processing
Use the provider event or delivery ID under a database unique constraint. A separate existence check and insert can race, so claim it atomically. Give downstream effects their own idempotency keys and handle a crash between effect and completion marking. Retain deduplication state for at least the provider replay and retry horizon.
Retries and response codes
Providers define which statuses and timeouts trigger retry. A late success response can still be retried because the provider did not observe it. Return a non-success status for transient inability to accept an event and monitor the provider's retry exhaustion behavior. For permanently invalid authenticated payloads, follow the provider contract to avoid an endless loop.
Ordering and source of truth
Delivery order is not guaranteed. Use an object version, sequence, timestamp with a defined tie-breaker, or fetch the current source object. Deduplication alone does not prevent an older event from overwriting newer state. Design event handlers as reconciliation steps when the provider offers a reliable source-of-truth API.
Testing and operations
Test invalid signatures, rotated secrets, stale timestamps, duplicate concurrent requests, delayed and reversed events, queue failure, process interruption, and lost acknowledgements. Expose redacted delivery age, verification failures, processing lag, retry count, and terminal errors. Inspect non-secret fields with the HTTP Header Analyzer.
Frequently Asked Questions
Are webhooks exactly once?
No. Design for at-least-once delivery.
When should success be returned?
After verification and durable acceptance.
Does signature prove freshness?
No. Add timestamp and deduplication checks.