Here's a scenario you will encounter in production: Stripe sends a payment_intent.succeeded event to your endpoint. Your handler processes it — charges the customer's account, sends a confirmation email, provisions their subscription. Your handler returns 200 OK.
Except Stripe never received that 200. A network blip dropped the response. So Stripe retries. Your handler runs again. The customer gets charged twice.
This is why idempotency is not optional for webhook handlers. Every webhook system retries. Your handler must be safe to call multiple times with the same event.
What Idempotency Means
An operation is idempotent if calling it multiple times produces the same result as calling it once.
DELETE /users/123 is idempotent — deleting a user that's already deleted returns 404 or 204, not a second deletion.
POST /charges with no idempotency key is not idempotent — calling it twice creates two charges.
For webhook handlers, you need to transform non-idempotent operations (charge, email, provision) into idempotent ones by tracking which events have already been processed.
The Idempotency Key Pattern
Every webhook event has a unique identifier. Your job is to track which IDs have been processed:
1. Receive event with ID evt_1234
2. Check: have we processed evt_1234 before?
3a. Yes → return 200 immediately, skip processing
3b. No → mark evt_1234 as "processing", run handler, mark as "processed"Where to find the event ID
| Provider | Header | Field |
|---|---|---|
| Stripe | Stripe-Signature (use payload id field) | event.id |
| GitHub | X-GitHub-Delivery | — |
| Shopify | X-Shopify-Webhook-Id | — |
| SendGrid | — | event[n].sg_event_id |
| GetHook | X-Webhook-Id | event.id |
For providers that don't include an ID header, use a hash of the payload as a best-effort deduplication key.
Implementation: Redis-Backed Idempotency
The simplest production-grade implementation uses Redis SET NX (set if not exists):
func HandleWebhook(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
eventID := extractEventID(r, body) // provider-specific
// Atomic check-and-set with TTL
key := "webhook:processed:" + eventID
set, err := redis.SetNX(ctx, key, "1", 7*24*time.Hour).Result()
if err != nil {
// Redis unavailable — fail open or fail closed based on risk tolerance
log.Error("idempotency check failed", "error", err)
// For critical operations: return 500 to trigger retry
// For non-critical: proceed and log
http.Error(w, "service unavailable", 503)
return
}
if !set {
// Already processed — acknowledge without reprocessing
w.WriteHeader(200)
return
}
// Process the event
if err := processEvent(body); err != nil {
// Remove the key so we can retry later
redis.Del(ctx, key)
http.Error(w, "processing failed", 500)
return
}
w.WriteHeader(200)
}TTL choice: Set TTL to the provider's maximum retry window plus a buffer. Stripe retries for 3 days → use 7-day TTL. GitHub retries for 72 hours → use 4-day TTL.
Implementation: Database-Backed Idempotency
If you don't have Redis, a database table works equally well:
CREATE TABLE processed_webhooks (
event_id TEXT PRIMARY KEY,
processed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
source TEXT NOT NULL
);
CREATE INDEX idx_processed_webhooks_processed_at ON processed_webhooks(processed_at);func markProcessed(ctx context.Context, db *sql.DB, eventID, source string) (bool, error) {
_, err := db.ExecContext(ctx, `
INSERT INTO processed_webhooks (event_id, source)
VALUES ($1, $2)
ON CONFLICT (event_id) DO NOTHING
`, eventID, source)
if err != nil {
return false, err
}
// Check if we inserted or it already existed
var count int
err = db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM processed_webhooks WHERE event_id = $1",
eventID,
).Scan(&count)
return count == 1, err
}Use ON CONFLICT DO NOTHING — this is atomic in Postgres. No race condition between check and insert.
Cleanup: Add a periodic job to delete processed_webhooks records older than 30 days to prevent table bloat.
The Three-State Model
Simple "processed / not processed" idempotency has a failure mode: what happens if your handler crashes halfway through?
The event is marked "processing" (or not yet marked), and on retry the handler will run again — potentially duplicating partial work.
The solution is a three-state model:
CREATE TYPE webhook_state AS ENUM ('processing', 'processed', 'failed');
CREATE TABLE webhook_idempotency (
event_id TEXT PRIMARY KEY,
state webhook_state NOT NULL DEFAULT 'processing',
started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
finished_at TIMESTAMPTZ,
error TEXT
);func handleWithStates(ctx context.Context, eventID string, handler func() error) error {
// Insert with 'processing' state
_, err := db.ExecContext(ctx, `
INSERT INTO webhook_idempotency (event_id, state)
VALUES ($1, 'processing')
ON CONFLICT (event_id) DO NOTHING
`, eventID)
// Check current state
var state string
db.QueryRowContext(ctx,
"SELECT state FROM webhook_idempotency WHERE event_id = $1",
eventID,
).Scan(&state)
switch state {
case "processed":
return nil // Already done, skip
case "processing":
// Was processing — check if it's stale (handler crashed)
// If started > 5 minutes ago, assume failure and retry
}
// Run the handler
if err := handler(); err != nil {
db.ExecContext(ctx, `
UPDATE webhook_idempotency SET state = 'failed', finished_at = NOW(), error = $2
WHERE event_id = $1
`, eventID, err.Error())
return err
}
db.ExecContext(ctx, `
UPDATE webhook_idempotency SET state = 'processed', finished_at = NOW()
WHERE event_id = $1
`, eventID)
return nil
}Idempotency for Multi-Step Handlers
Some webhook handlers do multiple things: update a database, send an email, call an external API. What if step 2 fails after step 1 succeeded?
Anti-pattern: all-or-nothing retry
Step 1: Update DB ✅
Step 2: Send email ❌ (SMTP timeout)
Return 500 → provider retries
Retry: Step 1 runs again → duplicate DB updatePattern: checkpoint-based processing
Track which steps have completed for each event:
type WebhookProgress struct {
EventID string
DBUpdated bool
EmailSent bool
CRMUpdated bool
}On retry, skip completed steps:
func processOrder(ctx context.Context, eventID string, payload OrderPayload) error {
progress := loadProgress(ctx, eventID)
if !progress.DBUpdated {
if err := updateDatabase(ctx, payload); err != nil {
return err
}
saveProgress(ctx, eventID, "db_updated")
}
if !progress.EmailSent {
if err := sendConfirmationEmail(ctx, payload); err != nil {
return err
}
saveProgress(ctx, eventID, "email_sent")
}
return nil
}This ensures each step runs exactly once, even across retries.
When to Skip Idempotency
Not every webhook operation needs strict idempotency. Weigh the cost vs. risk:
| Operation | Idempotency Required? | Risk if Duplicated |
|---|---|---|
| Charge a payment | Yes, absolutely | Customer charged twice |
| Send transactional email | Yes | Duplicate notification |
| Update database field (last seen, timestamp) | No | Harmless overwrite |
| Trigger a webhook test | No | Extra log entry |
| Provision a subscription | Yes | Duplicate subscription |
| Record an analytics event | No (count twice) | Slightly wrong metrics |
| Create a record (idempotent by design) | Check-then-insert | Duplicate record |
For operations with financial or user-facing impact, always implement idempotency. For read operations and analytics, it's typically not worth the complexity.
Testing Your Idempotency Implementation
Idempotency bugs are subtle and often only surface under load or during incidents. Write explicit tests:
func TestWebhookIdempotency(t *testing.T) {
eventID := "evt_test_123"
payload := `{"id": "evt_test_123", "type": "payment.succeeded"}`
// First delivery
resp1 := sendWebhook(t, eventID, payload)
assert.Equal(t, 200, resp1.StatusCode)
assert.Equal(t, 1, countCharges(t)) // One charge created
// Simulated retry (same event ID)
resp2 := sendWebhook(t, eventID, payload)
assert.Equal(t, 200, resp2.StatusCode)
assert.Equal(t, 1, countCharges(t)) // Still one charge — not two
}Test the crash-and-retry scenario explicitly: mark an event as "processing", simulate a crash, verify the retry handles the stale state correctly.
GetHook and Idempotency
GetHook assigns a stable event_id to every event at ingest time. This ID persists across delivery attempts and replays. Your handler can always use event_id as an idempotency key, regardless of which webhook provider sent the original event.
When you replay an event from the DLQ, GetHook uses the same event_id as the original — your idempotency logic naturally deduplicates replays that arrive after the event was already processed downstream.