Webhook Pagination and Cursor-Based Event Replay

When you build a webhook platform, two features turn out to be harder than they look: paginating the event list for customers and replaying past events reliably. Both problems look straightforward until you hit them at production volume. The root cause of most failures is the same: offset-based pagination.

This post explains why offset pagination fails for event streams, how cursor-based pagination fixes it, and how to design a replay API on top of it that holds up when customers need to backfill a new destination or recover from a failed integration.

Why Offset Pagination Breaks for Events

Offset pagination looks like this:

SELECT * FROM events
WHERE account_id = $1
ORDER BY created_at DESC
LIMIT 50 OFFSET 150;

For a static dataset, this works fine. For an event stream that has new rows inserted constantly, it breaks in two ways.

Problem 1: Rows shift between pages. If 10 new events arrive between your first and second page fetch, every row shifts by 10 positions. You skip 10 events or see 10 events twice, depending on which direction they shift. At high insert rates, you get consistent gaps.

Problem 2: Performance degrades with depth. OFFSET 150 tells Postgres to scan and discard 150 rows before returning your 50. At OFFSET 10000, Postgres scans and discards 10,000 rows. For customers paging through large event histories, this causes table scans that grow worse the deeper they go.

The fix for both is cursor-based pagination. Instead of telling the database how many rows to skip, you tell it where you left off.

Cursor-Based Pagination: The Core Pattern

A cursor is an opaque token that encodes the position of the last item the client received. On the next request, the client passes that cursor back, and the server uses it to fetch items strictly after that position.

For an event stream ordered by insertion time, the right cursor is a compound value: (created_at, id). Using both fields handles the case where two events have the same timestamp.

SELECT id, account_id, source_id, status, created_at
FROM events
WHERE account_id = $1
  AND (created_at, id) < ($cursor_created_at, $cursor_id)
ORDER BY created_at DESC, id DESC
LIMIT 50;

This query is stable regardless of concurrent inserts. New events arriving after the cursor position don't affect the result. And the performance is consistent: the (created_at, id) index lets Postgres seek directly to the cursor position rather than scanning from the beginning.

Encoding the Cursor

Expose the cursor as a base64-encoded opaque string, not raw timestamps or UUIDs. This gives you two benefits: clients can't construct cursors manually (reducing the surface for abuse), and you can change the internal format without breaking the API contract.

type EventCursor struct {
    CreatedAt time.Time `json:"ca"`
    ID        uuid.UUID `json:"id"`
}

func EncodeCursor(c EventCursor) string {
    b, _ := json.Marshal(c)
    return base64.URLEncoding.EncodeToString(b)
}

func DecodeCursor(s string) (EventCursor, error) {
    b, err := base64.URLEncoding.DecodeString(s)
    if err != nil {
        return EventCursor{}, fmt.Errorf("invalid cursor: %w", err)
    }
    var c EventCursor
    if err := json.Unmarshal(b, &c); err != nil {
        return EventCursor{}, fmt.Errorf("invalid cursor: %w", err)
    }
    return c, nil
}

The response from GET /v1/events should include the cursor for the next page:

{
  "data": [
    { "id": "evt_01HX...", "status": "delivered", "created_at": "2026-03-26T12:00:00Z" },
    { "id": "evt_01HW...", "status": "dead_letter", "created_at": "2026-03-26T11:58:00Z" }
  ],
  "next_cursor": "eyJjYSI6IjIwMjYtMDMtMjZUMTE6NTg6MDBaIiwiaWQiOiIwMUhXLi4uIn0=",
  "has_more": true
}

When has_more is false, the client has reached the end and should stop paginating. next_cursor is null when there are no more results.

The Event Listing API: Full Handler

Here is a complete Go handler for a cursor-paginated event listing endpoint:

func (h *EventHandler) List(w http.ResponseWriter, r *http.Request) {
    account := httpx.AccountFromContext(r.Context()).(*auth.Account)
    q := r.URL.Query()

    pageSize := 50
    if s := q.Get("limit"); s != "" {
        n, err := strconv.Atoi(s)
        if err != nil || n < 1 || n > 100 {
            httpx.BadRequest(w, "limit must be between 1 and 100")
            return
        }
        pageSize = n
    }

    var cursorCreatedAt time.Time
    var cursorID uuid.UUID
    hasCursor := false

    if raw := q.Get("cursor"); raw != "" {
        c, err := DecodeCursor(raw)
        if err != nil {
            httpx.BadRequest(w, "invalid cursor")
            return
        }
        cursorCreatedAt = c.CreatedAt
        cursorID = c.ID
        hasCursor = true
    }

    evts, err := h.store.ListEvents(r.Context(), account.ID, ListEventsParams{
        Cursor:          hasCursor,
        CursorCreatedAt: cursorCreatedAt,
        CursorID:        cursorID,
        Limit:           pageSize + 1, // fetch one extra to detect has_more
    })
    if err != nil {
        httpx.InternalError(w, err)
        return
    }

    hasMore := len(evts) > pageSize
    if hasMore {
        evts = evts[:pageSize]
    }

    var nextCursor *string
    if hasMore && len(evts) > 0 {
        last := evts[len(evts)-1]
        s := EncodeCursor(EventCursor{CreatedAt: last.CreatedAt, ID: last.ID})
        nextCursor = &s
    }

    httpx.Ok(w, map[string]any{
        "data":        evts,
        "next_cursor": nextCursor,
        "has_more":    hasMore,
    })
}

The "fetch one extra to detect has_more" trick avoids a separate COUNT(*) query, which is expensive on large tables. If you get back pageSize + 1 rows, there is a next page. Trim the extra row before responding.

Cursor Direction: Forward vs. Backward

Most event inspection workflows go backward through time: a customer wants to see recent events, then scroll back to investigate a problem. But replay workflows often go forward: start from a fixed point in the past and consume forward to the present.

Support both directions with a direction parameter:

The since parameter is particularly useful for replay: customers can say "re-deliver everything from the past 6 hours" without knowing any event IDs.

# Fetch all events since midnight UTC today, in ascending order
curl "https://api.yoursaas.com/v1/events?since=2026-03-26T00:00:00Z&direction=after&limit=100" \
  -H "Authorization: Bearer hk_..."

Event Replay: Triggering Redelivery

Pagination gives you the ability to find events. Replay gives you the ability to re-deliver them. These are different operations and should be separate API endpoints.

Single event replay:

POST /v1/events/{id}/replay

Bulk replay by time range:

POST /v1/events/replay
Content-Type: application/json

{
  "since": "2026-03-26T00:00:00Z",
  "until": "2026-03-26T06:00:00Z",
  "destination_id": "dst_xyz789",
  "status_filter": ["dead_letter", "failed"]
}

The bulk replay endpoint enqueues redelivery jobs, not immediate re-processing. This is important: a customer asking to replay 10,000 events should not block your API server or exhaust your delivery worker pool. Enqueue, acknowledge immediately, process asynchronously.

func (h *EventHandler) BulkReplay(w http.ResponseWriter, r *http.Request) {
    account := httpx.AccountFromContext(r.Context()).(*auth.Account)

    var body struct {
        Since         time.Time  `json:"since"`
        Until         time.Time  `json:"until"`
        DestinationID *uuid.UUID `json:"destination_id"`
        StatusFilter  []string   `json:"status_filter"`
    }
    if err := httpx.Decode(r, &body); err != nil {
        httpx.BadRequest(w, "invalid request body")
        return
    }

    if body.Since.IsZero() || body.Until.IsZero() {
        httpx.BadRequest(w, "since and until are required")
        return
    }
    if body.Until.Sub(body.Since) > 7*24*time.Hour {
        httpx.BadRequest(w, "replay window cannot exceed 7 days")
        return
    }

    job, err := h.store.EnqueueBulkReplay(r.Context(), account.ID, BulkReplayParams{
        Since:         body.Since,
        Until:         body.Until,
        DestinationID: body.DestinationID,
        StatusFilter:  body.StatusFilter,
    })
    if err != nil {
        httpx.InternalError(w, err)
        return
    }

    httpx.Created(w, map[string]any{
        "job_id":          job.ID,
        "status":          "queued",
        "estimated_count": job.EstimatedCount,
    })
}

The 7-day window limit is a practical guard. Without it, a single API call could enqueue millions of delivery jobs and overwhelm your worker pool. Enforce limits at the API layer; let customers chain multiple calls for longer replay windows.

The Postgres Schema for Replay Jobs

A simple replay job table that your Postgres-backed worker can pick up:

CREATE TABLE replay_jobs (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    account_id      UUID NOT NULL REFERENCES accounts(id),
    destination_id  UUID REFERENCES destinations(id),
    since           TIMESTAMPTZ NOT NULL,
    until           TIMESTAMPTZ NOT NULL,
    status_filter   TEXT[],
    status          TEXT NOT NULL DEFAULT 'queued',
    -- queued | processing | completed | failed
    events_enqueued INT NOT NULL DEFAULT 0,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    started_at      TIMESTAMPTZ,
    completed_at    TIMESTAMPTZ
);

CREATE INDEX replay_jobs_status ON replay_jobs (status)
    WHERE status = 'queued';

The worker queries FOR UPDATE SKIP LOCKED on status = 'queued', processes each job by paging through matching events using cursor pagination, re-enqueues each event for delivery, and marks the job complete. The cursor-based event listing you built is what powers the worker's internal paging — the same query used by the customer-facing API is used internally for bulk replay.

Common Mistakes

Index Design for Performance

Two indexes cover the access patterns described in this post:

-- Cursor pagination and replay range queries
CREATE INDEX events_account_created_id ON events (account_id, created_at DESC, id DESC);

-- Status-filtered replay (e.g., only dead_letter events)
CREATE INDEX events_account_status_created ON events (account_id, status, created_at DESC);

The first index handles all pagination queries. The second handles status-filtered replay. At 10M+ events per account, these indexes are what stand between a 5ms query and a 30-second full table scan.

Cursor-based pagination and reliable event replay are table-stakes features for any webhook platform that wants to be taken seriously by engineering teams. The offset-based alternative looks simpler to build and works fine in demos, but it degrades at exactly the moment customers need it most: during incident recovery, when they're replaying large volumes of failed events.

GetHook's event listing and replay API is built on the cursor pattern described here. You can paginate forward or backward through your event history, replay individual events or bulk ranges, and filter by status — all without the performance cliffs that come with offset pagination.

See the GetHook event replay docs →