Webhook Retry Policy

Here we provide an overview of how we attempt to retry sending webhook events.

3 minute read

In this guide we cover:

Why we attempt to retry
How the retry process works
Available interventions in the retry process

Why do we retry?

Marketplacer considers a successful webhook event to be one that generates a HTTP 2xx response back to Marketplacer. Webhook events are considered to be unsuccessful if:

The request timed out (currently 30s)
The request returns a non 2xx response code

In the case of a failure we enter into a retry process, we do this to give the webhook event the best chance possible of being successfully delivered.

The rest of this document details how we retry.

Retry Process

The retry process is shown below:

Retry Process

Some points to note about the retry process:

We use the “exponential backoff” pattern to retry:
- The time interval between retries increases in an exponential way
- Early retries are close together (e.g within seconds of each other)
- Later retires are far apart (e.g. within hours or days of each other)
We retry 25x
- This is the default value. You can configure this value, with a value range of: >=1 and <= 50
The complete retry cycle (25 attempts) is expected to span ~30 days
The retry cycle can be cancelled
The retry cycle can be requeued (restarted)
If you change the webhook config in between retries, (e.g. the endpoint) the new config will take effect on the next retry

Retry Formula

The retry formula can be expressed as:
(retry_count ** 4) + 15 + (rand(10) * (retry_count + 1))

Retry Interventions

In some circumstances you may be able to intervene in the webhook event / retry process, the ability to intervene is dependent on the “state” of the webhook event.

Note

Interventions are currently “manual”, i.e. they can only be performed via the UI, (not the API). For more information on how you can “intervene” refer to the Errors and Faults article.

Possible states are described below:

Unsent: The webhook event has been triggered but we have not attempted to send yet.
- This state only occurs prior to the 1st send attempt (not subsequent retries - see Rejected state) and should only exist for short period of time.
Sent: The webhook has successfully been sent.
Cancelled: The retry process has been stopped (by someone).
Failed: The retry process has ended unsuccessfully.
Rejected: The retry process is on-going and so far we have only received error responses from the webhook endpoint.
- You will see the current number of retry attempts along with this status.
Skipped: [Not shown above] If deduplication is turned on we have determined to skip sending this event.
- This happens outside the Retry Cycle so is not shown in the retry flow.

An example of the interventions you can perform for each state are shown below:

Retry Process