8.7 KiB
Stripe Poller
- Investigation
- Tasks
- Build Poller
- KILL Create an event processing endpoint
- Document how to find and replay a Stripe event
Identify webhook events tracked by Stripe that have not yet been processed by our service, and replay them against it.
Investigation
Fetching events on behalf of each connected account
On the afternoon of 2021-07-02 I fetched the 860 connected accounts and fetched webhook events from the previous two hours. Each request to Stripe took, on average, 0.38 seconds, requiring one request per account, totalling 323.77 seconds, or nearly five and a half minutes. No degradation nor failure of Stripe functionality was noted. From all of these requests, a total of 19 events were retrieved.
Alternatives
The Stripe dashboard
Viewing a particular webhook within the Stripe dashboard allows us to view the events sent specifically to that webhook, independent of the account to which it relates. Regrettably, this is fed from a dashboard-specific API which does not appear to be exposed elsewhere for consumption.
Email notification
We are emailed when a webhook event fails to process. It may make sense to alert when this occurs and correct the issue manually, or find a way to automate its replay.
Conclusion
Barring the release of an API to fetch all events sent to a single webhook, ideally filtered by delivery status, fetching these events from Stripe is terribly inefficient. Though we can process recent events in a timely manner, we will likely be further slowed as more customers connect with Stripe, and there is no good solution at this time for dealing with that problem. Given the infrequency with which Stripe fails to send us an event via their retry mechanisms, I expect we will be better served by reacting to notifications of those failures than to routinely slam their API with inefficient requests.
Tasks
Build Poller
Prepare a rundeck job for the Stripe poller
- Create the project
- Build the rundeck job
Add a stripe-payments endpoint to fetch logged events
- Needs only return event ids
- Events are partitioned by date and sorted by time in the time-index GSI, which projects keys only
paths:
/stripe/events:
get:
summary: Fetch webhook events
tags:
- Webhooks
parameters:
- name: last_id
in: query
schema:
$ref: '#/components/schemas/EventId'
- name: limit
in: query
description: Number of events to return per page of results
schema:
type: integer
minimum: 1
maximum: 1000
default: 100
responses:
'200':
description: Event list
headers:
Link:
description: Pagination links
schema:
type: string
example: >-
<{{ base_url }}/stripe/events?last_id=evt_1234abcdef>; rel="next"
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/EventId'
components:
schemas:
EventId:
type: string
description: Unique identifier for the webhook event
example: "evt_1IHAsBIoFf3wvXpR7VLvGfae"Add integrations endpoint enumerating connected Stripe accounts
Reprocess unlogged Stripe events
- Iterate processed event ids into memory
- Iterate over published events, sending them to the stripe-payments webhook endpoint if they are not in the processed set
loop until last page of results
Poller -> StripePayments : GET /stripe/events?since=<timestamp>[&last_id=<last_id>]
StripePayments --> Poller : Return list of event IDs
end
loop until last page of results
Poller -> Integrations : Get list of connected accounts
Integrations --> Poller : Return list of Stripe account IDs
loop for each account
loop until last page of results
Poller -> Stripe : GET /v1/events?created[gte]=<timestamp>&types[]=<...>[&starting_after=<last_id>]
StripePayments --> Poller : Return list of event IDs
alt is an AWeber Ecommerce event and event not in processed
Poller -> StripePayments : POST /stripe/webhooks
end
end
end
end
Determining how far back to look for events
Stripe events are retained for a maximum of thirty days, on their end and also in our event database.
The job could use consul to store the time of the latest fetched event to be referenced in subsequent runs. If it is set, it should collect all events newer than one hour prior to that time (this window may also be configurable). If that time is not set, it should collect all available events. This value should be updated in consul only when the job completes successfully.
Hit the Rundeck API to get the time of the last successful execution.
Retrieving webhook events
https://stripe.com/docs/api/events/list
-
Events can be filtered by type, creation time, and whether they were successfully delivered (the webhook endpoint returned a
200 OKresponse)- Wait, how does it know it was succesfully delivered to our webhook endpoint?
- Should we set up another "client" with its own rate limiting with Stripe for the poller's requests?
My team is hoping to build an automated process to check via the Stripe API for any events that weren't successfully delivered to our webhook endpoint and see that they're handled appropriately. I've been trying out the events list endpoint, and it does not seem to return all of the events that were sent to our webhook, nor does there appear to be any way to identify a webhook to that endpoint to find and filter events for it. We assume this is because the events are triggered from connected accounts. We noticed that the Stripe dashboard is able to show events sent to a webhook and their status, is there some way to achieve this through the API?
- Hector Otero has joined.
- Hi, there. I am glad to help.
Hello. Are you able to see my previous message?
- Yes, I am just reading through it.
- Ok, so your main question is if there is a way to show events sent to a webhook and their status using the API, similar to how it is done on the Stripe dashboard.
- Is that correct?
Correct
- Ok, my brief look into the documentation has not turned up anything regarding how to implement this. I am going to consult with my team members regarding whether any of them knows how to implement this functionality. Once I have more info I can reach out to you via email, is that alright?
Yes, thank you.
- Sure, no worries. Please keep an eye on your email inbox awapi@aweber.com for further correspondence regarding this issue.
- Have a nice day! we will be in contact soon.
Thanks!
- Hector Otero has left.
Signing the webhook request
Webhook events must be signed using a secret key, which we store in consul. The following should result in a valid signature header:
import hmac
import hashlib
import time
unix_timestamp = int(time.time())
json_payload = '{ ... }'
secret_key = 'secret'
message = f'{unix_timestamp}.{json_payload}'
signature = hmac.new(bytes(secret_key, 'utf-8'),
msg=bytes(message, 'utf-8'),
digestmod=hashlib.sha256).hexdigest()
print(f'Stripe-Signature: t={unix_timestamp},v1={signature}')Create playbook and dashboard for the Stripe poller
- Track how many events are resent for processing vs how many are already processed
KILL Create an event processing endpoint
Create an endpoint in the Stripe Payments service for internal use that will, given an event id, fetch that event from Stripe and process it as though it had been sent to the webhook endpoint.
TODO Document how to find and replay a Stripe event
- Use the Stripe UI to find failed events within the past 15 days
- Include steps for fetching an event from more than 15 days ago from the API and sending that to the webhook endpoint.