roam/aweber/20210316155320-stripe_payments_tracking_database.org

Stripe payments tracking database

• State Tracking
  • Legacy purchase
  • Product purchase
• Tables
  • Purchases
  • Subscriptions and Split Payments
  • Payments
  • Events
    • Modeling in DynamoDB
      • Table
      • Attributes
      • TTL
      • Time index (GSI)
      • Stripe Account index (GSI)
      • Subscription index (GSI)
      • IAM Policy
• Notes

Database for tracking payments and subscriptions managed by the Stripe Payments Service.

State Tracking

Legacy purchase

• Client initiates purchase

• Backend coordinates with Stripe and returns a payment intent

  • Track new purchase and related incomplete payment
• Client completes purchase with Stripe

• Stripe notifies backend of payment intent status (update purchase)

  • Track event
  • Track payment as completed
  • Fulfill purchase
  • Track purchase as fulfilled

  participant Stripe
  actor Client
  participant "stripe-payments" as Backend
  database Tracking

  Client -> Backend: Initiate purchase
  Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
  Backend -> Stripe : Create payment intent
  Backend -> Tracking : <font color="blue">Store incomplete payment</font>
  Backend -> Client : Return payment intent
  Client -> Stripe : Complete purchase
  ...
  alt Success
    Stripe ---> Backend : payment_intent.succeeded
    Backend -> Tracking : <font color="green">Store event</font>
    alt Event not previously handled
      Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
      Backend -> Backend : Fulfill purchase
      Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
    end
  else Failure
    Stripe --> Backend : payment_intent.payment_failed
    Backend -> Tracking : <font color="green">Store event</font>
  end

Legacy purchase flow with events

Product purchase

• Client prepares payment method with Stripe
• Client initiates purchase

• Backend coordinates with Stripe to complete the purchase

  • Attaches the payment method to the customer

  • For single products

    • Creates an invoice item
    • Creates an invoice
    • Pays the invoice

  • For subscriptions

    • Creates the subscription

  participant Stripe
  actor Client
  participant "stripe-payments" as Backend
  database Tracking

  Client -> Stripe : Create payment method
  Client -> Backend: Initiate purchase
  Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
  alt Non-recurring
          Backend -> Stripe : Create invoice item
          Backend -> Stripe : Create invoice
          Backend -> Stripe : Pay invoice
          Backend -> Tracking : <font color="blue">Store completed payment</font>
  else Recurring
          Backend -> Stripe : Create subscription
          Backend -> Tracking : <font color="blue">Store active subscription</font>
  end
  ...
  Stripe --> Backend : Subscription activated
  ...
  alt
          Stripe --> Backend : customer.subscription.updated (no longer active)
  else
          Stripe --> Backend : subscription_schedule.canceled
  end
  Backend -> Tracking : <font color="green">Store event</font>
  Backend -> Backend : Trigger unsubscribe actions
  Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>

Product purchase flow

  participant Stripe
  actor Client
  participant "stripe-payments" as Backend
  database Tracking

  == Payment succeeded ==
  Stripe --> Backend : payment_intent.succeeded
  Backend -> Tracking : <font color="green">Store event</font>
  alt Event not previously handled
          Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
          Backend -> Stripe : Look up invoice

          note over Backend
                  Recurring if an invoice exists
                  and has an associated subscription
          end note
          alt Non-recurring
                  Backend -> Backend : Fulfill purchase
                  Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
          else Recurring
          end
  end
  == Payment failed ==
  Stripe --> Backend : payment_intent.payment_failed
  Backend -> Tracking : <font color="green">Store event</font>

Payment events

  participant Stripe
  actor Client
  participant "stripe-payments" as Backend
  database Tracking

  == Subscription activated ==
  alt
          Stripe --> Backend : customer.subscription.created (active)
  else
          Stripe --> Backend : customer.subscription.updated (active)
  end
  Backend -> Tracking : <font color="green">Store event</font>
  alt New subscription
          Backend -> Tracking : <font color="blue">Mark subscription as active</font>
          Backend -> Backend : Fulfill purchase
          Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
  else Subscription already processed
          note over Backend
                  One of the following is true:

                  <font color="green">- Already received an activation event for this subscription</font>
                  <font color="green">- Already receieved a termination event for this subscription</font>
                  <font color="blue">- Subscription already marked as active or terminated</font>
                  <font color="blue">- Purchase already marked as fulfilled</font>
          end note
  end
  == Subscription terminated ==
  alt
          Stripe --> Backend : customer.subscription.updated (no longer active)
  else
          Stripe --> Backend : subscription_schedule.canceled
  end
  Backend -> Tracking : <font color="green">Store event</font>
  alt Active subscription
          Backend -> Backend : Trigger unsubscribe actions
          Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>
  else Subscription not yet processed
          note over Backend
                  One of the following is true:

                  <font color="green">- Did not receive an activation event for this subscription</font>
                  <font color="blue">- Subscription is not tracked</font>
          end note
  else Subscription already terminated
          note over Backend
                  One of the following is true:

                  <font color="green">- Already received a termination event for this subscription</font>
                  <font color="blue">- Subscription already marked as terminated</font>
          end note
  end

Subscription events

Tables

Purchases

Purchases made via the Stripe Payments Service.

  [*] -> New
  New -> Fulfilled

Purchase state diagram

Field	Type	Nullable	Description
id	UUID	N	Auto-generated purchase ID
created	timestamp	N	Time the purchase was initiated
last_updated	timestamp	N	Time the purchase was last updated
account	UUID	N	The AWeber customer account purchased from
stripe_account	text	N	The Stripe account of the AWeber customer
fulfilled	boolean	N	Was this purchase fulfilled

Purchases

• Store which automations were applied & when

Subscriptions and Split Payments

Purchases made with recurring payments, managed using a Stripe Subscription.

  [*] -> New
  New -> Active
  Active --> Terminated : Payment Failed
  Active --> Terminated : Unsubscribed
  New --> Terminated : Payment Failed

Subscription state diagram

Field	Type	Nullable	Description
id	text	N	The Stripe subscription ID
created	timestamp	N	Time the purchase was initiated
last_updated	timestamp	N	Time the purchase was last updated
account	UUID	N	The AWeber customer account the subscription belongs to
stripe_account	text	N	The Stripe account of the AWeber customer
status	subscription_status	N	Status of the subscription

Subscriptions

<<subscription_status>>

new

active

terminated

ENUM: Subscription Status

Payments

Payments collected from buyers.

  [*] -> New
  New --> Paid : Payment succeeded
  New --> Failed : Payment failed

Payment state diagram

Field	Type	Nullable	Description
id	text	N	The Stripe payment intent ID
created	timestamp	N	Time the purchase was initiated
last_updated	timestamp	N	Time the purchase was last updated
account	UUID	N	The AWeber customer account that was paid
stripe_account	text	N	The Stripe account of the AWeber customer
purchase	UUID	N
status	payment_status	N	Status of the payment intent

Payments

<<payment_status>>

new

paid

failed

ENUM: Payment Status

Events

Field	Type	Nullable	Description
id	text	N	Stripe event id
type	text	N	Stripe event type
account	UUID	N	The AWeber customer account the subscription belongs to
stripe_account	text	N	The Stripe account of the AWeber customer
timestamp	timestamp	N	Time event was published
subscription	text	Y	Related Stripe subscription ID

Events

Events should expire out of the database over time. Stripe maintains events in its database for up to 30 days.

Modeling in DynamoDB

Table

Partition Key

id

The table index provides for rapid lookup of individual events.

Attributes

Field	Type	Required	Description
id	String	Y	Stripe event id
type	String	Y	Stripe event type
stripe_account	String	Y	The Stripe account of the AWeber customer
timestamp	String (timestamp)	Y	Time event was published (e.g. "2020-01-01 01:02:34.567")
date	String (date)	Y	Date portion of timestamp (e.g. "2020-01-01")
time	String (time)	Y	Time portion of timestamp (e.g. "01:02:34.567")
expiration	Number (Unix timestamp)	Y	TTL expiration time of the event record
subscription	String	N	Related Stripe subscription ID

TTL

The events table will automatically expire and remove rows for which expiration has passed. We may set expiration to be the event creation time plus thirty days to match Stripe's own expiration. This will prevent old items from piling up in the table.

Time index (GSI)

Partition Key

date

Sort Key

time

Projection

Keys Only

The date and time portions of the event timestamp will be separated to allow for scanning all events within a day, filterable by time. This should support our polling job, which should run multiple times per day. We'll have to be careful around date boundaries when looking up events (e.g. by including a query for events from the previous day during the first run of the current day).

Stripe Account index (GSI)

Partition Key

stripe_account

Sort Key

timestamp

Projection

Include type, subscription

The account index provides for rapid lookup of events for a particular stripe account, filterable by timestamp.

Subscription index (GSI)

Partition Key

subscription

Sort Key

timestamp

Projection

Include type, stripe_account

The account index provides for rapid lookup of events for a particular subscription, filterable by timestamp.

IAM Policy

Put together by referencing DynamoDB IAM Policies.

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": "dynamodb:ListTables",
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": [
                  "dynamodb:BatchGetItem",
                  "dynamodb:BatchWriteItem",
                  "dynamodb:ConditionCheckItem",
                  "dynamodb:PutItem",
                  "dynamodb:DeleteItem",
                  "dynamodb:Scan",
                  "dynamodb:Query",
                  "dynamodb:UpdateItem",
                  "dynamodb:DescribeTimeToLive",
                  "dynamodb:CreateTable",
                  "dynamodb:DescribeTable",
                  "dynamodb:GetItem",
                  "dynamodb:UpdateTable",
                  "dynamodb:UpdateTimeToLive"
              ],
              "Resource": [
                  "arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*/index/*",
                  "arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*"
              ]
          },
      ]
  }

Stripe Payments DynamoDB IAM Policy

Notes

• How does the poller use these stored events?

  • To identify events retrieved from Stripe which do not need to be replayed

• How do we replay events?

  • Send it to the webhook endpoint? An internal webhook endpoint that shares code with the public one?

    • The same webhook endpoint would be preferable, requires signing the payload

• Do we expose the processed events as an internal endpoint in the stripe-payments service, or give the polling app a database connection?

  • API seems preferable

Following some discussion, the Events table should be sufficient for now.