Remove AWeber org files
This commit is contained in:
parent
74f9662c68
commit
ae2156ae09
220 changed files with 0 additions and 18175 deletions
|
|
@ -1,14 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: ac416861-ce45-49ac-8b60-f8ea39362135
|
||||
:END:
|
||||
#+title: Migration to common RabbitMQ
|
||||
|
||||
All services and consumers pointed at the legacy RabbitMQ cluster in
|
||||
Conshohocken should be migrated to the new common-rabbitmq cluster as a [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]].
|
||||
|
||||
The new servers are available at
|
||||
=common-rabbitmq.service.${ENVIRONMENT}.consul=.
|
||||
|
||||
* Legacy hostnames
|
||||
- =rabbitmq.service.${ENVIRONMENT}.consul=
|
||||
- =rabbit{1-3}.int.{stg,prd}.csh=
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e4d00c11-da8a-4c91-8f38-ce939846e5cb
|
||||
:END:
|
||||
#+title: CoreAPI
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: ddeea682-c8f0-4607-8e2b-0f8ee4fd6191
|
||||
:END:
|
||||
#+title: Puppet
|
||||
|
||||
- Repository :: [[https://gitlab.aweber.io/PSE/config-management/puppet/]]
|
||||
|
||||
* Applying changes on a node
|
||||
- Ensure changes are merged and tagged in the upstream repository.
|
||||
- As root, run =puppetd --test=.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 592aa825-154c-4659-8193-75b0ce1f2e5c
|
||||
:END:
|
||||
#+title: PGBouncer port migration
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: ebea379a-8fa6-4e22-9275-a9fc98c02804
|
||||
:END:
|
||||
#+title: Pagerduty
|
||||
|
||||
https://aweber.pagerduty.com/
|
||||
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 24578fe5-6ca0-4000-a7cd-201e952e4c76
|
||||
:END:
|
||||
#+title: Mail Relay
|
||||
|
||||
A postfix instance running in Kubernetes for the express purpose of supporting
|
||||
legacy emails in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] until they are all migrated to use [[id:32c66bc8-a397-4f50-96cd-2aec70dd14c5][Corporate
|
||||
Notifications]].
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e1b95d0e-366e-4ecf-b867-409b6b6c6ee8
|
||||
:END:
|
||||
#+title: Momentum
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89
|
||||
:END:
|
||||
#+title: Control Panel
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 32c66bc8-a397-4f50-96cd-2aec70dd14c5
|
||||
:END:
|
||||
#+title: Corporate Notifications
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 7a362881-875f-4f74-8053-55f63826da63
|
||||
:END:
|
||||
#+title: Refunding an Order
|
||||
|
||||
Refund options (via Admin)
|
||||
1. Use a balance item
|
||||
1. Insert a manual balance item, set Term at 1, a negative amount, and optionally a description.
|
||||
2. Add an invoice and pay it.
|
||||
2. Close the current package
|
||||
1. Open the current package, select cancellation in date closed, with a cancel reason
|
||||
|
|
@ -1,46 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d17e934b-b340-4246-88f0-9b36527100c0
|
||||
:END:
|
||||
#+title: Login Throttling
|
||||
|
||||
* CAPTCHA Throttling
|
||||
|
||||
We have login captcha throttling in place for the following:
|
||||
| Tracked behavior | CAPTCHA threshold | Time Interval |
|
||||
|-------------------------------------------------------+-------------------+---------------|
|
||||
| Repeated unsuccessful attempts with the same username | 3 attempts | 10 minutes |
|
||||
| Repeated attempts from the same IP address | 3 attempts | 12 hours |
|
||||
| Repeated attempts using the same Sift ID | 3 attempts | 30 minutes |
|
||||
| Invalid or missing CSRF token | Immediate | N/A |
|
||||
| Missing customer cookie | Immediate | N/A |
|
||||
|
||||
When a user meets one of the thresholds above, they will be presented with a
|
||||
CAPTCHA challenge. This does not necessarily mean a puzzle will have to be
|
||||
solved, only that the CAPTCHA script will attempt to determine if the user is a
|
||||
bot. Even if the user has correctly entered their credentials on the subsequent
|
||||
attempt, the CAPTCHA challenge will still occur.
|
||||
|
||||
All of the above thresholds are checked concurrently for each login attempt.
|
||||
|
||||
When a throttled user logs in successfully, the following occurs, the *username*
|
||||
threshold is reset. No other thresholds are cleared. This means that even after
|
||||
a user is able to successfully log in to an account, it is still possible for
|
||||
them to be throttled after failing to log in again because they are now being
|
||||
throttled by IP address.
|
||||
|
||||
* Sift ID Blocking
|
||||
During previous login attacks, we've documented a set of Sift IDs that have been
|
||||
used repeatedly during those attempts. Those IDs are blocked with CAPTCHA
|
||||
*immediately*, with a 20% chance that we will present them a faked successful
|
||||
response. This is done to throw off attackers using these IDs.
|
||||
|
||||
* Code
|
||||
All the captcha / throttling logic that’s currently in place lives in
|
||||
[[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][aweber_app/controllers/account_controller.php]], mainly in the =loginAjax= and
|
||||
=isThrottled= methods. The repeated actions are tracked using
|
||||
[[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/throttler.php][php5-vendors/vendors/throttler.php]], which uses counters in Redis with a TTL
|
||||
attached.
|
||||
|
||||
* Graphs
|
||||
Login attempts and throttling are graphed in Grafana on the [[https://grafana.aweber.io/d/000000530/account-logins][Account Logins
|
||||
dashboard]].
|
||||
|
|
@ -1,70 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: a81b2ff0-5ede-44b3-8f82-960357f15428
|
||||
:END:
|
||||
#+title: Python Services
|
||||
#+OPTIONS: ^:nil
|
||||
#+PROPERTY: header-args :exports code
|
||||
|
||||
* Platform
|
||||
- Python 3.9[fn:programming-languages]
|
||||
- Tornado 6[fn:frameworks]
|
||||
|
||||
* Code Style
|
||||
Code style must be enforced using flake8.[fn:python-lint-checking]
|
||||
|
||||
#+begin_example
|
||||
[flake8]
|
||||
application-import-names = PACKAGE_NAME,tests
|
||||
exclude = build,env
|
||||
import-order-style = pycharm
|
||||
#+end_example
|
||||
* Requirements
|
||||
** Uncaught errors are logged and alerted via Sentry
|
||||
#+NAME: packages
|
||||
- =sentry-sdk=
|
||||
|
||||
#+begin_src python
|
||||
from sentry_sdk import init
|
||||
init(SENTRY_DSN)
|
||||
#+end_src
|
||||
|
||||
** A status endpoint is exposed
|
||||
The endpoint should be provide the following fields:
|
||||
- application :: The name of the service
|
||||
- environment :: The operating environment the instance of the service is
|
||||
running in (i.e. "development", "testing", "staging" or "production")
|
||||
- status :: Current service status (e.g.: "ok", "starting")
|
||||
- version :: Packaged version of the service instance
|
||||
- python_version :: The python version running the service instance
|
||||
(=platform.python_version()=)
|
||||
|
||||
The endpoint should return =200= when the service is healthy, and =503= if the
|
||||
service is not ready to serve requests.
|
||||
** The service self-hosts its API documentation
|
||||
An OpenAPI specification is hosted using ReDoc at the root service URL.[fn:backend-services]
|
||||
** Test coverage reports are available in SonarQube
|
||||
** Structured logging
|
||||
json-scribe
|
||||
** Provides consistent error responses
|
||||
json-problem
|
||||
** The service represents itself using its service name and version
|
||||
- The service must include its name and version in its =Server= response header.[fn:response-headers]
|
||||
- The service must include its name and version in the =User-Agent= header for all its HTTP requests.[fn:user-agent]
|
||||
|
||||
Both of these should be presented as =${service-name}/${version}=, e.g.:
|
||||
=user-management/1.0.0=.
|
||||
* References
|
||||
- [[https://confluence.aweber.io/display/STD/Back+End+Services]]
|
||||
|
||||
* Footnotes
|
||||
|
||||
[fn:user-agent] https://confluence.aweber.io/display/STD/RESTful+APIs#heading-User-AgentRequestHeader
|
||||
[fn:response-headers] https://confluence.aweber.io/display/STD/RESTful+APIs#heading-ResponseHeaders
|
||||
|
||||
[fn:backend-services] https://confluence.aweber.io/display/STD/Back+End+Services
|
||||
|
||||
[fn:frameworks] https://confluence.aweber.io/display/STD/Development+Frameworks
|
||||
|
||||
[fn:python-lint-checking] https://confluence.aweber.io/display/STD/Python+Lint+Checking
|
||||
|
||||
[fn:programming-languages] https://confluence.aweber.io/display/STD/Programming+Languages
|
||||
|
|
@ -1,58 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: dcb2f0ad-72e8-41ff-84f5-07caf8c7fe8e
|
||||
:END:
|
||||
#+title: Easy Commerce MVP Brainstorm Notes
|
||||
#+TODO: ASK(a) FOLLOW-UP(f) | ANSWERED(d)
|
||||
#+DATE: <2020-10-29 Thu>
|
||||
|
||||
* ANSWERED How is a product identified and tracked?
|
||||
- The customer configures a product name and price on the landing page.
|
||||
- The product name is the "goal description" (page description or note in the
|
||||
DB) in our current sales tracking.
|
||||
- We'd like to include the product name, price, and URL in the subscriber's
|
||||
activity.
|
||||
- Full, separate product tracking could be something we build later if there is
|
||||
sufficient interest.
|
||||
|
||||
* ANSWERED How will the Stripe integration be configured?
|
||||
Will it be configured at the account level, and if so, could it leverage the
|
||||
existing system for linking things like FB & Twitter?
|
||||
|
||||
----------------------------------------------------------------------
|
||||
|
||||
- Stripe links via OAuth.
|
||||
- We will have only one Stripe account per AWeber account.
|
||||
- We will be logging charges to Stripe, not full orders (with product tracking
|
||||
information).
|
||||
- Investigation on whether the existing system will work is pending.
|
||||
* ANSWERED How does this interact with service limits?
|
||||
- Availability of the easy commerce feature
|
||||
- Subscriber limits?
|
||||
|
||||
----------------------------------------------------------------------
|
||||
|
||||
- Handle errors with free-tier subscriber limits.
|
||||
- Consider transaction service fee changes based on service limits.
|
||||
* ANSWERED When should the buyer be added as a subscriber?
|
||||
Immediately upon payment submission, or upon asynchronous payment confirmation?
|
||||
|
||||
----------------------------------------------------------------------
|
||||
|
||||
- The async web hook confirms the payment was made successfully.
|
||||
+ We're avoiding putting logic here only to limit the scope of adding storage
|
||||
for subscriber and purchase information to react to the event with.
|
||||
- If the subscriber already exists, we'll update them with any new information.
|
||||
* ANSWERED How will reports be handled?
|
||||
Is there a dependency on the upcoming Analytics View service?
|
||||
|
||||
----------------------------------------------------------------------
|
||||
|
||||
- Only add All Lists: Products Sold in MVP
|
||||
+ This seems like it'd just be a filter on event type in our current sales
|
||||
over time report.
|
||||
- The Analytics View service will provide report data via the authenticated
|
||||
public API interface.
|
||||
+ May not need to be coupled to the project, as we may be able to filter the
|
||||
existing report to distinguish between current sales tracking and landing
|
||||
page sales.
|
||||
+ Anything /new/ should be built using the new Analytics View service.
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 9b3ed74d-41d4-4784-89e3-6a9183903b9e
|
||||
:END:
|
||||
#+title: Stripe Payments Service
|
||||
* Overview
|
||||
The Stripe Payments service mediates purchases made by buyers from AWeber
|
||||
customers through the [[id:7d940785-68b9-4da7-bad1-4771d496168c][Stripe payment platform]] and delivers their goods via list
|
||||
subscription.
|
||||
* How It Works
|
||||
** Ordering
|
||||
The order endpoint prepares a [[https://stripe.com/docs/api/payment_intents][Payment Intent]] with Stripe that the buyer will pay
|
||||
directly using Stripe's public API. The payment intent captures the amount to be
|
||||
paid, any fees that AWeber will collect, and account and list metadata used to
|
||||
fulfill the order.
|
||||
*** Email validation
|
||||
Emails are checked against the [[https://gitlab.aweber.io/CP/Services/validation][CP Email Validation Service]] to ensure they can be
|
||||
added as subscribers successfully. The following steps are performed by the service:
|
||||
1. Validates that the format of the submitted email address matches AWeber's
|
||||
internal email formatting rules.
|
||||
2. Applies address normalization to remove any ISP specific markup.
|
||||
3. Extracts the domain-part from the normalized address and validates that there
|
||||
are MX records for the domain-part via DNS lookup.
|
||||
4. Checks if the both the submitted and normalized version of the email address
|
||||
would be blocked by the blocklist.
|
||||
|
||||
Should the validation service fail unexpectedly or be unavailable at the time an
|
||||
order is placed, a warning will be logged and only the email format check will
|
||||
be performed.
|
||||
*** CAPTCHA
|
||||
**** Domain validation
|
||||
***** Allowed domains
|
||||
The following root domains provided by AWeber for [[https://confluence.aweber.io/pages/viewpage.action?pageId=118885193][Web Content (a.k.a Landing
|
||||
Pages)]] always pass CAPTCHA domain validation:
|
||||
|
||||
- =aweberpages.com=
|
||||
- =aweb.page=
|
||||
***** Custom domains
|
||||
All other domains are checked against the [[https://confluence.aweber.io/display/AR/Custom+Domain+Service][Custom Domain Service]] to ensure they
|
||||
are owned and managed by the AWeber account from which the purchase is being
|
||||
made.
|
||||
*** Fees
|
||||
Fees are collected for each sale as a percentage of the sale value configured in
|
||||
the [[https://confluence.aweber.io/display/CT/Service+Limits+API][Service Limits API]] for the AWeber account, rounded to the nearest cent with
|
||||
a minimum fee of $0.01.
|
||||
** Fulfillment
|
||||
*** Adding the subscriber
|
||||
Subscribers are added to the list configured for the payment if they are not
|
||||
already subscribed. If the subscriber is currently on the list in an unconfirmed
|
||||
state, they will be marked as subscribed. Any tags configured for the sale will
|
||||
be added to the subscriber.
|
||||
*** Purchase Tracking
|
||||
A [[https://confluence.aweber.io/display/AR/Pageview][Pageview Event]] is emitted with details of the order so that the purchase is
|
||||
tracked in the subscriber's activity.
|
||||
|
|
@ -1,448 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: a9835afc-e0be-4436-8274-c3898fdf119c
|
||||
:END:
|
||||
#+title: Recurring and split Stripe payments
|
||||
#+options: prop:t
|
||||
|
||||
To be implemented in the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||
|
||||
Recurring and split payments are implementable in Stripe as [[https://stripe.com/docs/billing/subscriptions/overview#subscription-lifecycle][subscriptions]]. The
|
||||
key difference is that split payments have a limited number of payment
|
||||
iterations (e.g. 5 monthly installments) whereas recurring payments are ongoing.
|
||||
* Stripe objects
|
||||
** Products
|
||||
Represents something being sold.
|
||||
** Prices
|
||||
Amount and frequency to be charged for a product. Many prices may be available
|
||||
for a single product.
|
||||
** Customers
|
||||
Represents a buyer (subscriber?). Needs to be configured with a payment method.
|
||||
** Subscriptions
|
||||
Represents a [[*Products][Product]] being offered to a [[* Customers][Customer]] at a [[* Prices][Price]].
|
||||
** Invoices
|
||||
Generated at billing times.
|
||||
** Payment Intents
|
||||
Attempts to pay an [[* Invoices][Invoice]].
|
||||
* Easy Commerce Flow
|
||||
** Setup
|
||||
[[*Products][Products]] and [[* Prices][Prices]] should be set up in the customer's stripe account and
|
||||
referenced in their landing page.
|
||||
|
||||
#+begin_src plantuml :file "ecommerce-products.svg"
|
||||
actor "AWeber Customer" as customer
|
||||
participant "Landing Page Editor" as lp
|
||||
participant "Stripe Payments (Authenticated)" as sp
|
||||
participant "Stripe" as stripe
|
||||
|
||||
== Load product information ==
|
||||
customer -> lp : Edit landing page
|
||||
lp -> sp : GET /stripe-authenticated/products
|
||||
sp -> stripe : Get products with prices
|
||||
sp -> lp : Return list of products with pricing
|
||||
|
||||
== Save product information ==
|
||||
customer -> lp : Save landing page
|
||||
alt create a new product & price
|
||||
lp -> sp : POST /stripe-authenticated/products
|
||||
else update an existing product & price
|
||||
lp -> sp : PATCH /stripe-authenticated/products/{UUID}
|
||||
end
|
||||
sp -> stripe : Store product and price
|
||||
sp -> lp : 200 Return product and price IDs
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:ecommerce-products.svg]]
|
||||
** Purchase
|
||||
#+begin_src plantuml :file "ecommerce-subscribing.svg"
|
||||
actor "Buyer" as buyer
|
||||
participant "Landing Page" as lp
|
||||
participant "Stripe Payments (Unauthenticated)" as sp
|
||||
box "Internal"
|
||||
participant "Core API" as capi
|
||||
end box
|
||||
participant "Stripe" as stripe
|
||||
|
||||
buyer -> lp : Place order
|
||||
lp -> sp : POST /stripe/subscription with product and price IDs
|
||||
sp -> stripe : Create customer
|
||||
sp -> stripe : Create subscription
|
||||
sp -> stripe : Create initial invoice
|
||||
sp -> lp : Return payment intent client secret
|
||||
|
||||
lp -> stripe : Confirm payment
|
||||
stripe -> lp : Return payment intent
|
||||
|
||||
lp -> sp : POST /stripe/fulfillment
|
||||
sp -> stripe : Fetch payment intent and check status
|
||||
sp -> capi : Add subscriber with tags
|
||||
sp -> lp : 200 OK
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:ecommerce-subscribing.svg]]
|
||||
|
||||
*** Subscribing
|
||||
Unlike ordering, which creates a single payment intent, we'll instead need to
|
||||
take the payment information provided by the buyer and create a [[* Customers][Customer]] object
|
||||
in Stripe, as well as a [[* Subscriptions][Subscription]] to a [[*Products][Product]] at a specific [[* Prices][Price]].
|
||||
*** Fulfillment
|
||||
After a successful payment, the [[* Subscriptions][Subscription]] is marked as =active=, and the
|
||||
buyer should be subscribed to start receiving their content.
|
||||
** Automatic payments
|
||||
#+begin_src plantuml :file "ecommerce-payment-webhooks.svg"
|
||||
participant "Stripe Payments (Unauthenticated)" as sp
|
||||
box "Internal"
|
||||
participant "RabbitMQ" as amqp
|
||||
end box
|
||||
participant "Stripe" as stripe
|
||||
|
||||
stripe -> sp : POST /stripe/webhooks (payment_intent.succeeded)
|
||||
sp -> amqp : Sales tracking event (pageview.v4)
|
||||
sp -> amqp : Payment succeeded event (stripe_payment_succeeded.v1)
|
||||
sp -> stripe : 200 OK
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:ecommerce-payment-webhooks.svg]]
|
||||
|
||||
** Payment failure / Cancellation
|
||||
#+begin_src plantuml :file "ecommerce-cancellation-webhooks.svg"
|
||||
participant "Stripe Payments (Unauthenticated)" as sp
|
||||
box "Internal"
|
||||
participant "Core API" as capi
|
||||
end box
|
||||
participant "Stripe" as stripe
|
||||
|
||||
stripe -> sp : POST /stripe/webhooks (customer.subscription.updated)
|
||||
sp -> stripe : Fetch product metadata
|
||||
sp -> capi : Remove tags from subscriber or unsubscribe
|
||||
sp -> stripe : 200 OK
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:ecommerce-cancellation-webhooks.svg]]
|
||||
|
||||
* Questions
|
||||
- Should stripe-payments or the integrations service handle price and product management?
|
||||
+ stripe-payments would need authenticated endpoints exposed
|
||||
|
||||
* Project :taskjuggler_project:
|
||||
:PROPERTIES:
|
||||
:COLUMNS: %50ITEM(Task) %Effort %allocate %task_id %blocker
|
||||
:start: 2021-02-22
|
||||
:END:
|
||||
|
||||
#+begin: columnview
|
||||
| Task | Effort | allocate | task_id | blocker |
|
||||
|-------------------------------------------------------------------+--------+-------------------+-------------------------+-------------------------------------------------------------------------------|
|
||||
| Project | | | | |
|
||||
| Tasks | | | | |
|
||||
| Backend: Product Management | | | | |
|
||||
| Define API for managing products and prices | 1d | backend | api | |
|
||||
| Endpoint: Create products and prices | 3d | backend | create | api |
|
||||
| Endpoint: Update products and prices | 5d | backend | update | api |
|
||||
| Endpoint: Fetch products and prices | 2d | backend | fetch | api |
|
||||
| Endpoint: Update /order | 2d | backend | order_both | api |
|
||||
| Migrate products from existing landing pages into Stripe on save | | frontend | migrate | order_both |
|
||||
| Backfill products from existing landing pages into Stripe | 10d | backend, frontend | backfill | order_both |
|
||||
| Backend: Subscriptions | | | | |
|
||||
| Endpoint: Update /order to create customers for product purchases | 10d | backend | order_products | milestone_management |
|
||||
| Endpoint: Add subscription support to /order | 10d | backend | order_subscriptions | milestone_management |
|
||||
| Endpoint: Update /fulfill & /webhooks (iteration 1) | 3d | backend | webhooks_1 | milestone_management |
|
||||
| Endpoint: Update /webhooks to process subscriptions (iteration 2) | 8d | backend | webhooks_2 | webhooks_1, order_subscriptions |
|
||||
| Database: Add database for event tracking | 3d | backend | database | |
|
||||
| Endpoint: Add event tracking to /webhooks | 5d | backend | tracking | database, webhooks_2 |
|
||||
| Backend: Resiliency | | | | |
|
||||
| Poller: Create poller to check for unprocessed events | 10d | backend | poller | milestone_subscriptions |
|
||||
| Backend: Cleanup | | | | |
|
||||
| Backfill products from existing landing pages into Stripe | 10d | backend, frontend | backfill2 | milestone_subscriptions |
|
||||
| Endpoint: Update /order remove /fulfill | 1d | backend | cleanup | backfill2 |
|
||||
| Frontend | | | | |
|
||||
| Frontend: Plugin | 2d | frontend | plugin | |
|
||||
| Frontend: Templates | 1d | frontend | templates | plugin |
|
||||
| Frontend: Builder (Product Management) | 12d | frontend | builder_products | plugin, templates |
|
||||
| Frontend: Builder (Recurring Payments) | 17d | frontend | builder_recurring | milestone_management |
|
||||
| Milestones | | | | |
|
||||
| Milestone 1: Product Management | 5d | release | milestone_management | api, create, update, fetch, order_both, migrate, backfill, builder_products |
|
||||
| Milestone 2: Support Subscriptions | 5d | release | milestone_subscriptions | order_products, webhooks_1, webhooks_2, database, tracking, builder_recurring |
|
||||
| Milestone 3: Add Resiliency | | | milestone_resiliency | poller |
|
||||
| Milestone 4: Deprecate Product Name/Price | | | milestone_cleanup | milestone_subscriptions, cleanup |
|
||||
#+end:
|
||||
|
||||
#+begin_export html
|
||||
<iframe src="reports/recurring-and-split-stripe-payments/Plan.html" width="100%" height="1000"></iframe>
|
||||
#+end_export
|
||||
** Tasks
|
||||
|
||||
*** Backend: Product Management
|
||||
**** DONE Define API for managing products and prices
|
||||
:PROPERTIES:
|
||||
:EFFORT: 1d
|
||||
:ALLOCATE: backend
|
||||
:task_id: api
|
||||
:END:
|
||||
Probably makes sense to treat price + recurrence as product attributes, e.g.
|
||||
|
||||
- name :: String
|
||||
- Price :: Object
|
||||
+ amount :: Integer
|
||||
+ currency :: Enum["usd"]
|
||||
+ recurrence :: Optional[Object]
|
||||
- interval :: Enum["weekly", "monthly", "yearly"]
|
||||
- times :: Union["unlimited", Integer]
|
||||
|
||||
**** DONE Endpoint: Create products and prices
|
||||
:PROPERTIES:
|
||||
:EFFORT: 3d
|
||||
:ALLOCATE: backend
|
||||
:blocker: api
|
||||
:TASK_ID: create
|
||||
:END:
|
||||
- Define metadata schema for purchase and cancellation actions (add tags, unsubscribe)
|
||||
|
||||
**** DONE Endpoint: Update products and prices
|
||||
:PROPERTIES:
|
||||
:EFFORT: 5d
|
||||
:ALLOCATE: backend
|
||||
:blocker: api
|
||||
:TASK_ID: update
|
||||
:END:
|
||||
- Add new price object if the price or recurrence has changed, leaving the old one
|
||||
|
||||
**** DONE Endpoint: Fetch products and prices
|
||||
:PROPERTIES:
|
||||
:EFFORT: 2d
|
||||
:ALLOCATE: backend
|
||||
:blocker: api
|
||||
:TASK_ID: fetch
|
||||
:END:
|
||||
|
||||
**** DONE Endpoint: Update /order
|
||||
:PROPERTIES:
|
||||
:EFFORT: 2d
|
||||
:ALLOCATE: backend
|
||||
:blocker: api
|
||||
:task_id: order_both
|
||||
:END:
|
||||
|
||||
**** DONE Migrate products from existing landing pages into Stripe on save
|
||||
:PROPERTIES:
|
||||
:ALLOCATE: frontend
|
||||
:blocker: order_both
|
||||
:TASK_ID: migrate
|
||||
:END:
|
||||
**** CANCELLED Backfill products from existing landing pages into Stripe
|
||||
:PROPERTIES:
|
||||
:EFFORT: 10d
|
||||
:ALLOCATE: backend, frontend
|
||||
:blocker: order_both
|
||||
:task_id: backfill
|
||||
:END:
|
||||
**** DONE Separate Products and Prices
|
||||
*** Backend: Subscriptions
|
||||
**** TODO Endpoint: Create /purchase to create customers for product purchases
|
||||
:PROPERTIES:
|
||||
:ALLOCATE: backend
|
||||
:blocker: milestone_management
|
||||
:start: 2021-04-22
|
||||
:Effort: 12d
|
||||
:TASK_ID: order_products
|
||||
:END:
|
||||
If the product being ordered is specified by id:
|
||||
- Create the customer w/ the provided payment method
|
||||
- Create an invoice
|
||||
- Pay the invoice
|
||||
- Document the /order endpoint as deprecated
|
||||
- Create acceptance tests
|
||||
|
||||
**** TODO Endpoint: Update /fulfill and /webhooks (iteration 1)
|
||||
:PROPERTIES:
|
||||
:Effort: 5d
|
||||
:ALLOCATE: backend
|
||||
:BLOCKER: milestone_management
|
||||
:task_id: webhooks_1
|
||||
:start: 2021-04-22
|
||||
:END:
|
||||
- move add subscriber and sales tracking event emission to the webhook endpoint
|
||||
- no-op /fulfill
|
||||
- Document /fulfill endpoint as deprecated
|
||||
- Send =AWeber-Options: update-unconfirmed= header
|
||||
- Update acceptance tests to account for out-of-band fulfillment
|
||||
**** TODO Endpoint: Add subscription support to /purchase
|
||||
:PROPERTIES:
|
||||
:ALLOCATE: backend
|
||||
:blocker: milestone_management
|
||||
:start: 2021-04-22
|
||||
:Effort: 12d
|
||||
:TASK_ID: order_subscriptions
|
||||
:END:
|
||||
If the product being ordered has a recurring price type:
|
||||
- Create the customer w/ the provided payment method
|
||||
- Create a subscription
|
||||
- Create the initial invoice
|
||||
- Pay the initial invoice
|
||||
- Add acceptance tests for subscription purchases
|
||||
**** TODO Endpoint: Update /webhooks to process subscriptions (iteration 2)
|
||||
:PROPERTIES:
|
||||
:Effort: 10d
|
||||
:ALLOCATE: backend
|
||||
:blocker: webhooks_1, order_subscriptions
|
||||
:TASK_ID: webhooks_2
|
||||
:END:
|
||||
- Add logic to process a subscription
|
||||
- Add the events corresponding to cancellation of a subscription
|
||||
+ Process the remove tag/unsubscribe actions that are saved as metadata on the
|
||||
subscription
|
||||
+ Ensure that unconfirmed subscribers can be unsubscribed
|
||||
- Need a way to differentiate fulfillment of a payment intent for a subscription vs a single payment
|
||||
+ Pull the invoice for the payment intent with the expanded subscription details if they exist
|
||||
**** TODO Database: Add database for event tracking
|
||||
:PROPERTIES:
|
||||
:Effort: 3d
|
||||
:ALLOCATE: backend
|
||||
:BLOCKER: milestone_management
|
||||
:start: 2021-04-22
|
||||
:task_id: database
|
||||
:END:
|
||||
- new postgres users ( one for the poller, one for the stripe-payments service )
|
||||
- Needs the following information in an events table:
|
||||
+ timestamp from the event
|
||||
+ timestamp of action completion
|
||||
+ webhook type
|
||||
+ webhook id
|
||||
+ stripe account id
|
||||
+ aweber account id
|
||||
+ and whether it was successfully processed (process state)
|
||||
- dynamodb does not fit the use case because we need to retrieve all events in a time frame, and filter by account. We are also constantly deleting events.
|
||||
+ we also will have two services updating the database and need transaction safety
|
||||
- Scope assumes handing off the schema design to the DBA team
|
||||
**** TODO Endpoint: Add event tracking to /webhooks
|
||||
:PROPERTIES:
|
||||
:Effort: 7d
|
||||
:ALLOCATE: backend
|
||||
:BLOCKER: database, webhooks_2
|
||||
:TASK_ID: tracking
|
||||
:END:
|
||||
- Mark processed events in the database
|
||||
+ Do we need transactions? Assume yes.
|
||||
- Handling duplicate and out-of-order event cases
|
||||
+ Fulfill only once (adding subscriber + tags)
|
||||
+ Unsubscribe only once (removing subscriber / tags)
|
||||
+ Don't fulfill if unsubscribed
|
||||
*** Backend: Resiliency
|
||||
We will need a job to periodically poll Stripe for unhandled webhooks for
|
||||
processing.
|
||||
|
||||
- This job will need to share a data store with the stripe payments service to
|
||||
track which webhooks have been processed.
|
||||
+ Is this necessary, or does stripe expose the processed state of the
|
||||
webhooks? Webhook attempts and responses are logged in the console.
|
||||
- The job will need to maintain a lock to prevent concurrent runs.
|
||||
- How will unhandled webhooks get processed?
|
||||
+ Send them to the stripe payments service endpoint?
|
||||
- Would have to store status or update it in Stripe.
|
||||
+ Have Stripe send them?
|
||||
- Is this possible?
|
||||
**** TODO Poller: Create poller to check for unprocessed events
|
||||
:PROPERTIES:
|
||||
:EFFORT: 10d
|
||||
:ALLOCATE: backend
|
||||
:BLOCKER: milestone_subscriptions
|
||||
:TASK_ID: poller
|
||||
:END:
|
||||
*** Backend: Cleanup
|
||||
**** TODO Backfill products from existing landing pages into Stripe
|
||||
:PROPERTIES:
|
||||
:EFFORT: 10d
|
||||
:ALLOCATE: backend, frontend
|
||||
:blocker: milestone_subscriptions
|
||||
:task_id: backfill2
|
||||
:END:
|
||||
**** TODO Endpoint: Remove /order and /fulfill
|
||||
:PROPERTIES:
|
||||
:EFFORT: 1d
|
||||
:ALLOCATE: backend
|
||||
:blocker: backfill2
|
||||
:TASK_ID: cleanup
|
||||
:END:
|
||||
*** Frontend
|
||||
Builder originally estimated at 25d combined.
|
||||
**** TODO Frontend: Plugin
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:ALLOCATE: frontend
|
||||
:TASK_ID: plugin
|
||||
:BLOCKER:
|
||||
:END:
|
||||
**** TODO Frontend: Templates
|
||||
:PROPERTIES:
|
||||
:Effort: 1d
|
||||
:ALLOCATE: frontend
|
||||
:TASK_ID: templates
|
||||
:BLOCKER: plugin
|
||||
:END:
|
||||
**** TODO Frontend: Builder (Product Management)
|
||||
:PROPERTIES:
|
||||
:EFFORT: 12d
|
||||
:ALLOCATE: frontend
|
||||
:TASK_ID: builder_products
|
||||
:BLOCKER: plugin, templates
|
||||
:END:
|
||||
**** TODO Frontend: Builder (Recurring Payments)
|
||||
:PROPERTIES:
|
||||
:EFFORT: 28d
|
||||
:ALLOCATE: frontend
|
||||
:TASK_ID: builder_recurring
|
||||
:BLOCKER: milestone_management
|
||||
:start: 2021-04-22
|
||||
:END:
|
||||
** Milestones
|
||||
*** Milestone 1: Product Management
|
||||
:PROPERTIES:
|
||||
:BLOCKER: api, create, update, fetch, order_both, migrate, backfill, builder_products
|
||||
:TASK_ID: milestone_management
|
||||
:Effort: 5d
|
||||
:ALLOCATE: release
|
||||
:END:
|
||||
*** Milestone 2: Support Subscriptions
|
||||
:PROPERTIES:
|
||||
:BLOCKER: order_products, webhooks_1, webhooks_2, database, tracking, builder_recurring
|
||||
:TASK_ID: milestone_subscriptions
|
||||
:Effort: 10d
|
||||
:ALLOCATE: release
|
||||
:END:
|
||||
*** Milestone 3: Add Resiliency
|
||||
:PROPERTIES:
|
||||
:BLOCKER: poller
|
||||
:TASK_ID: milestone_resiliency
|
||||
:END:
|
||||
*** Milestone 4: Deprecate Product Name/Price
|
||||
:PROPERTIES:
|
||||
:BLOCKER: milestone_subscriptions, cleanup
|
||||
:TASK_ID: milestone_cleanup
|
||||
:END:
|
||||
* Resources :taskjuggler_resource:
|
||||
|
||||
*** Backend
|
||||
:PROPERTIES:
|
||||
:resource_id: backend
|
||||
:efficiency: 1.3
|
||||
:END:
|
||||
|
||||
*** Frontend
|
||||
:PROPERTIES:
|
||||
:resource_id: frontend
|
||||
:efficiency: 0.8
|
||||
:END:
|
||||
|
||||
*** Release Validation
|
||||
:PROPERTIES:
|
||||
:resource_id: release
|
||||
:efficiency: 1.0
|
||||
:END:
|
||||
|
||||
* Variables :noexport:
|
||||
Local Variables:
|
||||
org-taskjuggler-reports-directory: "reports/recurring-and-split-stripe-payments"
|
||||
End:
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 33e47957-b3d0-41c9-8977-7243b42a76dd
|
||||
:END:
|
||||
#+title: Control Panel HTTP Requests
|
||||
#+PROPERTY: header-args :exports both :eval no-export
|
||||
#+PROPERTY: header-args:http :cookie .cookies :cookie-jar .cookies
|
||||
|
||||
* Cookies
|
||||
| Name | Description |
|
||||
|-------------+-------------|
|
||||
| AUTORESPSID | Session ID |
|
||||
|
||||
Cookies for requests in this document are stored in cookie file by curl in
|
||||
=~/.cookies= (https://curl.se/docs/http-cookies.html).
|
||||
* AJAX Requests
|
||||
Control Panel controller actions that expect to be called as AJAX endpoints
|
||||
expect the =X-Requested-With= header to be present and set to =XMLHttpRequest=.
|
||||
* Logging In
|
||||
** Fetching a CSRF Token
|
||||
#+name: login-csrf
|
||||
#+begin_src http :pretty
|
||||
GET localhost:8080/users/pub/csrf
|
||||
X-Requested-With:XMLHttpRequest
|
||||
#+end_src
|
||||
|
||||
#+RESULTS: login-csrf
|
||||
: 63116e764c5d31cdd3e4f230ee3740527f6eb1c76aea1cb04e30da5d68e24d78
|
||||
|
||||
** Sending credentials
|
||||
#+begin_src http :pretty :var csrf=login-csrf
|
||||
POST localhost:8080/users/account/loginAjax
|
||||
X-Requested-With: XMLHttpRequest
|
||||
|
||||
username=lookatme@example.com&password=testing&_csrf=${csrf}
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
: {"submitStatus":{"code":200,"message":"\/users\/","category":"status_success"},"validationErrors":[]}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,383 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 6af95849-8f78-4697-ab48-3712ff2f5ee1
|
||||
:END:
|
||||
#+title: Stripe payments tracking database
|
||||
#+OPTIONS: ^:nil
|
||||
|
||||
Database for tracking payments and subscriptions managed by the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||
* State Tracking
|
||||
** Legacy purchase
|
||||
- Client initiates purchase
|
||||
- Backend coordinates with Stripe and returns a payment intent
|
||||
+ Track new purchase and related incomplete payment
|
||||
- Client completes purchase with Stripe
|
||||
- Stripe notifies backend of payment intent status (update purchase)
|
||||
+ Track event
|
||||
+ Track payment as completed
|
||||
+ Fulfill purchase
|
||||
+ Track purchase as fulfilled
|
||||
|
||||
#+begin_src plantuml :file stripe-legacy-purchase-tracking.svg
|
||||
participant Stripe
|
||||
actor Client
|
||||
participant "stripe-payments" as Backend
|
||||
database Tracking
|
||||
|
||||
Client -> Backend: Initiate purchase
|
||||
Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
|
||||
Backend -> Stripe : Create payment intent
|
||||
Backend -> Tracking : <font color="blue">Store incomplete payment</font>
|
||||
Backend -> Client : Return payment intent
|
||||
Client -> Stripe : Complete purchase
|
||||
...
|
||||
alt Success
|
||||
Stripe ---> Backend : payment_intent.succeeded
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
alt Event not previously handled
|
||||
Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
|
||||
Backend -> Backend : Fulfill purchase
|
||||
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||
end
|
||||
else Failure
|
||||
Stripe --> Backend : payment_intent.payment_failed
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
end
|
||||
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:stripe-legacy-purchase-tracking.svg]]
|
||||
|
||||
#+caption: Legacy purchase flow with events
|
||||
#+RESULTS:
|
||||
|
||||
** Product purchase
|
||||
- Client prepares payment method with Stripe
|
||||
- Client initiates purchase
|
||||
- Backend coordinates with Stripe to complete the purchase
|
||||
+ Attaches the payment method to the customer
|
||||
+ For single products
|
||||
- Creates an invoice item
|
||||
- Creates an invoice
|
||||
- Pays the invoice
|
||||
+ For subscriptions
|
||||
- Creates the subscription
|
||||
|
||||
#+begin_src plantuml :file stripe-purchase-tracking.svg
|
||||
participant Stripe
|
||||
actor Client
|
||||
participant "stripe-payments" as Backend
|
||||
database Tracking
|
||||
|
||||
Client -> Stripe : Create payment method
|
||||
Client -> Backend: Initiate purchase
|
||||
Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
|
||||
alt Non-recurring
|
||||
Backend -> Stripe : Create invoice item
|
||||
Backend -> Stripe : Create invoice
|
||||
Backend -> Stripe : Pay invoice
|
||||
Backend -> Tracking : <font color="blue">Store completed payment</font>
|
||||
else Recurring
|
||||
Backend -> Stripe : Create subscription
|
||||
Backend -> Tracking : <font color="blue">Store active subscription</font>
|
||||
end
|
||||
...
|
||||
Stripe --> Backend : Subscription activated
|
||||
...
|
||||
alt
|
||||
Stripe --> Backend : customer.subscription.updated (no longer active)
|
||||
else
|
||||
Stripe --> Backend : subscription_schedule.canceled
|
||||
end
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
Backend -> Backend : Trigger unsubscribe actions
|
||||
Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>
|
||||
#+end_src
|
||||
|
||||
#+caption: Product purchase flow
|
||||
#+RESULTS:
|
||||
[[file:stripe-purchase-tracking.svg]]
|
||||
|
||||
#+begin_src plantuml :file stripe-purchase-tracking-payment-events.svg
|
||||
participant Stripe
|
||||
actor Client
|
||||
participant "stripe-payments" as Backend
|
||||
database Tracking
|
||||
|
||||
== Payment succeeded ==
|
||||
Stripe --> Backend : payment_intent.succeeded
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
alt Event not previously handled
|
||||
Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
|
||||
Backend -> Stripe : Look up invoice
|
||||
|
||||
note over Backend
|
||||
Recurring if an invoice exists
|
||||
and has an associated subscription
|
||||
end note
|
||||
alt Non-recurring
|
||||
Backend -> Backend : Fulfill purchase
|
||||
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||
else Recurring
|
||||
end
|
||||
end
|
||||
== Payment failed ==
|
||||
Stripe --> Backend : payment_intent.payment_failed
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
#+end_src
|
||||
|
||||
#+caption: Payment events
|
||||
#+RESULTS:
|
||||
[[file:stripe-purchase-tracking-payment-events.svg]]
|
||||
|
||||
#+begin_src plantuml :file stripe-purchase-subscription-events.svg
|
||||
participant Stripe
|
||||
actor Client
|
||||
participant "stripe-payments" as Backend
|
||||
database Tracking
|
||||
|
||||
== Subscription activated ==
|
||||
alt
|
||||
Stripe --> Backend : customer.subscription.created (active)
|
||||
else
|
||||
Stripe --> Backend : customer.subscription.updated (active)
|
||||
end
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
alt New subscription
|
||||
Backend -> Tracking : <font color="blue">Mark subscription as active</font>
|
||||
Backend -> Backend : Fulfill purchase
|
||||
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||
else Subscription already processed
|
||||
note over Backend
|
||||
One of the following is true:
|
||||
|
||||
<font color="green">- Already received an activation event for this subscription</font>
|
||||
<font color="green">- Already receieved a termination event for this subscription</font>
|
||||
<font color="blue">- Subscription already marked as active or terminated</font>
|
||||
<font color="blue">- Purchase already marked as fulfilled</font>
|
||||
end note
|
||||
end
|
||||
== Subscription terminated ==
|
||||
alt
|
||||
Stripe --> Backend : customer.subscription.updated (no longer active)
|
||||
else
|
||||
Stripe --> Backend : subscription_schedule.canceled
|
||||
end
|
||||
Backend -> Tracking : <font color="green">Store event</font>
|
||||
alt Active subscription
|
||||
Backend -> Backend : Trigger unsubscribe actions
|
||||
Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>
|
||||
else Subscription not yet processed
|
||||
note over Backend
|
||||
One of the following is true:
|
||||
|
||||
<font color="green">- Did not receive an activation event for this subscription</font>
|
||||
<font color="blue">- Subscription is not tracked</font>
|
||||
end note
|
||||
else Subscription already terminated
|
||||
note over Backend
|
||||
One of the following is true:
|
||||
|
||||
<font color="green">- Already received a termination event for this subscription</font>
|
||||
<font color="blue">- Subscription already marked as terminated</font>
|
||||
end note
|
||||
end
|
||||
#+end_src
|
||||
|
||||
#+caption: Subscription events
|
||||
#+RESULTS:
|
||||
[[file:stripe-purchase-subscription-events.svg]]
|
||||
|
||||
* Tables
|
||||
** Purchases
|
||||
Purchases made via the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||
|
||||
#+begin_src plantuml :file stripe-purchase-states.svg
|
||||
[*] -> New
|
||||
New -> Fulfilled
|
||||
#+end_src
|
||||
|
||||
#+caption: Purchase state diagram
|
||||
#+RESULTS:
|
||||
[[file:stripe-purchase-states.svg]]
|
||||
|
||||
#+caption: Purchases
|
||||
| Field | Type | Nullable | Description |
|
||||
|----------------+-----------+----------+--------------------------------------------|
|
||||
| id | UUID | N | Auto-generated purchase ID |
|
||||
| created | timestamp | N | Time the purchase was initiated |
|
||||
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||
| account | UUID | N | The AWeber customer account purchased from |
|
||||
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||
| fulfilled | boolean | N | Was this purchase fulfilled |
|
||||
|
||||
- Store which automations were applied & when
|
||||
** Subscriptions and Split Payments
|
||||
Purchases made with recurring payments, managed using a Stripe Subscription.
|
||||
|
||||
#+begin_src plantuml :file stripe-subscription-states.svg
|
||||
[*] -> New
|
||||
New -> Active
|
||||
Active --> Terminated : Payment Failed
|
||||
Active --> Terminated : Unsubscribed
|
||||
New --> Terminated : Payment Failed
|
||||
#+end_src
|
||||
|
||||
#+caption: Subscription state diagram
|
||||
#+RESULTS:
|
||||
[[file:stripe-subscription-states.svg]]
|
||||
|
||||
#+caption: Subscriptions
|
||||
| Field | Type | Nullable | Description |
|
||||
|----------------+---------------------+----------+---------------------------------------------------------|
|
||||
| id | text | N | The Stripe subscription ID |
|
||||
| created | timestamp | N | Time the purchase was initiated |
|
||||
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||
| account | UUID | N | The AWeber customer account the subscription belongs to |
|
||||
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||
| status | [[subscription_status][subscription_status]] | N | Status of the subscription |
|
||||
|
||||
<<subscription_status>>
|
||||
#+caption: ENUM: Subscription Status
|
||||
| new |
|
||||
| active |
|
||||
| terminated |
|
||||
** Payments
|
||||
Payments collected from buyers.
|
||||
|
||||
#+begin_src plantuml :file stripe-payment-states.svg
|
||||
[*] -> New
|
||||
New --> Paid : Payment succeeded
|
||||
New --> Failed : Payment failed
|
||||
#+end_src
|
||||
|
||||
#+caption: Payment state diagram
|
||||
#+RESULTS:
|
||||
[[file:stripe-payment-states.svg]]
|
||||
|
||||
#+caption: Payments
|
||||
| Field | Type | Nullable | Description |
|
||||
|----------------+----------------+----------+-------------------------------------------|
|
||||
| id | text | N | The Stripe payment intent ID |
|
||||
| created | timestamp | N | Time the purchase was initiated |
|
||||
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||
| account | UUID | N | The AWeber customer account that was paid |
|
||||
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||
| purchase | UUID | N | |
|
||||
| status | [[payment_status][payment_status]] | N | Status of the payment intent |
|
||||
|
||||
<<payment_status>>
|
||||
#+caption: ENUM: Payment Status
|
||||
| new |
|
||||
| paid |
|
||||
| failed |
|
||||
** Events
|
||||
#+caption: Events
|
||||
| Field | Type | Nullable | Description |
|
||||
|----------------+-----------+----------+---------------------------------------------------------|
|
||||
| id | text | N | Stripe event id |
|
||||
| type | text | N | Stripe event type |
|
||||
| account | UUID | N | The AWeber customer account the subscription belongs to |
|
||||
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||
| timestamp | timestamp | N | Time event was published |
|
||||
| subscription | text | Y | Related Stripe subscription ID |
|
||||
|
||||
Events should expire out of the database over time. Stripe maintains events in
|
||||
its database for up to 30 days.
|
||||
*** Modeling in DynamoDB
|
||||
**** Table
|
||||
- Partition Key :: =id=
|
||||
|
||||
The table index provides for rapid lookup of individual events.
|
||||
**** Attributes
|
||||
| Field | Type | Required | Description |
|
||||
|----------------+-------------------------+----------+-----------------------------------------------------------|
|
||||
| id | String | Y | Stripe event id |
|
||||
| type | String | Y | Stripe event type |
|
||||
| stripe_account | String | Y | The Stripe account of the AWeber customer |
|
||||
| timestamp | String (timestamp) | Y | Time event was published (e.g. "2020-01-01 01:02:34.567") |
|
||||
| date | String (date) | Y | Date portion of timestamp (e.g. "2020-01-01") |
|
||||
| time | String (time) | Y | Time portion of timestamp (e.g. "01:02:34.567") |
|
||||
| expiration | Number (Unix timestamp) | Y | TTL expiration time of the event record |
|
||||
| subscription | String | N | Related Stripe subscription ID |
|
||||
**** TTL
|
||||
The events table will automatically expire and remove rows for which
|
||||
=expiration= has passed. We may set =expiration= to be the event creation time
|
||||
plus thirty days to match Stripe's own expiration. This will prevent old items
|
||||
from piling up in the table.
|
||||
**** Time index (GSI)
|
||||
- Partition Key :: =date=
|
||||
- Sort Key :: =time=
|
||||
- Projection :: Keys Only
|
||||
|
||||
|
||||
The date and time portions of the event timestamp will be separated to allow for
|
||||
scanning all events within a day, filterable by time. This should support our
|
||||
polling job, which should run multiple times per day. We'll have to be careful
|
||||
around date boundaries when looking up events (e.g. by including a query for
|
||||
events from the previous day during the first run of the current day).
|
||||
**** Stripe Account index (GSI)
|
||||
- Partition Key :: =stripe_account=
|
||||
- Sort Key :: =timestamp=
|
||||
- Projection :: Include type, subscription
|
||||
|
||||
The account index provides for rapid lookup of events for a particular stripe
|
||||
account, filterable by timestamp.
|
||||
**** Subscription index (GSI)
|
||||
- Partition Key :: =subscription=
|
||||
- Sort Key :: =timestamp=
|
||||
- Projection :: Include type, stripe_account
|
||||
|
||||
The account index provides for rapid lookup of events for a particular
|
||||
subscription, filterable by timestamp.
|
||||
**** IAM Policy
|
||||
Put together by referencing [[id:ab2d34bf-97b1-4e50-8e9a-597d0f8fcf01][DynamoDB IAM Policies]].
|
||||
#+caption: Stripe Payments DynamoDB IAM Policy
|
||||
#+begin_src json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": "dynamodb:ListTables",
|
||||
"Resource": "*"
|
||||
},
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"dynamodb:BatchGetItem",
|
||||
"dynamodb:BatchWriteItem",
|
||||
"dynamodb:ConditionCheckItem",
|
||||
"dynamodb:PutItem",
|
||||
"dynamodb:DeleteItem",
|
||||
"dynamodb:Scan",
|
||||
"dynamodb:Query",
|
||||
"dynamodb:UpdateItem",
|
||||
"dynamodb:DescribeTimeToLive",
|
||||
"dynamodb:CreateTable",
|
||||
"dynamodb:DescribeTable",
|
||||
"dynamodb:GetItem",
|
||||
"dynamodb:UpdateTable",
|
||||
"dynamodb:UpdateTimeToLive"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*/index/*",
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*"
|
||||
]
|
||||
},
|
||||
]
|
||||
}
|
||||
#+end_src
|
||||
* Notes
|
||||
- How does the [[id:83d61eef-0781-46e0-b959-1a739cff5ea3][poller]] use these stored events?
|
||||
+ /To identify events retrieved from Stripe which do not need to be replayed/
|
||||
- How do we replay events?
|
||||
+ Send it to the webhook endpoint? An internal webhook endpoint that shares
|
||||
code with the public one?
|
||||
- /The same webhook endpoint would be preferable, requires signing the
|
||||
payload/
|
||||
- Do we expose the processed events as an internal endpoint in the
|
||||
stripe-payments service, or give the polling app a database connection?
|
||||
+ /API seems preferable/
|
||||
|
||||
Following some discussion, the [[Events][Events]] table should be sufficient for now.
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: fe6374ac-8fa6-4579-bd42-6d4de92de86a
|
||||
:END:
|
||||
#+title: Finding number of subscribers with a tag
|
||||
|
||||
* Retrieve number of subscribers per tag
|
||||
- Tagging -> Kubernetes
|
||||
- Add appdb connection to tagging
|
||||
- Add endpoint to tagging
|
||||
- Expose tagging endpoints via authenticated kong
|
||||
* Hide tags
|
||||
* Delete tags
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 8435e743-c092-43e9-bcc6-a8098aa4110c
|
||||
:END:
|
||||
#+title: Tagging Roadmap
|
||||
|
||||
* Retrieve number of subscribers per tag
|
||||
** Migrate the tagging service from AWS to Kubernetes
|
||||
** Expose tagging endpoints via authenticated kong
|
||||
** Add appdb connection to tagging
|
||||
|
||||
** Create an AppDB user for the tagging service
|
||||
- Full access to the subscriber_tags table
|
||||
- Read access to the list.subscribers table
|
||||
|
||||
** Add endpoint to tagging to fetch subscribers on a tag
|
||||
e.g.:
|
||||
#+begin_example
|
||||
GET tagging.service.production.consul/{tag}/account/{account}/subscribers
|
||||
#+end_example
|
||||
|
||||
** Update the tagging service to update the subscriber_tags table directly.
|
||||
** Retire the subscriber-tag-sync consumer
|
||||
* Hide tags
|
||||
|
|
@ -1,46 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 193f7c04-0a03-4870-90c8-2b5e3c4c92ce
|
||||
:END:
|
||||
#+title: Moving pages out of Sites
|
||||
#+filetags: :project:
|
||||
|
||||
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to rewrite pages in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] still built in PHP.
|
||||
|
||||
* Pages in Sites
|
||||
|
||||
- Tracking document :: https://confluence.aweber.io/display/BETL/PHP+to+React+Page+Needs
|
||||
|
||||
** TODO Dashboard
|
||||
*** Broadcasts
|
||||
*** Lists
|
||||
*** Subscribers
|
||||
In progress.
|
||||
** Content Creation
|
||||
*** TODO Follow-ups
|
||||
*** TODO Blog Broadcasts
|
||||
*** TODO Email Template Manager
|
||||
** Subscribers
|
||||
*** TODO Manage Subscribers
|
||||
*** TODO Add Subscribers
|
||||
*** TODO Import History
|
||||
** Reports
|
||||
*** TODO Classic Reports
|
||||
*** TODO Report API
|
||||
*** TODO Tracking
|
||||
** Lists
|
||||
*** TODO Manage Lists
|
||||
*** TODO List Settings
|
||||
*** TODO Custom Fields
|
||||
*** TODO List Automations
|
||||
** Accounts
|
||||
*** TODO My Account
|
||||
*** Billing
|
||||
*** Notifications
|
||||
** Advocates
|
||||
- Create an affiliate management API
|
||||
- Create an affiliate frontend application
|
||||
- Embed the affiliate app in the CP and link them with CP accounts.
|
||||
** TODO Help
|
||||
Should this be separated?
|
||||
** Integrations
|
||||
😱
|
||||
|
|
@ -1,95 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: db322997-ff5e-416a-8dc8-f29e6a4928c8
|
||||
:END:
|
||||
#+title: Technical Initiative
|
||||
|
||||
- [[https://confluence.aweber.io/display/~scottm/CP+Technical+Work+Brainstorming][2022 Brainstorming Document]]
|
||||
- [[https://confluence.aweber.io/display/TCP/2022+Q1+CP+Priorities][2022 Q1 CP Priorities]]
|
||||
|
||||
- [[id:193f7c04-0a03-4870-90c8-2b5e3c4c92ce][Moving pages out of Sites]]
|
||||
|
||||
* Big
|
||||
** Analytics View
|
||||
- Coordinate on public URL structure with Dave S.
|
||||
- Update dashboard and reports to use new endpoints as they're made available.
|
||||
** [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]]
|
||||
*** Store and paginate search results
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-7148
|
||||
:END:
|
||||
*** Rebuild Subscriber Management in React
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11697
|
||||
:END:
|
||||
** Verifications
|
||||
*** Updating the existing verification flow to use email-verifications
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-9416
|
||||
:END:
|
||||
*** Decommission Verifications
|
||||
** Domain Validator
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-10554
|
||||
:END:
|
||||
** [[id:619b6c78-7be9-4ee4-a0b7-9d1a4d7536e2][Migrating services to use the new List service]]
|
||||
*** Audit remaining services
|
||||
*** Rebuild List Management in React
|
||||
*** Rebuild List Settings in React
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11694
|
||||
:END:
|
||||
*** Remove dependency on AWLists from Stripe
|
||||
**** Stripe master branch does not allow null values in product recurrence
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12072
|
||||
:END:
|
||||
*** Remove dependency on AWLists from Subscriber Import
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12071
|
||||
:END:
|
||||
**** Update Subscriber Import client to fetch list data from the new lists service
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12073
|
||||
:END:
|
||||
*** Remove dependency on AWLists from Sites
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12074
|
||||
:END:
|
||||
*** Remove dependency on AWLists from Email Verification
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12070
|
||||
:END:
|
||||
** Retire AWSubscribers in favor of Recipient
|
||||
*** Back Recipient with AppDB
|
||||
*** Retire sync consumers
|
||||
*** Identify gaps between AWSubs and Recipient
|
||||
Determine which endpoints need to have analogs in Recipient or could be replaced
|
||||
with calls to other, more appropriate services.
|
||||
*** Look into folding in edeliv's bulk subscriber service
|
||||
** [[id:03e00c18-99c0-477c-b7fb-95ddc538755e][Addlead]] Python rewrite
|
||||
https://jira.aweber.io/browse/TRAC-118
|
||||
- Find / Build a test suite that can be run against old and new addlead?
|
||||
- WHAT DOES IT DO?! https://jira.aweber.io/browse/CCPANEL-7614
|
||||
- ACP? https://jira.aweber.io/browse/CCPANEL-7613
|
||||
** Enlightener rewrite
|
||||
- Investigate how to rebuild this
|
||||
** Sites login / session management
|
||||
- Should advocate users be migrated to user management?
|
||||
*** Separate from the rest of the CP
|
||||
** Advocate CP
|
||||
*** Python service + react application
|
||||
** Verify Opt-in Python rewrite
|
||||
** Unsubscribe Python rewrite
|
||||
** [[id:b4f579f7-f848-4a7b-b7bc-f34fec36346a][Cleaning up public endpoints in proxy services]]
|
||||
|
||||
* Small
|
||||
** [[id:af4ae6ee-5201-49ee-aa01-6cf6a0801908][Migrating AWS services]]
|
||||
** [[id:96d1d218-60cd-41d9-91ba-48359137d239][Decommission the mail-relay service]]
|
||||
|
||||
** KTLO
|
||||
- User Management
|
||||
- Stripe Payments
|
||||
- Commissions Processor
|
||||
|
||||
* Ongoing
|
||||
** Update project configuration and gitlab pathing to match our taxonomies in Imbi.
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: b4f579f7-f848-4a7b-b7bc-f34fec36346a
|
||||
:END:
|
||||
#+title: Cleaning up public endpoints in proxy services
|
||||
|
||||
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to move endpoints exposed in "proxy" services into more
|
||||
relevant services and instead expose them via Kong.
|
||||
|
||||
Endpoints have been consolidated into Search Proxy.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 76933c22-fe7c-43e9-9ec9-62564377dd85
|
||||
:END:
|
||||
#+title: Imbi
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 9332ed8f-b669-4d3f-a25d-da751a8c2da1
|
||||
:END:
|
||||
#+title: Troubleshooting an unresolvable kubernetes service hostname
|
||||
|
||||
The service's selector wasn't matching any pods, therefore the service had no IP
|
||||
to respond with. This can be troubleshooted by inspecting the endpoints for the
|
||||
service. Due to the mismatch, none were available.
|
||||
|
||||
The ClusterIP service did not itself have an IP assigned even after fixing the
|
||||
mismatch. This appears to be an optimization in that [[id:3ba1f581-c66c-492f-80fc-e7d2e488b362][Kubernetes]] won't bother
|
||||
assigning an IP to the service if the service has only a single endpoint, as it
|
||||
is more expedient to return the sole endpoint's IP address.
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e2dab290-3e1b-4d5a-8628-61c2cb4896dc
|
||||
:END:
|
||||
#+title: Purchase tracking
|
||||
|
||||
* Related Confluence documents
|
||||
- [[https://confluence.aweber.io/pages/viewpage.action?spaceKey=PD&title=2021-03-16+Record+purchase+details+for+AWeber+integrations][2021-03-16 Record purchase details for AWeber integrations]]
|
||||
- [[https://confluence.aweber.io/pages/viewpage.action?spaceKey=API&title=Storing+PayPal+purchases+as+analytics+events][Storing PayPal purchases as analytics events]]
|
||||
|
||||
* Record purchases
|
||||
- Track a separate event type for sales tracking.
|
||||
- Changes to sales tracking URLs can break tracking currently.
|
||||
- Separate URL profiles for Stripe and Paypal? (Stripe is currently using
|
||||
ECommerce), and add an ECommerce event type for them.
|
||||
* Segmenting on purchases
|
||||
|
|
@ -1,65 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: ab2d34bf-97b1-4e50-8e9a-597d0f8fcf01
|
||||
:END:
|
||||
#+title: DynamoDB IAM Policies
|
||||
|
||||
#+caption: DynamoDB access for the k8s-labs-application role
|
||||
#+begin_src json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": "dynamodb:ListTables",
|
||||
"Resource": "*"
|
||||
},
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"dynamodb:BatchGetItem",
|
||||
"dynamodb:BatchWriteItem",
|
||||
"dynamodb:ConditionCheckItem",
|
||||
"dynamodb:PutItem",
|
||||
"dynamodb:DeleteItem",
|
||||
"dynamodb:Scan",
|
||||
"dynamodb:Query",
|
||||
"dynamodb:UpdateItem",
|
||||
"dynamodb:DescribeTimeToLive",
|
||||
"dynamodb:CreateTable",
|
||||
"dynamodb:DescribeTable",
|
||||
"dynamodb:GetItem",
|
||||
"dynamodb:UpdateTable"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-webhook-callbacks/index/*",
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-webhook-callbacks"
|
||||
]
|
||||
},
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"dynamodb:BatchGetItem",
|
||||
"dynamodb:BatchWriteItem",
|
||||
"dynamodb:ConditionCheckItem",
|
||||
"dynamodb:PutItem",
|
||||
"dynamodb:DeleteItem",
|
||||
"dynamodb:Scan",
|
||||
"dynamodb:Query",
|
||||
"dynamodb:UpdateItem",
|
||||
"dynamodb:DescribeTimeToLive",
|
||||
"dynamodb:CreateTable",
|
||||
"dynamodb:DescribeTable",
|
||||
"dynamodb:GetItem",
|
||||
"dynamodb:UpdateTable"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-webhooks",
|
||||
"arn:aws:dynamodb:*:018154689201:table/*-webhooks/index/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
#+end_src
|
||||
|
||||
- [[https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/access-control-overview.html][Overview of Managing Access Permissions to Your Amazon DynamoDB Resources]]
|
||||
- [[https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/api-permissions-reference.html][DynamoDB API Permissions: Actions, Resources, and Conditions Reference]]
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: bdc526d1-4f57-4210-93f2-12bb30d33ed9
|
||||
:END:
|
||||
#+title: Rebuild Unsubscribe Page as a React Application
|
||||
#+filetags: :project:
|
||||
|
||||
* Flows
|
||||
As described in [[https://confluence.aweber.io/display/~scottm/Unsubscribe+Flows][Unsubscribe Flows]]:
|
||||
|
||||
- View subscriptions
|
||||
- Edit subscriptions
|
||||
- Edit subscriber
|
||||
+ Name
|
||||
+ Email
|
||||
+ Custom fields, if present
|
||||
** Questions
|
||||
- What do we do when the backend (database) is unavailable?
|
||||
+ The backend could do some sort of queuing if the database is unavailable
|
||||
(similar to the disk-persisted queue currently in use).
|
||||
* Frontend
|
||||
A new React application to support subscriber subscription management actions.
|
||||
** Questions
|
||||
- What will host the frontend application?
|
||||
+ Deploy the application to S3 and have F5 route to it.
|
||||
* Backend
|
||||
A new python service to handle subscriber subscription management actions.
|
||||
|
||||
** Endpoints
|
||||
*** Configuration
|
||||
Returns account information and branding needed to display the page.
|
||||
*** Get subscriptions
|
||||
*** Edit subscriptions
|
||||
*** Edit subscriber
|
||||
Email verification needs to be processed if the subscriber updates their email
|
||||
address in order for that change to take effect (addresses [[https://jira.aweber.io/browse/CCPANEL-11269][CCPANEL-11269]]).
|
||||
Changes to other fields should be applied immediately.
|
||||
**** Questions
|
||||
- Do custom field changes need verification to change?
|
||||
+ Probably not.
|
||||
- Should we display a pending state until verification is complete?
|
||||
+ We do not plan to store a pending state.
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d0a802dd-3258-4a86-b53f-287f7f6df6e6
|
||||
:END:
|
||||
#+title: Cobrowse.io
|
||||
#+filetags: :project:
|
||||
|
||||
- Allows CS team to view a session of one of our customers.
|
||||
- This will initially be available to all customers as a link on the help page.
|
||||
- User-initiated, share the code with the AW agent.
|
||||
- We will be using their cloud solution.
|
||||
- Look into an API for adding admin notes that the session took place
|
||||
- Look into information to be redacted in session recordings
|
||||
+ Information displayed will have to be tagged with a CSS class to be identified
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: af4ae6ee-5201-49ee-aa01-6cf6a0801908
|
||||
:END:
|
||||
#+title: Migrating AWS services
|
||||
#+filetags: :project:
|
||||
|
||||
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to migrate services from ECS deployments to Kubernetes
|
||||
deployments. The purpose is to homogenize our deployments to use Kubernetes,
|
||||
which could theoretically be deployed to our local Kubernetes cluster or to EKS.
|
||||
|
||||
We currently maintain two separate sets of clusters in AWS. Services should be
|
||||
migrated to the new cluster, or to kubernetes, ideally.
|
||||
|
||||
- [[id:7e503917-646f-4275-aab9-3a125b99cbfd][Tagging Service]]
|
||||
- Mapping
|
||||
- Recipient
|
||||
|
|
@ -1,977 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 022406d2-0480-4470-90d0-9533f6b9fa32
|
||||
:END:
|
||||
#+title: Supporting multiple currencies in Stripe
|
||||
|
||||
To be completed after [[id:a9835afc-e0be-4436-8274-c3898fdf119c][Recurring and split Stripe payments]].
|
||||
* How does Stripe handle different currencies?
|
||||
- https://stripe.com/docs/currencies
|
||||
* Which currencies are supported by a particular customer?
|
||||
- [[https://stripe.com/docs/api/country_specs/object#country_spec_object-supported_payment_currencies][Supported payment currencies]] can be retrieved via the Stripe API by looking up
|
||||
their Stripe account's [[https://stripe.com/docs/api/accounts/object#account_object-country][configured country]].
|
||||
- Stripe accounts have a [[https://stripe.com/docs/api/accounts/object#account_object-default_currency][default currency]] configured that we could refer to when
|
||||
setting up an eCommerce widget.
|
||||
** What are my supported currencies?
|
||||
#+begin_src sh :results output :wrap src json :exports both :eval no-export
|
||||
stripe get /v1/country_specs/US
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
#+begin_src json
|
||||
{
|
||||
"id": "US",
|
||||
"object": "country_spec",
|
||||
"default_currency": "usd",
|
||||
"supported_bank_account_currencies": {
|
||||
"usd": [
|
||||
"US"
|
||||
]
|
||||
},
|
||||
"supported_payment_currencies": [
|
||||
"usd",
|
||||
"aed",
|
||||
"afn",
|
||||
"all",
|
||||
"amd",
|
||||
"ang",
|
||||
"aoa",
|
||||
"ars",
|
||||
"aud",
|
||||
"awg",
|
||||
"azn",
|
||||
"bam",
|
||||
"bbd",
|
||||
"bdt",
|
||||
"bgn",
|
||||
"bif",
|
||||
"bmd",
|
||||
"bnd",
|
||||
"bob",
|
||||
"brl",
|
||||
"bsd",
|
||||
"bwp",
|
||||
"bzd",
|
||||
"cad",
|
||||
"cdf",
|
||||
"chf",
|
||||
"clp",
|
||||
"cny",
|
||||
"cop",
|
||||
"crc",
|
||||
"cve",
|
||||
"czk",
|
||||
"djf",
|
||||
"dkk",
|
||||
"dop",
|
||||
"dzd",
|
||||
"egp",
|
||||
"etb",
|
||||
"eur",
|
||||
"fjd",
|
||||
"fkp",
|
||||
"gbp",
|
||||
"gel",
|
||||
"gip",
|
||||
"gmd",
|
||||
"gnf",
|
||||
"gtq",
|
||||
"gyd",
|
||||
"hkd",
|
||||
"hnl",
|
||||
"hrk",
|
||||
"htg",
|
||||
"huf",
|
||||
"idr",
|
||||
"ils",
|
||||
"inr",
|
||||
"isk",
|
||||
"jmd",
|
||||
"jpy",
|
||||
"kes",
|
||||
"kgs",
|
||||
"khr",
|
||||
"kmf",
|
||||
"krw",
|
||||
"kyd",
|
||||
"kzt",
|
||||
"lak",
|
||||
"lbp",
|
||||
"lkr",
|
||||
"lrd",
|
||||
"lsl",
|
||||
"mad",
|
||||
"mdl",
|
||||
"mga",
|
||||
"mkd",
|
||||
"mmk",
|
||||
"mnt",
|
||||
"mop",
|
||||
"mro",
|
||||
"mur",
|
||||
"mvr",
|
||||
"mwk",
|
||||
"mxn",
|
||||
"myr",
|
||||
"mzn",
|
||||
"nad",
|
||||
"ngn",
|
||||
"nio",
|
||||
"nok",
|
||||
"npr",
|
||||
"nzd",
|
||||
"pab",
|
||||
"pen",
|
||||
"pgk",
|
||||
"php",
|
||||
"pkr",
|
||||
"pln",
|
||||
"pyg",
|
||||
"qar",
|
||||
"ron",
|
||||
"rsd",
|
||||
"rub",
|
||||
"rwf",
|
||||
"sar",
|
||||
"sbd",
|
||||
"scr",
|
||||
"sek",
|
||||
"sgd",
|
||||
"shp",
|
||||
"sll",
|
||||
"sos",
|
||||
"srd",
|
||||
"std",
|
||||
"szl",
|
||||
"thb",
|
||||
"tjs",
|
||||
"top",
|
||||
"try",
|
||||
"ttd",
|
||||
"twd",
|
||||
"tzs",
|
||||
"uah",
|
||||
"ugx",
|
||||
"uyu",
|
||||
"uzs",
|
||||
"vnd",
|
||||
"vuv",
|
||||
"wst",
|
||||
"xaf",
|
||||
"xcd",
|
||||
"xof",
|
||||
"xpf",
|
||||
"yer",
|
||||
"zar",
|
||||
"zmw"
|
||||
],
|
||||
"supported_payment_methods": [
|
||||
"card",
|
||||
"stripe"
|
||||
],
|
||||
"supported_transfer_countries": [
|
||||
"US",
|
||||
"AT",
|
||||
"AR",
|
||||
"AU",
|
||||
"BE",
|
||||
"BG",
|
||||
"BO",
|
||||
"CA",
|
||||
"CH",
|
||||
"CR",
|
||||
"CY",
|
||||
"CZ",
|
||||
"DE",
|
||||
"DK",
|
||||
"DO",
|
||||
"EE",
|
||||
"EG",
|
||||
"ES",
|
||||
"FI",
|
||||
"FR",
|
||||
"GB",
|
||||
"GR",
|
||||
"HK",
|
||||
"HR",
|
||||
"HU",
|
||||
"ID",
|
||||
"IE",
|
||||
"IL",
|
||||
"IS",
|
||||
"IT",
|
||||
"LI",
|
||||
"LT",
|
||||
"LU",
|
||||
"LV",
|
||||
"MT",
|
||||
"MX",
|
||||
"NL",
|
||||
"NO",
|
||||
"NZ",
|
||||
"PE",
|
||||
"PL",
|
||||
"PT",
|
||||
"RO",
|
||||
"SE",
|
||||
"SG",
|
||||
"SI",
|
||||
"SK",
|
||||
"TH",
|
||||
"TT",
|
||||
"UY"
|
||||
],
|
||||
"verification_fields": {
|
||||
"company": {
|
||||
"additional": [
|
||||
"representative.verification.document"
|
||||
],
|
||||
"minimum": [
|
||||
"business_profile.mcc",
|
||||
"business_profile.url",
|
||||
"business_type",
|
||||
"company.address.city",
|
||||
"company.address.line1",
|
||||
"company.address.postal_code",
|
||||
"company.address.state",
|
||||
"company.name",
|
||||
"company.owners_provided",
|
||||
"company.phone",
|
||||
"company.tax_id",
|
||||
"external_account",
|
||||
"owners.address.city",
|
||||
"owners.address.line1",
|
||||
"owners.address.postal_code",
|
||||
"owners.address.state",
|
||||
"owners.dob.day",
|
||||
"owners.dob.month",
|
||||
"owners.dob.year",
|
||||
"owners.email",
|
||||
"owners.first_name",
|
||||
"owners.id_number",
|
||||
"owners.last_name",
|
||||
"owners.phone",
|
||||
"owners.relationship.title",
|
||||
"owners.verification.document",
|
||||
"representative.address.city",
|
||||
"representative.address.line1",
|
||||
"representative.address.postal_code",
|
||||
"representative.address.state",
|
||||
"representative.dob.day",
|
||||
"representative.dob.month",
|
||||
"representative.dob.year",
|
||||
"representative.email",
|
||||
"representative.first_name",
|
||||
"representative.id_number",
|
||||
"representative.last_name",
|
||||
"representative.phone",
|
||||
"representative.relationship.executive",
|
||||
"representative.relationship.title",
|
||||
"tos_acceptance.date",
|
||||
"tos_acceptance.ip"
|
||||
]
|
||||
},
|
||||
"individual": {
|
||||
"additional": [
|
||||
"individual.verification.document"
|
||||
],
|
||||
"minimum": [
|
||||
"business_profile.mcc",
|
||||
"business_profile.url",
|
||||
"business_type",
|
||||
"external_account",
|
||||
"individual.address.city",
|
||||
"individual.address.line1",
|
||||
"individual.address.postal_code",
|
||||
"individual.address.state",
|
||||
"individual.dob.day",
|
||||
"individual.dob.month",
|
||||
"individual.dob.year",
|
||||
"individual.email",
|
||||
"individual.first_name",
|
||||
"individual.id_number",
|
||||
"individual.last_name",
|
||||
"individual.phone",
|
||||
"tos_acceptance.date",
|
||||
"tos_acceptance.ip"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
|
||||
#+begin_notes
|
||||
Country data cannot, unfortunately, be expanded from an account:
|
||||
|
||||
#+begin_src sh :exports both :results output :wrap src json :eval no-export
|
||||
stripe get acct_1IGnMkIoFf3wvXpR -d "expand[]=country"
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
#+begin_src json
|
||||
{
|
||||
"error": {
|
||||
"message": "This property cannot be expanded (country).",
|
||||
"type": "invalid_request_error"
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
#+end_notes
|
||||
|
||||
* What happens when a payment is below the minimum allowed amount for a settlement currency?
|
||||
- Verify the error when making a price with an unsupported currenccy
|
||||
- Check that the flat fee is calculated correctly
|
||||
** What happens when I create a price with an unsupported currency?
|
||||
|
||||
#+name: price-bhd
|
||||
#+caption: Creating a price using Bahraini dinars (BHD)
|
||||
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||
#+header: :wrap src json
|
||||
#+begin_src shell :var product=product :results output :exports both :eval no-export
|
||||
stripe prices create \
|
||||
--stripe-account=$account \
|
||||
--product=$product \
|
||||
--unit-amount=350 \
|
||||
--currency=bhd \
|
||||
-d "nickname"="Price in BHD"
|
||||
#+end_src
|
||||
|
||||
#+RESULTS: price-bhd
|
||||
#+begin_src json
|
||||
{
|
||||
"error": {
|
||||
"message": "Invalid currency: bhd. Stripe accounts in US do not support bhd. Your account currently supports these currencies: usd, aed, afn, all, amd, ang, aoa, ars, aud, awg, azn, bam, bbd, bdt, bgn, bif, bmd, bnd, bob, brl, bsd, bwp, bzd, cad, cdf, chf, clp, cny, cop, crc, cve, czk, djf, dkk, dop, dzd, egp, etb, eur, fjd, fkp, gbp, gel, gip, gmd, gnf, gtq, gyd, hkd, hnl, hrk, htg, huf, idr, ils, inr, isk, jmd, jpy, kes, kgs, khr, kmf, krw, kyd, kzt, lak, lbp, lkr, lrd, lsl, mad, mdl, mga, mkd, mmk, mnt, mop, mro, mur, mvr, mwk, mxn, myr, mzn, nad, ngn, nio, nok, npr, nzd, pab, pen, pgk, php, pkr, pln, pyg, qar, ron, rsd, rub, rwf, sar, sbd, scr, sek, sgd, shp, sll, sos, srd, std, szl, thb, tjs, top, try, ttd, twd, tzs, uah, ugx, uyu, uzs, vnd, vuv, wst, xaf, xcd, xof, xpf, yer, zar, zmw.",
|
||||
"param": "currency",
|
||||
"type": "invalid_request_error"
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
** What do fees look like when using an alternate currency?
|
||||
#+name: price-jpy
|
||||
#+caption: Creating a price using Japanese yen (JPY)
|
||||
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||
#+header: :wrap src json
|
||||
#+begin_src shell :var product=product :results output :exports both :eval no-export
|
||||
stripe prices create \
|
||||
--stripe-account=$account \
|
||||
--product=$product \
|
||||
--unit-amount=3500 \
|
||||
--currency=jpy \
|
||||
-d "nickname"="Price in JPY"
|
||||
#+end_src
|
||||
|
||||
#+RESULTS: price-jpy
|
||||
#+begin_src json
|
||||
{
|
||||
"id": "price_1JBOBRIoFf3wvXpR24iMvhoG",
|
||||
"object": "price",
|
||||
"active": true,
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1625854337,
|
||||
"currency": "jpy",
|
||||
"livemode": false,
|
||||
"lookup_key": null,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Price in JPY",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"recurring": null,
|
||||
"tiers_mode": null,
|
||||
"transform_quantity": null,
|
||||
"type": "one_time",
|
||||
"unit_amount": 3500,
|
||||
"unit_amount_decimal": "3500"
|
||||
}
|
||||
#+end_src
|
||||
*** Flat fee in a one-time purchase invoice
|
||||
#+caption: Invoicing a purchase in JPY for a US (USD) account
|
||||
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||
#+header: :var price="price_1JBOBRIoFf3wvXpR24iMvhoG"
|
||||
#+begin_src sh :exports both :results output :wrap src json :eval no-export
|
||||
customer=$(stripe customers create \
|
||||
--stripe-account=$account \
|
||||
--description="JPY Invoice Customer" \
|
||||
--name="Correl Roush" \
|
||||
--email="correl+stripe.roam.docs@gmail.com" \
|
||||
| jq -r .id)
|
||||
|
||||
stripe invoiceitems create \
|
||||
--stripe-account=$account \
|
||||
--customer=$customer \
|
||||
--price=$price >/dev/null
|
||||
|
||||
invoice=$(stripe invoices create \
|
||||
--stripe-account="$account" \
|
||||
--customer="$customer" \
|
||||
--application-fee-amount=500 \
|
||||
| jq -r .id)
|
||||
|
||||
intent=$(stripe invoices finalize_invoice \
|
||||
--stripe-account=$account \
|
||||
$invoice \
|
||||
| jq -r .payment_intent)
|
||||
|
||||
stripe payment_intents confirm $intent \
|
||||
--stripe-account="$account" \
|
||||
--payment-method=pm_card_visa
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
#+begin_src json
|
||||
{
|
||||
"id": "pi_1JBOVQIoFf3wvXpRRJIbx8xf",
|
||||
"object": "payment_intent",
|
||||
"amount": 3500,
|
||||
"amount_capturable": 0,
|
||||
"amount_received": 3500,
|
||||
"application": "ca_IIJmzFRoIG2rIdny5psryEvYurgcOmEP",
|
||||
"application_fee_amount": 500,
|
||||
"canceled_at": null,
|
||||
"cancellation_reason": null,
|
||||
"capture_method": "automatic",
|
||||
"charges": {
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "ch_1JBOVRIoFf3wvXpRCYVnRz0y",
|
||||
"object": "charge",
|
||||
"amount": 3500,
|
||||
"amount_captured": 3500,
|
||||
"amount_refunded": 0,
|
||||
"application": "ca_IIJmzFRoIG2rIdny5psryEvYurgcOmEP",
|
||||
"application_fee": "fee_1JBOVSIoFf3wvXpR2DbO4Xxe",
|
||||
"application_fee_amount": 500,
|
||||
"balance_transaction": "txn_1JBOVSIoFf3wvXpRXaKfOsWB",
|
||||
"billing_details": {
|
||||
"address": {
|
||||
"city": null,
|
||||
"country": null,
|
||||
"line1": null,
|
||||
"line2": null,
|
||||
"postal_code": null,
|
||||
"state": null
|
||||
},
|
||||
"email": null,
|
||||
"name": null,
|
||||
"phone": null
|
||||
},
|
||||
"calculated_statement_descriptor": "CORRELS STUFF",
|
||||
"captured": true,
|
||||
"created": 1625855577,
|
||||
"currency": "jpy",
|
||||
"customer": "cus_Jp2a7jolV9KaII",
|
||||
"description": "Payment for Invoice",
|
||||
"destination": null,
|
||||
"dispute": null,
|
||||
"disputed": false,
|
||||
"failure_code": null,
|
||||
"failure_message": null,
|
||||
"fraud_details": {
|
||||
},
|
||||
"invoice": "in_1JBOVPIoFf3wvXpRuM28s36R",
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"on_behalf_of": null,
|
||||
"order": null,
|
||||
"outcome": {
|
||||
"network_status": "approved_by_network",
|
||||
"reason": null,
|
||||
"risk_level": "normal",
|
||||
"risk_score": 15,
|
||||
"seller_message": "Payment complete.",
|
||||
"type": "authorized"
|
||||
},
|
||||
"paid": true,
|
||||
"payment_intent": "pi_1JBOVQIoFf3wvXpRRJIbx8xf",
|
||||
"payment_method": "pm_1JBOVRIoFf3wvXpR3JKo6m2a",
|
||||
"payment_method_details": {
|
||||
"card": {
|
||||
"brand": "visa",
|
||||
"checks": {
|
||||
"address_line1_check": null,
|
||||
"address_postal_code_check": null,
|
||||
"cvc_check": null
|
||||
},
|
||||
"country": "US",
|
||||
"exp_month": 7,
|
||||
"exp_year": 2022,
|
||||
"fingerprint": "C5qD8oSCGvVCbtcH",
|
||||
"funding": "credit",
|
||||
"installments": null,
|
||||
"last4": "4242",
|
||||
"network": "visa",
|
||||
"three_d_secure": null,
|
||||
"wallet": null
|
||||
},
|
||||
"type": "card"
|
||||
},
|
||||
"receipt_email": null,
|
||||
"receipt_number": null,
|
||||
"receipt_url": "https://pay.stripe.com/receipts/acct_1IGnMkIoFf3wvXpR/ch_1JBOVRIoFf3wvXpRCYVnRz0y/rcpt_Jp2aVQIwbWqn8CZ7T5uF9Wr7MX016cT",
|
||||
"refunded": false,
|
||||
"refunds": {
|
||||
"object": "list",
|
||||
"data": [
|
||||
|
||||
],
|
||||
"has_more": false,
|
||||
"total_count": 0,
|
||||
"url": "/v1/charges/ch_1JBOVRIoFf3wvXpRCYVnRz0y/refunds"
|
||||
},
|
||||
"review": null,
|
||||
"shipping": null,
|
||||
"source": null,
|
||||
"source_transfer": null,
|
||||
"statement_descriptor": null,
|
||||
"statement_descriptor_suffix": null,
|
||||
"status": "succeeded",
|
||||
"transfer_data": null,
|
||||
"transfer_group": null
|
||||
}
|
||||
],
|
||||
"has_more": false,
|
||||
"total_count": 1,
|
||||
"url": "/v1/charges?payment_intent=pi_1JBOVQIoFf3wvXpRRJIbx8xf"
|
||||
},
|
||||
"client_secret": "pi_1JBOVQIoFf3wvXpRRJIbx8xf_secret_sP4fa41NT86GIrhcyfEZEmFJE",
|
||||
"confirmation_method": "automatic",
|
||||
"created": 1625855576,
|
||||
"currency": "jpy",
|
||||
"customer": "cus_Jp2a7jolV9KaII",
|
||||
"description": "Payment for Invoice",
|
||||
"invoice": "in_1JBOVPIoFf3wvXpRuM28s36R",
|
||||
"last_payment_error": null,
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"next_action": null,
|
||||
"on_behalf_of": null,
|
||||
"payment_method": "pm_1JBOVRIoFf3wvXpR3JKo6m2a",
|
||||
"payment_method_options": {
|
||||
"card": {
|
||||
"installments": null,
|
||||
"network": null,
|
||||
"request_three_d_secure": "automatic"
|
||||
}
|
||||
},
|
||||
"payment_method_types": [
|
||||
"card"
|
||||
],
|
||||
"receipt_email": null,
|
||||
"review": null,
|
||||
"setup_future_usage": null,
|
||||
"shipping": null,
|
||||
"source": null,
|
||||
"statement_descriptor": null,
|
||||
"statement_descriptor_suffix": null,
|
||||
"status": "succeeded",
|
||||
"transfer_data": null,
|
||||
"transfer_group": null
|
||||
}
|
||||
#+end_src
|
||||
|
||||
*** Percentage fee in a subscription
|
||||
#+caption: Creating a subscription with a recurring JPY fee for a US (USD) account
|
||||
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||
#+header: :var price="price_1JBOBRIoFf3wvXpR24iMvhoG"
|
||||
#+header: :exports both :results output :wrap src json :eval no-export
|
||||
#+begin_src sh
|
||||
price=$(stripe prices create \
|
||||
--stripe-account=$account \
|
||||
--product=$product \
|
||||
--unit-amount=11500 \
|
||||
--currency=vnd \
|
||||
-d "recurring[interval]"="month" \
|
||||
-d "nickname"="Recurring Price in VND" \
|
||||
| jq -r .id)
|
||||
payment_method=$(stripe payment_methods create \
|
||||
--stripe-account=$account \
|
||||
--type=card \
|
||||
-d "card[number]"=4242424242424242 \
|
||||
-d "card[exp_month]"=3 \
|
||||
-d "card[exp_year]"=2022 \
|
||||
-d "card[cvc]"=314 \
|
||||
| jq -r '.id')
|
||||
customer=$(stripe customers create \
|
||||
--stripe-account=$account \
|
||||
--description="JPY Subscription Customer" \
|
||||
--name="Correl Roush" \
|
||||
--email="correl+stripe.roam.docs@gmail.com" \
|
||||
| jq -r .id)
|
||||
stripe payment_methods attach "$payment_method" \
|
||||
--stripe-account=$account \
|
||||
--customer="$customer" >/dev/null
|
||||
stripe customers update "$customer" \
|
||||
--stripe-account=$account \
|
||||
-d "invoice_settings[default_payment_method]=$payment_method" >/dev/null
|
||||
stripe subscriptions create \
|
||||
--stripe-account=$account \
|
||||
--customer="$customer" \
|
||||
--application-fee-percent=1 \
|
||||
-d "items[0][price]"=$price
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
#+begin_src json
|
||||
{
|
||||
"id": "sub_JqAatXRV9aIjfK",
|
||||
"object": "subscription",
|
||||
"application_fee_percent": 1.0,
|
||||
"automatic_tax": {
|
||||
"enabled": false
|
||||
},
|
||||
"billing_cycle_anchor": 1626115944,
|
||||
"billing_thresholds": null,
|
||||
"cancel_at": null,
|
||||
"cancel_at_period_end": false,
|
||||
"canceled_at": null,
|
||||
"collection_method": "charge_automatically",
|
||||
"created": 1626115944,
|
||||
"current_period_end": 1628794344,
|
||||
"current_period_start": 1626115944,
|
||||
"customer": "cus_JqAaapAUdcr46Z",
|
||||
"days_until_due": null,
|
||||
"default_payment_method": null,
|
||||
"default_source": null,
|
||||
"default_tax_rates": [
|
||||
|
||||
],
|
||||
"discount": null,
|
||||
"ended_at": null,
|
||||
"items": {
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "si_JqAaoXdAhBEisj",
|
||||
"object": "subscription_item",
|
||||
"billing_thresholds": null,
|
||||
"created": 1626115944,
|
||||
"metadata": {
|
||||
},
|
||||
"plan": {
|
||||
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||
"object": "plan",
|
||||
"active": true,
|
||||
"aggregate_usage": null,
|
||||
"amount": 11500,
|
||||
"amount_decimal": "11500",
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1626115939,
|
||||
"currency": "vnd",
|
||||
"interval": "month",
|
||||
"interval_count": 1,
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Recurring Price in VND",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"tiers_mode": null,
|
||||
"transform_usage": null,
|
||||
"trial_period_days": null,
|
||||
"usage_type": "licensed"
|
||||
},
|
||||
"price": {
|
||||
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||
"object": "price",
|
||||
"active": true,
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1626115939,
|
||||
"currency": "vnd",
|
||||
"livemode": false,
|
||||
"lookup_key": null,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Recurring Price in VND",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"recurring": {
|
||||
"aggregate_usage": null,
|
||||
"interval": "month",
|
||||
"interval_count": 1,
|
||||
"trial_period_days": null,
|
||||
"usage_type": "licensed"
|
||||
},
|
||||
"tiers_mode": null,
|
||||
"transform_quantity": null,
|
||||
"type": "recurring",
|
||||
"unit_amount": 11500,
|
||||
"unit_amount_decimal": "11500"
|
||||
},
|
||||
"quantity": 1,
|
||||
"subscription": "sub_JqAatXRV9aIjfK",
|
||||
"tax_rates": [
|
||||
|
||||
]
|
||||
}
|
||||
],
|
||||
"has_more": false,
|
||||
"total_count": 1,
|
||||
"url": "/v1/subscription_items?subscription=sub_JqAatXRV9aIjfK"
|
||||
},
|
||||
"latest_invoice": "in_1JCUEuIoFf3wvXpRl1be4Ymf",
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"next_pending_invoice_item_invoice": null,
|
||||
"pause_collection": null,
|
||||
"pending_invoice_item_interval": null,
|
||||
"pending_setup_intent": null,
|
||||
"pending_update": null,
|
||||
"plan": {
|
||||
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||
"object": "plan",
|
||||
"active": true,
|
||||
"aggregate_usage": null,
|
||||
"amount": 11500,
|
||||
"amount_decimal": "11500",
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1626115939,
|
||||
"currency": "vnd",
|
||||
"interval": "month",
|
||||
"interval_count": 1,
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Recurring Price in VND",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"tiers_mode": null,
|
||||
"transform_usage": null,
|
||||
"trial_period_days": null,
|
||||
"usage_type": "licensed"
|
||||
},
|
||||
"quantity": 1,
|
||||
"schedule": null,
|
||||
"start_date": 1626115944,
|
||||
"status": "active",
|
||||
"transfer_data": null,
|
||||
"trial_end": null,
|
||||
"trial_start": null
|
||||
}
|
||||
#+end_src
|
||||
|
||||
#+caption: Fetching the initial invoice
|
||||
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||
#+header: :exports both :results output :wrap src json :eval no-export
|
||||
#+begin_src sh
|
||||
stripe get in_1JCUEuIoFf3wvXpRl1be4Ymf \
|
||||
--stripe-account=$account
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
#+begin_src json
|
||||
{
|
||||
"id": "in_1JCUEuIoFf3wvXpRl1be4Ymf",
|
||||
"object": "invoice",
|
||||
"account_country": "US",
|
||||
"account_name": "Correl's Stuff",
|
||||
"account_tax_ids": null,
|
||||
"amount_due": 11500,
|
||||
"amount_paid": 11500,
|
||||
"amount_remaining": 0,
|
||||
"application_fee_amount": 115,
|
||||
"attempt_count": 1,
|
||||
"attempted": true,
|
||||
"auto_advance": false,
|
||||
"automatic_tax": {
|
||||
"enabled": false,
|
||||
"status": null
|
||||
},
|
||||
"billing_reason": "subscription_create",
|
||||
"charge": "ch_1JCUEuIoFf3wvXpRB1M6kEQz",
|
||||
"collection_method": "charge_automatically",
|
||||
"created": 1626115944,
|
||||
"currency": "vnd",
|
||||
"custom_fields": null,
|
||||
"customer": "cus_JqAaapAUdcr46Z",
|
||||
"customer_address": null,
|
||||
"customer_email": "correl+stripe.roam.docs@gmail.com",
|
||||
"customer_name": "Correl Roush",
|
||||
"customer_phone": null,
|
||||
"customer_shipping": null,
|
||||
"customer_tax_exempt": "none",
|
||||
"customer_tax_ids": [
|
||||
|
||||
],
|
||||
"default_payment_method": null,
|
||||
"default_source": null,
|
||||
"default_tax_rates": [
|
||||
|
||||
],
|
||||
"description": null,
|
||||
"discount": null,
|
||||
"discounts": [
|
||||
|
||||
],
|
||||
"due_date": null,
|
||||
"ending_balance": 0,
|
||||
"footer": null,
|
||||
"hosted_invoice_url": "https://invoice.stripe.com/i/acct_1IGnMkIoFf3wvXpR/invst_JqAahNswSL7xgr1R8bfwCjJA4V0UouX",
|
||||
"invoice_pdf": "https://pay.stripe.com/invoice/acct_1IGnMkIoFf3wvXpR/invst_JqAahNswSL7xgr1R8bfwCjJA4V0UouX/pdf",
|
||||
"last_finalization_error": null,
|
||||
"lines": {
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "il_1JCUEuIoFf3wvXpRcrDHYRXJ",
|
||||
"object": "line_item",
|
||||
"amount": 11500,
|
||||
"currency": "vnd",
|
||||
"description": "1 × Example Product (Org-Roam Doc) (at ₫11,500 / month)",
|
||||
"discount_amounts": [
|
||||
|
||||
],
|
||||
"discountable": true,
|
||||
"discounts": [
|
||||
|
||||
],
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"period": {
|
||||
"end": 1628794344,
|
||||
"start": 1626115944
|
||||
},
|
||||
"plan": {
|
||||
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||
"object": "plan",
|
||||
"active": true,
|
||||
"aggregate_usage": null,
|
||||
"amount": 11500,
|
||||
"amount_decimal": "11500",
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1626115939,
|
||||
"currency": "vnd",
|
||||
"interval": "month",
|
||||
"interval_count": 1,
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Recurring Price in VND",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"tiers_mode": null,
|
||||
"transform_usage": null,
|
||||
"trial_period_days": null,
|
||||
"usage_type": "licensed"
|
||||
},
|
||||
"price": {
|
||||
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||
"object": "price",
|
||||
"active": true,
|
||||
"billing_scheme": "per_unit",
|
||||
"created": 1626115939,
|
||||
"currency": "vnd",
|
||||
"livemode": false,
|
||||
"lookup_key": null,
|
||||
"metadata": {
|
||||
},
|
||||
"nickname": "Recurring Price in VND",
|
||||
"product": "prod_JgnmcPS2MRwDtp",
|
||||
"recurring": {
|
||||
"aggregate_usage": null,
|
||||
"interval": "month",
|
||||
"interval_count": 1,
|
||||
"trial_period_days": null,
|
||||
"usage_type": "licensed"
|
||||
},
|
||||
"tiers_mode": null,
|
||||
"transform_quantity": null,
|
||||
"type": "recurring",
|
||||
"unit_amount": 11500,
|
||||
"unit_amount_decimal": "11500"
|
||||
},
|
||||
"proration": false,
|
||||
"quantity": 1,
|
||||
"subscription": "sub_JqAatXRV9aIjfK",
|
||||
"subscription_item": "si_JqAaoXdAhBEisj",
|
||||
"tax_amounts": [
|
||||
|
||||
],
|
||||
"tax_rates": [
|
||||
|
||||
],
|
||||
"type": "subscription"
|
||||
}
|
||||
],
|
||||
"has_more": false,
|
||||
"total_count": 1,
|
||||
"url": "/v1/invoices/in_1JCUEuIoFf3wvXpRl1be4Ymf/lines"
|
||||
},
|
||||
"livemode": false,
|
||||
"metadata": {
|
||||
},
|
||||
"next_payment_attempt": null,
|
||||
"number": "CC4DCDC8-0001",
|
||||
"on_behalf_of": null,
|
||||
"paid": true,
|
||||
"payment_intent": "pi_1JCUEuIoFf3wvXpRYSMrrCoL",
|
||||
"payment_settings": {
|
||||
"payment_method_options": null,
|
||||
"payment_method_types": null
|
||||
},
|
||||
"period_end": 1626115944,
|
||||
"period_start": 1626115944,
|
||||
"post_payment_credit_notes_amount": 0,
|
||||
"pre_payment_credit_notes_amount": 0,
|
||||
"receipt_number": "2336-2115",
|
||||
"starting_balance": 0,
|
||||
"statement_descriptor": null,
|
||||
"status": "paid",
|
||||
"status_transitions": {
|
||||
"finalized_at": 1626115944,
|
||||
"marked_uncollectible_at": null,
|
||||
"paid_at": 1626115944,
|
||||
"voided_at": null
|
||||
},
|
||||
"subscription": "sub_JqAatXRV9aIjfK",
|
||||
"subtotal": 11500,
|
||||
"tax": null,
|
||||
"total": 11500,
|
||||
"total_discount_amounts": [
|
||||
|
||||
],
|
||||
"total_tax_amounts": [
|
||||
|
||||
],
|
||||
"transfer_data": null,
|
||||
"webhooks_delivered_at": 1626115947
|
||||
}
|
||||
#+end_src
|
||||
|
||||
* What do we do about sales tracking?
|
||||
- Start storing currency with sales
|
||||
+ Add a currency column to the analytics database
|
||||
+ Add a currency field to the avro event
|
||||
- +Hide non-usd transactions from reports, etc. until they are ready?+
|
||||
- Beware of mixed-currency totals
|
||||
|
||||
* Tasks
|
||||
** Add an endpoint exposing an account's supported currencies
|
||||
** Remove =usd= restriction to the currency field in Stripe requests
|
||||
** Add acceptance tests for multiple currencies
|
||||
** Reporting
|
||||
*** Add a currency column to the analytics database
|
||||
*** Add a currency field to the pageview avro event
|
||||
*** Update page hits consumer to store the currency field
|
||||
*** Update reports
|
||||
Both classic and redesigned reports need to be updated to handle multiple
|
||||
currencies.
|
||||
|
||||
- Totals must be broken down by currency
|
||||
- Values must be displayed in a manner appropriate for its currency
|
||||
|
||||
It'll be more straightforward to duplicate the graph/table for each currency or
|
||||
otherwise filter them rather than attempt to restructure them to accomodate
|
||||
another dimension (currency).
|
||||
|
||||
- Do we want to eliminate the old reports? Or at least the old sales over time
|
||||
report? The new sales over time report will need the activity detail section.
|
||||
+ /Yes, provided it's doable in a similar or shorter amount of time./
|
||||
**** Revenue Over Time
|
||||
https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/analytics_charts_controller.php
|
||||
**** Sales over time
|
||||
*** Update display of sale value in subscriber details
|
||||
We'll want to either find a way to mimic the behavior of the frontend currency
|
||||
display library, or pass the value through in such a way that we could use that
|
||||
library.
|
||||
|
|
@ -1,199 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 83d61eef-0781-46e0-b959-1a739cff5ea3
|
||||
:END:
|
||||
#+title: Stripe Poller
|
||||
|
||||
Identify webhook events tracked by Stripe that have not yet been processed by
|
||||
our service, and replay them against it.
|
||||
|
||||
* Investigation
|
||||
** Fetching events on behalf of each connected account
|
||||
On the afternoon of [[id:8c85f055-9d0c-4b7b-991e-1e32905d38ba][2021-07-02]] I fetched the 860 connected accounts and fetched
|
||||
webhook events from the previous two hours. Each request to Stripe took, on
|
||||
average, 0.38 seconds, requiring one request per account, totalling 323.77
|
||||
seconds, or nearly five and a half minutes. No degradation nor failure of Stripe
|
||||
functionality was noted. From all of these requests, a total of 19 events were
|
||||
retrieved.
|
||||
** Alternatives
|
||||
*** The Stripe dashboard
|
||||
Viewing a particular webhook within the Stripe dashboard allows us to view the
|
||||
events sent specifically to that webhook, independent of the account to which it
|
||||
relates. Regrettably, this is fed from a dashboard-specific API which does not
|
||||
appear to be exposed elsewhere for consumption.
|
||||
*** Email notification
|
||||
We are emailed when a webhook event fails to process. It may make sense to alert
|
||||
when this occurs and correct the issue manually, or find a way to automate its
|
||||
replay.
|
||||
** Conclusion
|
||||
Barring the release of an API to fetch all events sent to a single webhook,
|
||||
ideally filtered by delivery status, fetching these events from Stripe is
|
||||
terribly inefficient. Though we can process recent events in a timely manner, we
|
||||
will likely be further slowed as more customers connect with Stripe, and there
|
||||
is no good solution at this time for dealing with that problem. Given the
|
||||
infrequency with which Stripe fails to send us an event via their retry
|
||||
mechanisms, I expect we will be better served by reacting to notifications of
|
||||
those failures than to routinely slam their API with inefficient requests.
|
||||
* Tasks
|
||||
** Build Poller
|
||||
*** Prepare a rundeck job for the Stripe poller
|
||||
- Create the project
|
||||
- Build the rundeck job
|
||||
*** Add a stripe-payments endpoint to fetch logged events
|
||||
- Needs only return event ids
|
||||
- Events are partitioned by date and sorted by time in the time-index GSI, which projects keys only
|
||||
|
||||
#+begin_src yaml
|
||||
paths:
|
||||
/stripe/events:
|
||||
get:
|
||||
summary: Fetch webhook events
|
||||
tags:
|
||||
- Webhooks
|
||||
parameters:
|
||||
- name: last_id
|
||||
in: query
|
||||
schema:
|
||||
$ref: '#/components/schemas/EventId'
|
||||
- name: limit
|
||||
in: query
|
||||
description: Number of events to return per page of results
|
||||
schema:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 1000
|
||||
default: 100
|
||||
responses:
|
||||
'200':
|
||||
description: Event list
|
||||
headers:
|
||||
Link:
|
||||
description: Pagination links
|
||||
schema:
|
||||
type: string
|
||||
example: >-
|
||||
<{{ base_url }}/stripe/events?last_id=evt_1234abcdef>; rel="next"
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: '#/components/schemas/EventId'
|
||||
components:
|
||||
schemas:
|
||||
EventId:
|
||||
type: string
|
||||
description: Unique identifier for the webhook event
|
||||
example: "evt_1IHAsBIoFf3wvXpR7VLvGfae"
|
||||
|
||||
#+end_src
|
||||
*** Add integrations endpoint enumerating connected Stripe accounts
|
||||
*** Reprocess unlogged Stripe events
|
||||
- Iterate processed event ids into memory
|
||||
- Iterate over published events, sending them to the stripe-payments webhook
|
||||
endpoint if they are not in the processed set
|
||||
|
||||
#+begin_src plantuml :file stripe-poller.svg
|
||||
loop until last page of results
|
||||
Poller -> StripePayments : GET /stripe/events?since=<timestamp>[&last_id=<last_id>]
|
||||
StripePayments --> Poller : Return list of event IDs
|
||||
end
|
||||
loop until last page of results
|
||||
Poller -> Integrations : Get list of connected accounts
|
||||
Integrations --> Poller : Return list of Stripe account IDs
|
||||
loop for each account
|
||||
loop until last page of results
|
||||
Poller -> Stripe : GET /v1/events?created[gte]=<timestamp>&types[]=<...>[&starting_after=<last_id>]
|
||||
StripePayments --> Poller : Return list of event IDs
|
||||
alt is an AWeber Ecommerce event and event not in processed
|
||||
Poller -> StripePayments : POST /stripe/webhooks
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:stripe-poller.svg]]
|
||||
|
||||
**** Determining how far back to look for events
|
||||
Stripe events are retained for a maximum of thirty days, on their end and also
|
||||
in our event database.
|
||||
|
||||
The job could use consul to store the time of the latest fetched event to be
|
||||
referenced in subsequent runs. If it is set, it should collect all events newer
|
||||
than one hour prior to that time (this window may also be configurable). If that
|
||||
time is not set, it should collect all available events. This value should be
|
||||
updated in consul only when the job completes successfully.
|
||||
|
||||
Hit the Rundeck API to get the time of the last successful execution.
|
||||
|
||||
**** Retrieving webhook events
|
||||
https://stripe.com/docs/api/events/list
|
||||
|
||||
+ Events can be filtered by type, creation time, and whether they were
|
||||
successfully delivered (the webhook endpoint returned a =200 OK= response)
|
||||
- Wait, how does it know it was succesfully delivered to /our/ webhook
|
||||
endpoint?
|
||||
+ Should we set up another "client" with its own rate limiting with Stripe for
|
||||
the poller's requests?
|
||||
|
||||
#+CAPTION: Support chat on [2021-07-01 Thu]
|
||||
#+begin_quote
|
||||
- My team is hoping to build an automated process to check via the Stripe API
|
||||
for any events that weren't successfully delivered to our webhook endpoint and
|
||||
see that they're handled appropriately. I've been trying out the events list
|
||||
endpoint, and it does not seem to return all of the events that were sent to
|
||||
our webhook, nor does there appear to be any way to identify a webhook to that
|
||||
endpoint to find and filter events for it. We assume this is because the
|
||||
events are triggered from connected accounts. We noticed that the Stripe
|
||||
dashboard is able to show events sent to a webhook and their status, is there
|
||||
some way to achieve this through the API?
|
||||
+ /Hector Otero has joined./
|
||||
+ Hi, there. I am glad to help.
|
||||
- Hello. Are you able to see my previous message?
|
||||
+ Yes, I am just reading through it.
|
||||
+ Ok, so your main question is if there is a way to show events sent to a webhook and their status using the API, similar to how it is done on the Stripe dashboard.
|
||||
+ Is that correct?
|
||||
- Correct
|
||||
+ Ok, my brief look into the documentation has not turned up anything regarding how to implement this. I am going to consult with my team members regarding whether any of them knows how to implement this functionality. Once I have more info I can reach out to you via email, is that alright?
|
||||
- Yes, thank you.
|
||||
+ Sure, no worries. Please keep an eye on your email inbox awapi@aweber.com for further correspondence regarding this issue.
|
||||
+ Have a nice day! we will be in contact soon.
|
||||
- Thanks!
|
||||
+ /Hector Otero has left./
|
||||
#+end_quote
|
||||
|
||||
**** Signing the webhook request
|
||||
Webhook events [[https://stripe.com/docs/webhooks/signatures][must be signed using a secret key]], which we store [[http://consul.service.production.consul/ui/production/kv/services/cp/services/stripe-payments/stripe_webhook_secret/edit][in consul]]. The
|
||||
following /should/ result in a valid signature header:
|
||||
|
||||
#+CAPTION: Stripe signature generation example
|
||||
#+begin_src python :results output :exports code
|
||||
import hmac
|
||||
import hashlib
|
||||
import time
|
||||
|
||||
unix_timestamp = int(time.time())
|
||||
json_payload = '{ ... }'
|
||||
secret_key = 'secret'
|
||||
|
||||
message = f'{unix_timestamp}.{json_payload}'
|
||||
signature = hmac.new(bytes(secret_key, 'utf-8'),
|
||||
msg=bytes(message, 'utf-8'),
|
||||
digestmod=hashlib.sha256).hexdigest()
|
||||
|
||||
print(f'Stripe-Signature: t={unix_timestamp},v1={signature}')
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
: Stripe-Signature: t=1625084385,v1=dce1ef0332969bce98fd76b5fd08d1b07af0d0fd5f9788d9f8435537e5c3cd12
|
||||
*** Create playbook and dashboard for the Stripe poller
|
||||
- Track how many events are resent for processing vs how many are already processed
|
||||
** KILL Create an event processing endpoint
|
||||
Create an endpoint in the Stripe Payments service for internal use that will,
|
||||
given an event id, fetch that event from Stripe and process it as though it had
|
||||
been sent to the webhook endpoint.
|
||||
** TODO Document how to find and replay a Stripe event
|
||||
- Use the Stripe UI to find failed events within the past 15 days
|
||||
- Include steps for fetching an event from more than 15 days ago from the API
|
||||
and sending that to the webhook endpoint.
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 89c2dda6-46d7-41c9-8af7-18ce604a2daf
|
||||
:END:
|
||||
#+title: Supporting multiple time zones
|
||||
* Initial investigation
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e06b26c8-9227-4fcc-8f0a-9b83c64693b4
|
||||
:END:
|
||||
#+title: Deploying projects
|
||||
* Environments
|
||||
- Testing :: Newly merged code is automatically deployed to this environment to
|
||||
be tested.
|
||||
- Staging :: Tagged releases are automatically deployed to this environment for
|
||||
spot-checking prior to production release.
|
||||
- Production :: Tagged releases are manually deployed to the live production
|
||||
environment.
|
||||
|
||||
* Deployment methods
|
||||
** Gitlab CI
|
||||
Projects define pipelines in a =.gitlab-ci.yml= file to automate running tests,
|
||||
building the project, and deploying it to our three platform environments.
|
||||
** Jenkins :deprecated:
|
||||
Pipelines are defined in Jenkins to react to pushed and tagged code in source
|
||||
control to run tests and deploy projects to our platform environments.
|
||||
** Chef / Puppet :deprecated:
|
||||
* Deployment targets
|
||||
** Buzzops (Local Kubernetes)
|
||||
https://confluence.aweber.io/display/STD/Kubernetes+Application+Deployment
|
||||
** Amazon Web Services
|
||||
When appropriate, dockerized applications may be deployed to Amazon ECS
|
||||
** Chef and Puppet managed virtual machines :deprecated:
|
||||
* Further information
|
||||
** Front-End Applications
|
||||
https://confluence.aweber.io/display/FEBOF/Web+App+Deployment
|
||||
** Control Panel (Sites)
|
||||
https://confluence.aweber.io/display/AR/Control+Panel+%28Sites%29+Playbook
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: bf5d1146-9481-4710-8143-61086f263a7a
|
||||
:END:
|
||||
#+title: Team Member Onboarding
|
||||
|
||||
- [[id:e06b26c8-9227-4fcc-8f0a-9b83c64693b4][Deploying projects]]
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 9cfd85fd-998e-4f21-b82e-c7963576c202
|
||||
:END:
|
||||
#+title: Deploying S4 to Kubernetes
|
||||
|
||||
Tasks for deploying [[id:c7322400-c6e6-4595-87e2-7db6e57b6a2b][S4]] to Kubernetes.
|
||||
|
||||
* DONE Deploying the service
|
||||
* Migrating the Redis database
|
||||
* Deploying the workers
|
||||
** DONE Use environment variables for configuration rather than consul
|
||||
https://gitlab.aweber.io/CP/Rundeck/s4-utils/-/commit/8d4dd3b8196bcd716a70e1c751b4743fdaa62646
|
||||
** DONE Set up all the rundeck jobs in testing
|
||||
- Configure the rundeck jobs
|
||||
- Ensure all ACL tokens are accounted for
|
||||
** TODO Update the appdb credentials
|
||||
- [ ] Testing
|
||||
- [ ] Staging
|
||||
- [ ] Production
|
||||
** TODO Set up all the rundeck jobs in staging
|
||||
** TODO Set up all the rundeck jobs in production
|
||||
** TODO Add missing config values
|
||||
- IP ranges =lock_key= and =lock_minutes=
|
||||
|
|
@ -1,108 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 207560cc-7700-4d06-918d-cc01ae530146
|
||||
:END:
|
||||
#+title: Projects
|
||||
#+STARTUP: indent logdrawer
|
||||
#+COLUMNS: %50ITEM %JIRA_ID
|
||||
#+PROPERTY: Effort_ALL 0:30 1:00 0.5d 1d 2d 3d 4d 1w
|
||||
#+PROPERTY: ClassificationOfWork_ALL backend frontend ops product design
|
||||
#+TODO: BACKLOG(b!) TODO(t!) | DONE(d@!) CANCELLED(c@!)
|
||||
#+TAGS: { SPRINT(S) EPIC(e) STORY(s) BUG(b) TASK(t) }
|
||||
#+OPTIONS: num:nil toc:t arch:nil p:t prop:t
|
||||
#+LINK: jira https://jira.aweber.io/browse/
|
||||
|
||||
* Priorities
|
||||
#+BEGIN: columnview :id global :match "TODO=\"TODO\""
|
||||
| ITEM | JIRA_ID |
|
||||
|------------------------------------------------+---------|
|
||||
| Create the [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]] | |
|
||||
| Create the [[id:c45881de-46f2-4f76-9579-063626c5956c][Analytics View Service]] | |
|
||||
| [[id:619b6c78-7be9-4ee4-a0b7-9d1a4d7536e2][Migrating services to use the new List service]] | |
|
||||
#+END:
|
||||
|
||||
* Service Upgrades
|
||||
** DONE Deploy GeoIP to Kubernetes
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11592
|
||||
:END:
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-09-01 Wed 13:44]
|
||||
:END:
|
||||
** DONE [[id:9cfd85fd-998e-4f21-b82e-c7963576c202][Deploying S4 to Kubernetes]]
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-10549
|
||||
:END:
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||
:END:
|
||||
** DONE [[id:6413d680-ee2e-43e6-b7c7-10f14e0873c2][Deploying Bulk Tagging to Kubernetes]]
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11615
|
||||
:END:
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||
:END:
|
||||
** BACKLOG Deploying Domain Validator to Kubernetes
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-10554
|
||||
:END:
|
||||
:LOGBOOK:
|
||||
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:53]
|
||||
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||
:END:
|
||||
** BACKLOG Deploy Notification Bar to Kubernetes
|
||||
:LOGBOOK:
|
||||
- State "BACKLOG" from [2021-10-26 Tue 14:02]
|
||||
:END:
|
||||
** DONE Deploying Recipient Service to Kubernetes
|
||||
:LOGBOOK:
|
||||
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:53]
|
||||
- State "TODO" from [2021-10-13 Wed 16:26]
|
||||
:END:
|
||||
** BACKLOG Deploying Tagging Service to Kubernetes
|
||||
:LOGBOOK:
|
||||
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:53]
|
||||
- State "TODO" from [2021-10-13 Wed 16:26]
|
||||
:END:
|
||||
* [[id:f633f967-11d2-432c-b5ff-ad842c88a51c][Decommissioning Sites]]
|
||||
** [[id:3cc8bd09-dd02-4950-8c89-a737f92809fd][Tracking progress of moving pages out of Sites]]
|
||||
** Creating a new Control Panel shell application
|
||||
* TODO Create the [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]]
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-10-20 Wed 15:57]
|
||||
:END:
|
||||
* TODO Create the [[id:c45881de-46f2-4f76-9579-063626c5956c][Analytics View Service]]
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-10-20 Wed 15:57]
|
||||
:END:
|
||||
* [[id:ee5b8d5f-e3d4-45c2-9ce6-bcd8c7a63376][Retire Redcache]]
|
||||
* [[id:4df15f2f-d2e1-40f4-8acd-dbfb78fe304f][Deploy CoreAPI to Kubernetes]]
|
||||
* Replacing CAPI Services
|
||||
** TODO [[id:619b6c78-7be9-4ee4-a0b7-9d1a4d7536e2][Migrating services to use the new List service]]
|
||||
:LOGBOOK:
|
||||
- State "TODO" from [2021-10-20 Wed 15:58]
|
||||
:END:
|
||||
*** DONE Set EOL date for awlists
|
||||
- [2021-08-13 Fri 15:21] :: Discussed this. Also talked about separation of
|
||||
concerns about account status vs list status. Also discussed how an
|
||||
entitlements service might fit into our architecture and how we handle state
|
||||
transitions and reverals (e.g. cancellations).
|
||||
- [2021-08-17 Tue 16:44] :: Set a one-year time limit? Should the public list
|
||||
endpoints be in the new service as well, deprecating public api lists?
|
||||
- [2021-10-18 Mon] :: The expectation is set to be migrated to the new list service exclusively by the end of Q2 2022
|
||||
** Subscribers API
|
||||
*** [[id:2c1a7b1d-8726-4b88-9534-2f5abfec35f0][Use AppDB as the source of truth for subscriber data in Recipient]]
|
||||
* Frontend Client Upgrades
|
||||
** Upgrade Dashboard to React
|
||||
*** BACKLOG Create an API for broadcasts and sent messages across lists
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11609
|
||||
:END:
|
||||
:LOGBOOK:
|
||||
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:57]
|
||||
- State "TODO" from [2021-09-01 Wed 13:33]
|
||||
:END:
|
||||
** Upgrade other non-React projects to React
|
||||
*** Add subscriber
|
||||
** [[id:fab0cf8f-7c54-4848-882b-dba5e087760d][Redesigned Reports]]
|
||||
* New List Management Interface
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 6413d680-ee2e-43e6-b7c7-10f14e0873c2
|
||||
:END:
|
||||
#+title: Deploying Bulk Tagging to Kubernetes
|
||||
#+filetags: project
|
||||
|
||||
* DONE Deploy to kubernetes in production
|
||||
|
||||
* Update services to use the new consul hostname
|
||||
** DONE Subscriber Proxy
|
||||
Configured in consul
|
||||
** DONE Bulk Tagging Consumers
|
||||
Configured in consul
|
||||
** TODO AWSubscribers
|
||||
https://gitlab.aweber.io/CP/Services/awsubscribers
|
||||
** TODO Bulk Tagging Acceptance Tests
|
||||
[[https://gitlab.aweber.io/CP/automation/bulk-tagging-acceptance/]]
|
||||
** TODO QA Regression Tests
|
||||
- https://gitlab.aweber.io/BoFs/QA/QAWeber
|
||||
|
|
@ -1,202 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 11edd6c9-b976-403b-a419-b5542ddedaae
|
||||
:END:
|
||||
#+title: Subscriber Search Service
|
||||
#+LINK: jira https://jira.aweber.io/browse/
|
||||
|
||||
A replacement for the current [[id:d9cb2b55-3b0e-4ab3-8369-f71ebc3cd882][Sites Subscriber Search]] and [[id:f74e335d-577f-4749-bf32-1c025795b039][Broadcast Segment Search]] implementations.
|
||||
|
||||
- [[https://jira.aweber.io/issues/?jql=project%20%3D%20CCPANEL%20AND%20component%20%3D%20%22Search%20Service%22][JIRA tickets]]
|
||||
|
||||
* Architecture Notes
|
||||
#+begin_quote
|
||||
Added: [2019-08-06 Tue 10:34]
|
||||
|
||||
- Broadcast sending manipulated queries before performing them to suit
|
||||
its needs
|
||||
- *Handle list exclusion in the search service?* ([[jira:CCPANEL-9557][CCPANEL-9557]])
|
||||
- Blocked emails
|
||||
- De-duplication
|
||||
- *Lead view ids -> segment ids -> search service segments?* ([[jira:CCPANEL-9556][CCPANEL-9556]])
|
||||
- Side-by-side comparison
|
||||
- Use broadcast-segment to compare its results to the search api
|
||||
results
|
||||
- *Recipient-style representation or leads?*
|
||||
- *Define API* ([[jira:CCPANEL-9554][CCPANEL-9554]])
|
||||
- Include saved searches (segments)
|
||||
- *Iterate release* ([[jira:CCPANEL-9555][CCPANEL-9555]])
|
||||
- Current UI
|
||||
- New UI with same capabilities of old UI
|
||||
- Still writing old-style segments
|
||||
- Research getting off the leads table (aurora?)
|
||||
- https://confluence.aweber.io/display/AR/Search+Service+Using+Existing+Databases
|
||||
- Distinct from a materialized search-optimized db
|
||||
- Deal with =subscriber_tags= table bloat in AppDB
|
||||
#+end_quote
|
||||
** Component Diagram
|
||||
#+BEGIN_SRC plantuml :file search-components.svg
|
||||
database Analytics {
|
||||
database Ana as Ana01
|
||||
database Ana as Ana02
|
||||
database Ana as Ana03
|
||||
}
|
||||
database App
|
||||
database "Results Cache" as ResultsCache
|
||||
|
||||
component "Search Service" as Service {
|
||||
|
||||
component Search
|
||||
component Results
|
||||
|
||||
Search -- Analytics
|
||||
Search -- App
|
||||
Search --> ResultsCache
|
||||
|
||||
Results <-- ResultsCache
|
||||
|
||||
}
|
||||
#+END_SRC
|
||||
|
||||
#+RESULTS:
|
||||
[[file:search-components.svg]]
|
||||
|
||||
* Concerns
|
||||
** Performance
|
||||
Email delivery has cases
|
||||
|
||||
*** Analytics query speedups for broadcasts
|
||||
https://jira.aweber.io/browse/EDELIV-8318
|
||||
** Fitness to Purpose
|
||||
This service will need to fulfill the needs of both end-user subscriber searches
|
||||
and segment emailing.
|
||||
|
||||
* Plan
|
||||
:PROPERTIES:
|
||||
:COLUMNS: %40ITEM %Effort{:}
|
||||
:END:
|
||||
#+BEGIN: columnview :id local
|
||||
| ITEM | Effort |
|
||||
|--------------------------------------------------------+----------|
|
||||
| Plan | 18d 0:00 |
|
||||
| Search Centralization | 18d 0:00 |
|
||||
| Expose search inputs backed with the existing database | 2d |
|
||||
| Enable dblink on the search master database | 2d |
|
||||
| Create new unlogged search results table | 2d |
|
||||
| Define the search result format | 1d |
|
||||
| Perform search using new search DSL | 5d |
|
||||
| Perform search using legacy segment ID | 3d |
|
||||
| Manage segments using the existing database | 3d |
|
||||
| Migrate to an updated schema | |
|
||||
| Migrate to new search service | |
|
||||
| Create new subscriber management React application | |
|
||||
| Update broadcast-segment to use new search service | |
|
||||
| Milestone 3: Add new search features | |
|
||||
#+END:
|
||||
|
||||
** A Dedicated Service for Subscriber Search
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-10169
|
||||
:END:
|
||||
Create a dedicated, publicly exposed service for performing searches on
|
||||
subscribers using subscriber and analytics criteria. The goal of this project is
|
||||
to replace the current implementation from sites and in the broadcast segment
|
||||
service.
|
||||
|
||||
*** Expose search inputs backed with the existing database
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:END:
|
||||
- Include IDs required to build existing POST format
|
||||
|
||||
*** Enable dblink on the search master database
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:END:
|
||||
[[jira:CCPANEL-7147][CCPANEL-7147]]
|
||||
|
||||
*** NO Create new unlogged search results table
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:END:
|
||||
https://jira.aweber.io/browse/CCPANEL-7077
|
||||
|
||||
*** Define the search result format
|
||||
:PROPERTIES:
|
||||
:Effort: 1d
|
||||
:JIRA_ID: CCPANEL-10440
|
||||
:END:
|
||||
https://xd.adobe.com/view/ae8fb2b2-c039-4e88-8ade-ff2562a8c8cf-fbdc/screen/c03f09c7-187e-4a6f-8b8c-571d131daee1/ (ignore engagement column)
|
||||
|
||||
- name
|
||||
- email
|
||||
- source
|
||||
- status
|
||||
- date added
|
||||
- last updated
|
||||
*** Perform search using new search DSL
|
||||
:PROPERTIES:
|
||||
:Effort: 5d
|
||||
:END:
|
||||
**** DONE Perform search using text comparisons
|
||||
- Is / Is Not
|
||||
- Contains / Does not contain
|
||||
- Starts with / Does not start with
|
||||
- Ends with / Does not end with
|
||||
**** DONE Perform search using numeric comparisons
|
||||
**** DONE Perform search using tag sets
|
||||
**** DONE Perform search using enumerated values
|
||||
**** Add support for remaining static AppDB filters
|
||||
- Date range fields
|
||||
- Remaining string / numeric fields
|
||||
**** Add support for custom field filters
|
||||
- Add all custom field columns as supported filters
|
||||
- Fetch the custom fields for the list and use them when building the list of
|
||||
available filters.
|
||||
**** Add support for Analytics filters
|
||||
- Clicks
|
||||
- Opens
|
||||
- Messages
|
||||
- Web pages (links)
|
||||
*** Perform search using legacy segment ID
|
||||
:PROPERTIES:
|
||||
:Effort: 3d
|
||||
:END:
|
||||
|
||||
Provided with a legacy segment ID, execute a search using its stored parameters.
|
||||
*** Manage segments using the existing database
|
||||
:PROPERTIES:
|
||||
:Effort: 3d
|
||||
:END:
|
||||
Create, retrieve, update, and delete legacy segments using the new search DSL.
|
||||
*** Manage segments stored using the new DSL
|
||||
*** Migrate legacy segments to the new DSL
|
||||
*** Migrate to an updated schema
|
||||
*** Support filtering options used by broadcast-segment
|
||||
** Centralizing Subscriber Search
|
||||
Applications making use of subscriber search will be updated to use the new
|
||||
dedicated service, eliminating multiple search implementations.
|
||||
*** Create new subscriber management React application
|
||||
*** Update broadcast-segment to use new search service
|
||||
** Milestone 3: Add new search features
|
||||
|
||||
* Implementation
|
||||
|
||||
** [[id:7b0f97f3-9037-4d05-9170-a478e97c8d1f][Modeling the new search DSL]]
|
||||
|
||||
** Constructing SQL queries programmatically
|
||||
|
||||
** Translating legacy segments
|
||||
|
||||
** Gathering results
|
||||
|
||||
** Reaching into Analytics
|
||||
|
||||
* Resources
|
||||
- [[https://confluence.aweber.io/display/AR/PostgreSQL+Backed+Search][PostgreSQL Backed Search]] (Rejected ACP)
|
||||
- [[https://confluence.aweber.io/display/AR/Search+Proxy+Service][Search Proxy Service]]
|
||||
- +[[https://confluence.aweber.io/display/~robink/SoT+-+ElasticSearch+Next+Steps][SoT - ElasticSearch Next Steps]]+
|
||||
- [[https://confluence.aweber.io/display/~robink/Alternative+Search+Proposal][Alternative Search Proposal]]
|
||||
- [[https://confluence.aweber.io/display/AR/Search+Service+Using+Existing+Databases][Search Service Using Existing Databases]] (Approved ACP)
|
||||
+ [[https://confluence.aweber.io/display/~victorc/Search+Service+-+Proof+of+Concept+Findings][Search Service - Proof of Concept Findings]] (Benchmarks different approaches)
|
||||
+ [[https://gitlab.aweber.io/CP/archive/victorc-search-prototype][Search Prototype]]
|
||||
- [[https://confluence.aweber.io/display/AR/Search+DSL+JSON+Schema][Search DSL JSON Schema]]
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d9cb2b55-3b0e-4ab3-8369-f71ebc3cd882
|
||||
:END:
|
||||
#+title: Sites Subscriber Search
|
||||
|
||||
* Sorting
|
||||
Added: [2020-04-14 Tue 13:34]
|
||||
|
||||
The current sites search code includes the following functioning code for
|
||||
setting a sort order on a search based on form input:
|
||||
|
||||
https://gitlab.aweber.io/CP/applications/sites/blob/52d1d944854554c5818ef9a46c8a12493599eb55/aweber_app/controllers/queries_controller.php#L386-402
|
||||
#+begin_src php :exports code :eval never
|
||||
// Look up the column for the order by clause. There are no SQL column name values passed publicly.
|
||||
if (!empty($this->data['SearchOrder']['SearchInput'])){
|
||||
$this->SearchInput->recursive = -1;
|
||||
if ($time = $this->SearchMutex->lock($aId, '6')) {
|
||||
$orderCol = $this->SearchInput->find(array('SearchInput.id' => $this->data['SearchOrder']['SearchInput']));
|
||||
$this->SearchMutex->unlock($aId, '6', $time);
|
||||
}
|
||||
if (!empty($orderCol['SearchInput']['column'])){
|
||||
//Case-insensitive text sorting.
|
||||
// Lower text fields so that ordering is case insensitive. SearchInputs 5, 23, and 24 are actually integers, despite
|
||||
// having a text search input. refs #3275
|
||||
if($orderCol['SearchInput']['input_type'] == 'text' && !in_array($orderCol['SearchInput']['id'], array(5,23,24))) {
|
||||
$orderCol['SearchInput']['column'] = 'lower('.$orderCol['SearchInput']['column'].')';
|
||||
}
|
||||
$this->data['SearchOrder']['column'] = $orderCol['SearchInput']['column'];
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
|
||||
- It is saved with the segment
|
||||
- It is passed back to the front-end when loading a saved segment
|
||||
- The =SearchCriteria= class incorporates the selected ordering and column when
|
||||
building its query for a targeted search database.
|
||||
- The broadcast segment service ignores the selected ordering, opting for its
|
||||
own for deliverability reasons.
|
||||
- All search inputs NOT in the analytics database are available for sorting (https://gitlab.aweber.io/CP/applications/sites/blob/f7ea2e9431e3ed2e694730f6446b4b3828d7c8fe/aweber_app/views/helpers/search_form.php#L54-62).
|
||||
- Performance degrades with list size, likely due to memory constraints and
|
||||
unindexed sort fields
|
||||
(https://www.cybertec-postgresql.com/en/postgresql-improving-sort-performance/).
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: f74e335d-577f-4749-bf32-1c025795b039
|
||||
:END:
|
||||
#+title: Broadcast Segment Search
|
||||
|
||||
Performs a search using a stored segment and builds an iterable list of
|
||||
recipients for a broadcast email. Implemented in the service's [[https://gitlab.aweber.io/edeliv/Applications/broadcast-segment/-/blob/master/broadcastsegment/handlers.py#L357][BroadcastHandler]].
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: fab0cf8f-7c54-4848-882b-dba5e087760d
|
||||
:END:
|
||||
#+title: Redesigned Reports
|
||||
|
||||
Currently dependent on the [[id:3ddc4e32-932f-4748-bfe9-7025d4d6b352][Report API Controller]] in [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][sites]] for data, which we
|
||||
hope to move into an [[id:c45881de-46f2-4f76-9579-063626c5956c][Analytics View Service]].
|
||||
|
|
@ -1,737 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: c45881de-46f2-4f76-9579-063626c5956c
|
||||
:END:
|
||||
#+title: Analytics View Service
|
||||
#+TODO: WAITING(w) READY(r) | DONE(d)
|
||||
|
||||
The Analytics View Service provides a collection of report endpoints. These
|
||||
endpoints handle querying the analytics databases or, in the longer term,
|
||||
exposing efficient materialized data views.
|
||||
|
||||
* Plan
|
||||
- Parent ticket :: [[https://jira.aweber.io/browse/CCPANEL-11781][CCPANEL-11781]]
|
||||
** Create the analytics view service
|
||||
- New project using cookie cutter
|
||||
- Deployed to kubernetes
|
||||
- Grafana dashboard created
|
||||
** Create the analytics view service playbook
|
||||
** Plan API structure
|
||||
- Pathing (=/reports/*=)?
|
||||
- Report versioning? (=/reports/$NAME.v$VERSION=)?
|
||||
** Create endpoints for existing reports
|
||||
- Based on the endpoints provided in the [[id:3ddc4e32-932f-4748-bfe9-7025d4d6b352][Report API Controller]]
|
||||
- Are all of these report endpoints in use?
|
||||
*** Opens over time
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: daily-opens
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Date range (default: last 30 days)
|
||||
- Report API controller endpoints ::
|
||||
- opens_all_range
|
||||
- opens_list_range
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-11-02T00:00:00Z": {
|
||||
"broadcasts": 2499,
|
||||
"followups": 2547,
|
||||
"unique": 2923,
|
||||
"total": 5046
|
||||
},
|
||||
"2021-11-03T00:00:00Z": {
|
||||
"broadcasts": 25808,
|
||||
"followups": 2430,
|
||||
"unique": 24876,
|
||||
"total": 28238
|
||||
},
|
||||
"2021-11-04T00:00:00Z": {
|
||||
"broadcasts": 16733,
|
||||
"followups": 1437,
|
||||
"unique": 14780,
|
||||
"total": 18170
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
|
||||
*** Clicks over time
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: daily-clicks
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Date range (default: last 30 days)
|
||||
- Report API controller endpoints ::
|
||||
- clicks_all_range
|
||||
- clicks_list_range
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-11-02T00:00:00Z": {
|
||||
"broadcasts": 105,
|
||||
"followups": 137,
|
||||
"unique": 130,
|
||||
"total": 242
|
||||
},
|
||||
"2021-11-03T00:00:00Z": {
|
||||
"broadcasts": 636,
|
||||
"followups": 185,
|
||||
"unique": 622,
|
||||
"total": 821
|
||||
},
|
||||
"2021-11-04T00:00:00Z": {
|
||||
"broadcasts": 480,
|
||||
"followups": 109,
|
||||
"unique": 426,
|
||||
"total": 589
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
|
||||
*** Sales over time (events)
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: sale-events
|
||||
- Parameters ::
|
||||
- Date range (default: last 30 days)
|
||||
- Currency (default: USD)
|
||||
- Report API controller endpoints ::
|
||||
- sales_tracked_events
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
[
|
||||
{
|
||||
"time": "2021-11-02 09:37:36-04",
|
||||
"type": "followup",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "team@harmoniamedia.com"
|
||||
},
|
||||
{
|
||||
"time": "2021-11-02 09:37:37-04",
|
||||
"type": "followup",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "team@harmoniamedia.com"
|
||||
},
|
||||
{
|
||||
"time": "2021-11-02 12:01:17-04",
|
||||
"type": "followup",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "giuliagiardino12@gmail.com"
|
||||
},
|
||||
{
|
||||
"time": "2021-11-02 12:01:19-04",
|
||||
"type": "followup",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "giuliagiardino12@gmail.com"
|
||||
},
|
||||
{
|
||||
"time": "2021-11-04 05:21:35-04",
|
||||
"type": "broadcast",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "jeremy@jeremy-quick.com"
|
||||
},
|
||||
{
|
||||
"time": "2021-11-04 05:21:36-04",
|
||||
"type": "broadcast",
|
||||
"currency": "USD",
|
||||
"revenue": "19.00",
|
||||
"note": "",
|
||||
"description": "Upgraded to Pro",
|
||||
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||
"email": "jeremy@jeremy-quick.com"
|
||||
}
|
||||
]
|
||||
#+end_src
|
||||
*** Sales over time (summary)
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: daily-sales
|
||||
- Parameters ::
|
||||
- Date range (default: last 60 days)
|
||||
- Currency (default: USD)
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-11-02T00:00:00Z": {
|
||||
"broadcast": 0,
|
||||
"followup": 76,
|
||||
"pageview": 76,
|
||||
"ecommerce": 0,
|
||||
"total": 76
|
||||
},
|
||||
"2021-11-03T00:00:00Z": {
|
||||
"broadcast": 0,
|
||||
"followup": 0,
|
||||
"pageview": 0,
|
||||
"ecommerce": 0,
|
||||
"total": 0
|
||||
},
|
||||
"2021-11-04T00:00:00Z": {
|
||||
"broadcast": 38,
|
||||
"followup": 0,
|
||||
"pageview": 38,
|
||||
"ecommerce": 0,
|
||||
"total": 38
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
*** Lifetime Sale Totals
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: sale-totals
|
||||
- Parameters ::
|
||||
- Currency (default: USD)
|
||||
- Report API controller endpoints ::
|
||||
- sales_tracked_total
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
{
|
||||
"count": 94924,
|
||||
"pagehits": 1820122.79,
|
||||
"revenue": 2181498.96
|
||||
}
|
||||
#+end_src
|
||||
*** Sale Currencies
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: sale-currencies
|
||||
- Parameters ::
|
||||
- Date range (default: last 60 days)
|
||||
- Report API controller endpoints ::
|
||||
- sales_tracked_currencies
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
["USD", "CAD"]
|
||||
#+end_src
|
||||
*** Pending Broadcasts
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: pending-broadcasts
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- broadcasts_pending
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
[
|
||||
{
|
||||
"unit_id": "3854",
|
||||
"send_date": "01/01/25 12:00am",
|
||||
"broadcast_id": "1800243",
|
||||
"subject": "Scheduled Broadcast Test",
|
||||
"status": "Queue",
|
||||
"percent_done": "0",
|
||||
"mesg_encoding": "utf-8",
|
||||
"campaign_id": "1802217",
|
||||
"unit": "awlist3854",
|
||||
"orig_send_date": "2025-01-01 00:00:00-05",
|
||||
"friendly_list_name": "Fluff Cafe",
|
||||
"id": "1802217",
|
||||
"list_id": "3854",
|
||||
"account_id": "778",
|
||||
"campaign_type_id": "b",
|
||||
"uses_block_editor": "t"
|
||||
}
|
||||
]
|
||||
#+end_src
|
||||
*** Completed Broadcasts
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: completed-broadcasts
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- broadcasts_completed
|
||||
- broadcasts_completed_all
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
[
|
||||
{
|
||||
"a_id": "778",
|
||||
"broadcast_id": "1258855",
|
||||
"for_sent_date": "08/14/20 03:41 PM",
|
||||
"track_click_rate": "1",
|
||||
"unit_id": "3854",
|
||||
"assassin_pts": "0",
|
||||
"mesg_type": "HTML",
|
||||
"mesg_encoding": "utf-8",
|
||||
"sent_date": "2020-08-14 15:41:05.24086-04",
|
||||
"num_emailed": "1",
|
||||
"num_undeliv": 0,
|
||||
"num_opened": 0,
|
||||
"num_attachments": 0,
|
||||
"num_complaints": 0,
|
||||
"subject": "Testing a bad segment",
|
||||
"created_date": "2020-08-14 15:38:57.59072-04",
|
||||
"status": "Sent Composer",
|
||||
"lead_view_id": "42564",
|
||||
"unit": "awlist3854",
|
||||
"pct_opened": "0",
|
||||
"show_opens_warning": true,
|
||||
"pct_undeliv": "0",
|
||||
"total_clicks": 0,
|
||||
"clicks_analytics_type": "premium",
|
||||
"pct_click": "0",
|
||||
"pct_complaints": "0",
|
||||
"segment_id": "42564",
|
||||
"extra_lists": "",
|
||||
"excluded_lists": "",
|
||||
"extra_lists_count": 0,
|
||||
"excluded_lists_count": 0,
|
||||
"friendly_list_name": "Fluff Cafe"
|
||||
},
|
||||
{
|
||||
"a_id": "778",
|
||||
"broadcast_id": "1243650",
|
||||
"for_sent_date": "08/05/20 06:33 PM",
|
||||
"track_click_rate": "1",
|
||||
"unit_id": "3854",
|
||||
"assassin_pts": "0",
|
||||
"mesg_type": "Text/HTML",
|
||||
"mesg_encoding": "utf-8",
|
||||
"sent_date": "2020-08-05 18:33:48.638875-04",
|
||||
"num_emailed": "27",
|
||||
"num_undeliv": 0,
|
||||
"num_opened": 0,
|
||||
"num_attachments": 0,
|
||||
"num_complaints": 0,
|
||||
"subject": "Buggssss 🐛🐛🐛🐛🐛🐛🐛",
|
||||
"created_date": "2020-08-05 18:31:10.567418-04",
|
||||
"status": "Sent Composer",
|
||||
"lead_view_id": "8",
|
||||
"unit": "awlist3854",
|
||||
"pct_opened": "0",
|
||||
"show_opens_warning": true,
|
||||
"pct_undeliv": "0",
|
||||
"total_clicks": 0,
|
||||
"clicks_analytics_type": "premium",
|
||||
"pct_click": "0",
|
||||
"pct_complaints": "0",
|
||||
"segment_id": "8",
|
||||
"extra_lists": "",
|
||||
"excluded_lists": "",
|
||||
"extra_lists_count": 0,
|
||||
"excluded_lists_count": 0,
|
||||
"friendly_list_name": "Fluff Cafe"
|
||||
}
|
||||
]
|
||||
#+end_src
|
||||
*** Cities, States, and Countries
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: subscribers-by-location
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- city_state_countries
|
||||
|
||||
#+caption: Sample response
|
||||
#+begin_src json
|
||||
[
|
||||
{
|
||||
"country": null,
|
||||
"state": null,
|
||||
"city": null,
|
||||
"pending": 0,
|
||||
"unsubscribed": 457262,
|
||||
"subscribed": 276245,
|
||||
"total": 733507
|
||||
},
|
||||
{
|
||||
"country": "US",
|
||||
"state": "PA",
|
||||
"city": "Newtown",
|
||||
"pending": 0,
|
||||
"unsubscribed": 279304,
|
||||
"subscribed": 3236,
|
||||
"total": 282540
|
||||
},
|
||||
{
|
||||
"country": "US",
|
||||
"state": null,
|
||||
"city": null,
|
||||
"pending": 0,
|
||||
"unsubscribed": 109074,
|
||||
"subscribed": 2456,
|
||||
"total": 111530
|
||||
},
|
||||
{
|
||||
"country": "US",
|
||||
"state": "PA",
|
||||
"city": "Philadelphia",
|
||||
"pending": 0,
|
||||
"unsubscribed": 48485,
|
||||
"subscribed": 657,
|
||||
"total": 49142
|
||||
},
|
||||
{
|
||||
"country": "US",
|
||||
"state": "PA",
|
||||
"city": "Huntingdon Valley",
|
||||
"pending": 0,
|
||||
"unsubscribed": 4839,
|
||||
"subscribed": 90,
|
||||
"total": 4929
|
||||
},
|
||||
{
|
||||
"country": "VN",
|
||||
"state": "64",
|
||||
"city": "Hanoi",
|
||||
"pending": 0,
|
||||
"unsubscribed": 204,
|
||||
"subscribed": 2,
|
||||
"total": 206
|
||||
},
|
||||
{
|
||||
"country": "VN",
|
||||
"state": "65",
|
||||
"city": "Ho Chi Minh City",
|
||||
"pending": 0,
|
||||
"unsubscribed": 176,
|
||||
"subscribed": 1,
|
||||
"total": 177
|
||||
}
|
||||
]
|
||||
#+end_src
|
||||
*** Campaign Statistics
|
||||
Add the campaign start endpoint into analytics-view including
|
||||
|
||||
- DynamoDB fixtures
|
||||
- Dynamo dbhelpers (currently there is no dynamo connectivity in analytics-view)
|
||||
- CampaignStarted Handler
|
||||
https://gitlab.aweber.io/CP/Services/campaignstats/-/blob/master/campaignstats/handlers.py
|
||||
- Grafana dashboard updated
|
||||
- Alerts configured
|
||||
- Confluence docs updated
|
||||
|
||||
*** Followups
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: followup-totals
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- followups
|
||||
|
||||
Uses the =freq_mesg= and =freq_mesg_stats= tables in AppDB combined with data
|
||||
from the =messages= table in Analytics.
|
||||
|
||||
=editor_path= is hard-coded in the current report endpoint.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
[
|
||||
{
|
||||
"followup_message": "1",
|
||||
"num_emailed": 83024,
|
||||
"num_opened": 89950,
|
||||
"open_percentage": 23.659423781075,
|
||||
"clicks": "13198",
|
||||
"message_id": "28613652",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "19643",
|
||||
"unique_clicks": "1584",
|
||||
"clicks_percentage": 23.659423781075,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "2",
|
||||
"num_emailed": 73714,
|
||||
"num_opened": 49979,
|
||||
"open_percentage": 13.675828200884,
|
||||
"clicks": "6169",
|
||||
"message_id": "28613655",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "10081",
|
||||
"unique_clicks": "649",
|
||||
"clicks_percentage": 13.675828200884,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "3",
|
||||
"num_emailed": 70880,
|
||||
"num_opened": 38472,
|
||||
"open_percentage": 10.272291196388,
|
||||
"clicks": "4322",
|
||||
"message_id": "28613659",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "7281",
|
||||
"unique_clicks": "409",
|
||||
"clicks_percentage": 10.272291196388,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "4",
|
||||
"num_emailed": 69375,
|
||||
"num_opened": 37496,
|
||||
"open_percentage": 10.105945945946,
|
||||
"clicks": "4419",
|
||||
"message_id": "28613661",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "7011",
|
||||
"unique_clicks": "447",
|
||||
"clicks_percentage": 10.105945945946,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "5",
|
||||
"num_emailed": 67478,
|
||||
"num_opened": 30996,
|
||||
"open_percentage": 8.2752897240582,
|
||||
"clicks": "3236",
|
||||
"message_id": "28613662",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "5584",
|
||||
"unique_clicks": "257",
|
||||
"clicks_percentage": 8.2752897240582,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "6",
|
||||
"num_emailed": 65414,
|
||||
"num_opened": 32483,
|
||||
"open_percentage": 9.2334974164552,
|
||||
"clicks": "3127",
|
||||
"message_id": "28613664",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "6040",
|
||||
"unique_clicks": "323",
|
||||
"clicks_percentage": 9.2334974164552,
|
||||
"editor_path": "messages#/active"
|
||||
},
|
||||
{
|
||||
"followup_message": "7",
|
||||
"num_emailed": 63871,
|
||||
"num_opened": 28756,
|
||||
"open_percentage": 8.1883797028385,
|
||||
"clicks": "1964",
|
||||
"message_id": "28613665",
|
||||
"click_percentage": 0,
|
||||
"unique_opens": "5230",
|
||||
"unique_clicks": "166",
|
||||
"clicks_percentage": 8.1883797028385,
|
||||
"editor_path": "messages#/active"
|
||||
}
|
||||
]
|
||||
#+end_src
|
||||
*** New Subscribers Daily
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: daily-new-subscribers
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- new_subscribers_daily
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2022-01-09T00:00:00Z": {
|
||||
"subscribed": 324,
|
||||
"unsubscribed": 83
|
||||
},
|
||||
"2022-01-10T00:00:00Z": {
|
||||
"subscribed": 417,
|
||||
"unsubscribed": 80
|
||||
},
|
||||
"2022-01-11T00:00:00Z": {
|
||||
"subscribed": 433,
|
||||
"unsubscribed": 92
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
*** New Subscribers Weekly
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: weekly-new-subscribers
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- new_subscribers_weekly
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-03-26T00:00:00Z": {
|
||||
"subscribed": 3182,
|
||||
"unsubscribed": 1249
|
||||
},
|
||||
"2021-04-02T00:00:00Z": {
|
||||
"subscribed": 3423,
|
||||
"unsubscribed": 1497
|
||||
},
|
||||
"2021-04-09T00:00:00Z": {
|
||||
"subscribed": 3052,
|
||||
"unsubscribed": 1217
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
*** New Subscribers Monthly
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: monthly-new-subscribers
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- new_subscribers_monthly
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-03-01T00:00:00Z": {
|
||||
"subscribed": 14972,
|
||||
"unsubscribed": 4770
|
||||
},
|
||||
"2021-04-01T00:00:00Z": {
|
||||
"subscribed": 14973,
|
||||
"unsubscribed": 7181
|
||||
},
|
||||
"2021-05-01T00:00:00Z": {
|
||||
"subscribed": 12652,
|
||||
"unsubscribed": 5243
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
*** Subscriber Totals Daily
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: subscribers-by-location
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- subscriber_totals_daily
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2022-01-09T00:00:00Z": {
|
||||
"subscribed": 289192,
|
||||
"unsubscribed": 920812
|
||||
},
|
||||
"2022-01-10T00:00:00Z": {
|
||||
"subscribed": 289609,
|
||||
"unsubscribed": 920892
|
||||
},
|
||||
"2022-01-11T00:00:00Z": {
|
||||
"subscribed": 290042,
|
||||
"unsubscribed": 920984
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
*** Subscriber Totals Weekly
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: subscribers-by-location
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- subscriber_totals_weekly
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-02-08T00:00:00Z": {
|
||||
"subscribed": 191745,
|
||||
"unsubscribed": 887713
|
||||
},
|
||||
"2021-02-15T00:00:00Z": {
|
||||
"subscribed": 193972,
|
||||
"unsubscribed": 888733
|
||||
},
|
||||
"2021-02-22T00:00:00Z": {
|
||||
"subscribed": 196200,
|
||||
"unsubscribed": 889717
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
*** Subscriber Totals Monthly
|
||||
#+attr_confluence: :as-table t
|
||||
- Name :: subscribers-by-location
|
||||
- Parameters ::
|
||||
- List (default: all lists)
|
||||
- Report API controller endpoints ::
|
||||
- subscriber_totals_monthly
|
||||
|
||||
Uses the =public.leads_stats_day= table in AppDB.
|
||||
|
||||
#+caption: Sample Response (Production AID 91)
|
||||
#+begin_src json
|
||||
{
|
||||
"2021-02-01T00:00:00Z": {
|
||||
"subscribed": 196200,
|
||||
"unsubscribed": 889717
|
||||
},
|
||||
"2021-03-01T00:00:00Z": {
|
||||
"subscribed": 206569,
|
||||
"unsubscribed": 894298
|
||||
},
|
||||
"2021-04-01T00:00:00Z": {
|
||||
"subscribed": 215813,
|
||||
"unsubscribed": 900020
|
||||
},
|
||||
...
|
||||
}
|
||||
#+end_src
|
||||
|
||||
** Migrate reports to the Analytics View Service
|
||||
*** READY Use the new analytics view endpoints for the opens over time reports
|
||||
Update the opens over time report for [[https://www.aweber.com/users/report/opens_all][all lists]] and the [[https://www.aweber.com/users/report/opens][current list]] to use the
|
||||
new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1opens/get][daily opens endpoint]] in the Analytics View service.
|
||||
*** READY Use the new analytics view endpoints for the clicks over time reports
|
||||
Update the clicks over time report for [[https://www.aweber.com/users/report/clicks_all][all lists]] and the [[https://www.aweber.com/users/report/clicks][current list]] to use the
|
||||
new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1clicks/get][daily clicks endpoint]] in the Analytics View service.
|
||||
*** WAITING Use the new analytics view endpoints for the sales over time report
|
||||
Update the [[https://www.aweber.com/users/report/sales_tracked_all][sales over time report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1sales-by-day/get][daily sales endpoint]] for graph
|
||||
summary data and the sale events endpoint for the sales table.
|
||||
|
||||
#+begin_notes
|
||||
The sale events endpoint present, but is not yet documented!
|
||||
#+end_notes
|
||||
*** DONE New subscribers
|
||||
Update the new subscribers report to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-daily/get][daily]], [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-weekly/get][weekly]], and [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-monthly/get][monthly]] new
|
||||
subscriber endpoints in the Analytics View service.
|
||||
*** DONE Subscriber totals
|
||||
Update the subscriber totals report to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-daily/get][daily]], [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-weekly/get][weekly]], and [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-monthly/get][monthly]]
|
||||
total subscriber endpoints in the Analytics View service.
|
||||
*** WAITING Use the new analytics view endpoints for the broadcast totals report
|
||||
Update the [[https://www.aweber.com/users/report/broadcast_totals][broadcast totals report]] to use the new completed broadcasts endpoint
|
||||
in the Analytics View service.
|
||||
|
||||
#+begin_notes
|
||||
The completed broadcasts endpoint is [[https://jira.aweber.io/browse/CCPANEL-11788][not yet complete]].
|
||||
#+end_notes
|
||||
*** Use the new analytics view endpoints for the follow-up totals report
|
||||
Update the [[https://www.aweber.com/users/report/followup_totals][followup totals report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1followup-totals/get][followup totals]] endpoint in the
|
||||
Analytics View service.
|
||||
*** Use the new analytics view endpoints for the location totals report
|
||||
Update the [[https://www.aweber.com/users/report/subscribers_by_location][location totals report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1subscribers-by-location/get][subscribers by location]]
|
||||
endpoint in the Analytics View service.
|
||||
|
|
@ -1,39 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 3ddc4e32-932f-4748-bfe9-7025d4d6b352
|
||||
:ROAM_REFS: https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/report_api_controller.php
|
||||
:END:
|
||||
#+title: Report API Controller
|
||||
|
||||
Contains JSON report endpoints used by the [[id:fab0cf8f-7c54-4848-882b-dba5e087760d][Redesigned Reports]] and the [[id:0d24c57a-fd56-43b0-8209-497320bf79f7][Dashboard]].
|
||||
These make use of the App and Analytics databases.
|
||||
|
||||
| Endpoint | Description | Report | Dashboard |
|
||||
|--------------------------------------+-----------------+-----------------------+----------------------|
|
||||
| opens_all | | | |
|
||||
| opens_all_range | All lists | Opens over time | |
|
||||
| opens | | | |
|
||||
| opens_list_range | Current list | Opens over time | |
|
||||
| clicks_all | | | |
|
||||
| clicks_all_range | All lists | Clicks over time | |
|
||||
| clicks | | | |
|
||||
| clicks_list_range | Current list | Clicks over time | |
|
||||
| sales_tracked_all | | | |
|
||||
| sales_tracked_all_range | | | |
|
||||
| sales_tracked_summary | | Sales over time | |
|
||||
| sales_tracked_currencies | | Sales over time | Sales |
|
||||
| sales_tracked_events | | Sales over time | |
|
||||
| sales_tracked_total | Lifetime totals | | Sales |
|
||||
| broadcasts_pending | | | Scheduled Broadcasts |
|
||||
| broadcasts_completed | Current list | Broadcast totals | |
|
||||
| broadcasts_completed_all | All lists | | Sent Broadcasts |
|
||||
| broadcasts_completed_range | | | |
|
||||
| city_state_countries | | Location totals | |
|
||||
| followups | | Followup totals | |
|
||||
| new_subscribers_daily | | New subscribers added | |
|
||||
| new_subscribers_weekly | | New subscribers added | |
|
||||
| new_subscribers_monthly | | New subscribers added | |
|
||||
| subscriber_totals_daily | | Subscriber totals | |
|
||||
| subscriber_totals_weekly | | Subscriber totals | |
|
||||
| subscriber_totals_monthly | | Subscriber totals | |
|
||||
| account_subscriber_totals | | | People |
|
||||
| account_subscriber_totals_past_month | | | People |
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 6ee0e3f3-1df2-4f8a-bc1b-659d8c01e2b5
|
||||
:ROAM_ALIASES: CEEECS
|
||||
:END:
|
||||
#+title: Customer Empathy and Excellence via Customer Solutions
|
||||
|
||||
A program in which AWeber team members shadow a CS specialist as they field
|
||||
customer support work, with the goal of finding areas in our platform where we
|
||||
can improve the customer experience.
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: c7322400-c6e6-4595-87e2-7db6e57b6a2b
|
||||
:ROAM_ALIASES: S4
|
||||
:END:
|
||||
#+title: Suspicious Submission Spam Service
|
||||
|
|
@ -1,88 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 321075e7-db53-4676-b785-7c77ed9d1150
|
||||
:END:
|
||||
#+title: Bulk Tagging Service
|
||||
|
||||
The Bulk Tagging service queues the application of tags to a set of subscribers
|
||||
and tracks the overall progress of the operation. This allows customers to
|
||||
select a large number of subscribers and apply tags to all of them in a single
|
||||
operation, while also giving them visibility into how their tagging request is
|
||||
progressing.
|
||||
|
||||
* Problems Solved
|
||||
|
||||
** Background processing of tagging operations
|
||||
Tagging operations are assigned a bulk-tagging job with one task per subscriber
|
||||
to add or remove the specified tags. These operations are performed
|
||||
asynchronously by consumers such that the end-user need not wait for all
|
||||
operations to be completed before moving on.
|
||||
** Rate-limited application of tags
|
||||
Consumers are configured to consume tasks no faster than a configured maximum
|
||||
rate to control the load placed upon downstream services (e.g. rules engine,
|
||||
etc.). The ideal rate is divided amongst the number of consumers available.
|
||||
** Serial application of operations within an account
|
||||
There is an expectation that operations a customer applies to subscribers will
|
||||
be performed sequentially. To allow this while still allowing jobs for /other/
|
||||
accounts to be worked upon simultaneously, jobs are divided into queues
|
||||
deterministically using a consistent hash of their account IDs.
|
||||
** Tracking overall job progress
|
||||
The job itself may be in one of the following states, which is updated as tasks
|
||||
are acted upon:
|
||||
- Pending :: No tasks have yet been acted upon
|
||||
- Processing :: Some, but not all, tasks have been acted upon
|
||||
- Succeeded :: All tasks have completed successfully
|
||||
- Failed :: All tasks have completed, but at least one task was not successful
|
||||
** Tracking of individual task status
|
||||
Any particular task, once acted upon, will be updated as having succeeded or
|
||||
failed with a message explaining the issue.
|
||||
** Jobs must survive queue failures
|
||||
A guarantee of the service is such that if a job is successfully submitted to
|
||||
the Bulk Tagging service, we will not lose it, and can take steps manually if
|
||||
necessary to ensure its completion. To account for unexpected failures when
|
||||
submitting tasks to a queue or consuming a task from the queue, the job request
|
||||
is archived to S3. This archive contains sufficient information to requeue the
|
||||
job for processing as it was requested without further input.
|
||||
** Customer visibility into job progress
|
||||
End users are able to, via the service API, fetch any and all jobs stored for
|
||||
their account, as well as their associated tasks.
|
||||
** Administrative visibility into job progress
|
||||
Administrative (internal) users are able to, via the service API, fetch any and
|
||||
all jobs stored for any or all accounts. This is used to populate dashboards for
|
||||
insight into the progress of Bulk Tagging as a whole, and whether jobs from
|
||||
different accounts are holding each other up.
|
||||
|
||||
Administrative users also have access to delete or requeue jobs.
|
||||
** Prioritizing smaller jobs over larger jobs
|
||||
Tasks for jobs affecting a number of subscribers lower than a defined threshold
|
||||
are assigned higher priority, causing them to be processed /ahead/ of any other
|
||||
ongoing jobs in their respective queues. This avoids leaving a customer waiting
|
||||
long periods of time for quick operations, which they may not expect to be held
|
||||
up by other jobs outside their control.
|
||||
* Known Problems
|
||||
** Jobs appearing "stuck"
|
||||
Jobs may appear stuck (either failing to start processing, or failing to
|
||||
complete processing).
|
||||
*** Job has not yet begun processing
|
||||
This is caused by one of two things. Either a job is stuck behind other jobs
|
||||
(this or another account's job could be in front of it in the same queue), or
|
||||
the tasks failed to be written successfully to the queue. In the former case,
|
||||
the job will complete normally once the tasks ahead of it in the queue are
|
||||
processed. In the latter case, the job will need to be requeued.
|
||||
*** Job is stuck in an in-progress state
|
||||
This is typically due to the overall job progress being out of sync with the
|
||||
actual state of its consituent tasks ([[https://jira.aweber.io/browse/CCPANEL-8660][CCPANEL-8660]]).
|
||||
|
||||
The overall status of a job and the status of its individual tasks are stored in
|
||||
separate Dynamo tables. Because Dynamo tables are independent, there is no way
|
||||
to update an individual task and the job's overall status in a single, atomic
|
||||
operation. This means they may (and do) at times get out of sync, if
|
||||
infrequently. The indexes on the tables are not designed to make reconciliation
|
||||
easy, nor does an automated reconciliation process exist.
|
||||
|
||||
This could be solved by adding indexes to allow rapid computation of the actual
|
||||
state of all tasks in a job and creating a scheduled task to synchronize these
|
||||
counts, or by changing the underlying storage to a relational database which
|
||||
would make it easier to compute these states.
|
||||
* Resources
|
||||
- ACP :: https://confluence.aweber.io/display/AR/ACP+Bulk+Tagging
|
||||
- Playbook :: https://confluence.aweber.io/display/AR/Bulk+Tagging+Service+Playbook
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 0d24c57a-fd56-43b0-8209-497320bf79f7
|
||||
:END:
|
||||
#+title: Dashboard
|
||||
|
|
@ -1,253 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 7b0f97f3-9037-4d05-9170-a478e97c8d1f
|
||||
:END:
|
||||
#+title: Modeling the new search DSL
|
||||
|
||||
Defining and translating the Search DSL for the [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]].
|
||||
|
||||
* Searches
|
||||
** A search is a collection of groupings
|
||||
#+begin_src python :noweb-ref search
|
||||
@dataclasses.dataclass
|
||||
class Search:
|
||||
group: Group
|
||||
# TODO: sorting : Sorting
|
||||
#+end_src
|
||||
|
||||
#+begin_src yaml :noweb-ref search-yaml
|
||||
Search:
|
||||
type: object
|
||||
properties:
|
||||
group:
|
||||
$ref: "#/components/schemas/Group"
|
||||
#+end_src
|
||||
** A grouping is a collection of conditions
|
||||
#+begin_src python :noweb-ref group
|
||||
class GroupType(enum.Enum):
|
||||
AND = 1
|
||||
# TODO: OR = 2
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Group:
|
||||
group_type: GroupType
|
||||
conditions: typing.List[Condition]
|
||||
#+end_src
|
||||
|
||||
#+begin_src yaml :noweb-ref group-yaml
|
||||
Group:
|
||||
type: object
|
||||
properties:
|
||||
group_type:
|
||||
enum:
|
||||
- "AND"
|
||||
conditions:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/Condition"
|
||||
|
||||
#+end_src
|
||||
** A condition is a filter applied to a field
|
||||
#+begin_src python :noweb-ref condition
|
||||
@dataclasses.dataclass
|
||||
class Condition:
|
||||
filter: Filter
|
||||
match : str
|
||||
#+end_src
|
||||
|
||||
#+begin_src yaml :noweb-ref condition-yaml
|
||||
Condition:
|
||||
type: object
|
||||
properties:
|
||||
filter:
|
||||
$ref: "#/components/schemas/Filter"
|
||||
match:
|
||||
type: string
|
||||
#+end_src
|
||||
** A filter is a boolean expression applied to a field with an optional argument
|
||||
|
||||
#+begin_src python :noweb-ref filter
|
||||
class InputType(enum.Enum):
|
||||
Nothing = 1
|
||||
String = 2
|
||||
Date = 3
|
||||
Tag = 4
|
||||
TagSet = 5
|
||||
Message = 6
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Filter:
|
||||
operator: str
|
||||
field: Field
|
||||
input_type: InputType
|
||||
#+end_src
|
||||
|
||||
** A field refers to a specific database field somewhere in our system
|
||||
#+begin_src python :noweb-ref field
|
||||
class Database(enum.Enum):
|
||||
AppDB = 1
|
||||
Analytics = 2
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class FieldType:
|
||||
name: str
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Field:
|
||||
name: str
|
||||
column: str
|
||||
table: str
|
||||
database: Database
|
||||
#+end_src
|
||||
|
||||
** Available filters
|
||||
*** Subscriber email is x
|
||||
#+begin_src python :noweb-ref fields
|
||||
email = Field(
|
||||
name="email",
|
||||
column="email",
|
||||
table="subscribers",
|
||||
database=Database.AppDB,
|
||||
)
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :noweb-ref filters
|
||||
email = Filter(field=fields.email, operator="is", input_type=InputType.String)
|
||||
#+end_src
|
||||
|
||||
#+begin_src yaml :noweb-ref filters-spec
|
||||
#+end_src
|
||||
** Sample searches
|
||||
|
||||
*** Match subscriber email
|
||||
#+begin_src python :noweb-ref searches
|
||||
Search(
|
||||
group=Group(
|
||||
group_type=GroupType.AND,
|
||||
conditions=[Condition(filter=filters.email, match="test@example.org")],
|
||||
)
|
||||
)
|
||||
#+end_src
|
||||
|
||||
* SQL Generation
|
||||
|
||||
#+begin_src python :noweb-ref builder
|
||||
def to_sql(search: Search) -> str:
|
||||
tables: typing.Set[str] = {"subscribers"}
|
||||
tables = tables | {
|
||||
condition.filter.field.table for condition in search.group.conditions
|
||||
}
|
||||
|
||||
def condition_to_sql(condition: Condition):
|
||||
field = ".".join([condition.filter.field.table, condition.filter.field.column])
|
||||
return f"{field} {condition.filter.operator} {condition.match}"
|
||||
|
||||
def group_to_sql(group: Group) -> str:
|
||||
operator = "AND" if search.group.group_type == GroupType.AND else "OR"
|
||||
clauses = f" {operator} ".join(
|
||||
[condition_to_sql(condition) for condition in group.conditions]
|
||||
)
|
||||
return f"({clauses})"
|
||||
|
||||
where = group_to_sql(search.group)
|
||||
return f"""SELECT * FROM {', '.join(tables)} WHERE {where}"""
|
||||
#+end_src
|
||||
* Decisions
|
||||
|
||||
** DONE Should the input type presented to the end-user be tied to the database field or the conditional operator?
|
||||
Seems it should be the operator, as an "equals" operator would match a single
|
||||
value, whereas an "in" operator would match against multiple. That said, it
|
||||
could be /parameterized/ by the field's type (e.g. a tag has type =str=, its
|
||||
"equals" operator has type =str=, its "in" operator has type =List[str]=).
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
The input type will be defined as a property of the filter being applied.
|
||||
|
||||
** DONE Should the search service maintain a set of filters, or field types and operators?
|
||||
- A filter is a combination of a field, an operator, and a type
|
||||
- A field has a type, and operators could be defined that work with a type or set of types
|
||||
|
||||
For the former, the service would have total control over the search filters
|
||||
available to the UI, and the UI would be coupled to the filter collection. With
|
||||
the latter, the UI would have total control over which fields it's able to
|
||||
search on and how, provided the fields are available.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
The search service will maintain a set of filters.
|
||||
** TODO How should the values of each filter be represented in the request schema?
|
||||
Should they be normalized to strings, or should we allow any type and validate
|
||||
it when we attempt to build the search data model? If the latter, could the
|
||||
available filters be baked into the OpenAPI schema?
|
||||
** TODO How should the SQL be generated for each filter?
|
||||
Should a SQL template or generation function be attached to each filter?
|
||||
** TODO How do we want to define the joins for the various tables that may come into play?
|
||||
We'll have to know, one way or another, how to narrow the records from the
|
||||
joined table. Will they all be joined by the subscriber id, or will we need to
|
||||
maintain a map?
|
||||
|
||||
* Code
|
||||
** Python
|
||||
#+begin_src python :noweb yes :noweb-ref final :exports code :results silent
|
||||
import dataclasses
|
||||
import enum
|
||||
import typing
|
||||
|
||||
|
||||
<<field>>
|
||||
|
||||
|
||||
<<filter>>
|
||||
|
||||
|
||||
<<condition>>
|
||||
|
||||
|
||||
<<group>>
|
||||
|
||||
|
||||
<<search>>
|
||||
|
||||
|
||||
<<builder>>
|
||||
|
||||
|
||||
class fields:
|
||||
<<fields>>
|
||||
|
||||
|
||||
class filters:
|
||||
<<filters>>
|
||||
|
||||
|
||||
searches = [
|
||||
<<searches>>,
|
||||
]
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
|
||||
#+caption: Mypy analysis
|
||||
#+begin_src bash :noweb yes :results output :exports results
|
||||
mypy <(cat <<'EOF'
|
||||
<<final>>
|
||||
EOF) 2>&1 || true
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
: Success: no issues found in 1 source file
|
||||
** OpenAPI
|
||||
* Output
|
||||
#+caption: Generated queries
|
||||
#+begin_src python :noweb yes :exports results
|
||||
<<final>>
|
||||
|
||||
return [[to_sql(search)] for search in searches]
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
| SELECT * FROM subscribers WHERE (subscribers.email is test@example.org) |
|
||||
|
|
@ -1,166 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 3cc8bd09-dd02-4950-8c89-a737f92809fd
|
||||
:header-args:bash: :dir ~/sites-clean :exports both :eval no-export
|
||||
:header-args:python: :exports results :eval no-export
|
||||
:END:
|
||||
#+title: Tracking progress of moving pages out of Sites
|
||||
|
||||
- [[https://jira.aweber.io/browse/CCPANEL-11608][Initiative parent ticket in JIRA]]
|
||||
|
||||
* Metrics
|
||||
|
||||
#+caption: Migrated controllers in the CP
|
||||
#+begin_src python :var total=controller-count done=js-controller-count :results file
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
total = float(total)
|
||||
fig1, ax1 = plt.subplots()
|
||||
ax1.pie(
|
||||
[100 * (total - done) / total, 100 * done / total],
|
||||
explode=[0.0, 0.1],
|
||||
labels=["Legacy", "JavaScript"],
|
||||
autopct="%1.1f%%",
|
||||
shadow=True,
|
||||
startangle=90,
|
||||
)
|
||||
ax1.axis("equal")
|
||||
plt.title("Controller Types")
|
||||
filename = "controllers-migrated-in-sites.png"
|
||||
plt.savefig(filename)
|
||||
return filename
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:None]]
|
||||
|
||||
** Controllers in Sites
|
||||
#+caption: Identifying the total number of public controllers in the CP
|
||||
#+name: controller-count
|
||||
#+begin_src bash
|
||||
grep -l AppController aweber_app/controllers/*_controller.php | wc -l
|
||||
#+end_src
|
||||
|
||||
#+RESULTS: controller-count
|
||||
: 85
|
||||
|
||||
** Controllers loading JavaScript applications
|
||||
#+caption: Identifying the number of controllers loading JS applications
|
||||
#+name: js-controller-count
|
||||
#+begin_src bash
|
||||
egrep -l '\bappName\b' aweber_app/controllers/*_controller.php | wc -l
|
||||
#+end_src
|
||||
|
||||
#+RESULTS: js-controller-count
|
||||
: 25
|
||||
|
||||
* Progress over time
|
||||
|
||||
#+caption: Percentage of controllers migrated over time
|
||||
#+begin_src python :var progress=progress :results file
|
||||
from datetime import date
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
progress = [[date.fromisoformat(row[0]), 100.0 * row[2] / row[1]] for row in progress]
|
||||
x = [p[0] for p in progress]
|
||||
y = [p[1] for p in progress]
|
||||
plt.plot(x, y)
|
||||
plt.fill_between(x, y, alpha=0.3)
|
||||
plt.gcf().autofmt_xdate()
|
||||
|
||||
plt.title("% Controllers Migrated Over Time")
|
||||
|
||||
filename = "controllers-migrated-in-sites-over-time.png"
|
||||
plt.savefig(filename)
|
||||
return filename
|
||||
|
||||
#+end_src
|
||||
|
||||
#+RESULTS:
|
||||
[[file:None]]
|
||||
|
||||
#+caption: Identifying the last tagged release each month
|
||||
#+name: tags
|
||||
#+begin_src bash :results silent :exports code
|
||||
git log --tags \
|
||||
--simplify-by-decoration \
|
||||
--pretty="format:%as#%S" \
|
||||
--after="2018-01-01" \
|
||||
| sort -r -u -t- -k1,2 # Last tag of each month
|
||||
#+end_src
|
||||
|
||||
#+caption: Gathering progress over time
|
||||
#+name: progress
|
||||
#+begin_src bash :noweb yes :cache yes :exports code
|
||||
controller_total () {
|
||||
<<controller-count>>
|
||||
}
|
||||
|
||||
controller_done () {
|
||||
<<js-controller-count>>
|
||||
}
|
||||
|
||||
tags () {
|
||||
<<tags>>
|
||||
}
|
||||
|
||||
git checkout -q master
|
||||
for taginfo in $(tags); do
|
||||
date=$(echo $taginfo | cut -d '#' -f 1)
|
||||
tag=$(echo $taginfo | cut -d '#' -f 2)
|
||||
|
||||
git checkout $tag
|
||||
echo $date $(controller_total) $(controller_done)
|
||||
done
|
||||
git checkout -q master
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[b2f17a7946c030068f7ef85189b10bd0c5cb6a0a]: progress
|
||||
| 2021-09-30 | 85 | 24 |
|
||||
| 2021-08-31 | 85 | 23 |
|
||||
| 2021-07-28 | 85 | 23 |
|
||||
| 2021-06-28 | 85 | 23 |
|
||||
| 2021-05-27 | 85 | 23 |
|
||||
| 2021-04-28 | 85 | 23 |
|
||||
| 2021-03-04 | 84 | 22 |
|
||||
| 2021-02-25 | 84 | 22 |
|
||||
| 2021-01-28 | 83 | 16 |
|
||||
| 2020-12-29 | 83 | 16 |
|
||||
| 2020-11-20 | 83 | 16 |
|
||||
| 2020-10-29 | 83 | 16 |
|
||||
| 2020-09-30 | 83 | 16 |
|
||||
| 2020-08-27 | 83 | 16 |
|
||||
| 2020-07-31 | 83 | 16 |
|
||||
| 2020-06-30 | 82 | 15 |
|
||||
| 2020-05-29 | 81 | 15 |
|
||||
| 2020-04-30 | 81 | 15 |
|
||||
| 2020-03-31 | 81 | 15 |
|
||||
| 2020-02-28 | 81 | 15 |
|
||||
| 2020-01-30 | 82 | 15 |
|
||||
| 2019-12-18 | 82 | 15 |
|
||||
| 2019-11-25 | 82 | 15 |
|
||||
| 2019-10-31 | 81 | 14 |
|
||||
| 2019-09-30 | 81 | 14 |
|
||||
| 2019-08-27 | 81 | 14 |
|
||||
| 2019-07-31 | 81 | 14 |
|
||||
| 2019-06-27 | 81 | 14 |
|
||||
| 2019-05-31 | 81 | 14 |
|
||||
| 2019-04-26 | 80 | 13 |
|
||||
| 2019-03-29 | 79 | 13 |
|
||||
| 2019-02-28 | 78 | 12 |
|
||||
| 2019-01-30 | 78 | 12 |
|
||||
| 2018-12-27 | 77 | 10 |
|
||||
| 2018-11-29 | 76 | 10 |
|
||||
| 2018-10-31 | 75 | 9 |
|
||||
| 2018-09-28 | 75 | 9 |
|
||||
| 2018-08-31 | 74 | 8 |
|
||||
| 2018-07-26 | 74 | 8 |
|
||||
| 2018-06-29 | 73 | 6 |
|
||||
| 2018-05-31 | 73 | 6 |
|
||||
| 2018-04-30 | 73 | 6 |
|
||||
| 2018-03-29 | 73 | 6 |
|
||||
| 2018-02-28 | 73 | 6 |
|
||||
| 2018-01-24 | 73 | 6 |
|
||||
|
||||
* Comparing with the CP URL Inventory
|
||||
https://docs.google.com/spreadsheets/d/1bRKL1zRe_SjePD1QKSHrbgiW9paMjLyCIX_2wu2FXwE/edit#gid=1209260269
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 071f551f-56d9-425c-bfde-af80cd7c26f7
|
||||
:END:
|
||||
#+title: Tech Initiative Workshop
|
||||
|
||||
A bi-weekly meeting to swarm on various [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] issues.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 7e503917-646f-4275-aab9-3a125b99cbfd
|
||||
:END:
|
||||
#+title: Tagging Service
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 131dde93-60d3-4813-a16e-7568c79ba6c4
|
||||
:END:
|
||||
#+title: Tag Publisher Consumer
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: bdea0611-e377-4378-a118-aef6d4a70bdf
|
||||
:ROAM_ALIASES: CREASE
|
||||
:END:
|
||||
#+title: Creating Remarkable Experiences via Application Support Engineering
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 87f76e97-fbdc-49e4-9af8-588ceee85a5c
|
||||
:END:
|
||||
#+title: Staging account information
|
||||
|
||||
[[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] staging account.
|
||||
|
||||
* Lists
|
||||
- Fluff Cafe :: 9f1db623-fbc3-4112-a2f3-ab563e37e131 (3854)
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 16298b74-f9a2-48ac-a84c-118af70d834c
|
||||
:END:
|
||||
#+title: Validating and sanitizing tags
|
||||
|
||||
* Sanitizing tag display
|
||||
|
||||
** DONE In the autocomplete of the tag input box
|
||||
Fixes [[https://jira.aweber.io/browse/CCPANEL-11654][CCPANEL-11654]].
|
||||
|
||||
https://gitlab.aweber.io/BoFs/FE/libraries/tagbox/-/merge_requests/29
|
||||
|
||||
* Validating tags on creation
|
||||
|
||||
** TODO Lead controller in [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]]
|
||||
** TODO [[id:03e00c18-99c0-477c-b7fb-95ddc538755e][Addlead]]
|
||||
** TODO [[id:cd4a8a83-be53-4ec9-8cca-b6f34b59ba35][Subscriber Proxy]]
|
||||
** TODO [[id:321075e7-db53-4676-b785-7c77ed9d1150][Bulk Tagging]]
|
||||
** TODO [[id:7e503917-646f-4275-aab9-3a125b99cbfd][Tagging]]
|
||||
*** TODO Add inbound validation
|
||||
*** TODO Remove outbound sanitization
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 03e00c18-99c0-477c-b7fb-95ddc538755e
|
||||
:END:
|
||||
#+title: Addlead
|
||||
|
||||
A business-critical nightmare in Perl.
|
||||
|
||||
https://confluence.aweber.io/display/~erict/Addlead+Notes
|
||||
* Create ACP to rewrite Addlead as a Python service
|
||||
* Break down tickets
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: cd4a8a83-be53-4ec9-8cca-b6f34b59ba35
|
||||
:END:
|
||||
#+title: Subscriber Proxy Service
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 0e5f578f-96a2-47d8-8dd9-d0d7f1e4fc35
|
||||
:END:
|
||||
#+title: CP Leads and Product Sync-Up
|
||||
|
||||
A weekly discussion on team priorities.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 0a1e48ec-e132-4ec4-81a1-124711330b5a
|
||||
:END:
|
||||
#+title: Manager one-on-one
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 619b6c78-7be9-4ee4-a0b7-9d1a4d7536e2
|
||||
:END:
|
||||
#+title: Migrating services to use the new List service
|
||||
|
||||
- Parent ticket :: [[https://jira.aweber.io/browse/CCPANEL-11745][CCPANEL-11745]]
|
||||
|
||||
As part of our effort to deprecate the old Core API services and iterate towards
|
||||
modern, domain-oriented APIs, the Control Panel team is deprecating usage of
|
||||
AWLists in favor of a new List API.
|
||||
|
||||
AWLists is planned to be sunsetted at the end of Q2 2022. Applications and
|
||||
services dependent upon AWLists must be migrated to use the new List API by that
|
||||
time.
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 4df15f2f-d2e1-40f4-8acd-dbfb78fe304f
|
||||
:END:
|
||||
#+title: Deploy CoreAPI to Kubernetes
|
||||
|
||||
- Merge the sub-projects into CAPI?
|
||||
- API Suspenders replacement?
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 77ea54db-0c35-47ad-84b3-5c08ae5ac347
|
||||
:END:
|
||||
#+title: Redash
|
||||
|
||||
Tool for interrogating and graphing data points from the event stream.
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 38457ac3-ba81-4727-9a65-5de22059c175
|
||||
:END:
|
||||
#+title: Validation and Sanitization Guidelines
|
||||
|
||||
- [[id:2ba04972-f498-41c2-970e-a64c7f3f1c3b][Data sanitization]]
|
||||
- [[id:9914d09e-99fe-46a6-95be-676c5b78ed90][Input validation]]
|
||||
|
||||
- All content being displayed to a web browser MUST be appropriately sanitized
|
||||
(unsafe characters should be escaped using their respective html entities)
|
||||
-
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: f633f967-11d2-432c-b5ff-ad842c88a51c
|
||||
:END:
|
||||
#+title: Decommissioning Sites
|
||||
|
||||
The goal of this project is the elimination of the [[https://gitlab.aweber.io/CP/applications/sites][sites repository]], which is
|
||||
built upon Perl and PHP code that is well past it's end-of-life date, and a
|
||||
modernization of our public-facing application stack.
|
||||
|
||||
The project will engage multiple teams to coordinate the following three
|
||||
efforts:
|
||||
|
||||
* [[id:193f7c04-0a03-4870-90c8-2b5e3c4c92ce][Moving applications out of Sites]]
|
||||
- Individual pages will be replaced with React applications using public APIs
|
||||
- Static content will be moved to CDN hosting
|
||||
- Independent applications will be broken out into separate services
|
||||
|
||||
* Replacing the top-level application
|
||||
The CakePHP Control Panel application will be replaced with a modern alternative
|
||||
handling routing and React application loading.
|
||||
|
||||
* [[id:0328a202-376d-4e97-b0e3-031eaad2a557][Overhauling logins and session management]]
|
||||
The session-based login mechanism of the legacy Control Panel application is to
|
||||
be replaced with OAuth, which is used for public API requests.
|
||||
|
|
@ -1,82 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d399955b-894c-4d44-82ed-892009b4aa4f
|
||||
:END:
|
||||
#+title: Updating projects using Tagbox
|
||||
#+OPTIONS: prop:("JIRA_ID")
|
||||
#+todo: TODO(t) INVESTIGATE(i) TESTING(s) AWAITING-RELEASE(a) | DONE(d) NO-ACTION(n)
|
||||
|
||||
The Tagbox component has been updated with a fix addressing an XSS security
|
||||
vulnerability in its tag label auto-completion which needs to be propogated out
|
||||
to projects using the widget ([[https://jira.aweber.io/browse/CCPANEL-11654][CCPANEL-11654]]).
|
||||
|
||||
- New version of Tagbox is 5.0.4 (AWeberUI 8.7)
|
||||
- Aweber UI is updated, includes other breaking changes
|
||||
- Adobe spectrum resource picker breaks stuff (present in some versions)
|
||||
- Sites is already updated
|
||||
* Notes on upgrading AWeberUI
|
||||
Target versions:
|
||||
- =@aw-int/components= :: =^9.0.5=
|
||||
- =@aw-int/icons= :: =^8.3.0=
|
||||
- =@aw-int/aweber-webapp-scripts= :: =^10.8.0=
|
||||
* Projects
|
||||
Parent ticket: [[https://jira.aweber.io/browse/CCPANEL-11762][CCPANEL-11762]]
|
||||
|
||||
** TESTING Campaign Builder (Direct)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CC-7550
|
||||
:END:
|
||||
- Tagbox :: 5.0.3
|
||||
** NO-ACTION GoToWebinar Client (Direct)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: INT-5508
|
||||
:END:
|
||||
- Tagbox :: 3.0.0
|
||||
|
||||
This project does not use the Tagbox component.
|
||||
** TESTING Subscriber Import (Direct)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11768
|
||||
:END:
|
||||
- Tagbox :: 5.0.3
|
||||
- AWeberUI :: 2
|
||||
|
||||
Uses ramda, etc.
|
||||
** TESTING List Automation Client (AWeberUI)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11769
|
||||
:END:
|
||||
- AWeberUI :: 8.3
|
||||
** NO-ACTION User Management Client (AWeberUI)
|
||||
This project does not use the Tagbox component.
|
||||
** TESTING Landing Page Editor (AWeberUI)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CC-7551
|
||||
:END:
|
||||
** TESTING Draft Bin (Direct)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CC-7552
|
||||
:END:
|
||||
- Tagbox :: 5.0.0
|
||||
** TESTING Subscribers Client (AWeber UI)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11770
|
||||
:END:
|
||||
- AWeberUI :: 8.6.1
|
||||
** TESTING Webform Generator (Standalone file)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CC-7553
|
||||
:END:
|
||||
- Tagbox :: 5.0.3
|
||||
** NO-ACTION Integrations Platform Client (AWeberUI)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: INT-5509
|
||||
:END:
|
||||
- AWeberUI :: 5.14.0
|
||||
This is a prototype project that has not been deployed anywhere.
|
||||
** NO-ACTION Tag Management Client (Direct)
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-11771
|
||||
:END:
|
||||
- Tagbox :: 5.0.3
|
||||
|
||||
This project does not use the Tagbox component.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 1463cf0a-e2b2-490c-b1b3-40249b483ca8
|
||||
:END:
|
||||
#+title: Bulk Job Service
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea
|
||||
:END:
|
||||
#+title: AppDB
|
||||
|
|
@ -1,517 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 2f42c362-ae96-41d5-988b-329f8c162e45
|
||||
:END:
|
||||
#+title: Legacy Search Filters
|
||||
|
||||
Search filters provided by [[id:d9cb2b55-3b0e-4ab3-8369-f71ebc3cd882][Sites Subscriber Search]] and supported in the new
|
||||
[[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]].
|
||||
|
||||
#+name: search-boxes
|
||||
#+header: :engine postgresql
|
||||
#+header: :dbhost 127.0.0.1 :dbport 56893
|
||||
#+header: :dbuser postgres
|
||||
#+begin_src sql :cache yes :eval no-export
|
||||
select box.id, input.description, input.column, term.description
|
||||
from search_boxes as box
|
||||
join search_inputs as input on (box.search_input_id = input.id)
|
||||
join search_terms as term on (box.search_term_id = term.id)
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[50d2c2ce018c9fd37f4093af3646725bcd98131f]: search-boxes
|
||||
| id | description | column | description |
|
||||
|-----+-----------------------+----------------------+-----------------------------|
|
||||
| 1 | Email | email | is |
|
||||
| 2 | Email | email | is not |
|
||||
| 3 | Email | email | contains |
|
||||
| 4 | Email | email | does not contain |
|
||||
| 5 | Email | email | starts with |
|
||||
| 6 | Email | email | ends with |
|
||||
| 7 | Email | email | does not start with |
|
||||
| 8 | Email | email | does not end with |
|
||||
| 9 | Name | name | is |
|
||||
| 10 | Name | name | is not |
|
||||
| 11 | Name | name | contains |
|
||||
| 12 | Name | name | does not contain |
|
||||
| 13 | Name | name | starts with |
|
||||
| 14 | Name | name | ends with |
|
||||
| 15 | Name | name | does not start with |
|
||||
| 16 | Name | name | does not end with |
|
||||
| 17 | Ad Tracking | note | is |
|
||||
| 18 | Ad Tracking | note | is not |
|
||||
| 19 | Ad Tracking | note | contains |
|
||||
| 20 | Ad Tracking | note | does not contain |
|
||||
| 21 | Ad Tracking | note | starts with |
|
||||
| 22 | Ad Tracking | note | ends with |
|
||||
| 23 | Ad Tracking | note | does not start with |
|
||||
| 24 | Ad Tracking | note | does not end with |
|
||||
| 25 | Additional Notes | name2 | is |
|
||||
| 26 | Additional Notes | name2 | is not |
|
||||
| 27 | Additional Notes | name2 | contains |
|
||||
| 28 | Additional Notes | name2 | does not contain |
|
||||
| 29 | Additional Notes | name2 | starts with |
|
||||
| 30 | Additional Notes | name2 | ends with |
|
||||
| 31 | Additional Notes | name2 | does not start with |
|
||||
| 32 | Additional Notes | name2 | does not end with |
|
||||
| 33 | Last Message # | message | is |
|
||||
| 34 | Last Message # | message | is not |
|
||||
| 35 | Last Message # | message | contains |
|
||||
| 36 | Last Message # | message | does not contain |
|
||||
| 37 | Last Message # | message | starts with |
|
||||
| 38 | Last Message # | message | ends with |
|
||||
| 39 | Last Message # | message | does not start with |
|
||||
| 40 | Last Message # | message | does not end with |
|
||||
| 41 | Last Message # | message | is less than |
|
||||
| 42 | Last Message # | message | is less than or equal to |
|
||||
| 43 | Last Message # | message | is greater than |
|
||||
| 44 | Last Message # | message | is greater than or equal to |
|
||||
| 45 | Stop Status | stop_status | is |
|
||||
| 46 | Stop Status | stop_status | is not |
|
||||
| 47 | Stop Method | stop_method | is |
|
||||
| 48 | Stop Method | stop_method | is not |
|
||||
| 49 | Confirmed? | verified | is |
|
||||
| 50 | Confirmed? | verified | is not |
|
||||
| 51 | Add Method | add_method | is |
|
||||
| 52 | Add Method | add_method | is not |
|
||||
| 53 | Add URL | add_url | is |
|
||||
| 54 | Add URL | add_url | is not |
|
||||
| 55 | Add URL | add_url | contains |
|
||||
| 56 | Add URL | add_url | does not contain |
|
||||
| 57 | Add URL | add_url | starts with |
|
||||
| 58 | Add URL | add_url | ends with |
|
||||
| 59 | Add URL | add_url | does not start with |
|
||||
| 60 | Add URL | add_url | does not end with |
|
||||
| 86 | Date Stopped | stop_time | date is on or before |
|
||||
| 61 | Add IP | add_notes | is |
|
||||
| 62 | Add IP | add_notes | is not |
|
||||
| 63 | Add IP | add_notes | contains |
|
||||
| 64 | Add IP | add_notes | does not contain |
|
||||
| 65 | Add IP | add_notes | starts with |
|
||||
| 66 | Add IP | add_notes | ends with |
|
||||
| 67 | Add IP | add_notes | does not start with |
|
||||
| 68 | Add IP | add_notes | does not end with |
|
||||
| 69 | Confirmation IP | verification_notes | is |
|
||||
| 70 | Confirmation IP | verification_notes | is not |
|
||||
| 71 | Confirmation IP | verification_notes | contains |
|
||||
| 72 | Confirmation IP | verification_notes | does not contain |
|
||||
| 73 | Confirmation IP | verification_notes | starts with |
|
||||
| 74 | Confirmation IP | verification_notes | ends with |
|
||||
| 75 | Confirmation IP | verification_notes | does not start with |
|
||||
| 76 | Confirmation IP | verification_notes | does not end with |
|
||||
| 87 | Date Stopped | stop_time | date is after |
|
||||
| 88 | Date Stopped | stop_time | date is on or after |
|
||||
| 89 | Date Confirmed | verification_time | date is before |
|
||||
| 90 | Date Confirmed | verification_time | date is on or before |
|
||||
| 91 | Date Confirmed | verification_time | date is after |
|
||||
| 92 | Date Confirmed | verification_time | date is on or after |
|
||||
| 127 | Latitude (from IP) | geog_lat | is |
|
||||
| 128 | Latitude (from IP) | geog_lat | is not |
|
||||
| 129 | Latitude (from IP) | geog_lat | contains |
|
||||
| 130 | Latitude (from IP) | geog_lat | does not contain |
|
||||
| 131 | Latitude (from IP) | geog_lat | starts with |
|
||||
| 132 | Latitude (from IP) | geog_lat | ends with |
|
||||
| 133 | Latitude (from IP) | geog_lat | does not start with |
|
||||
| 134 | Latitude (from IP) | geog_lat | does not end with |
|
||||
| 135 | Latitude (from IP) | geog_lat | is less than |
|
||||
| 136 | Latitude (from IP) | geog_lat | is less than or equal to |
|
||||
| 93 | Country (from IP) | geog_country | is |
|
||||
| 94 | Country (from IP) | geog_country | is not |
|
||||
| 95 | Country (from IP) | geog_country | contains |
|
||||
| 96 | Country (from IP) | geog_country | does not contain |
|
||||
| 97 | Country (from IP) | geog_country | starts with |
|
||||
| 98 | Country (from IP) | geog_country | ends with |
|
||||
| 99 | Country (from IP) | geog_country | does not start with |
|
||||
| 100 | Country (from IP) | geog_country | does not end with |
|
||||
| 101 | Region (from IP) | geog_region | is |
|
||||
| 102 | Region (from IP) | geog_region | is not |
|
||||
| 103 | Region (from IP) | geog_region | contains |
|
||||
| 104 | Region (from IP) | geog_region | does not contain |
|
||||
| 105 | Region (from IP) | geog_region | starts with |
|
||||
| 106 | Region (from IP) | geog_region | ends with |
|
||||
| 107 | Region (from IP) | geog_region | does not start with |
|
||||
| 108 | Region (from IP) | geog_region | does not end with |
|
||||
| 137 | Latitude (from IP) | geog_lat | is greater than |
|
||||
| 138 | Latitude (from IP) | geog_lat | is greater than or equal to |
|
||||
| 111 | City (from IP) | geog_city | is |
|
||||
| 112 | City (from IP) | geog_city | is not |
|
||||
| 113 | City (from IP) | geog_city | contains |
|
||||
| 114 | City (from IP) | geog_city | does not contain |
|
||||
| 115 | City (from IP) | geog_city | starts with |
|
||||
| 116 | City (from IP) | geog_city | ends with |
|
||||
| 117 | City (from IP) | geog_city | does not start with |
|
||||
| 118 | City (from IP) | geog_city | does not end with |
|
||||
| 119 | Postal Code (from IP) | geog_postal | is |
|
||||
| 120 | Postal Code (from IP) | geog_postal | is not |
|
||||
| 121 | Postal Code (from IP) | geog_postal | contains |
|
||||
| 122 | Postal Code (from IP) | geog_postal | does not contain |
|
||||
| 123 | Postal Code (from IP) | geog_postal | starts with |
|
||||
| 124 | Postal Code (from IP) | geog_postal | ends with |
|
||||
| 125 | Postal Code (from IP) | geog_postal | does not start with |
|
||||
| 126 | Postal Code (from IP) | geog_postal | does not end with |
|
||||
| 77 | Date Added | timehit | date is before |
|
||||
| 78 | Date Added | timehit | date is on or before |
|
||||
| 79 | Date Added | timehit | date is after |
|
||||
| 80 | Date Added | timehit | date is on or after |
|
||||
| 81 | Date Last Follow Up | followuptime | date is before |
|
||||
| 82 | Date Last Follow Up | followuptime | date is on or before |
|
||||
| 83 | Date Last Follow Up | followuptime | date is after |
|
||||
| 84 | Date Last Follow Up | followuptime | date is on or after |
|
||||
| 85 | Date Stopped | stop_time | date is before |
|
||||
| 139 | Longitude (from IP) | geog_lon | is |
|
||||
| 144 | Longitude (from IP) | geog_lon | starts with |
|
||||
| 140 | Longitude (from IP) | geog_lon | is not |
|
||||
| 141 | Longitude (from IP) | geog_lon | contains |
|
||||
| 142 | Longitude (from IP) | geog_lon | does not contain |
|
||||
| 145 | Longitude (from IP) | geog_lon | ends with |
|
||||
| 146 | Longitude (from IP) | geog_lon | does not start with |
|
||||
| 147 | Longitude (from IP) | geog_lon | does not end with |
|
||||
| 148 | Longitude (from IP) | geog_lon | is less than |
|
||||
| 149 | Longitude (from IP) | geog_lon | is less than or equal to |
|
||||
| 150 | Longitude (from IP) | geog_lon | is greater than |
|
||||
| 151 | Longitude (from IP) | geog_lon | is greater than or equal to |
|
||||
| 152 | Area Code (from IP) | geog_area_code | is |
|
||||
| 153 | Area Code (from IP) | geog_area_code | is not |
|
||||
| 154 | Area Code (from IP) | geog_area_code | contains |
|
||||
| 155 | Area Code (from IP) | geog_area_code | does not contain |
|
||||
| 156 | Area Code (from IP) | geog_area_code | starts with |
|
||||
| 157 | Area Code (from IP) | geog_area_code | ends with |
|
||||
| 158 | Area Code (from IP) | geog_area_code | does not start with |
|
||||
| 159 | Area Code (from IP) | geog_area_code | does not end with |
|
||||
| 160 | Area Code (from IP) | geog_area_code | is less than |
|
||||
| 161 | Area Code (from IP) | geog_area_code | is less than or equal to |
|
||||
| 162 | Area Code (from IP) | geog_area_code | is greater than |
|
||||
| 163 | Area Code (from IP) | geog_area_code | is greater than or equal to |
|
||||
| 164 | DMA Code (from IP) | geog_dma_code | is |
|
||||
| 165 | DMA Code (from IP) | geog_dma_code | is not |
|
||||
| 166 | DMA Code (from IP) | geog_dma_code | contains |
|
||||
| 167 | DMA Code (from IP) | geog_dma_code | does not contain |
|
||||
| 168 | DMA Code (from IP) | geog_dma_code | starts with |
|
||||
| 169 | DMA Code (from IP) | geog_dma_code | ends with |
|
||||
| 170 | DMA Code (from IP) | geog_dma_code | does not start with |
|
||||
| 171 | DMA Code (from IP) | geog_dma_code | does not end with |
|
||||
| 172 | DMA Code (from IP) | geog_dma_code | is less than |
|
||||
| 173 | DMA Code (from IP) | geog_dma_code | is less than or equal to |
|
||||
| 174 | DMA Code (from IP) | geog_dma_code | is greater than |
|
||||
| 175 | DMA Code (from IP) | geog_dma_code | is greater than or equal to |
|
||||
| 176 | Message not opened | app_message_id | is |
|
||||
| 177 | Message opened | app_message_id | is |
|
||||
| 226 | datum1 | datum1 | is |
|
||||
| 227 | datum1 | datum1 | is not |
|
||||
| 228 | datum1 | datum1 | contains |
|
||||
| 229 | datum1 | datum1 | does not contain |
|
||||
| 230 | datum1 | datum1 | starts with |
|
||||
| 231 | datum1 | datum1 | ends with |
|
||||
| 232 | datum1 | datum1 | does not start with |
|
||||
| 233 | datum1 | datum1 | does not end with |
|
||||
| 234 | datum2 | datum2 | is |
|
||||
| 235 | datum2 | datum2 | is not |
|
||||
| 236 | datum2 | datum2 | contains |
|
||||
| 237 | datum2 | datum2 | does not contain |
|
||||
| 238 | datum2 | datum2 | starts with |
|
||||
| 239 | datum2 | datum2 | ends with |
|
||||
| 240 | datum2 | datum2 | does not start with |
|
||||
| 241 | datum2 | datum2 | does not end with |
|
||||
| 242 | datum3 | datum3 | is |
|
||||
| 243 | datum3 | datum3 | is not |
|
||||
| 244 | datum3 | datum3 | contains |
|
||||
| 245 | datum3 | datum3 | does not contain |
|
||||
| 246 | datum3 | datum3 | starts with |
|
||||
| 247 | datum3 | datum3 | ends with |
|
||||
| 248 | datum3 | datum3 | does not start with |
|
||||
| 249 | datum3 | datum3 | does not end with |
|
||||
| 250 | datum4 | datum4 | is |
|
||||
| 251 | datum4 | datum4 | is not |
|
||||
| 252 | datum4 | datum4 | contains |
|
||||
| 253 | datum4 | datum4 | does not contain |
|
||||
| 254 | datum4 | datum4 | starts with |
|
||||
| 255 | datum4 | datum4 | ends with |
|
||||
| 256 | datum4 | datum4 | does not start with |
|
||||
| 257 | datum4 | datum4 | does not end with |
|
||||
| 258 | datum5 | datum5 | is |
|
||||
| 259 | datum5 | datum5 | is not |
|
||||
| 260 | datum5 | datum5 | contains |
|
||||
| 261 | datum5 | datum5 | does not contain |
|
||||
| 262 | datum5 | datum5 | starts with |
|
||||
| 263 | datum5 | datum5 | ends with |
|
||||
| 264 | datum5 | datum5 | does not start with |
|
||||
| 265 | datum5 | datum5 | does not end with |
|
||||
| 266 | datum6 | datum6 | is |
|
||||
| 267 | datum6 | datum6 | is not |
|
||||
| 268 | datum6 | datum6 | contains |
|
||||
| 269 | datum6 | datum6 | does not contain |
|
||||
| 270 | datum6 | datum6 | starts with |
|
||||
| 271 | datum6 | datum6 | ends with |
|
||||
| 272 | datum6 | datum6 | does not start with |
|
||||
| 273 | datum6 | datum6 | does not end with |
|
||||
| 274 | datum7 | datum7 | is |
|
||||
| 275 | datum7 | datum7 | is not |
|
||||
| 276 | datum7 | datum7 | contains |
|
||||
| 277 | datum7 | datum7 | does not contain |
|
||||
| 278 | datum7 | datum7 | starts with |
|
||||
| 279 | datum7 | datum7 | ends with |
|
||||
| 280 | datum7 | datum7 | does not start with |
|
||||
| 281 | datum7 | datum7 | does not end with |
|
||||
| 282 | datum8 | datum8 | is |
|
||||
| 283 | datum8 | datum8 | is not |
|
||||
| 284 | datum8 | datum8 | contains |
|
||||
| 285 | datum8 | datum8 | does not contain |
|
||||
| 286 | datum8 | datum8 | starts with |
|
||||
| 287 | datum8 | datum8 | ends with |
|
||||
| 288 | datum8 | datum8 | does not start with |
|
||||
| 289 | datum8 | datum8 | does not end with |
|
||||
| 290 | datum9 | datum9 | is |
|
||||
| 291 | datum9 | datum9 | is not |
|
||||
| 292 | datum9 | datum9 | contains |
|
||||
| 293 | datum9 | datum9 | does not contain |
|
||||
| 294 | datum9 | datum9 | starts with |
|
||||
| 295 | datum9 | datum9 | ends with |
|
||||
| 296 | datum9 | datum9 | does not start with |
|
||||
| 297 | datum9 | datum9 | does not end with |
|
||||
| 298 | datum10 | datum10 | is |
|
||||
| 299 | datum10 | datum10 | is not |
|
||||
| 300 | datum10 | datum10 | contains |
|
||||
| 301 | datum10 | datum10 | does not contain |
|
||||
| 302 | datum10 | datum10 | starts with |
|
||||
| 303 | datum10 | datum10 | ends with |
|
||||
| 304 | datum10 | datum10 | does not start with |
|
||||
| 305 | datum10 | datum10 | does not end with |
|
||||
| 306 | datum11 | datum11 | is |
|
||||
| 307 | datum11 | datum11 | is not |
|
||||
| 308 | datum11 | datum11 | contains |
|
||||
| 309 | datum11 | datum11 | does not contain |
|
||||
| 310 | datum11 | datum11 | starts with |
|
||||
| 311 | datum11 | datum11 | ends with |
|
||||
| 312 | datum11 | datum11 | does not start with |
|
||||
| 313 | datum11 | datum11 | does not end with |
|
||||
| 314 | datum12 | datum12 | is |
|
||||
| 315 | datum12 | datum12 | is not |
|
||||
| 316 | datum12 | datum12 | contains |
|
||||
| 317 | datum12 | datum12 | does not contain |
|
||||
| 318 | datum12 | datum12 | starts with |
|
||||
| 319 | datum12 | datum12 | ends with |
|
||||
| 320 | datum12 | datum12 | does not start with |
|
||||
| 321 | datum12 | datum12 | does not end with |
|
||||
| 322 | datum13 | datum13 | is |
|
||||
| 323 | datum13 | datum13 | is not |
|
||||
| 324 | datum13 | datum13 | contains |
|
||||
| 325 | datum13 | datum13 | does not contain |
|
||||
| 326 | datum13 | datum13 | starts with |
|
||||
| 327 | datum13 | datum13 | ends with |
|
||||
| 328 | datum13 | datum13 | does not start with |
|
||||
| 329 | datum13 | datum13 | does not end with |
|
||||
| 330 | datum14 | datum14 | is |
|
||||
| 331 | datum14 | datum14 | is not |
|
||||
| 332 | datum14 | datum14 | contains |
|
||||
| 333 | datum14 | datum14 | does not contain |
|
||||
| 334 | datum14 | datum14 | starts with |
|
||||
| 335 | datum14 | datum14 | ends with |
|
||||
| 336 | datum14 | datum14 | does not start with |
|
||||
| 337 | datum14 | datum14 | does not end with |
|
||||
| 338 | datum15 | datum15 | is |
|
||||
| 339 | datum15 | datum15 | is not |
|
||||
| 340 | datum15 | datum15 | contains |
|
||||
| 341 | datum15 | datum15 | does not contain |
|
||||
| 342 | datum15 | datum15 | starts with |
|
||||
| 343 | datum15 | datum15 | ends with |
|
||||
| 344 | datum15 | datum15 | does not start with |
|
||||
| 345 | datum15 | datum15 | does not end with |
|
||||
| 346 | datum16 | datum16 | is |
|
||||
| 347 | datum16 | datum16 | is not |
|
||||
| 348 | datum16 | datum16 | contains |
|
||||
| 349 | datum16 | datum16 | does not contain |
|
||||
| 350 | datum16 | datum16 | starts with |
|
||||
| 351 | datum16 | datum16 | ends with |
|
||||
| 352 | datum16 | datum16 | does not start with |
|
||||
| 353 | datum16 | datum16 | does not end with |
|
||||
| 354 | datum17 | datum17 | is |
|
||||
| 355 | datum17 | datum17 | is not |
|
||||
| 356 | datum17 | datum17 | contains |
|
||||
| 357 | datum17 | datum17 | does not contain |
|
||||
| 358 | datum17 | datum17 | starts with |
|
||||
| 359 | datum17 | datum17 | ends with |
|
||||
| 360 | datum17 | datum17 | does not start with |
|
||||
| 361 | datum17 | datum17 | does not end with |
|
||||
| 362 | datum18 | datum18 | is |
|
||||
| 363 | datum18 | datum18 | is not |
|
||||
| 364 | datum18 | datum18 | contains |
|
||||
| 365 | datum18 | datum18 | does not contain |
|
||||
| 366 | datum18 | datum18 | starts with |
|
||||
| 367 | datum18 | datum18 | ends with |
|
||||
| 368 | datum18 | datum18 | does not start with |
|
||||
| 369 | datum18 | datum18 | does not end with |
|
||||
| 370 | datum19 | datum19 | is |
|
||||
| 371 | datum19 | datum19 | is not |
|
||||
| 372 | datum19 | datum19 | contains |
|
||||
| 373 | datum19 | datum19 | does not contain |
|
||||
| 374 | datum19 | datum19 | starts with |
|
||||
| 375 | datum19 | datum19 | ends with |
|
||||
| 376 | datum19 | datum19 | does not start with |
|
||||
| 377 | datum19 | datum19 | does not end with |
|
||||
| 534 | Link clicked | link_id | is |
|
||||
| 535 | Link not clicked | link_id | is |
|
||||
| 429 | datum1 | datum1 | is greater than |
|
||||
| 433 | datum2 | datum2 | is greater than |
|
||||
| 437 | datum3 | datum3 | is greater than |
|
||||
| 441 | datum4 | datum4 | is greater than |
|
||||
| 445 | datum5 | datum5 | is greater than |
|
||||
| 449 | datum6 | datum6 | is greater than |
|
||||
| 453 | datum7 | datum7 | is greater than |
|
||||
| 457 | datum8 | datum8 | is greater than |
|
||||
| 461 | datum9 | datum9 | is greater than |
|
||||
| 465 | datum10 | datum10 | is greater than |
|
||||
| 469 | datum11 | datum11 | is greater than |
|
||||
| 473 | datum12 | datum12 | is greater than |
|
||||
| 477 | datum13 | datum13 | is greater than |
|
||||
| 430 | datum1 | datum1 | is greater than or equal to |
|
||||
| 434 | datum2 | datum2 | is greater than or equal to |
|
||||
| 438 | datum3 | datum3 | is greater than or equal to |
|
||||
| 442 | datum4 | datum4 | is greater than or equal to |
|
||||
| 446 | datum5 | datum5 | is greater than or equal to |
|
||||
| 450 | datum6 | datum6 | is greater than or equal to |
|
||||
| 454 | datum7 | datum7 | is greater than or equal to |
|
||||
| 458 | datum8 | datum8 | is greater than or equal to |
|
||||
| 462 | datum9 | datum9 | is greater than or equal to |
|
||||
| 466 | datum10 | datum10 | is greater than or equal to |
|
||||
| 470 | datum11 | datum11 | is greater than or equal to |
|
||||
| 474 | datum12 | datum12 | is greater than or equal to |
|
||||
| 478 | datum13 | datum13 | is greater than or equal to |
|
||||
| 482 | datum14 | datum14 | is greater than or equal to |
|
||||
| 486 | datum15 | datum15 | is greater than or equal to |
|
||||
| 490 | datum16 | datum16 | is greater than or equal to |
|
||||
| 494 | datum17 | datum17 | is greater than or equal to |
|
||||
| 498 | datum18 | datum18 | is greater than or equal to |
|
||||
| 502 | datum19 | datum19 | is greater than or equal to |
|
||||
| 432 | datum1 | datum1 | is less than or equal to |
|
||||
| 436 | datum2 | datum2 | is less than or equal to |
|
||||
| 440 | datum3 | datum3 | is less than or equal to |
|
||||
| 444 | datum4 | datum4 | is less than or equal to |
|
||||
| 448 | datum5 | datum5 | is less than or equal to |
|
||||
| 452 | datum6 | datum6 | is less than or equal to |
|
||||
| 456 | datum7 | datum7 | is less than or equal to |
|
||||
| 460 | datum8 | datum8 | is less than or equal to |
|
||||
| 464 | datum9 | datum9 | is less than or equal to |
|
||||
| 468 | datum10 | datum10 | is less than or equal to |
|
||||
| 472 | datum11 | datum11 | is less than or equal to |
|
||||
| 476 | datum12 | datum12 | is less than or equal to |
|
||||
| 480 | datum13 | datum13 | is less than or equal to |
|
||||
| 484 | datum14 | datum14 | is less than or equal to |
|
||||
| 488 | datum15 | datum15 | is less than or equal to |
|
||||
| 492 | datum16 | datum16 | is less than or equal to |
|
||||
| 496 | datum17 | datum17 | is less than or equal to |
|
||||
| 500 | datum18 | datum18 | is less than or equal to |
|
||||
| 504 | datum19 | datum19 | is less than or equal to |
|
||||
| 431 | datum1 | datum1 | is less than |
|
||||
| 435 | datum2 | datum2 | is less than |
|
||||
| 439 | datum3 | datum3 | is less than |
|
||||
| 443 | datum4 | datum4 | is less than |
|
||||
| 447 | datum5 | datum5 | is less than |
|
||||
| 451 | datum6 | datum6 | is less than |
|
||||
| 455 | datum7 | datum7 | is less than |
|
||||
| 459 | datum8 | datum8 | is less than |
|
||||
| 463 | datum9 | datum9 | is less than |
|
||||
| 467 | datum10 | datum10 | is less than |
|
||||
| 471 | datum11 | datum11 | is less than |
|
||||
| 475 | datum12 | datum12 | is less than |
|
||||
| 479 | datum13 | datum13 | is less than |
|
||||
| 483 | datum14 | datum14 | is less than |
|
||||
| 487 | datum15 | datum15 | is less than |
|
||||
| 491 | datum16 | datum16 | is less than |
|
||||
| 495 | datum17 | datum17 | is less than |
|
||||
| 499 | datum18 | datum18 | is less than |
|
||||
| 503 | datum19 | datum19 | is less than |
|
||||
| 529 | Sale Amount | monetary_value | is |
|
||||
| 530 | Sale Amount | monetary_value | is less than |
|
||||
| 531 | Sale Amount | monetary_value | is less than or equal to |
|
||||
| 532 | Sale Amount | monetary_value | is greater than |
|
||||
| 533 | Sale Amount | monetary_value | is greater than or equal to |
|
||||
| 481 | datum14 | datum14 | is greater than |
|
||||
| 485 | datum15 | datum15 | is greater than |
|
||||
| 489 | datum16 | datum16 | is greater than |
|
||||
| 493 | datum17 | datum17 | is greater than |
|
||||
| 497 | datum18 | datum18 | is greater than |
|
||||
| 501 | datum19 | datum19 | is greater than |
|
||||
| 536 | Undeliverable | unit_id | is |
|
||||
| 537 | Web Page Visited | web_page_id | is |
|
||||
| 538 | No Opens | created | since |
|
||||
| 587 | datum20 | datum20 | is |
|
||||
| 588 | datum20 | datum20 | is not |
|
||||
| 589 | datum20 | datum20 | contains |
|
||||
| 590 | datum20 | datum20 | does not contain |
|
||||
| 591 | datum20 | datum20 | starts with |
|
||||
| 592 | datum20 | datum20 | ends with |
|
||||
| 593 | datum20 | datum20 | does not start with |
|
||||
| 594 | datum20 | datum20 | does not end with |
|
||||
| 595 | datum20 | datum20 | is less than |
|
||||
| 596 | datum20 | datum20 | is less than or equal to |
|
||||
| 597 | datum20 | datum20 | is greater than |
|
||||
| 598 | datum20 | datum20 | is greater than or equal to |
|
||||
| 599 | datum21 | datum21 | is |
|
||||
| 600 | datum21 | datum21 | is not |
|
||||
| 601 | datum21 | datum21 | contains |
|
||||
| 602 | datum21 | datum21 | does not contain |
|
||||
| 603 | datum21 | datum21 | starts with |
|
||||
| 604 | datum21 | datum21 | ends with |
|
||||
| 605 | datum21 | datum21 | does not start with |
|
||||
| 606 | datum21 | datum21 | does not end with |
|
||||
| 607 | datum21 | datum21 | is less than |
|
||||
| 608 | datum21 | datum21 | is less than or equal to |
|
||||
| 609 | datum21 | datum21 | is greater than |
|
||||
| 610 | datum21 | datum21 | is greater than or equal to |
|
||||
| 611 | datum22 | datum22 | is |
|
||||
| 612 | datum22 | datum22 | is not |
|
||||
| 613 | datum22 | datum22 | contains |
|
||||
| 614 | datum22 | datum22 | does not contain |
|
||||
| 615 | datum22 | datum22 | starts with |
|
||||
| 616 | datum22 | datum22 | ends with |
|
||||
| 617 | datum22 | datum22 | does not start with |
|
||||
| 618 | datum22 | datum22 | does not end with |
|
||||
| 619 | datum22 | datum22 | is less than |
|
||||
| 620 | datum22 | datum22 | is less than or equal to |
|
||||
| 621 | datum22 | datum22 | is greater than |
|
||||
| 622 | datum22 | datum22 | is greater than or equal to |
|
||||
| 623 | datum23 | datum23 | is |
|
||||
| 624 | datum23 | datum23 | is not |
|
||||
| 625 | datum23 | datum23 | contains |
|
||||
| 626 | datum23 | datum23 | does not contain |
|
||||
| 627 | datum23 | datum23 | starts with |
|
||||
| 628 | datum23 | datum23 | ends with |
|
||||
| 629 | datum23 | datum23 | does not start with |
|
||||
| 630 | datum23 | datum23 | does not end with |
|
||||
| 631 | datum23 | datum23 | is less than |
|
||||
| 632 | datum23 | datum23 | is less than or equal to |
|
||||
| 633 | datum23 | datum23 | is greater than |
|
||||
| 634 | datum23 | datum23 | is greater than or equal to |
|
||||
| 635 | datum24 | datum24 | is |
|
||||
| 636 | datum24 | datum24 | is not |
|
||||
| 637 | datum24 | datum24 | contains |
|
||||
| 638 | datum24 | datum24 | does not contain |
|
||||
| 639 | datum24 | datum24 | starts with |
|
||||
| 640 | datum24 | datum24 | ends with |
|
||||
| 641 | datum24 | datum24 | does not start with |
|
||||
| 642 | datum24 | datum24 | does not end with |
|
||||
| 643 | datum24 | datum24 | is less than |
|
||||
| 644 | datum24 | datum24 | is less than or equal to |
|
||||
| 645 | datum24 | datum24 | is greater than |
|
||||
| 646 | datum24 | datum24 | is greater than or equal to |
|
||||
| 647 | datum25 | datum25 | is |
|
||||
| 648 | datum25 | datum25 | is not |
|
||||
| 649 | datum25 | datum25 | contains |
|
||||
| 650 | datum25 | datum25 | does not contain |
|
||||
| 651 | datum25 | datum25 | starts with |
|
||||
| 652 | datum25 | datum25 | ends with |
|
||||
| 653 | datum25 | datum25 | does not start with |
|
||||
| 654 | datum25 | datum25 | does not end with |
|
||||
| 655 | datum25 | datum25 | is less than |
|
||||
| 656 | datum25 | datum25 | is less than or equal to |
|
||||
| 657 | datum25 | datum25 | is greater than |
|
||||
| 658 | datum25 | datum25 | is greater than or equal to |
|
||||
| 659 | Tag | subscriber_tags.tags | is |
|
||||
| 660 | Tag | subscriber_tags.tags | is not |
|
||||
| 661 | Any Opens | opens | since |
|
||||
| 662 | Any Clicks | clicks | since |
|
||||
| 663 | Any Clicks | clicks | before |
|
||||
| 664 | Any Opens | opens | before |
|
||||
| 665 | No Opens | created | before |
|
||||
| 666 | Tag | subscriber_tags.tags | is any of these |
|
||||
| 667 | Tag | subscriber_tags.tags | includes all of these |
|
||||
|
||||
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 4bf81e33-8020-40f2-b6ed-ce4e1eae2234
|
||||
:ROAM_ALIASES: "Restoring deleted subscribers"
|
||||
:END:
|
||||
#+title: Admin restore deleted controller
|
||||
|
||||
Lists have a deleted subscriber history view, from which deleted subscribers can
|
||||
be restored.
|
||||
|
|
@ -1,74 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e6a2c650-ff59-4b72-b073-970731796888
|
||||
:END:
|
||||
#+title: Legacy search segment construction
|
||||
|
||||
Thoughts on constructing a segment representation from legacy data in the
|
||||
[[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]], referencing a [[https://gitlab.aweber.io/CP/Services/subscriber-search/-/merge_requests/15/diffs?commit_id=4d203dfec800cf49825d32a7ea663c4b862693bb][WIP commit]] loading legacy segments from
|
||||
the database.
|
||||
|
||||
- The legacy segment stores field and operator information encoded in a search
|
||||
box ID, and a match value in a list view criteria field.
|
||||
- Legacy segments only support a top-level AND grouping of criteria
|
||||
|
||||
Therefore, given a list of criteria on a segment, a Search can be formed as
|
||||
follows:
|
||||
#+begin_src python
|
||||
import dataclasses
|
||||
|
||||
from subscribersearch import legacy, search
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Segment:
|
||||
"""A saved search on a list representing a segment of subscribers.
|
||||
|
||||
Likely belongs in the search module.
|
||||
"""
|
||||
id: typing.Optional[int]
|
||||
name: str
|
||||
search: search.Search
|
||||
|
||||
@staticmethod
|
||||
def from_dict(value: dict) -> Search:
|
||||
return Segment(
|
||||
id=value.get("id"),
|
||||
name=value["name"],
|
||||
search=search.from_dict(value["search"]),
|
||||
)
|
||||
|
||||
def to_dict(self) -> typing.Dict[str, typing.Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"name": self.name,
|
||||
"search": self.search.to_dict(),
|
||||
}
|
||||
|
||||
|
||||
# segment = fetch legacy segment info
|
||||
# rows = fetch legacy segment criteria
|
||||
segment = Segment(
|
||||
id=segment["id"],
|
||||
name=segment["name"],
|
||||
search=search.Search(
|
||||
list_id=segment["list_id"],
|
||||
group=search.Group(
|
||||
group_type=search.GroupType.AND,
|
||||
conditions=[
|
||||
search.Condition(
|
||||
field=filter.field,
|
||||
operator=filter.operator,
|
||||
match=match,
|
||||
)
|
||||
for filter, match in [
|
||||
(legacy.filter_from_id(row["sb_id"]), row["lvc_criteria"]) in rows
|
||||
]
|
||||
],
|
||||
),
|
||||
),
|
||||
)
|
||||
#+end_src
|
||||
|
||||
By having functions that can retrieve a Segment object from the database and
|
||||
save one back to it, we have an interface that we can then copy later for
|
||||
storing and retrieving segments in an updated schema, and migrate between the
|
||||
two.
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: b6a4fa1c-c07b-4834-8ab8-5ebe79334729
|
||||
:END:
|
||||
#+title: Move Cache1-Cache4 to MDF
|
||||
#+filetags: :project:
|
||||
|
||||
- https://jira.aweber.io/browse/DCM-259
|
||||
- https://confluence.aweber.io/pages/viewpage.action?spaceKey=PSE&title=DCM2+Analysis%3A+Cache+Servers
|
||||
|
||||
| Team | Project | Redis Usage |
|
||||
|------+------------------------------+----------------------------------------|
|
||||
| cp | bulk-tagging | aiomappinglib |
|
||||
| cp | bulk-tagging-consumer | aiomappinglib |
|
||||
| cp | coiconsumer | aiomappinglib |
|
||||
| cp | newsubnotifier | aiomappinglib |
|
||||
| cp | sosconsumer | aiomappinglib |
|
||||
| cp | stripe-payments | aiomappinglib |
|
||||
| cp | subscriber-import-evaluation | imports |
|
||||
| cp | subscriber-import-processor | imports |
|
||||
| cp | subscriber-rebuild | aiomappinglib |
|
||||
| cp | subscriber-sync-create | aiomappinglib, subscriber-lead-mapping |
|
||||
| cp | subscriber-sync-delete | aiomappinglib, subscriber-lead-mapping |
|
||||
| cp | subscriber-sync-update | aiomappinglib, subscriber-lead-mapping |
|
||||
| cp | subscriber-tag-sync | aiomappinglib |
|
||||
| cp | subscriberimportapi | imports |
|
||||
| cp | subscriberproxy | aiomappinglib |
|
||||
| cp | uosconsumer | aiomappinglib |
|
||||
| cp | uouconsumer | aiomappinglib |
|
||||
|
||||
TTLs:
|
||||
- imports :: 86400 seconds
|
||||
- subscriber-lead-mapping :: None
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 46515cfd-3e6c-46ac-a8f7-7fc722141338
|
||||
:END:
|
||||
#+title: Retire CAPI
|
||||
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 96d1d218-60cd-41d9-91ba-48359137d239
|
||||
:END:
|
||||
#+title: Decommission the mail-relay service
|
||||
|
||||
- https://jira.aweber.io/browse/CCPANEL-10580
|
||||
|
||||
Tackle remaining legacy emails and retire the mail relay container.
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 311f56fc-4404-4e25-a764-d7e455cd406e
|
||||
:END:
|
||||
#+title: CS Lead Ticket Questionairre
|
||||
|
||||
- *What was broken?* :: _
|
||||
- *When was it broken?* :: _
|
||||
- *What did you do to fix the problem?* :: _
|
||||
- *How many customers did it likely impact?* :: _
|
||||
- *Is the issue automatically fixed for all customers now?* :: _
|
||||
- *Does the customer or CS need to manually do something to fix their account?* :: _
|
||||
- *Should a new monitoring check, metric, or test be created to prevent this from happening again?* :: _
|
||||
|
|
@ -1,148 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d06d3ab4-c2d0-47c3-aae1-4395567fc3d2
|
||||
:END:
|
||||
#+title: Tag Normalization
|
||||
#+OPTIONS: prop:t
|
||||
|
||||
https://jira.aweber.io/browse/CCPANEL-11888
|
||||
|
||||
Tags allow extra spaces between words and characters like commas and quotes and
|
||||
then rendering is misinterpreting the actual tag save in the database.
|
||||
|
||||
A lot of campaign issues come from inconsistencies in how tags are saved and
|
||||
visualized.
|
||||
|
||||
* Normalization Rules
|
||||
- Spaces
|
||||
- Multiple spaces turned into a single space.
|
||||
- No leading or trailing spaces.
|
||||
- No non-printable characters (nbsp, line breaks, etc).
|
||||
- Other Characters
|
||||
- No commas.
|
||||
- No quotes (single or double).
|
||||
- Lowercase all characters.
|
||||
|
||||
* Roll-out
|
||||
:PROPERTIES:
|
||||
:COLUMNS: %50ITEM %JIRA_ID %Effort{:}
|
||||
:END:
|
||||
|
||||
#+BEGIN: columnview :indent t
|
||||
| ITEM | JIRA_ID | Effort |
|
||||
|-----------------------------------------------------------------------------------------+---------------+----------|
|
||||
| Roll-out | | 21d 0:00 |
|
||||
| \_ Normalize when comparing tags | CCPANEL-12033 | 4d 4:00 |
|
||||
| \_ Update Campaign Engine to apply normalization rules when comparing tags | CCPANEL-12031 | 1.5d |
|
||||
| \_ Update Analytics Search DB Terms to apply normalization rules when comparing tags | CCPANEL-12034 | 3d |
|
||||
| \_ Normalize incoming data | CCPANEL-12009 | 5d 0:00 |
|
||||
| \_ Update [[id:7e503917-646f-4275-aab9-3a125b99cbfd][Tagging Service]] to normalize tags | CCPANEL-12010 | 2d |
|
||||
| \_ Update [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] to normalize tags | CCPANEL-12011 | 2d |
|
||||
| \_ Update TagBox to normalize tags | CCPANEL-12035 | |
|
||||
| \_ Normalize tags when storing in rule sets | CCPANEL-12036 | 1d |
|
||||
| \_ Normalize existing tag data | CCPANEL-12013 | 10d 0:00 |
|
||||
| \_ Update tags in rulesets to conform to normalization rules | CCPANEL-12014 | 3d |
|
||||
| \_ Update tags in segments to conform to normalization rules | CCPANEL-12015 | 3d |
|
||||
| \_ Update tags in the tagging database to conform to normalization rules | CCPANEL-12016 | 2d |
|
||||
| \_ Update tags in the subscriber tags table to conform to normalization rules | CCPANEL-12037 | 2d |
|
||||
| \_ Remove comparison normalization logic | CCPANEL-12038 | 1d 4:00 |
|
||||
| \_ Remove normalized comparisons from Campaign Engine | CCPANEL-12039 | 4h |
|
||||
| \_ Remove normalized comparisons from Analytics DB Search Terms | CCPANEL-12040 | 1d |
|
||||
| \_ Normalize tags stored outside of tagging and campaigns | CCPANEL-12041 | |
|
||||
| \_ Normalize tags when storing in Stripe | | |
|
||||
| \_ Integrations | | |
|
||||
#+END:
|
||||
|
||||
** Normalize when comparing tags
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12033
|
||||
:END:
|
||||
*** Update Campaign Engine to apply normalization rules when comparing tags
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12031
|
||||
:EFFORT: 1.5d
|
||||
:END:
|
||||
*** Update Analytics Search DB Terms to apply normalization rules when comparing tags
|
||||
:PROPERTIES:
|
||||
:EFFORT: 3d
|
||||
:JIRA_ID: CCPANEL-12034
|
||||
:END:
|
||||
** Normalize incoming data
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12009
|
||||
:END:
|
||||
*** Update [[id:7e503917-646f-4275-aab9-3a125b99cbfd][Tagging Service]] to normalize tags
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:JIRA_ID: CCPANEL-12010
|
||||
:END:
|
||||
*** Update [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] to normalize tags
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:JIRA_ID: CCPANEL-12011
|
||||
:END:
|
||||
*** Update TagBox to normalize tags
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12035
|
||||
:END:
|
||||
- Needs UX
|
||||
- Should this also change the display of non-normalized tags?
|
||||
*** Normalize tags when storing in rule sets
|
||||
:PROPERTIES:
|
||||
:EFFORT: 1d
|
||||
:JIRA_ID: CCPANEL-12036
|
||||
:END:
|
||||
Campaign proxy / rule service
|
||||
** Normalize existing tag data
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12013
|
||||
:EFFORT: 10d 0:00
|
||||
:END:
|
||||
|
||||
- Takes an account as input and updates tags in all tables.
|
||||
- Store existing records before modification.
|
||||
*** Update tags in rulesets to conform to normalization rules
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12014
|
||||
:EFFORT: 3d
|
||||
:END:
|
||||
*** Update tags in segments to conform to normalization rules
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12015
|
||||
:EFFORT: 3d
|
||||
:END:
|
||||
*** Update tags in the tagging database to conform to normalization rules
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12016
|
||||
:EFFORT: 2d
|
||||
:END:
|
||||
*** Update tags in the subscriber tags table to conform to normalization rules
|
||||
:PROPERTIES:
|
||||
:Effort: 2d
|
||||
:JIRA_ID: CCPANEL-12037
|
||||
:END:
|
||||
** Remove comparison normalization logic
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12038
|
||||
:END:
|
||||
*** Remove normalized comparisons from Campaign Engine
|
||||
:PROPERTIES:
|
||||
:EFFORT: 4h
|
||||
:JIRA_ID: CCPANEL-12039
|
||||
:END:
|
||||
*** Remove normalized comparisons from Analytics DB Search Terms
|
||||
:PROPERTIES:
|
||||
:EFFORT: 1d
|
||||
:JIRA_ID: CCPANEL-12040
|
||||
:END:
|
||||
** [#B] Normalize tags stored outside of tagging and campaigns
|
||||
:PROPERTIES:
|
||||
:JIRA_ID: CCPANEL-12041
|
||||
:END:
|
||||
Optional based on product decisions. Tags will be normalized when ingested.
|
||||
|
||||
This should be done, but is not a blocker for the tag normalization project.
|
||||
|
||||
(Could this be managed just by updating the display of non-normalized tags in
|
||||
TagBox?)
|
||||
*** Normalize tags when storing in Stripe
|
||||
*** Integrations
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: aa9b1fdc-d766-41bb-ab7b-11c35bca54fa
|
||||
:END:
|
||||
#+title: AWSubscribers
|
||||
|
||||
A [[id:e4d00c11-da8a-4c91-8f38-ce939846e5cb][CoreAPI]] service dedicated to mailing list subscriber data.
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 0328a202-376d-4e97-b0e3-031eaad2a557
|
||||
:END:
|
||||
#+title: Overhauling logins and session management
|
||||
|
|
@ -1,29 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: e97adcf4-86ad-4d97-9c63-41476b52b111
|
||||
:END:
|
||||
#+title: Identifying active accounts
|
||||
|
||||
A query to identify active accounts in [[id:dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea][AppDB]]
|
||||
|
||||
Per [[file:~/git/appdb/functions/coreapi_account/get_account_status.yaml][coreapi_account.get_account_status]]:
|
||||
|
||||
#+begin_src sql :exports code :eval never
|
||||
CREATE FUNCTION coreapi_account.get_account_status(in_account_id integer) RETURNS coreapi_account.account_status
|
||||
LANGUAGE sql STRICT
|
||||
AS $_$
|
||||
|
||||
SELECT accounts.status_id,
|
||||
accounts.status_id = 7,
|
||||
CASE accounts.status_id
|
||||
WHEN 1 THEN 'New Order'
|
||||
WHEN 4 THEN 'Unpaid - Un-notified'
|
||||
WHEN 5 THEN 'Paid'
|
||||
WHEN 6 THEN 'Unpaid - Notified'
|
||||
WHEN 7 THEN 'Cancelled'
|
||||
WHEN 8 THEN 'Unpaid - Overdue'
|
||||
WHEN 9 THEN 'Place Holder'
|
||||
ELSE 'Unknown'
|
||||
END
|
||||
FROM public.accounts
|
||||
WHERE accounts.a_id = $1;
|
||||
#+end_src
|
||||
|
|
@ -1,136 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 00cf1628-bc60-451d-bd30-d11d6b92992f
|
||||
:header-args:sql: :engine postgresql :cmdline "-U postgres postgres" :dir /docker:postgres: :exports both :cache yes :eval no-export
|
||||
:END:
|
||||
#+title: Null and PostgreSQL array operators
|
||||
|
||||
Researching an issue in subscriber search and broadcast sending resulted in
|
||||
learning some interesting things about [[id:af84ed59-96a4-4f9c-b34c-b79178ad20cb][PostgreSQL]]'s generation and handling of
|
||||
=NULL= values, despite its otherwise strict type checking.
|
||||
|
||||
* The symptom
|
||||
[[https://jira.aweber.io/browse/ASE-8617][ASE-8617]] describes a scenario where a customer ([[https://admin.aweber.io/account/index/1018872][AID 1018872]]) sends broadcasts to
|
||||
a segment ("no tags subscribers") on their list (List ID 5830776), defined with
|
||||
the following criteria:
|
||||
|
||||
#+caption: Segment definition: "no tags subscribers"
|
||||
| Tag | is not | wr |
|
||||
| Tag | is not | cb |
|
||||
| Tag | is not | cb2 |
|
||||
| Tag | is not | fsc |
|
||||
| Tag | is not | flp |
|
||||
| Tag | is not | lsa |
|
||||
|
||||
Normally, they expect this segment to reach >1000 subscribers. Following a
|
||||
release of our [[id:d06d3ab4-c2d0-47c3-aae1-4395567fc3d2][Tag Normalization]] changes, they sent a broadcast to this segment
|
||||
that only reached 7 subscribers. Rolling back the [[https://gitlab.aweber.io/DBA/ddl/schema-deploy/-/merge_requests/257][change to segment search terms]]
|
||||
corrected the issue for the customer.
|
||||
|
||||
* The implementation
|
||||
|
||||
** The "Tag Is Not" search filter
|
||||
The change updated the tag "is not" filter from
|
||||
#+begin_src sql :exports code :eval never
|
||||
(NOT(tags @> ARRAY[=:$1:=]) or tags is null)
|
||||
#+end_src
|
||||
to
|
||||
#+begin_src sql :exports code :eval never
|
||||
(NOT(public.normalize_tags(tags) @> public.normalize_tags(ARRAY[=:$1:=])) or tags is null)
|
||||
#+end_src
|
||||
** The =normalize_tags= function
|
||||
The =normalize_tags= function is defined as:
|
||||
#+begin_src sql :exports code :eval never
|
||||
CREATE OR REPLACE FUNCTION public.normalize_tags(in_tags text[])
|
||||
RETURNS text[]
|
||||
LANGUAGE SQL STRICT IMMUTABLE AS $$
|
||||
SELECT ARRAY_AGG(public.normalize_tag(tag))
|
||||
FROM UNNEST(in_tags) AS tag;
|
||||
$$;
|
||||
#+end_src
|
||||
* The cause
|
||||
When testing to see what the criteria SQL would evaluate to, I discovered
|
||||
something ... interesting.
|
||||
|
||||
#+begin_src sql
|
||||
SELECT (NOT(public.normalize_tags(ARRAY[]::text[]) @> ARRAY['foo']::text[]))
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[b0e468c2d8fc47126ee77eb463b2525ff9dca266]:
|
||||
| ?column? |
|
||||
|----------|
|
||||
| |
|
||||
|
||||
That hardly seems right. It's not even returning a boolean. Could it be?
|
||||
|
||||
#+begin_src sql
|
||||
SELECT (NOT(public.normalize_tags(ARRAY[]::text[]) @> ARRAY['foo']::text[])) IS NULL
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[c30ad761b00b6d3c038ec008010f17ae5228befd]:
|
||||
| ?column? |
|
||||
|----------|
|
||||
| t |
|
||||
|
||||
Yep. It's returning =NULL=, Tony Hoare's "[[https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare/][billion-dollar mistake]]". The only
|
||||
thing different in this implementation is the call to =normalize_tags=, so let's
|
||||
take a look at that.
|
||||
|
||||
#+begin_src sql
|
||||
SELECT public.normalize_tags(ARRAY[]::text[]) IS NULL
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[8a85412be393d1e19d576db8ef321f0ab1380261]:
|
||||
| ?column? |
|
||||
|----------|
|
||||
| t |
|
||||
|
||||
Sure enough, it is returning =NULL= when called with an empty array. That =NULL= then escapes to our array comparison...
|
||||
|
||||
#+begin_src sql
|
||||
SELECT (NULL @> ARRAY['foo'::text]) IS NULL
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[80d59a923b98d6ec7adc7af29a53b7557c7c0efc]:
|
||||
| ?column? |
|
||||
|----------|
|
||||
| t |
|
||||
|
||||
... which also returns =NULL=. That then gets passed to =NOT=...
|
||||
|
||||
#+begin_src sql
|
||||
SELECT (NOT(NULL)) IS NULL
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[77051c05ab0f6df48e3c13be6ff0651904f8b6da]:
|
||||
| ?column? |
|
||||
|----------|
|
||||
| t |
|
||||
|
||||
... which also returns =NULL=. =NULL= is treated as falsy by [[id:af84ed59-96a4-4f9c-b34c-b79178ad20cb][PostgreSQL]],
|
||||
and so the search fails to match subscribers without tags.
|
||||
* The resolution
|
||||
Using =COALESCE= on the result of =ARRAY_AGG= to ensure we get an empty array
|
||||
when the result is =NULL=, which lets us avoid all of the above problems caused
|
||||
by =NULL= escaping into our comparisons.
|
||||
#+begin_src sql :exports code :eval no-export
|
||||
CREATE OR REPLACE FUNCTION public.normalize_tags(in_tags text[])
|
||||
RETURNS text[]
|
||||
LANGUAGE SQL STRICT IMMUTABLE AS $$
|
||||
SELECT COALESCE(ARRAY_AGG(public.normalize_tag(tag)), ARRAY[]::text[])
|
||||
FROM UNNEST(in_tags) AS tag
|
||||
WHERE public.normalize_tag(tag) <> '';
|
||||
$$;
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[f49761b0fa0e3074cf739102361da7272672f0a1]:
|
||||
| CREATE FUNCTION |
|
||||
|-----------------|
|
||||
|
||||
#+begin_src sql
|
||||
SELECT public.normalize_tags(ARRAY[]::text[])
|
||||
#+end_src
|
||||
|
||||
#+RESULTS[a37fd5ae0eaedaf2b33b4886d60d2db0cf632fb5]:
|
||||
| normalize_tags |
|
||||
|----------------|
|
||||
| {} |
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 1ff6586e-2dba-41a2-a887-753cc5ac27c9
|
||||
:ROAM_REFS: https://gitlab.aweber.io/CP/Services/recipient
|
||||
:END:
|
||||
#+title: Recipient Service
|
||||
|
|
@ -1,150 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: ee5b8d5f-e3d4-45c2-9ce6-bcd8c7a63376
|
||||
:ROAM_REFS: https://jira.aweber.io/browse/AWEB-378
|
||||
:END:
|
||||
#+title: Retire RedCache
|
||||
|
||||
The goal of this initiative is to identify uses of the shared RedCache servers
|
||||
and eliminate them. If a key/value store is required, use a sidecar Redis or
|
||||
memcached instance as appropriate.
|
||||
|
||||
Any keys that will require cross-team coordination to remove should be
|
||||
documented in the [[https://confluence.aweber.io/display/AWD/RedCache+Inventory][RedCache Inventory]] page in Confluence.
|
||||
|
||||
* Usage in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]]
|
||||
** Caching
|
||||
These use cases treat Redis as a temporary cache. They could be safely and
|
||||
seamlessly switched over to a new Redis instance.
|
||||
|
||||
*** Cake Cache
|
||||
The default Cake framework cache engine for the application is [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/config/core.php#L236-243][configured to use
|
||||
RedCache]].
|
||||
*** Mapping
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/mapping.php][Mapping component]] caches mapping lookups.
|
||||
*** Avro Schemas
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/avro.php][Avro component]] caches schema documents.
|
||||
*** Private Labeling
|
||||
A [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/purge_pl_memcache.php][script to purge the private label cache]] exists, though it does not appear to
|
||||
be used.
|
||||
|
||||
#+begin_notes
|
||||
Private labeling is no longer used in the CP.
|
||||
#+end_notes
|
||||
*** Click Tracking
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/click_tracker.php][Click tracking component]] caches tracking url lookups in AppDB.
|
||||
*** Showcased Applications
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/application.php][Application model]] caches the number of customers using each application from
|
||||
queries to AppDB for six hours.
|
||||
*** Web Form Templates
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/web_form_template_category.php][Web Form Template Category model]] caches the top ten web form template
|
||||
families using the =template_directory_web_form_popular= key.
|
||||
*** Active Lists
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/auto_responder.php][Auto Responder model]] caches the active lists for an account using the key
|
||||
format =aweber_app_lists_{$a_id}_{$aServId}=, where =$a_id= is the integer
|
||||
account ID and =$aServId= is an integer list ID or =false=.
|
||||
*** Web Form Chicklet Images
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/form/ci/index.php][Web form chicklet image handler]] caches image data stored in AppDB using the
|
||||
key format =aweber_app_chicklet_$id= where =$id= is the integer ID of the
|
||||
chicklet image.
|
||||
*** Template Gallery
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/views/helpers/template_gallery.php][Template Gallery view helper]] caches email templates from the [[http://template-directory.service.production.consul][Template
|
||||
Directory Service]] using the =template_directory_{$type}_family_data= key format,
|
||||
where =$type= is =web_form= or =block=.
|
||||
*** List Settings
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/settings_controller.php][Settings controller]] clears cached list settings using the key pattern
|
||||
=aweber_app_{$namespace}_remove_options_{$list_id}=, where =$namespace= is the
|
||||
value retrieved from the key =aweber_app_remove_options_namespace=.
|
||||
*** Web Form Serving
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_server.php][Web Form Server component]] caches web form split tests loaded from AppDB
|
||||
using the key format =aweber_app_{$namespace}_web-form_split_{$split_id}= where
|
||||
=$namespace= is the value retrieved from the key
|
||||
=aweber_app_web_form_namespace=.
|
||||
|
||||
** Other
|
||||
These use cases treat Redis as a key/value store with specific expectations
|
||||
around if/when the key is cleared. Data migration may be necessary for a move to
|
||||
a separate Redis instance.
|
||||
*** Throttling
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/throttler.php][Throttler component]] uses cache keys with a TTL to rate-limit various actions
|
||||
in the CP including logins. An older [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/throttle.php][Throttle component]] also exists with
|
||||
references to redcache, but appears unused.
|
||||
*** Verify-Optin Processing
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine
|
||||
whether the control panel is under scheduled maintenance.
|
||||
|
||||
This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][Verify-Optin]] project, doing the
|
||||
same thing, and should be removed from the sites repository.
|
||||
*** Unsubscribe / Manage Subscriptions
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/index.htm][Manage Subscriptions page]] uses the =aweber_app_db_down= cache key to
|
||||
determine whether the control panel is under scheduled maintenance.
|
||||
|
||||
This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/unsubscribe][Unsubscribe]] project, doing the
|
||||
same thing, and should be removed from the sites repository.
|
||||
*** Refer a Friend
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/refer_a_friend.php][Refer a Friend component]] uses the =refer_a_friend:access_token= key to store
|
||||
and verify an access token.
|
||||
*** One-Click Unsubscribe
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/one_click_remove.php][One-click remove handler]] uses the =aweber_app_db_down= cache key to
|
||||
determine whether the control panel is under scheduled maintenance.
|
||||
*** Blocked Orders
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] checks and stores IP addresses for blocking orders with the
|
||||
key pattern =orders_blocked_for_{$_SERVER['REMOTE_ADDR']}=
|
||||
*** Feed Broadcasts
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/feedbroadcaster_controller.php][Feed Broadcasts controller]] tracks feed gearman job status by setting and
|
||||
retrieving the Redis key patterns =aweber_app_process_feed_{$sid}_complete= and
|
||||
=aweber_app_process_feed_{$sid}_status=, where =$sid= is the current session ID.
|
||||
*** Preferences
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/preferences.php][Preferences controller component]] stores and retrieves account-level
|
||||
preferences using the key pattern
|
||||
=aweber_app_{$namespace}_cp_preference_{$aId}=.
|
||||
*** Subscriber Search Locking
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/search_mutex.php][Search Mutex controller component]] sets and clears a lock preventing
|
||||
concurrent searches within an account using the key pattern
|
||||
=subscriber_search_lock_{$accountId}=.
|
||||
*** Login Email Verification
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores an email verification token using the key pattern
|
||||
=login_email:verification_token:{$user['id']}=
|
||||
*** Sift Login Verification
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores a flag allowing previously verified logins to
|
||||
bypass the Sift score check using the key pattern
|
||||
=login_verification:verified_bypass:{$login}:{$ipAddress}=.
|
||||
*** Application maintenance
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/app_controller.php][App controller]] uses the =aweber_app_db_down= cache key to determine whether
|
||||
the control panel is under scheduled maintenance.
|
||||
*** Web Form Generation
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_generator.php][Web Form Generator component]] stores and retrieves web form settings using the key pattern ={$namespace}_web_form_{$webFormId}{$type}= where =$namespace= is the value retrieved from the key =aweber_app_web_form_namespace=, and type is one of the following:
|
||||
|
||||
- =_js= (JavaScript)
|
||||
- =s_js= (split JavaScript)
|
||||
- =_htm=
|
||||
- =_html=
|
||||
** Unclear
|
||||
These use cases read or clear keys in the key/value store, but the keys may be
|
||||
managed elsewhere. It is unsafe to migrate these to another Redis instance until
|
||||
they are fully understood.
|
||||
*** List Twitter Account
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/vendor_account_list.php][Vendor Account List model]] clears the twitter account on a list using the key
|
||||
format =orm.list.by_id.$listId.twitter_account=. It is unclear what sets that
|
||||
key.
|
||||
*** Flagged Credit Card Numbers
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] retrieves flagged credit card numbers using the key
|
||||
=aweber_app_flagged_cc_bins=. Used to check the first six digits of a card for
|
||||
flagging.
|
||||
*** Message Templates
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/message_templates_controller.php][Message Templates controller]] clears the key
|
||||
=template_directory_block_family_data= when a message template is saved.
|
||||
*** Lead Editing
|
||||
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/leads_controller.php][Leads controller]] checks for blocked emails and email domains using the key
|
||||
patterns =0-$email= and =0-$domain=.
|
||||
* Usage in Verify-Optin
|
||||
** Control Panel Maintenance
|
||||
The [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine
|
||||
whether the control panel is under scheduled maintenance.
|
||||
* Usage in Unsubscribe
|
||||
** Control Panel Maintenance
|
||||
The [[https://gitlab.aweber.io/CP/applications/unsubscribe/-/blob/master/unsubscribe/webroot/z/r/index.php][Manage Subscriptions page]] uses the =aweber_app_db_down= cache key to
|
||||
determine whether the control panel is under scheduled maintenance.
|
||||
* Usage in [[id:03e00c18-99c0-477c-b7fb-95ddc538755e][Addlead]]
|
||||
** Control Panel Maintenance
|
||||
Uses the =aweber_app_db_down= cache key to determine whether the control panel
|
||||
is under scheduled maintenance.
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 2c1a7b1d-8726-4b88-9534-2f5abfec35f0
|
||||
:END:
|
||||
#+title: Use AppDB as the source of truth for subscriber data in Recipient
|
||||
|
||||
The [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]] shall replace [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] as the API for mailing list
|
||||
subscribers. Currently, [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] is responsible for all subscriber creation
|
||||
and updating and uses [[id:dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea][AppDB]] as its data store, which is the current source of
|
||||
truth for subscriber data. The [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]] maintains its own copy of
|
||||
subscriber data in DynamoDB along with its additional recipient data, which is
|
||||
kept up to date via the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync Consumer]]. Having both services sharing
|
||||
the same data store will allow us to migrate calls towards the [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]]
|
||||
without risking data becoming desynchronized between them.
|
||||
|
||||
|
||||
|
||||
* Read subscriber data from AppDB by default
|
||||
Campaigns may fail as they race to retrieve information from the [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient
|
||||
Service]] for a newly created subscriber, which may not have synchronized over
|
||||
from [[id:dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea][AppDB]]. Reads are also the least impactful change to make, as it will only
|
||||
affect the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync Consumer]] as it checks whether to create or update a
|
||||
record. This makes this change the ideal starting point for this project.
|
||||
|
||||
The current DynamoDB behavior will be kept intact so that the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync
|
||||
Consumer]] can continue to keep the data stored in DynamoDB up to date.
|
||||
** DONE Add support for reading subscriber data from AppDB to Recipient
|
||||
** DONE Update subscriber-sync to use flag to read from DynamoDB
|
||||
** DONE Switch recipient to read subscriber data from AppDB by default
|
||||
|
||||
* Write subscriber data to AppDB
|
||||
|
||||
* Retire subscriber data in DynamoDB
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: b285adee-2dab-48ed-b2d8-2df5594f9d30
|
||||
:ROAM_REFS: https://gitlab.aweber.io/CP/Consumers/subscriber-sync
|
||||
:END:
|
||||
#+title: Subscriber Sync Consumer
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: 9adaef5d-9cfa-424f-966b-64fd558a5122
|
||||
:END:
|
||||
#+title: Addlead Rewrite
|
||||
|
||||
A [[id:207560cc-7700-4d06-918d-cc01ae530146][Project]] to rewrite [[id:03e00c18-99c0-477c-b7fb-95ddc538755e][Addlead]] in Python as a modern, highly available service.
|
||||
|
||||
- [[id:d30d8d19-f9bd-4c98-827a-5895e3902688][Addlead Rewrite ACP]]
|
||||
|
||||
* New Product Requirements
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
:PROPERTIES:
|
||||
:ID: d30d8d19-f9bd-4c98-827a-5895e3902688
|
||||
:END:
|
||||
#+title: Addlead Rewrite ACP
|
||||
|
||||
* Purpose
|
||||
The purpose of this ACP is to replace the existing subscriber web form endpoint
|
||||
with a modern implementation using our current language, library, and deployment
|
||||
standards.
|
||||
* Problem
|
||||
Addlead is currently written in Perl, relying on old libraries, and is running
|
||||
on deprecated VM hardware deployed with Chef.
|
||||
* Current State
|
||||
Addlead is far behind our current technology stack and standard practices:
|
||||
- It is written in Perl, which is not one of our supported languages.
|
||||
Increasingly fewer engineers are familiar enough with it to make modifications
|
||||
to it.
|
||||
- Its libraries are no longer being actively maintained.
|
||||
- It is running on old hardware which would be difficult to reprovision if lost.
|
||||
- It is deployed using Chef, which we have deprecated and are actively removing
|
||||
from our stack.
|
||||
* Requirements
|
||||
* Terminology
|
||||
* Proposed Solution
|
||||
** Addlead HTTP Endpoint
|
||||
- Accept web form POST
|
||||
- Validate request
|
||||
- If validation services are unavailable, proceed as though validation
|
||||
succeeded. Validation will be applied when the event is processed.
|
||||
- Submit subscriber event to RabbitMQ
|
||||
- If RabbitMQ is unavailable, store the event in a failover queue (S3, shared
|
||||
volume?)
|
||||
** Failover Queue Consumer
|
||||
This will be a job that runs periodically to take any events in the failover
|
||||
queue and deliver them to RabbitMQ, removing them once they've been accepted by
|
||||
RabbitMQ.
|
||||
** Addlead Consumer
|
||||
Listens for subscription requests and processes them using the subscriber API.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 92 KiB |
|
|
@ -1,22 +0,0 @@
|
|||
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="242px" preserveAspectRatio="none" style="width:554px;height:242px;background:#FFFFFF;" version="1.1" viewBox="0 0 554 242" width="554px" zoomAndPan="magnify"><defs><filter height="300%" id="f1g5bqu9vn3r6s" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><rect fill="#DDDDDD" height="230.5293" style="stroke:#A80036;stroke-width:1.0;" width="85" x="395.5" y="6"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="53" x="411.5" y="18.5684">Internal</text><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="132" x2="132" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="437.5" x2="437.5" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="515.5" x2="515.5" y1="60.7988" y2="198.041"/><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="45.8457">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="217.5762">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="73" x="399.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="59" x="406.5" y="45.8457">Core API</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="73" x="399.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="59" x="406.5" y="217.5762">Core API</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="486.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="493.5" y="45.8457">Stripe</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="486.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="493.5" y="217.5762">Stripe</text><polygon fill="#A80036" points="143,88.1094,133,92.1094,143,96.1094,139,92.1094" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="137" x2="514.5" y1="92.1094" y2="92.1094"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="359" x="149" y="87.3672">POST /stripe/webhooks (customer.subscription.updated)</text><polygon fill="#A80036" points="503.5,117.4199,513.5,121.4199,503.5,125.4199,507.5,121.4199" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="509.5" y1="121.4199" y2="121.4199"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="149" x="139" y="116.6777">Fetch product metadata</text><polygon fill="#A80036" points="426,146.7305,436,150.7305,426,154.7305,430,150.7305" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="432" y1="150.7305" y2="150.7305"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="282" x="139" y="145.9883">Remove tags from subscriber or unsubscribe</text><polygon fill="#A80036" points="503.5,176.041,513.5,180.041,503.5,184.041,507.5,180.041" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="509.5" y1="180.041" y2="180.041"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="46" x="139" y="175.2988">200 OK</text><!--MD5=[550573097974f7680661b673e70c4d3c]
|
||||
@startuml
|
||||
participant "Stripe Payments (Unauthenticated)" as sp
|
||||
box "Internal"
|
||||
participant "Core API" as capi
|
||||
end box
|
||||
participant "Stripe" as stripe
|
||||
|
||||
stripe -> sp : POST /stripe/webhooks (customer.subscription.updated)
|
||||
sp -> stripe : Fetch product metadata
|
||||
sp -> capi : Remove tags from subscriber or unsubscribe
|
||||
sp -> stripe : 200 OK
|
||||
@enduml
|
||||
|
||||
PlantUML version 1.2021.10(Mon Aug 30 09:43:48 EDT 2021)
|
||||
(GPL source distribution)
|
||||
Java Runtime: Java(TM) SE Runtime Environment
|
||||
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||
Default Encoding: UTF-8
|
||||
Language: en
|
||||
Country: US
|
||||
--></g></svg>
|
||||
|
Before Width: | Height: | Size: 5.1 KiB |
|
|
@ -1,22 +0,0 @@
|
|||
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="242px" preserveAspectRatio="none" style="width:638px;height:242px;background:#FFFFFF;" version="1.1" viewBox="0 0 638 242" width="638px" zoomAndPan="magnify"><defs><filter height="300%" id="fd0b0cv41yrbd" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><rect fill="#DDDDDD" height="230.5293" style="stroke:#A80036;stroke-width:1.0;" width="93" x="471.5" y="6"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="53" x="491.5" y="18.5684">Internal</text><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="132" x2="132" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="517.5" x2="517.5" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="599.5" x2="599.5" y1="60.7988" y2="198.041"/><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="45.8457">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="217.5762">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="81" x="475.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="67" x="482.5" y="45.8457">RabbitMQ</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="81" x="475.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="67" x="482.5" y="217.5762">RabbitMQ</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="570.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="577.5" y="45.8457">Stripe</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="570.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="577.5" y="217.5762">Stripe</text><polygon fill="#A80036" points="143,88.1094,133,92.1094,143,96.1094,139,92.1094" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="137" x2="598.5" y1="92.1094" y2="92.1094"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="330" x="149" y="87.3672">POST /stripe/webhooks (payment_intent.succeeded)</text><polygon fill="#A80036" points="506,117.4199,516,121.4199,506,125.4199,510,121.4199" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="512" y1="121.4199" y2="121.4199"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="215" x="139" y="116.6777">Sales tracking event (pageview.v4)</text><polygon fill="#A80036" points="506,146.7305,516,150.7305,506,154.7305,510,150.7305" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="512" y1="150.7305" y2="150.7305"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="362" x="139" y="145.9883">Payment succeeded event (stripe_payment_succeeded.v1)</text><polygon fill="#A80036" points="587.5,176.041,597.5,180.041,587.5,184.041,591.5,180.041" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="593.5" y1="180.041" y2="180.041"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="46" x="139" y="175.2988">200 OK</text><!--MD5=[75ac5feda4e5cecbf340b2717033f540]
|
||||
@startuml
|
||||
participant "Stripe Payments (Unauthenticated)" as sp
|
||||
box "Internal"
|
||||
participant "RabbitMQ" as amqp
|
||||
end box
|
||||
participant "Stripe" as stripe
|
||||
|
||||
stripe -> sp : POST /stripe/webhooks (payment_intent.succeeded)
|
||||
sp -> amqp : Sales tracking event (pageview.v4)
|
||||
sp -> amqp : Payment succeeded event (stripe_payment_succeeded.v1)
|
||||
sp -> stripe : 200 OK
|
||||
@enduml
|
||||
|
||||
PlantUML version 1.2021.10(Mon Aug 30 09:43:48 EDT 2021)
|
||||
(GPL source distribution)
|
||||
Java Runtime: Java(TM) SE Runtime Environment
|
||||
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||
Default Encoding: UTF-8
|
||||
Language: en
|
||||
Country: US
|
||||
--></g></svg>
|
||||
|
Before Width: | Height: | Size: 5.2 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 11 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 9.9 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 744 B |
Binary file not shown.
|
Before Width: | Height: | Size: 583 B |
Binary file not shown.
|
Before Width: | Height: | Size: 578 B |
Binary file not shown.
|
Before Width: | Height: | Size: 577 B |
Binary file not shown.
|
Before Width: | Height: | Size: 722 B |
Binary file not shown.
|
Before Width: | Height: | Size: 866 B |
Binary file not shown.
|
Before Width: | Height: | Size: 494 B |
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue