Add aweber roam files
This commit is contained in:
parent
23301f89d1
commit
9775046d26
90 changed files with 7001 additions and 0 deletions
14
aweber/20200713131302-migration_to_common_rabbitmq.org
Normal file
14
aweber/20200713131302-migration_to_common_rabbitmq.org
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: ac416861-ce45-49ac-8b60-f8ea39362135
|
||||||
|
:END:
|
||||||
|
#+title: Migration to common RabbitMQ
|
||||||
|
|
||||||
|
All services and consumers pointed at the legacy RabbitMQ cluster in
|
||||||
|
Conshohocken should be migrated to the new common-rabbitmq cluster.
|
||||||
|
|
||||||
|
The new servers are available at
|
||||||
|
=common-rabbitmq.service.${ENVIRONMENT}.consul=.
|
||||||
|
|
||||||
|
* Legacy hostnames
|
||||||
|
- =rabbitmq.service.${ENVIRONMENT}.consul=
|
||||||
|
- =rabbit{1-3}.int.{stg,prd}.csh=
|
||||||
4
aweber/20200713131512-coreapi.org
Normal file
4
aweber/20200713131512-coreapi.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: e4d00c11-da8a-4c91-8f38-ce939846e5cb
|
||||||
|
:END:
|
||||||
|
#+title: CoreAPI
|
||||||
10
aweber/20200713162108-puppet.org
Normal file
10
aweber/20200713162108-puppet.org
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: ddeea682-c8f0-4607-8e2b-0f8ee4fd6191
|
||||||
|
:END:
|
||||||
|
#+title: Puppet
|
||||||
|
|
||||||
|
- Repository :: [[https://gitlab.aweber.io/PSE/config-management/puppet/]]
|
||||||
|
|
||||||
|
* Applying changes on a node
|
||||||
|
- Ensure changes are merged and tagged in the upstream repository.
|
||||||
|
- As root, run =puppetd --test=.
|
||||||
4
aweber/20200714100836-pgbouncer_port_migration.org
Normal file
4
aweber/20200714100836-pgbouncer_port_migration.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 592aa825-154c-4659-8193-75b0ce1f2e5c
|
||||||
|
:END:
|
||||||
|
#+title: PGBouncer port migration
|
||||||
7
aweber/20200714125028-pagerduty.org
Normal file
7
aweber/20200714125028-pagerduty.org
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: ebea379a-8fa6-4e22-9275-a9fc98c02804
|
||||||
|
:END:
|
||||||
|
#+title: Pagerduty
|
||||||
|
|
||||||
|
https://aweber.pagerduty.com/
|
||||||
|
|
||||||
8
aweber/20200714130151-mail_relay.org
Normal file
8
aweber/20200714130151-mail_relay.org
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 24578fe5-6ca0-4000-a7cd-201e952e4c76
|
||||||
|
:END:
|
||||||
|
#+title: Mail Relay
|
||||||
|
|
||||||
|
A postfix instance running in Kubernetes for the express purpose of supporting
|
||||||
|
legacy emails in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] until they are all migrated to use [[id:32c66bc8-a397-4f50-96cd-2aec70dd14c5][Corporate
|
||||||
|
Notifications]].
|
||||||
4
aweber/20200714141221-momentum.org
Normal file
4
aweber/20200714141221-momentum.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: e1b95d0e-366e-4ecf-b867-409b6b6c6ee8
|
||||||
|
:END:
|
||||||
|
#+title: Momentum
|
||||||
4
aweber/20200714141314-control_panel.org
Normal file
4
aweber/20200714141314-control_panel.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89
|
||||||
|
:END:
|
||||||
|
#+title: Control Panel
|
||||||
4
aweber/20200714141333-corporate_notifications.org
Normal file
4
aweber/20200714141333-corporate_notifications.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 32c66bc8-a397-4f50-96cd-2aec70dd14c5
|
||||||
|
:END:
|
||||||
|
#+title: Corporate Notifications
|
||||||
11
aweber/20200714143255-refunding_an_order.org
Normal file
11
aweber/20200714143255-refunding_an_order.org
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 7a362881-875f-4f74-8053-55f63826da63
|
||||||
|
:END:
|
||||||
|
#+title: Refunding an Order
|
||||||
|
|
||||||
|
Refund options (via Admin)
|
||||||
|
1. Use a balance item
|
||||||
|
1. Insert a manual balance item, set Term at 1, a negative amount, and optionally a description.
|
||||||
|
2. Add an invoice and pay it.
|
||||||
|
2. Close the current package
|
||||||
|
1. Open the current package, select cancellation in date closed, with a cancel reason
|
||||||
46
aweber/20200714212153-login_throttling.org
Normal file
46
aweber/20200714212153-login_throttling.org
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: d17e934b-b340-4246-88f0-9b36527100c0
|
||||||
|
:END:
|
||||||
|
#+title: Login Throttling
|
||||||
|
|
||||||
|
* CAPTCHA Throttling
|
||||||
|
|
||||||
|
We have login captcha throttling in place for the following:
|
||||||
|
| Tracked behavior | CAPTCHA threshold | Time Interval |
|
||||||
|
|-------------------------------------------------------+-------------------+---------------|
|
||||||
|
| Repeated unsuccessful attempts with the same username | 3 attempts | 10 minutes |
|
||||||
|
| Repeated attempts from the same IP address | 3 attempts | 12 hours |
|
||||||
|
| Repeated attempts using the same Sift ID | 3 attempts | 30 minutes |
|
||||||
|
| Invalid or missing CSRF token | Immediate | N/A |
|
||||||
|
| Missing customer cookie | Immediate | N/A |
|
||||||
|
|
||||||
|
When a user meets one of the thresholds above, they will be presented with a
|
||||||
|
CAPTCHA challenge. This does not necessarily mean a puzzle will have to be
|
||||||
|
solved, only that the CAPTCHA script will attempt to determine if the user is a
|
||||||
|
bot. Even if the user has correctly entered their credentials on the subsequent
|
||||||
|
attempt, the CAPTCHA challenge will still occur.
|
||||||
|
|
||||||
|
All of the above thresholds are checked concurrently for each login attempt.
|
||||||
|
|
||||||
|
When a throttled user logs in successfully, the following occurs, the *username*
|
||||||
|
threshold is reset. No other thresholds are cleared. This means that even after
|
||||||
|
a user is able to successfully log in to an account, it is still possible for
|
||||||
|
them to be throttled after failing to log in again because they are now being
|
||||||
|
throttled by IP address.
|
||||||
|
|
||||||
|
* Sift ID Blocking
|
||||||
|
During previous login attacks, we've documented a set of Sift IDs that have been
|
||||||
|
used repeatedly during those attempts. Those IDs are blocked with CAPTCHA
|
||||||
|
*immediately*, with a 20% chance that we will present them a faked successful
|
||||||
|
response. This is done to throw off attackers using these IDs.
|
||||||
|
|
||||||
|
* Code
|
||||||
|
All the captcha / throttling logic that’s currently in place lives in
|
||||||
|
[[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][aweber_app/controllers/account_controller.php]], mainly in the =loginAjax= and
|
||||||
|
=isThrottled= methods. The repeated actions are tracked using
|
||||||
|
[[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/throttler.php][php5-vendors/vendors/throttler.php]], which uses counters in Redis with a TTL
|
||||||
|
attached.
|
||||||
|
|
||||||
|
* Graphs
|
||||||
|
Login attempts and throttling are graphed in Grafana on the [[https://grafana.aweber.io/d/000000530/account-logins][Account Logins
|
||||||
|
dashboard]].
|
||||||
70
aweber/20201022141542-python_services.org
Normal file
70
aweber/20201022141542-python_services.org
Normal file
|
|
@ -0,0 +1,70 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: a81b2ff0-5ede-44b3-8f82-960357f15428
|
||||||
|
:END:
|
||||||
|
#+title: Python Services
|
||||||
|
#+OPTIONS: ^:nil
|
||||||
|
#+PROPERTY: header-args :exports code
|
||||||
|
|
||||||
|
* Platform
|
||||||
|
- Python 3.9[fn:programming-languages]
|
||||||
|
- Tornado 6[fn:frameworks]
|
||||||
|
|
||||||
|
* Code Style
|
||||||
|
Code style must be enforced using flake8.[fn:python-lint-checking]
|
||||||
|
|
||||||
|
#+begin_example
|
||||||
|
[flake8]
|
||||||
|
application-import-names = PACKAGE_NAME,tests
|
||||||
|
exclude = build,env
|
||||||
|
import-order-style = pycharm
|
||||||
|
#+end_example
|
||||||
|
* Requirements
|
||||||
|
** Uncaught errors are logged and alerted via Sentry
|
||||||
|
#+NAME: packages
|
||||||
|
- =sentry-sdk=
|
||||||
|
|
||||||
|
#+begin_src python
|
||||||
|
from sentry_sdk import init
|
||||||
|
init(SENTRY_DSN)
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
** A status endpoint is exposed
|
||||||
|
The endpoint should be provide the following fields:
|
||||||
|
- application :: The name of the service
|
||||||
|
- environment :: The operating environment the instance of the service is
|
||||||
|
running in (i.e. "development", "testing", "staging" or "production")
|
||||||
|
- status :: Current service status (e.g.: "ok", "starting")
|
||||||
|
- version :: Packaged version of the service instance
|
||||||
|
- python_version :: The python version running the service instance
|
||||||
|
(=platform.python_version()=)
|
||||||
|
|
||||||
|
The endpoint should return =200= when the service is healthy, and =503= if the
|
||||||
|
service is not ready to serve requests.
|
||||||
|
** The service self-hosts its API documentation
|
||||||
|
An OpenAPI specification is hosted using ReDoc at the root service URL.[fn:backend-services]
|
||||||
|
** Test coverage reports are available in SonarQube
|
||||||
|
** Structured logging
|
||||||
|
json-scribe
|
||||||
|
** Provides consistent error responses
|
||||||
|
json-problem
|
||||||
|
** The service represents itself using its service name and version
|
||||||
|
- The service must include its name and version in its =Server= response header.[fn:response-headers]
|
||||||
|
- The service must include its name and version in the =User-Agent= header for all its HTTP requests.[fn:user-agent]
|
||||||
|
|
||||||
|
Both of these should be presented as =${service-name}/${version}=, e.g.:
|
||||||
|
=user-management/1.0.0=.
|
||||||
|
* References
|
||||||
|
- [[https://confluence.aweber.io/display/STD/Back+End+Services]]
|
||||||
|
|
||||||
|
* Footnotes
|
||||||
|
|
||||||
|
[fn:user-agent] https://confluence.aweber.io/display/STD/RESTful+APIs#heading-User-AgentRequestHeader
|
||||||
|
[fn:response-headers] https://confluence.aweber.io/display/STD/RESTful+APIs#heading-ResponseHeaders
|
||||||
|
|
||||||
|
[fn:backend-services] https://confluence.aweber.io/display/STD/Back+End+Services
|
||||||
|
|
||||||
|
[fn:frameworks] https://confluence.aweber.io/display/STD/Development+Frameworks
|
||||||
|
|
||||||
|
[fn:python-lint-checking] https://confluence.aweber.io/display/STD/Python+Lint+Checking
|
||||||
|
|
||||||
|
[fn:programming-languages] https://confluence.aweber.io/display/STD/Programming+Languages
|
||||||
58
aweber/20201028130030-easy_commerce_mvp_brainstorm_notes.org
Normal file
58
aweber/20201028130030-easy_commerce_mvp_brainstorm_notes.org
Normal file
|
|
@ -0,0 +1,58 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: dcb2f0ad-72e8-41ff-84f5-07caf8c7fe8e
|
||||||
|
:END:
|
||||||
|
#+title: Easy Commerce MVP Brainstorm Notes
|
||||||
|
#+TODO: ASK(a) FOLLOW-UP(f) | ANSWERED(d)
|
||||||
|
#+DATE: <2020-10-29 Thu>
|
||||||
|
|
||||||
|
* ANSWERED How is a product identified and tracked?
|
||||||
|
- The customer configures a product name and price on the landing page.
|
||||||
|
- The product name is the "goal description" (page description or note in the
|
||||||
|
DB) in our current sales tracking.
|
||||||
|
- We'd like to include the product name, price, and URL in the subscriber's
|
||||||
|
activity.
|
||||||
|
- Full, separate product tracking could be something we build later if there is
|
||||||
|
sufficient interest.
|
||||||
|
|
||||||
|
* ANSWERED How will the Stripe integration be configured?
|
||||||
|
Will it be configured at the account level, and if so, could it leverage the
|
||||||
|
existing system for linking things like FB & Twitter?
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
- Stripe links via OAuth.
|
||||||
|
- We will have only one Stripe account per AWeber account.
|
||||||
|
- We will be logging charges to Stripe, not full orders (with product tracking
|
||||||
|
information).
|
||||||
|
- Investigation on whether the existing system will work is pending.
|
||||||
|
* ANSWERED How does this interact with service limits?
|
||||||
|
- Availability of the easy commerce feature
|
||||||
|
- Subscriber limits?
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
- Handle errors with free-tier subscriber limits.
|
||||||
|
- Consider transaction service fee changes based on service limits.
|
||||||
|
* ANSWERED When should the buyer be added as a subscriber?
|
||||||
|
Immediately upon payment submission, or upon asynchronous payment confirmation?
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
- The async web hook confirms the payment was made successfully.
|
||||||
|
+ We're avoiding putting logic here only to limit the scope of adding storage
|
||||||
|
for subscriber and purchase information to react to the event with.
|
||||||
|
- If the subscriber already exists, we'll update them with any new information.
|
||||||
|
* ANSWERED How will reports be handled?
|
||||||
|
Is there a dependency on the upcoming Analytics View service?
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
- Only add All Lists: Products Sold in MVP
|
||||||
|
+ This seems like it'd just be a filter on event type in our current sales
|
||||||
|
over time report.
|
||||||
|
- The Analytics View service will provide report data via the authenticated
|
||||||
|
public API interface.
|
||||||
|
+ May not need to be coupled to the project, as we may be able to filter the
|
||||||
|
existing report to distinguish between current sales tracking and landing
|
||||||
|
page sales.
|
||||||
|
+ Anything /new/ should be built using the new Analytics View service.
|
||||||
53
aweber/20210112104838-stripe_payments_service.org
Normal file
53
aweber/20210112104838-stripe_payments_service.org
Normal file
|
|
@ -0,0 +1,53 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 9b3ed74d-41d4-4784-89e3-6a9183903b9e
|
||||||
|
:END:
|
||||||
|
#+title: Stripe Payments Service
|
||||||
|
* Overview
|
||||||
|
The Stripe Payments service mediates purchases made by buyers from AWeber
|
||||||
|
customers through the [[id:7d940785-68b9-4da7-bad1-4771d496168c][Stripe payment platform]] and delivers their goods via list
|
||||||
|
subscription.
|
||||||
|
* How It Works
|
||||||
|
** Ordering
|
||||||
|
The order endpoint prepares a [[https://stripe.com/docs/api/payment_intents][Payment Intent]] with Stripe that the buyer will pay
|
||||||
|
directly using Stripe's public API. The payment intent captures the amount to be
|
||||||
|
paid, any fees that AWeber will collect, and account and list metadata used to
|
||||||
|
fulfill the order.
|
||||||
|
*** Email validation
|
||||||
|
Emails are checked against the [[https://gitlab.aweber.io/CP/Services/validation][CP Email Validation Service]] to ensure they can be
|
||||||
|
added as subscribers successfully. The following steps are performed by the service:
|
||||||
|
1. Validates that the format of the submitted email address matches AWeber's
|
||||||
|
internal email formatting rules.
|
||||||
|
2. Applies address normalization to remove any ISP specific markup.
|
||||||
|
3. Extracts the domain-part from the normalized address and validates that there
|
||||||
|
are MX records for the domain-part via DNS lookup.
|
||||||
|
4. Checks if the both the submitted and normalized version of the email address
|
||||||
|
would be blocked by the blocklist.
|
||||||
|
|
||||||
|
Should the validation service fail unexpectedly or be unavailable at the time an
|
||||||
|
order is placed, a warning will be logged and only the email format check will
|
||||||
|
be performed.
|
||||||
|
*** CAPTCHA
|
||||||
|
**** Domain validation
|
||||||
|
***** Allowed domains
|
||||||
|
The following root domains provided by AWeber for [[https://confluence.aweber.io/pages/viewpage.action?pageId=118885193][Web Content (a.k.a Landing
|
||||||
|
Pages)]] always pass CAPTCHA domain validation:
|
||||||
|
|
||||||
|
- =aweberpages.com=
|
||||||
|
- =aweb.page=
|
||||||
|
***** Custom domains
|
||||||
|
All other domains are checked against the [[https://confluence.aweber.io/display/AR/Custom+Domain+Service][Custom Domain Service]] to ensure they
|
||||||
|
are owned and managed by the AWeber account from which the purchase is being
|
||||||
|
made.
|
||||||
|
*** Fees
|
||||||
|
Fees are collected for each sale as a percentage of the sale value configured in
|
||||||
|
the [[https://confluence.aweber.io/display/CT/Service+Limits+API][Service Limits API]] for the AWeber account, rounded to the nearest cent with
|
||||||
|
a minimum fee of $0.01.
|
||||||
|
** Fulfillment
|
||||||
|
*** Adding the subscriber
|
||||||
|
Subscribers are added to the list configured for the payment if they are not
|
||||||
|
already subscribed. If the subscriber is currently on the list in an unconfirmed
|
||||||
|
state, they will be marked as subscribed. Any tags configured for the sale will
|
||||||
|
be added to the subscriber.
|
||||||
|
*** Purchase Tracking
|
||||||
|
A [[https://confluence.aweber.io/display/AR/Pageview][Pageview Event]] is emitted with details of the order so that the purchase is
|
||||||
|
tracked in the subscriber's activity.
|
||||||
448
aweber/20210127141844-recurring_and_split_stripe_payments.org
Normal file
448
aweber/20210127141844-recurring_and_split_stripe_payments.org
Normal file
|
|
@ -0,0 +1,448 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: a9835afc-e0be-4436-8274-c3898fdf119c
|
||||||
|
:END:
|
||||||
|
#+title: Recurring and split Stripe payments
|
||||||
|
#+options: prop:t
|
||||||
|
|
||||||
|
To be implemented in the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||||
|
|
||||||
|
Recurring and split payments are implementable in Stripe as [[https://stripe.com/docs/billing/subscriptions/overview#subscription-lifecycle][subscriptions]]. The
|
||||||
|
key difference is that split payments have a limited number of payment
|
||||||
|
iterations (e.g. 5 monthly installments) whereas recurring payments are ongoing.
|
||||||
|
* Stripe objects
|
||||||
|
** Products
|
||||||
|
Represents something being sold.
|
||||||
|
** Prices
|
||||||
|
Amount and frequency to be charged for a product. Many prices may be available
|
||||||
|
for a single product.
|
||||||
|
** Customers
|
||||||
|
Represents a buyer (subscriber?). Needs to be configured with a payment method.
|
||||||
|
** Subscriptions
|
||||||
|
Represents a [[*Products][Product]] being offered to a [[* Customers][Customer]] at a [[* Prices][Price]].
|
||||||
|
** Invoices
|
||||||
|
Generated at billing times.
|
||||||
|
** Payment Intents
|
||||||
|
Attempts to pay an [[* Invoices][Invoice]].
|
||||||
|
* Easy Commerce Flow
|
||||||
|
** Setup
|
||||||
|
[[*Products][Products]] and [[* Prices][Prices]] should be set up in the customer's stripe account and
|
||||||
|
referenced in their landing page.
|
||||||
|
|
||||||
|
#+begin_src plantuml :file "ecommerce-products.svg"
|
||||||
|
actor "AWeber Customer" as customer
|
||||||
|
participant "Landing Page Editor" as lp
|
||||||
|
participant "Stripe Payments (Authenticated)" as sp
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
== Load product information ==
|
||||||
|
customer -> lp : Edit landing page
|
||||||
|
lp -> sp : GET /stripe-authenticated/products
|
||||||
|
sp -> stripe : Get products with prices
|
||||||
|
sp -> lp : Return list of products with pricing
|
||||||
|
|
||||||
|
== Save product information ==
|
||||||
|
customer -> lp : Save landing page
|
||||||
|
alt create a new product & price
|
||||||
|
lp -> sp : POST /stripe-authenticated/products
|
||||||
|
else update an existing product & price
|
||||||
|
lp -> sp : PATCH /stripe-authenticated/products/{UUID}
|
||||||
|
end
|
||||||
|
sp -> stripe : Store product and price
|
||||||
|
sp -> lp : 200 Return product and price IDs
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:ecommerce-products.svg]]
|
||||||
|
** Purchase
|
||||||
|
#+begin_src plantuml :file "ecommerce-subscribing.svg"
|
||||||
|
actor "Buyer" as buyer
|
||||||
|
participant "Landing Page" as lp
|
||||||
|
participant "Stripe Payments (Unauthenticated)" as sp
|
||||||
|
box "Internal"
|
||||||
|
participant "Core API" as capi
|
||||||
|
end box
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
buyer -> lp : Place order
|
||||||
|
lp -> sp : POST /stripe/subscription with product and price IDs
|
||||||
|
sp -> stripe : Create customer
|
||||||
|
sp -> stripe : Create subscription
|
||||||
|
sp -> stripe : Create initial invoice
|
||||||
|
sp -> lp : Return payment intent client secret
|
||||||
|
|
||||||
|
lp -> stripe : Confirm payment
|
||||||
|
stripe -> lp : Return payment intent
|
||||||
|
|
||||||
|
lp -> sp : POST /stripe/fulfillment
|
||||||
|
sp -> stripe : Fetch payment intent and check status
|
||||||
|
sp -> capi : Add subscriber with tags
|
||||||
|
sp -> lp : 200 OK
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:ecommerce-subscribing.svg]]
|
||||||
|
|
||||||
|
*** Subscribing
|
||||||
|
Unlike ordering, which creates a single payment intent, we'll instead need to
|
||||||
|
take the payment information provided by the buyer and create a [[* Customers][Customer]] object
|
||||||
|
in Stripe, as well as a [[* Subscriptions][Subscription]] to a [[*Products][Product]] at a specific [[* Prices][Price]].
|
||||||
|
*** Fulfillment
|
||||||
|
After a successful payment, the [[* Subscriptions][Subscription]] is marked as =active=, and the
|
||||||
|
buyer should be subscribed to start receiving their content.
|
||||||
|
** Automatic payments
|
||||||
|
#+begin_src plantuml :file "ecommerce-payment-webhooks.svg"
|
||||||
|
participant "Stripe Payments (Unauthenticated)" as sp
|
||||||
|
box "Internal"
|
||||||
|
participant "RabbitMQ" as amqp
|
||||||
|
end box
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
stripe -> sp : POST /stripe/webhooks (payment_intent.succeeded)
|
||||||
|
sp -> amqp : Sales tracking event (pageview.v4)
|
||||||
|
sp -> amqp : Payment succeeded event (stripe_payment_succeeded.v1)
|
||||||
|
sp -> stripe : 200 OK
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:ecommerce-payment-webhooks.svg]]
|
||||||
|
|
||||||
|
** Payment failure / Cancellation
|
||||||
|
#+begin_src plantuml :file "ecommerce-cancellation-webhooks.svg"
|
||||||
|
participant "Stripe Payments (Unauthenticated)" as sp
|
||||||
|
box "Internal"
|
||||||
|
participant "Core API" as capi
|
||||||
|
end box
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
stripe -> sp : POST /stripe/webhooks (customer.subscription.updated)
|
||||||
|
sp -> stripe : Fetch product metadata
|
||||||
|
sp -> capi : Remove tags from subscriber or unsubscribe
|
||||||
|
sp -> stripe : 200 OK
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:ecommerce-cancellation-webhooks.svg]]
|
||||||
|
|
||||||
|
* Questions
|
||||||
|
- Should stripe-payments or the integrations service handle price and product management?
|
||||||
|
+ stripe-payments would need authenticated endpoints exposed
|
||||||
|
|
||||||
|
* Project :taskjuggler_project:
|
||||||
|
:PROPERTIES:
|
||||||
|
:COLUMNS: %50ITEM(Task) %Effort %allocate %task_id %blocker
|
||||||
|
:start: 2021-02-22
|
||||||
|
:END:
|
||||||
|
|
||||||
|
#+begin: columnview
|
||||||
|
| Task | Effort | allocate | task_id | blocker |
|
||||||
|
|-------------------------------------------------------------------+--------+-------------------+-------------------------+-------------------------------------------------------------------------------|
|
||||||
|
| Project | | | | |
|
||||||
|
| Tasks | | | | |
|
||||||
|
| Backend: Product Management | | | | |
|
||||||
|
| Define API for managing products and prices | 1d | backend | api | |
|
||||||
|
| Endpoint: Create products and prices | 3d | backend | create | api |
|
||||||
|
| Endpoint: Update products and prices | 5d | backend | update | api |
|
||||||
|
| Endpoint: Fetch products and prices | 2d | backend | fetch | api |
|
||||||
|
| Endpoint: Update /order | 2d | backend | order_both | api |
|
||||||
|
| Migrate products from existing landing pages into Stripe on save | | frontend | migrate | order_both |
|
||||||
|
| Backfill products from existing landing pages into Stripe | 10d | backend, frontend | backfill | order_both |
|
||||||
|
| Backend: Subscriptions | | | | |
|
||||||
|
| Endpoint: Update /order to create customers for product purchases | 10d | backend | order_products | milestone_management |
|
||||||
|
| Endpoint: Add subscription support to /order | 10d | backend | order_subscriptions | milestone_management |
|
||||||
|
| Endpoint: Update /fulfill & /webhooks (iteration 1) | 3d | backend | webhooks_1 | milestone_management |
|
||||||
|
| Endpoint: Update /webhooks to process subscriptions (iteration 2) | 8d | backend | webhooks_2 | webhooks_1, order_subscriptions |
|
||||||
|
| Database: Add database for event tracking | 3d | backend | database | |
|
||||||
|
| Endpoint: Add event tracking to /webhooks | 5d | backend | tracking | database, webhooks_2 |
|
||||||
|
| Backend: Resiliency | | | | |
|
||||||
|
| Poller: Create poller to check for unprocessed events | 10d | backend | poller | milestone_subscriptions |
|
||||||
|
| Backend: Cleanup | | | | |
|
||||||
|
| Backfill products from existing landing pages into Stripe | 10d | backend, frontend | backfill2 | milestone_subscriptions |
|
||||||
|
| Endpoint: Update /order remove /fulfill | 1d | backend | cleanup | backfill2 |
|
||||||
|
| Frontend | | | | |
|
||||||
|
| Frontend: Plugin | 2d | frontend | plugin | |
|
||||||
|
| Frontend: Templates | 1d | frontend | templates | plugin |
|
||||||
|
| Frontend: Builder (Product Management) | 12d | frontend | builder_products | plugin, templates |
|
||||||
|
| Frontend: Builder (Recurring Payments) | 17d | frontend | builder_recurring | milestone_management |
|
||||||
|
| Milestones | | | | |
|
||||||
|
| Milestone 1: Product Management | 5d | release | milestone_management | api, create, update, fetch, order_both, migrate, backfill, builder_products |
|
||||||
|
| Milestone 2: Support Subscriptions | 5d | release | milestone_subscriptions | order_products, webhooks_1, webhooks_2, database, tracking, builder_recurring |
|
||||||
|
| Milestone 3: Add Resiliency | | | milestone_resiliency | poller |
|
||||||
|
| Milestone 4: Deprecate Product Name/Price | | | milestone_cleanup | milestone_subscriptions, cleanup |
|
||||||
|
#+end:
|
||||||
|
|
||||||
|
#+begin_export html
|
||||||
|
<iframe src="reports/recurring-and-split-stripe-payments/Plan.html" width="100%" height="1000"></iframe>
|
||||||
|
#+end_export
|
||||||
|
** Tasks
|
||||||
|
|
||||||
|
*** Backend: Product Management
|
||||||
|
**** DONE Define API for managing products and prices
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 1d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:task_id: api
|
||||||
|
:END:
|
||||||
|
Probably makes sense to treat price + recurrence as product attributes, e.g.
|
||||||
|
|
||||||
|
- name :: String
|
||||||
|
- Price :: Object
|
||||||
|
+ amount :: Integer
|
||||||
|
+ currency :: Enum["usd"]
|
||||||
|
+ recurrence :: Optional[Object]
|
||||||
|
- interval :: Enum["weekly", "monthly", "yearly"]
|
||||||
|
- times :: Union["unlimited", Integer]
|
||||||
|
|
||||||
|
**** DONE Endpoint: Create products and prices
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 3d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: api
|
||||||
|
:TASK_ID: create
|
||||||
|
:END:
|
||||||
|
- Define metadata schema for purchase and cancellation actions (add tags, unsubscribe)
|
||||||
|
|
||||||
|
**** DONE Endpoint: Update products and prices
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 5d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: api
|
||||||
|
:TASK_ID: update
|
||||||
|
:END:
|
||||||
|
- Add new price object if the price or recurrence has changed, leaving the old one
|
||||||
|
|
||||||
|
**** DONE Endpoint: Fetch products and prices
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 2d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: api
|
||||||
|
:TASK_ID: fetch
|
||||||
|
:END:
|
||||||
|
|
||||||
|
**** DONE Endpoint: Update /order
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 2d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: api
|
||||||
|
:task_id: order_both
|
||||||
|
:END:
|
||||||
|
|
||||||
|
**** DONE Migrate products from existing landing pages into Stripe on save
|
||||||
|
:PROPERTIES:
|
||||||
|
:ALLOCATE: frontend
|
||||||
|
:blocker: order_both
|
||||||
|
:TASK_ID: migrate
|
||||||
|
:END:
|
||||||
|
**** CANCELLED Backfill products from existing landing pages into Stripe
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 10d
|
||||||
|
:ALLOCATE: backend, frontend
|
||||||
|
:blocker: order_both
|
||||||
|
:task_id: backfill
|
||||||
|
:END:
|
||||||
|
**** DONE Separate Products and Prices
|
||||||
|
*** Backend: Subscriptions
|
||||||
|
**** TODO Endpoint: Create /purchase to create customers for product purchases
|
||||||
|
:PROPERTIES:
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: milestone_management
|
||||||
|
:start: 2021-04-22
|
||||||
|
:Effort: 12d
|
||||||
|
:TASK_ID: order_products
|
||||||
|
:END:
|
||||||
|
If the product being ordered is specified by id:
|
||||||
|
- Create the customer w/ the provided payment method
|
||||||
|
- Create an invoice
|
||||||
|
- Pay the invoice
|
||||||
|
- Document the /order endpoint as deprecated
|
||||||
|
- Create acceptance tests
|
||||||
|
|
||||||
|
**** TODO Endpoint: Update /fulfill and /webhooks (iteration 1)
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 5d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:BLOCKER: milestone_management
|
||||||
|
:task_id: webhooks_1
|
||||||
|
:start: 2021-04-22
|
||||||
|
:END:
|
||||||
|
- move add subscriber and sales tracking event emission to the webhook endpoint
|
||||||
|
- no-op /fulfill
|
||||||
|
- Document /fulfill endpoint as deprecated
|
||||||
|
- Send =AWeber-Options: update-unconfirmed= header
|
||||||
|
- Update acceptance tests to account for out-of-band fulfillment
|
||||||
|
**** TODO Endpoint: Add subscription support to /purchase
|
||||||
|
:PROPERTIES:
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: milestone_management
|
||||||
|
:start: 2021-04-22
|
||||||
|
:Effort: 12d
|
||||||
|
:TASK_ID: order_subscriptions
|
||||||
|
:END:
|
||||||
|
If the product being ordered has a recurring price type:
|
||||||
|
- Create the customer w/ the provided payment method
|
||||||
|
- Create a subscription
|
||||||
|
- Create the initial invoice
|
||||||
|
- Pay the initial invoice
|
||||||
|
- Add acceptance tests for subscription purchases
|
||||||
|
**** TODO Endpoint: Update /webhooks to process subscriptions (iteration 2)
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 10d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: webhooks_1, order_subscriptions
|
||||||
|
:TASK_ID: webhooks_2
|
||||||
|
:END:
|
||||||
|
- Add logic to process a subscription
|
||||||
|
- Add the events corresponding to cancellation of a subscription
|
||||||
|
+ Process the remove tag/unsubscribe actions that are saved as metadata on the
|
||||||
|
subscription
|
||||||
|
+ Ensure that unconfirmed subscribers can be unsubscribed
|
||||||
|
- Need a way to differentiate fulfillment of a payment intent for a subscription vs a single payment
|
||||||
|
+ Pull the invoice for the payment intent with the expanded subscription details if they exist
|
||||||
|
**** TODO Database: Add database for event tracking
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 3d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:BLOCKER: milestone_management
|
||||||
|
:start: 2021-04-22
|
||||||
|
:task_id: database
|
||||||
|
:END:
|
||||||
|
- new postgres users ( one for the poller, one for the stripe-payments service )
|
||||||
|
- Needs the following information in an events table:
|
||||||
|
+ timestamp from the event
|
||||||
|
+ timestamp of action completion
|
||||||
|
+ webhook type
|
||||||
|
+ webhook id
|
||||||
|
+ stripe account id
|
||||||
|
+ aweber account id
|
||||||
|
+ and whether it was successfully processed (process state)
|
||||||
|
- dynamodb does not fit the use case because we need to retrieve all events in a time frame, and filter by account. We are also constantly deleting events.
|
||||||
|
+ we also will have two services updating the database and need transaction safety
|
||||||
|
- Scope assumes handing off the schema design to the DBA team
|
||||||
|
**** TODO Endpoint: Add event tracking to /webhooks
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 7d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:BLOCKER: database, webhooks_2
|
||||||
|
:TASK_ID: tracking
|
||||||
|
:END:
|
||||||
|
- Mark processed events in the database
|
||||||
|
+ Do we need transactions? Assume yes.
|
||||||
|
- Handling duplicate and out-of-order event cases
|
||||||
|
+ Fulfill only once (adding subscriber + tags)
|
||||||
|
+ Unsubscribe only once (removing subscriber / tags)
|
||||||
|
+ Don't fulfill if unsubscribed
|
||||||
|
*** Backend: Resiliency
|
||||||
|
We will need a job to periodically poll Stripe for unhandled webhooks for
|
||||||
|
processing.
|
||||||
|
|
||||||
|
- This job will need to share a data store with the stripe payments service to
|
||||||
|
track which webhooks have been processed.
|
||||||
|
+ Is this necessary, or does stripe expose the processed state of the
|
||||||
|
webhooks? Webhook attempts and responses are logged in the console.
|
||||||
|
- The job will need to maintain a lock to prevent concurrent runs.
|
||||||
|
- How will unhandled webhooks get processed?
|
||||||
|
+ Send them to the stripe payments service endpoint?
|
||||||
|
- Would have to store status or update it in Stripe.
|
||||||
|
+ Have Stripe send them?
|
||||||
|
- Is this possible?
|
||||||
|
**** TODO Poller: Create poller to check for unprocessed events
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 10d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:BLOCKER: milestone_subscriptions
|
||||||
|
:TASK_ID: poller
|
||||||
|
:END:
|
||||||
|
*** Backend: Cleanup
|
||||||
|
**** TODO Backfill products from existing landing pages into Stripe
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 10d
|
||||||
|
:ALLOCATE: backend, frontend
|
||||||
|
:blocker: milestone_subscriptions
|
||||||
|
:task_id: backfill2
|
||||||
|
:END:
|
||||||
|
**** TODO Endpoint: Remove /order and /fulfill
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 1d
|
||||||
|
:ALLOCATE: backend
|
||||||
|
:blocker: backfill2
|
||||||
|
:TASK_ID: cleanup
|
||||||
|
:END:
|
||||||
|
*** Frontend
|
||||||
|
Builder originally estimated at 25d combined.
|
||||||
|
**** TODO Frontend: Plugin
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 2d
|
||||||
|
:ALLOCATE: frontend
|
||||||
|
:TASK_ID: plugin
|
||||||
|
:BLOCKER:
|
||||||
|
:END:
|
||||||
|
**** TODO Frontend: Templates
|
||||||
|
:PROPERTIES:
|
||||||
|
:Effort: 1d
|
||||||
|
:ALLOCATE: frontend
|
||||||
|
:TASK_ID: templates
|
||||||
|
:BLOCKER: plugin
|
||||||
|
:END:
|
||||||
|
**** TODO Frontend: Builder (Product Management)
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 12d
|
||||||
|
:ALLOCATE: frontend
|
||||||
|
:TASK_ID: builder_products
|
||||||
|
:BLOCKER: plugin, templates
|
||||||
|
:END:
|
||||||
|
**** TODO Frontend: Builder (Recurring Payments)
|
||||||
|
:PROPERTIES:
|
||||||
|
:EFFORT: 28d
|
||||||
|
:ALLOCATE: frontend
|
||||||
|
:TASK_ID: builder_recurring
|
||||||
|
:BLOCKER: milestone_management
|
||||||
|
:start: 2021-04-22
|
||||||
|
:END:
|
||||||
|
** Milestones
|
||||||
|
*** Milestone 1: Product Management
|
||||||
|
:PROPERTIES:
|
||||||
|
:BLOCKER: api, create, update, fetch, order_both, migrate, backfill, builder_products
|
||||||
|
:TASK_ID: milestone_management
|
||||||
|
:Effort: 5d
|
||||||
|
:ALLOCATE: release
|
||||||
|
:END:
|
||||||
|
*** Milestone 2: Support Subscriptions
|
||||||
|
:PROPERTIES:
|
||||||
|
:BLOCKER: order_products, webhooks_1, webhooks_2, database, tracking, builder_recurring
|
||||||
|
:TASK_ID: milestone_subscriptions
|
||||||
|
:Effort: 10d
|
||||||
|
:ALLOCATE: release
|
||||||
|
:END:
|
||||||
|
*** Milestone 3: Add Resiliency
|
||||||
|
:PROPERTIES:
|
||||||
|
:BLOCKER: poller
|
||||||
|
:TASK_ID: milestone_resiliency
|
||||||
|
:END:
|
||||||
|
*** Milestone 4: Deprecate Product Name/Price
|
||||||
|
:PROPERTIES:
|
||||||
|
:BLOCKER: milestone_subscriptions, cleanup
|
||||||
|
:TASK_ID: milestone_cleanup
|
||||||
|
:END:
|
||||||
|
* Resources :taskjuggler_resource:
|
||||||
|
|
||||||
|
*** Backend
|
||||||
|
:PROPERTIES:
|
||||||
|
:resource_id: backend
|
||||||
|
:efficiency: 1.3
|
||||||
|
:END:
|
||||||
|
|
||||||
|
*** Frontend
|
||||||
|
:PROPERTIES:
|
||||||
|
:resource_id: frontend
|
||||||
|
:efficiency: 0.8
|
||||||
|
:END:
|
||||||
|
|
||||||
|
*** Release Validation
|
||||||
|
:PROPERTIES:
|
||||||
|
:resource_id: release
|
||||||
|
:efficiency: 1.0
|
||||||
|
:END:
|
||||||
|
|
||||||
|
* Variables :noexport:
|
||||||
|
Local Variables:
|
||||||
|
org-taskjuggler-reports-directory: "reports/recurring-and-split-stripe-payments"
|
||||||
|
End:
|
||||||
38
aweber/20210210113027-control_panel_http_requests.org
Normal file
38
aweber/20210210113027-control_panel_http_requests.org
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 33e47957-b3d0-41c9-8977-7243b42a76dd
|
||||||
|
:END:
|
||||||
|
#+title: Control Panel HTTP Requests
|
||||||
|
#+PROPERTY: header-args :exports both :eval no-export
|
||||||
|
#+PROPERTY: header-args:http :cookie .cookies :cookie-jar .cookies
|
||||||
|
|
||||||
|
* Cookies
|
||||||
|
| Name | Description |
|
||||||
|
|-------------+-------------|
|
||||||
|
| AUTORESPSID | Session ID |
|
||||||
|
|
||||||
|
Cookies for requests in this document are stored in cookie file by curl in
|
||||||
|
=~/.cookies= (https://curl.se/docs/http-cookies.html).
|
||||||
|
* AJAX Requests
|
||||||
|
Control Panel controller actions that expect to be called as AJAX endpoints
|
||||||
|
expect the =X-Requested-With= header to be present and set to =XMLHttpRequest=.
|
||||||
|
* Logging In
|
||||||
|
** Fetching a CSRF Token
|
||||||
|
#+name: login-csrf
|
||||||
|
#+begin_src http :pretty
|
||||||
|
GET localhost:8080/users/pub/csrf
|
||||||
|
X-Requested-With:XMLHttpRequest
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS: login-csrf
|
||||||
|
: 63116e764c5d31cdd3e4f230ee3740527f6eb1c76aea1cb04e30da5d68e24d78
|
||||||
|
|
||||||
|
** Sending credentials
|
||||||
|
#+begin_src http :pretty :var csrf=login-csrf
|
||||||
|
POST localhost:8080/users/account/loginAjax
|
||||||
|
X-Requested-With: XMLHttpRequest
|
||||||
|
|
||||||
|
username=lookatme@example.com&password=testing&_csrf=${csrf}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: {"submitStatus":{"code":200,"message":"\/users\/","category":"status_success"},"validationErrors":[]}
|
||||||
1166
aweber/20210302135423-stripe_api.org
Normal file
1166
aweber/20210302135423-stripe_api.org
Normal file
File diff suppressed because it is too large
Load diff
383
aweber/20210316155320-stripe_payments_tracking_database.org
Normal file
383
aweber/20210316155320-stripe_payments_tracking_database.org
Normal file
|
|
@ -0,0 +1,383 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 6af95849-8f78-4697-ab48-3712ff2f5ee1
|
||||||
|
:END:
|
||||||
|
#+title: Stripe payments tracking database
|
||||||
|
#+OPTIONS: ^:nil
|
||||||
|
|
||||||
|
Database for tracking payments and subscriptions managed by the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||||
|
* State Tracking
|
||||||
|
** Legacy purchase
|
||||||
|
- Client initiates purchase
|
||||||
|
- Backend coordinates with Stripe and returns a payment intent
|
||||||
|
+ Track new purchase and related incomplete payment
|
||||||
|
- Client completes purchase with Stripe
|
||||||
|
- Stripe notifies backend of payment intent status (update purchase)
|
||||||
|
+ Track event
|
||||||
|
+ Track payment as completed
|
||||||
|
+ Fulfill purchase
|
||||||
|
+ Track purchase as fulfilled
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-legacy-purchase-tracking.svg
|
||||||
|
participant Stripe
|
||||||
|
actor Client
|
||||||
|
participant "stripe-payments" as Backend
|
||||||
|
database Tracking
|
||||||
|
|
||||||
|
Client -> Backend: Initiate purchase
|
||||||
|
Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
|
||||||
|
Backend -> Stripe : Create payment intent
|
||||||
|
Backend -> Tracking : <font color="blue">Store incomplete payment</font>
|
||||||
|
Backend -> Client : Return payment intent
|
||||||
|
Client -> Stripe : Complete purchase
|
||||||
|
...
|
||||||
|
alt Success
|
||||||
|
Stripe ---> Backend : payment_intent.succeeded
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
alt Event not previously handled
|
||||||
|
Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
|
||||||
|
Backend -> Backend : Fulfill purchase
|
||||||
|
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||||
|
end
|
||||||
|
else Failure
|
||||||
|
Stripe --> Backend : payment_intent.payment_failed
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
end
|
||||||
|
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-legacy-purchase-tracking.svg]]
|
||||||
|
|
||||||
|
#+caption: Legacy purchase flow with events
|
||||||
|
#+RESULTS:
|
||||||
|
|
||||||
|
** Product purchase
|
||||||
|
- Client prepares payment method with Stripe
|
||||||
|
- Client initiates purchase
|
||||||
|
- Backend coordinates with Stripe to complete the purchase
|
||||||
|
+ Attaches the payment method to the customer
|
||||||
|
+ For single products
|
||||||
|
- Creates an invoice item
|
||||||
|
- Creates an invoice
|
||||||
|
- Pays the invoice
|
||||||
|
+ For subscriptions
|
||||||
|
- Creates the subscription
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-purchase-tracking.svg
|
||||||
|
participant Stripe
|
||||||
|
actor Client
|
||||||
|
participant "stripe-payments" as Backend
|
||||||
|
database Tracking
|
||||||
|
|
||||||
|
Client -> Stripe : Create payment method
|
||||||
|
Client -> Backend: Initiate purchase
|
||||||
|
Backend -> Tracking : <font color="blue">Store incomplete purchase</font>
|
||||||
|
alt Non-recurring
|
||||||
|
Backend -> Stripe : Create invoice item
|
||||||
|
Backend -> Stripe : Create invoice
|
||||||
|
Backend -> Stripe : Pay invoice
|
||||||
|
Backend -> Tracking : <font color="blue">Store completed payment</font>
|
||||||
|
else Recurring
|
||||||
|
Backend -> Stripe : Create subscription
|
||||||
|
Backend -> Tracking : <font color="blue">Store active subscription</font>
|
||||||
|
end
|
||||||
|
...
|
||||||
|
Stripe --> Backend : Subscription activated
|
||||||
|
...
|
||||||
|
alt
|
||||||
|
Stripe --> Backend : customer.subscription.updated (no longer active)
|
||||||
|
else
|
||||||
|
Stripe --> Backend : subscription_schedule.canceled
|
||||||
|
end
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
Backend -> Backend : Trigger unsubscribe actions
|
||||||
|
Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Product purchase flow
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-purchase-tracking.svg]]
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-purchase-tracking-payment-events.svg
|
||||||
|
participant Stripe
|
||||||
|
actor Client
|
||||||
|
participant "stripe-payments" as Backend
|
||||||
|
database Tracking
|
||||||
|
|
||||||
|
== Payment succeeded ==
|
||||||
|
Stripe --> Backend : payment_intent.succeeded
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
alt Event not previously handled
|
||||||
|
Backend -> Tracking : <font color="blue">Mark payment as succeeded</font>
|
||||||
|
Backend -> Stripe : Look up invoice
|
||||||
|
|
||||||
|
note over Backend
|
||||||
|
Recurring if an invoice exists
|
||||||
|
and has an associated subscription
|
||||||
|
end note
|
||||||
|
alt Non-recurring
|
||||||
|
Backend -> Backend : Fulfill purchase
|
||||||
|
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||||
|
else Recurring
|
||||||
|
end
|
||||||
|
end
|
||||||
|
== Payment failed ==
|
||||||
|
Stripe --> Backend : payment_intent.payment_failed
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Payment events
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-purchase-tracking-payment-events.svg]]
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-purchase-subscription-events.svg
|
||||||
|
participant Stripe
|
||||||
|
actor Client
|
||||||
|
participant "stripe-payments" as Backend
|
||||||
|
database Tracking
|
||||||
|
|
||||||
|
== Subscription activated ==
|
||||||
|
alt
|
||||||
|
Stripe --> Backend : customer.subscription.created (active)
|
||||||
|
else
|
||||||
|
Stripe --> Backend : customer.subscription.updated (active)
|
||||||
|
end
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
alt New subscription
|
||||||
|
Backend -> Tracking : <font color="blue">Mark subscription as active</font>
|
||||||
|
Backend -> Backend : Fulfill purchase
|
||||||
|
Backend -> Tracking : <font color="blue">Mark purchase as fulfilled</font>
|
||||||
|
else Subscription already processed
|
||||||
|
note over Backend
|
||||||
|
One of the following is true:
|
||||||
|
|
||||||
|
<font color="green">- Already received an activation event for this subscription</font>
|
||||||
|
<font color="green">- Already receieved a termination event for this subscription</font>
|
||||||
|
<font color="blue">- Subscription already marked as active or terminated</font>
|
||||||
|
<font color="blue">- Purchase already marked as fulfilled</font>
|
||||||
|
end note
|
||||||
|
end
|
||||||
|
== Subscription terminated ==
|
||||||
|
alt
|
||||||
|
Stripe --> Backend : customer.subscription.updated (no longer active)
|
||||||
|
else
|
||||||
|
Stripe --> Backend : subscription_schedule.canceled
|
||||||
|
end
|
||||||
|
Backend -> Tracking : <font color="green">Store event</font>
|
||||||
|
alt Active subscription
|
||||||
|
Backend -> Backend : Trigger unsubscribe actions
|
||||||
|
Backend -> Tracking : <font color="blue">Mark subscription as terminated</font>
|
||||||
|
else Subscription not yet processed
|
||||||
|
note over Backend
|
||||||
|
One of the following is true:
|
||||||
|
|
||||||
|
<font color="green">- Did not receive an activation event for this subscription</font>
|
||||||
|
<font color="blue">- Subscription is not tracked</font>
|
||||||
|
end note
|
||||||
|
else Subscription already terminated
|
||||||
|
note over Backend
|
||||||
|
One of the following is true:
|
||||||
|
|
||||||
|
<font color="green">- Already received a termination event for this subscription</font>
|
||||||
|
<font color="blue">- Subscription already marked as terminated</font>
|
||||||
|
end note
|
||||||
|
end
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Subscription events
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-purchase-subscription-events.svg]]
|
||||||
|
|
||||||
|
* Tables
|
||||||
|
** Purchases
|
||||||
|
Purchases made via the [[id:9b3ed74d-41d4-4784-89e3-6a9183903b9e][Stripe Payments Service]].
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-purchase-states.svg
|
||||||
|
[*] -> New
|
||||||
|
New -> Fulfilled
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Purchase state diagram
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-purchase-states.svg]]
|
||||||
|
|
||||||
|
#+caption: Purchases
|
||||||
|
| Field | Type | Nullable | Description |
|
||||||
|
|----------------+-----------+----------+--------------------------------------------|
|
||||||
|
| id | UUID | N | Auto-generated purchase ID |
|
||||||
|
| created | timestamp | N | Time the purchase was initiated |
|
||||||
|
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||||
|
| account | UUID | N | The AWeber customer account purchased from |
|
||||||
|
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||||
|
| fulfilled | boolean | N | Was this purchase fulfilled |
|
||||||
|
|
||||||
|
- Store which automations were applied & when
|
||||||
|
** Subscriptions and Split Payments
|
||||||
|
Purchases made with recurring payments, managed using a Stripe Subscription.
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-subscription-states.svg
|
||||||
|
[*] -> New
|
||||||
|
New -> Active
|
||||||
|
Active --> Terminated : Payment Failed
|
||||||
|
Active --> Terminated : Unsubscribed
|
||||||
|
New --> Terminated : Payment Failed
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Subscription state diagram
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-subscription-states.svg]]
|
||||||
|
|
||||||
|
#+caption: Subscriptions
|
||||||
|
| Field | Type | Nullable | Description |
|
||||||
|
|----------------+---------------------+----------+---------------------------------------------------------|
|
||||||
|
| id | text | N | The Stripe subscription ID |
|
||||||
|
| created | timestamp | N | Time the purchase was initiated |
|
||||||
|
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||||
|
| account | UUID | N | The AWeber customer account the subscription belongs to |
|
||||||
|
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||||
|
| status | [[subscription_status][subscription_status]] | N | Status of the subscription |
|
||||||
|
|
||||||
|
<<subscription_status>>
|
||||||
|
#+caption: ENUM: Subscription Status
|
||||||
|
| new |
|
||||||
|
| active |
|
||||||
|
| terminated |
|
||||||
|
** Payments
|
||||||
|
Payments collected from buyers.
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-payment-states.svg
|
||||||
|
[*] -> New
|
||||||
|
New --> Paid : Payment succeeded
|
||||||
|
New --> Failed : Payment failed
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Payment state diagram
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-payment-states.svg]]
|
||||||
|
|
||||||
|
#+caption: Payments
|
||||||
|
| Field | Type | Nullable | Description |
|
||||||
|
|----------------+----------------+----------+-------------------------------------------|
|
||||||
|
| id | text | N | The Stripe payment intent ID |
|
||||||
|
| created | timestamp | N | Time the purchase was initiated |
|
||||||
|
| last_updated | timestamp | N | Time the purchase was last updated |
|
||||||
|
| account | UUID | N | The AWeber customer account that was paid |
|
||||||
|
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||||
|
| purchase | UUID | N | |
|
||||||
|
| status | [[payment_status][payment_status]] | N | Status of the payment intent |
|
||||||
|
|
||||||
|
<<payment_status>>
|
||||||
|
#+caption: ENUM: Payment Status
|
||||||
|
| new |
|
||||||
|
| paid |
|
||||||
|
| failed |
|
||||||
|
** Events
|
||||||
|
#+caption: Events
|
||||||
|
| Field | Type | Nullable | Description |
|
||||||
|
|----------------+-----------+----------+---------------------------------------------------------|
|
||||||
|
| id | text | N | Stripe event id |
|
||||||
|
| type | text | N | Stripe event type |
|
||||||
|
| account | UUID | N | The AWeber customer account the subscription belongs to |
|
||||||
|
| stripe_account | text | N | The Stripe account of the AWeber customer |
|
||||||
|
| timestamp | timestamp | N | Time event was published |
|
||||||
|
| subscription | text | Y | Related Stripe subscription ID |
|
||||||
|
|
||||||
|
Events should expire out of the database over time. Stripe maintains events in
|
||||||
|
its database for up to 30 days.
|
||||||
|
*** Modeling in DynamoDB
|
||||||
|
**** Table
|
||||||
|
- Partition Key :: =id=
|
||||||
|
|
||||||
|
The table index provides for rapid lookup of individual events.
|
||||||
|
**** Attributes
|
||||||
|
| Field | Type | Required | Description |
|
||||||
|
|----------------+-------------------------+----------+-----------------------------------------------------------|
|
||||||
|
| id | String | Y | Stripe event id |
|
||||||
|
| type | String | Y | Stripe event type |
|
||||||
|
| stripe_account | String | Y | The Stripe account of the AWeber customer |
|
||||||
|
| timestamp | String (timestamp) | Y | Time event was published (e.g. "2020-01-01 01:02:34.567") |
|
||||||
|
| date | String (date) | Y | Date portion of timestamp (e.g. "2020-01-01") |
|
||||||
|
| time | String (time) | Y | Time portion of timestamp (e.g. "01:02:34.567") |
|
||||||
|
| expiration | Number (Unix timestamp) | Y | TTL expiration time of the event record |
|
||||||
|
| subscription | String | N | Related Stripe subscription ID |
|
||||||
|
**** TTL
|
||||||
|
The events table will automatically expire and remove rows for which
|
||||||
|
=expiration= has passed. We may set =expiration= to be the event creation time
|
||||||
|
plus thirty days to match Stripe's own expiration. This will prevent old items
|
||||||
|
from piling up in the table.
|
||||||
|
**** Time index (GSI)
|
||||||
|
- Partition Key :: =date=
|
||||||
|
- Sort Key :: =time=
|
||||||
|
- Projection :: Keys Only
|
||||||
|
|
||||||
|
|
||||||
|
The date and time portions of the event timestamp will be separated to allow for
|
||||||
|
scanning all events within a day, filterable by time. This should support our
|
||||||
|
polling job, which should run multiple times per day. We'll have to be careful
|
||||||
|
around date boundaries when looking up events (e.g. by including a query for
|
||||||
|
events from the previous day during the first run of the current day).
|
||||||
|
**** Stripe Account index (GSI)
|
||||||
|
- Partition Key :: =stripe_account=
|
||||||
|
- Sort Key :: =timestamp=
|
||||||
|
- Projection :: Include type, subscription
|
||||||
|
|
||||||
|
The account index provides for rapid lookup of events for a particular stripe
|
||||||
|
account, filterable by timestamp.
|
||||||
|
**** Subscription index (GSI)
|
||||||
|
- Partition Key :: =subscription=
|
||||||
|
- Sort Key :: =timestamp=
|
||||||
|
- Projection :: Include type, stripe_account
|
||||||
|
|
||||||
|
The account index provides for rapid lookup of events for a particular
|
||||||
|
subscription, filterable by timestamp.
|
||||||
|
**** IAM Policy
|
||||||
|
Put together by referencing [[id:ab2d34bf-97b1-4e50-8e9a-597d0f8fcf01][DynamoDB IAM Policies]].
|
||||||
|
#+caption: Stripe Payments DynamoDB IAM Policy
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"Version": "2012-10-17",
|
||||||
|
"Statement": [
|
||||||
|
{
|
||||||
|
"Effect": "Allow",
|
||||||
|
"Action": "dynamodb:ListTables",
|
||||||
|
"Resource": "*"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"Effect": "Allow",
|
||||||
|
"Action": [
|
||||||
|
"dynamodb:BatchGetItem",
|
||||||
|
"dynamodb:BatchWriteItem",
|
||||||
|
"dynamodb:ConditionCheckItem",
|
||||||
|
"dynamodb:PutItem",
|
||||||
|
"dynamodb:DeleteItem",
|
||||||
|
"dynamodb:Scan",
|
||||||
|
"dynamodb:Query",
|
||||||
|
"dynamodb:UpdateItem",
|
||||||
|
"dynamodb:DescribeTimeToLive",
|
||||||
|
"dynamodb:CreateTable",
|
||||||
|
"dynamodb:DescribeTable",
|
||||||
|
"dynamodb:GetItem",
|
||||||
|
"dynamodb:UpdateTable",
|
||||||
|
"dynamodb:UpdateTimeToLive"
|
||||||
|
],
|
||||||
|
"Resource": [
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*/index/*",
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-stripe-payments-*"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
* Notes
|
||||||
|
- How does the [[id:83d61eef-0781-46e0-b959-1a739cff5ea3][poller]] use these stored events?
|
||||||
|
+ /To identify events retrieved from Stripe which do not need to be replayed/
|
||||||
|
- How do we replay events?
|
||||||
|
+ Send it to the webhook endpoint? An internal webhook endpoint that shares
|
||||||
|
code with the public one?
|
||||||
|
- /The same webhook endpoint would be preferable, requires signing the
|
||||||
|
payload/
|
||||||
|
- Do we expose the processed events as an internal endpoint in the
|
||||||
|
stripe-payments service, or give the polling app a database connection?
|
||||||
|
+ /API seems preferable/
|
||||||
|
|
||||||
|
Following some discussion, the [[Events][Events]] table should be sufficient for now.
|
||||||
|
|
@ -0,0 +1,12 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: fe6374ac-8fa6-4579-bd42-6d4de92de86a
|
||||||
|
:END:
|
||||||
|
#+title: Finding number of subscribers with a tag
|
||||||
|
|
||||||
|
* Retrieve number of subscribers per tag
|
||||||
|
- Tagging -> Kubernetes
|
||||||
|
- Add appdb connection to tagging
|
||||||
|
- Add endpoint to tagging
|
||||||
|
- Expose tagging endpoints via authenticated kong
|
||||||
|
* Hide tags
|
||||||
|
* Delete tags
|
||||||
23
aweber/20210318112158-tagging_roadmap.org
Normal file
23
aweber/20210318112158-tagging_roadmap.org
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 8435e743-c092-43e9-bcc6-a8098aa4110c
|
||||||
|
:END:
|
||||||
|
#+title: Tagging Roadmap
|
||||||
|
|
||||||
|
* Retrieve number of subscribers per tag
|
||||||
|
** Migrate the tagging service from AWS to Kubernetes
|
||||||
|
** Expose tagging endpoints via authenticated kong
|
||||||
|
** Add appdb connection to tagging
|
||||||
|
|
||||||
|
** Create an AppDB user for the tagging service
|
||||||
|
- Full access to the subscriber_tags table
|
||||||
|
- Read access to the list.subscribers table
|
||||||
|
|
||||||
|
** Add endpoint to tagging to fetch subscribers on a tag
|
||||||
|
e.g.:
|
||||||
|
#+begin_example
|
||||||
|
GET tagging.service.production.consul/{tag}/account/{account}/subscribers
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
** Update the tagging service to update the subscriber_tags table directly.
|
||||||
|
** Retire the subscriber-tag-sync consumer
|
||||||
|
* Hide tags
|
||||||
43
aweber/20210323161636-moving_pages_out_of_sites.org
Normal file
43
aweber/20210323161636-moving_pages_out_of_sites.org
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 193f7c04-0a03-4870-90c8-2b5e3c4c92ce
|
||||||
|
:END:
|
||||||
|
#+title: Moving pages out of Sites
|
||||||
|
#+filetags: :project:
|
||||||
|
|
||||||
|
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to rewrite pages in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] still built in PHP.
|
||||||
|
|
||||||
|
* Pages in Sites
|
||||||
|
** TODO Dashboard
|
||||||
|
*** Broadcasts
|
||||||
|
*** Lists
|
||||||
|
*** Subscribers
|
||||||
|
In progress.
|
||||||
|
** Content Creation
|
||||||
|
*** TODO Follow-ups
|
||||||
|
*** TODO Blog Broadcasts
|
||||||
|
*** TODO Email Template Manager
|
||||||
|
** Subscribers
|
||||||
|
*** TODO Manage Subscribers
|
||||||
|
*** TODO Add Subscribers
|
||||||
|
*** TODO Import History
|
||||||
|
** Reports
|
||||||
|
*** TODO Classic Reports
|
||||||
|
*** TODO Report API
|
||||||
|
*** TODO Tracking
|
||||||
|
** Lists
|
||||||
|
*** TODO Manage Lists
|
||||||
|
*** TODO List Settings
|
||||||
|
*** TODO Custom Fields
|
||||||
|
*** TODO List Automations
|
||||||
|
** Accounts
|
||||||
|
*** TODO My Account
|
||||||
|
*** Billing
|
||||||
|
*** Notifications
|
||||||
|
** Advocates
|
||||||
|
- Create an affiliate management API
|
||||||
|
- Create an affiliate frontend application
|
||||||
|
- Embed the affiliate app in the CP and link them with CP accounts.
|
||||||
|
** TODO Help
|
||||||
|
Should this be separated?
|
||||||
|
** Integrations
|
||||||
|
😱
|
||||||
8
aweber/20210323162325-technical_initiative.org
Normal file
8
aweber/20210323162325-technical_initiative.org
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: db322997-ff5e-416a-8dc8-f29e6a4928c8
|
||||||
|
:END:
|
||||||
|
#+title: Technical Initiative
|
||||||
|
|
||||||
|
* Active
|
||||||
|
- [[id:193f7c04-0a03-4870-90c8-2b5e3c4c92ce][Moving pages out of Sites]]
|
||||||
|
- [[id:b4f579f7-f848-4a7b-b7bc-f34fec36346a][Cleaning up public endpoints in proxy services]]
|
||||||
|
|
@ -0,0 +1,29 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: b4f579f7-f848-4a7b-b7bc-f34fec36346a
|
||||||
|
:END:
|
||||||
|
#+title: Cleaning up public endpoints in proxy services
|
||||||
|
|
||||||
|
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to move endpoints exposed in "proxy" services into more
|
||||||
|
relevant services and instead expose them via Kong.
|
||||||
|
|
||||||
|
* Proxy Services
|
||||||
|
|
||||||
|
** subscriber-proxy
|
||||||
|
|
||||||
|
*** TODO =/<subscriber_id>=
|
||||||
|
|
||||||
|
*** TODO =/batch_delete=
|
||||||
|
|
||||||
|
*** TODO =/batch_tag=
|
||||||
|
|
||||||
|
*** TODO =/batch_unsubscribe=
|
||||||
|
|
||||||
|
*** TODO =/bulk-tagging/jobs=
|
||||||
|
|
||||||
|
*** TODO =/bulk-tagging/jobs/<id>=
|
||||||
|
|
||||||
|
** search-proxy
|
||||||
|
|
||||||
|
** tagging-proxy
|
||||||
|
|
||||||
|
** search-recipients
|
||||||
4
aweber/20210324133247-imbi.org
Normal file
4
aweber/20210324133247-imbi.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 76933c22-fe7c-43e9-9ec9-62564377dd85
|
||||||
|
:END:
|
||||||
|
#+title: Imbi
|
||||||
|
|
@ -0,0 +1,13 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 9332ed8f-b669-4d3f-a25d-da751a8c2da1
|
||||||
|
:END:
|
||||||
|
#+title: Troubleshooting an unresolvable kubernetes service hostname
|
||||||
|
|
||||||
|
The service's selector wasn't matching any pods, therefore the service had no IP
|
||||||
|
to respond with. This can be troubleshooted by inspecting the endpoints for the
|
||||||
|
service. Due to the mismatch, none were available.
|
||||||
|
|
||||||
|
The ClusterIP service did not itself have an IP assigned even after fixing the
|
||||||
|
mismatch. This appears to be an optimization in that Kubernetes won't bother
|
||||||
|
assigning an IP to the service if the service has only a single endpoint, as it
|
||||||
|
is more expedient to return the sole endpoint's IP address.
|
||||||
15
aweber/20210412153138-purchase_tracking.org
Normal file
15
aweber/20210412153138-purchase_tracking.org
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: e2dab290-3e1b-4d5a-8628-61c2cb4896dc
|
||||||
|
:END:
|
||||||
|
#+title: Purchase tracking
|
||||||
|
|
||||||
|
* Related Confluence documents
|
||||||
|
- [[https://confluence.aweber.io/pages/viewpage.action?spaceKey=PD&title=2021-03-16+Record+purchase+details+for+AWeber+integrations][2021-03-16 Record purchase details for AWeber integrations]]
|
||||||
|
- [[https://confluence.aweber.io/pages/viewpage.action?spaceKey=API&title=Storing+PayPal+purchases+as+analytics+events][Storing PayPal purchases as analytics events]]
|
||||||
|
|
||||||
|
* Record purchases
|
||||||
|
- Track a separate event type for sales tracking.
|
||||||
|
- Changes to sales tracking URLs can break tracking currently.
|
||||||
|
- Separate URL profiles for Stripe and Paypal? (Stripe is currently using
|
||||||
|
ECommerce), and add an ECommerce event type for them.
|
||||||
|
* Segmenting on purchases
|
||||||
65
aweber/20210506110533-dynamodb_iam_policies.org
Normal file
65
aweber/20210506110533-dynamodb_iam_policies.org
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: ab2d34bf-97b1-4e50-8e9a-597d0f8fcf01
|
||||||
|
:END:
|
||||||
|
#+title: DynamoDB IAM Policies
|
||||||
|
|
||||||
|
#+caption: DynamoDB access for the k8s-labs-application role
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"Version": "2012-10-17",
|
||||||
|
"Statement": [
|
||||||
|
{
|
||||||
|
"Effect": "Allow",
|
||||||
|
"Action": "dynamodb:ListTables",
|
||||||
|
"Resource": "*"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"Effect": "Allow",
|
||||||
|
"Action": [
|
||||||
|
"dynamodb:BatchGetItem",
|
||||||
|
"dynamodb:BatchWriteItem",
|
||||||
|
"dynamodb:ConditionCheckItem",
|
||||||
|
"dynamodb:PutItem",
|
||||||
|
"dynamodb:DeleteItem",
|
||||||
|
"dynamodb:Scan",
|
||||||
|
"dynamodb:Query",
|
||||||
|
"dynamodb:UpdateItem",
|
||||||
|
"dynamodb:DescribeTimeToLive",
|
||||||
|
"dynamodb:CreateTable",
|
||||||
|
"dynamodb:DescribeTable",
|
||||||
|
"dynamodb:GetItem",
|
||||||
|
"dynamodb:UpdateTable"
|
||||||
|
],
|
||||||
|
"Resource": [
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-webhook-callbacks/index/*",
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-webhook-callbacks"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"Effect": "Allow",
|
||||||
|
"Action": [
|
||||||
|
"dynamodb:BatchGetItem",
|
||||||
|
"dynamodb:BatchWriteItem",
|
||||||
|
"dynamodb:ConditionCheckItem",
|
||||||
|
"dynamodb:PutItem",
|
||||||
|
"dynamodb:DeleteItem",
|
||||||
|
"dynamodb:Scan",
|
||||||
|
"dynamodb:Query",
|
||||||
|
"dynamodb:UpdateItem",
|
||||||
|
"dynamodb:DescribeTimeToLive",
|
||||||
|
"dynamodb:CreateTable",
|
||||||
|
"dynamodb:DescribeTable",
|
||||||
|
"dynamodb:GetItem",
|
||||||
|
"dynamodb:UpdateTable"
|
||||||
|
],
|
||||||
|
"Resource": [
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-webhooks",
|
||||||
|
"arn:aws:dynamodb:*:018154689201:table/*-webhooks/index/*"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
- [[https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/access-control-overview.html][Overview of Managing Access Permissions to Your Amazon DynamoDB Resources]]
|
||||||
|
- [[https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/api-permissions-reference.html][DynamoDB API Permissions: Actions, Resources, and Conditions Reference]]
|
||||||
|
|
@ -0,0 +1,41 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: bdc526d1-4f57-4210-93f2-12bb30d33ed9
|
||||||
|
:END:
|
||||||
|
#+title: Rebuild Unsubscribe Page as a React Application
|
||||||
|
#+filetags: :project:
|
||||||
|
|
||||||
|
* Flows
|
||||||
|
As described in [[https://confluence.aweber.io/display/~scottm/Unsubscribe+Flows][Unsubscribe Flows]]:
|
||||||
|
|
||||||
|
- View subscriptions
|
||||||
|
- Edit subscriptions
|
||||||
|
- Edit subscriber
|
||||||
|
+ Name
|
||||||
|
+ Email
|
||||||
|
+ Custom fields, if present
|
||||||
|
** Questions
|
||||||
|
- What do we do when the backend (database) is unavailable?
|
||||||
|
+ The backend could do some sort of queuing if the database is unavailable
|
||||||
|
(similar to the disk-persisted queue currently in use).
|
||||||
|
* Frontend
|
||||||
|
A new React application to support subscriber subscription management actions.
|
||||||
|
** Questions
|
||||||
|
- What will host the frontend application?
|
||||||
|
+ Deploy the application to S3 and have F5 route to it.
|
||||||
|
* Backend
|
||||||
|
A new python service to handle subscriber subscription management actions.
|
||||||
|
|
||||||
|
** Endpoints
|
||||||
|
*** Configuration
|
||||||
|
Returns account information and branding needed to display the page.
|
||||||
|
*** Get subscriptions
|
||||||
|
*** Edit subscriptions
|
||||||
|
*** Edit subscriber
|
||||||
|
Email verification needs to be processed if the subscriber updates their email
|
||||||
|
address in order for that change to take effect (addresses [[https://jira.aweber.io/browse/CCPANEL-11269][CCPANEL-11269]]).
|
||||||
|
Changes to other fields should be applied immediately.
|
||||||
|
**** Questions
|
||||||
|
- Do custom field changes need verification to change?
|
||||||
|
+ Probably not.
|
||||||
|
- Should we display a pending state until verification is complete?
|
||||||
|
+ We do not plan to store a pending state.
|
||||||
13
aweber/20210512133000-cobrowse_io.org
Normal file
13
aweber/20210512133000-cobrowse_io.org
Normal file
|
|
@ -0,0 +1,13 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: d0a802dd-3258-4a86-b53f-287f7f6df6e6
|
||||||
|
:END:
|
||||||
|
#+title: Cobrowse.io
|
||||||
|
#+filetags: :project:
|
||||||
|
|
||||||
|
- Allows CS team to view a session of one of our customers.
|
||||||
|
- This will initially be available to all customers as a link on the help page.
|
||||||
|
- User-initiated, share the code with the AW agent.
|
||||||
|
- We will be using their cloud solution.
|
||||||
|
- Look into an API for adding admin notes that the session took place
|
||||||
|
- Look into information to be redacted in session recordings
|
||||||
|
+ Information displayed will have to be tagged with a CSS class to be identified
|
||||||
12
aweber/20210520114208-migrating_aws_services.org
Normal file
12
aweber/20210520114208-migrating_aws_services.org
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: af4ae6ee-5201-49ee-aa01-6cf6a0801908
|
||||||
|
:END:
|
||||||
|
#+title: Migrating AWS services
|
||||||
|
#+filetags: :project:
|
||||||
|
|
||||||
|
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to migrate services from ECS deployments to Kubernetes
|
||||||
|
deployments. The purpose is to homogenize our deployments to use Kubernetes,
|
||||||
|
which could theoretically be deployed to our local Kubernetes cluster or to EKS.
|
||||||
|
|
||||||
|
We currently maintain two separate sets of clusters in AWS. Services should be
|
||||||
|
migrated to the new cluster, or to kubernetes, ideally.
|
||||||
|
|
@ -0,0 +1,977 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 022406d2-0480-4470-90d0-9533f6b9fa32
|
||||||
|
:END:
|
||||||
|
#+title: Supporting multiple currencies in Stripe
|
||||||
|
|
||||||
|
To be completed after [[id:a9835afc-e0be-4436-8274-c3898fdf119c][Recurring and split Stripe payments]].
|
||||||
|
* How does Stripe handle different currencies?
|
||||||
|
- https://stripe.com/docs/currencies
|
||||||
|
* Which currencies are supported by a particular customer?
|
||||||
|
- [[https://stripe.com/docs/api/country_specs/object#country_spec_object-supported_payment_currencies][Supported payment currencies]] can be retrieved via the Stripe API by looking up
|
||||||
|
their Stripe account's [[https://stripe.com/docs/api/accounts/object#account_object-country][configured country]].
|
||||||
|
- Stripe accounts have a [[https://stripe.com/docs/api/accounts/object#account_object-default_currency][default currency]] configured that we could refer to when
|
||||||
|
setting up an eCommerce widget.
|
||||||
|
** What are my supported currencies?
|
||||||
|
#+begin_src sh :results output :wrap src json :exports both :eval no-export
|
||||||
|
stripe get /v1/country_specs/US
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"id": "US",
|
||||||
|
"object": "country_spec",
|
||||||
|
"default_currency": "usd",
|
||||||
|
"supported_bank_account_currencies": {
|
||||||
|
"usd": [
|
||||||
|
"US"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"supported_payment_currencies": [
|
||||||
|
"usd",
|
||||||
|
"aed",
|
||||||
|
"afn",
|
||||||
|
"all",
|
||||||
|
"amd",
|
||||||
|
"ang",
|
||||||
|
"aoa",
|
||||||
|
"ars",
|
||||||
|
"aud",
|
||||||
|
"awg",
|
||||||
|
"azn",
|
||||||
|
"bam",
|
||||||
|
"bbd",
|
||||||
|
"bdt",
|
||||||
|
"bgn",
|
||||||
|
"bif",
|
||||||
|
"bmd",
|
||||||
|
"bnd",
|
||||||
|
"bob",
|
||||||
|
"brl",
|
||||||
|
"bsd",
|
||||||
|
"bwp",
|
||||||
|
"bzd",
|
||||||
|
"cad",
|
||||||
|
"cdf",
|
||||||
|
"chf",
|
||||||
|
"clp",
|
||||||
|
"cny",
|
||||||
|
"cop",
|
||||||
|
"crc",
|
||||||
|
"cve",
|
||||||
|
"czk",
|
||||||
|
"djf",
|
||||||
|
"dkk",
|
||||||
|
"dop",
|
||||||
|
"dzd",
|
||||||
|
"egp",
|
||||||
|
"etb",
|
||||||
|
"eur",
|
||||||
|
"fjd",
|
||||||
|
"fkp",
|
||||||
|
"gbp",
|
||||||
|
"gel",
|
||||||
|
"gip",
|
||||||
|
"gmd",
|
||||||
|
"gnf",
|
||||||
|
"gtq",
|
||||||
|
"gyd",
|
||||||
|
"hkd",
|
||||||
|
"hnl",
|
||||||
|
"hrk",
|
||||||
|
"htg",
|
||||||
|
"huf",
|
||||||
|
"idr",
|
||||||
|
"ils",
|
||||||
|
"inr",
|
||||||
|
"isk",
|
||||||
|
"jmd",
|
||||||
|
"jpy",
|
||||||
|
"kes",
|
||||||
|
"kgs",
|
||||||
|
"khr",
|
||||||
|
"kmf",
|
||||||
|
"krw",
|
||||||
|
"kyd",
|
||||||
|
"kzt",
|
||||||
|
"lak",
|
||||||
|
"lbp",
|
||||||
|
"lkr",
|
||||||
|
"lrd",
|
||||||
|
"lsl",
|
||||||
|
"mad",
|
||||||
|
"mdl",
|
||||||
|
"mga",
|
||||||
|
"mkd",
|
||||||
|
"mmk",
|
||||||
|
"mnt",
|
||||||
|
"mop",
|
||||||
|
"mro",
|
||||||
|
"mur",
|
||||||
|
"mvr",
|
||||||
|
"mwk",
|
||||||
|
"mxn",
|
||||||
|
"myr",
|
||||||
|
"mzn",
|
||||||
|
"nad",
|
||||||
|
"ngn",
|
||||||
|
"nio",
|
||||||
|
"nok",
|
||||||
|
"npr",
|
||||||
|
"nzd",
|
||||||
|
"pab",
|
||||||
|
"pen",
|
||||||
|
"pgk",
|
||||||
|
"php",
|
||||||
|
"pkr",
|
||||||
|
"pln",
|
||||||
|
"pyg",
|
||||||
|
"qar",
|
||||||
|
"ron",
|
||||||
|
"rsd",
|
||||||
|
"rub",
|
||||||
|
"rwf",
|
||||||
|
"sar",
|
||||||
|
"sbd",
|
||||||
|
"scr",
|
||||||
|
"sek",
|
||||||
|
"sgd",
|
||||||
|
"shp",
|
||||||
|
"sll",
|
||||||
|
"sos",
|
||||||
|
"srd",
|
||||||
|
"std",
|
||||||
|
"szl",
|
||||||
|
"thb",
|
||||||
|
"tjs",
|
||||||
|
"top",
|
||||||
|
"try",
|
||||||
|
"ttd",
|
||||||
|
"twd",
|
||||||
|
"tzs",
|
||||||
|
"uah",
|
||||||
|
"ugx",
|
||||||
|
"uyu",
|
||||||
|
"uzs",
|
||||||
|
"vnd",
|
||||||
|
"vuv",
|
||||||
|
"wst",
|
||||||
|
"xaf",
|
||||||
|
"xcd",
|
||||||
|
"xof",
|
||||||
|
"xpf",
|
||||||
|
"yer",
|
||||||
|
"zar",
|
||||||
|
"zmw"
|
||||||
|
],
|
||||||
|
"supported_payment_methods": [
|
||||||
|
"card",
|
||||||
|
"stripe"
|
||||||
|
],
|
||||||
|
"supported_transfer_countries": [
|
||||||
|
"US",
|
||||||
|
"AT",
|
||||||
|
"AR",
|
||||||
|
"AU",
|
||||||
|
"BE",
|
||||||
|
"BG",
|
||||||
|
"BO",
|
||||||
|
"CA",
|
||||||
|
"CH",
|
||||||
|
"CR",
|
||||||
|
"CY",
|
||||||
|
"CZ",
|
||||||
|
"DE",
|
||||||
|
"DK",
|
||||||
|
"DO",
|
||||||
|
"EE",
|
||||||
|
"EG",
|
||||||
|
"ES",
|
||||||
|
"FI",
|
||||||
|
"FR",
|
||||||
|
"GB",
|
||||||
|
"GR",
|
||||||
|
"HK",
|
||||||
|
"HR",
|
||||||
|
"HU",
|
||||||
|
"ID",
|
||||||
|
"IE",
|
||||||
|
"IL",
|
||||||
|
"IS",
|
||||||
|
"IT",
|
||||||
|
"LI",
|
||||||
|
"LT",
|
||||||
|
"LU",
|
||||||
|
"LV",
|
||||||
|
"MT",
|
||||||
|
"MX",
|
||||||
|
"NL",
|
||||||
|
"NO",
|
||||||
|
"NZ",
|
||||||
|
"PE",
|
||||||
|
"PL",
|
||||||
|
"PT",
|
||||||
|
"RO",
|
||||||
|
"SE",
|
||||||
|
"SG",
|
||||||
|
"SI",
|
||||||
|
"SK",
|
||||||
|
"TH",
|
||||||
|
"TT",
|
||||||
|
"UY"
|
||||||
|
],
|
||||||
|
"verification_fields": {
|
||||||
|
"company": {
|
||||||
|
"additional": [
|
||||||
|
"representative.verification.document"
|
||||||
|
],
|
||||||
|
"minimum": [
|
||||||
|
"business_profile.mcc",
|
||||||
|
"business_profile.url",
|
||||||
|
"business_type",
|
||||||
|
"company.address.city",
|
||||||
|
"company.address.line1",
|
||||||
|
"company.address.postal_code",
|
||||||
|
"company.address.state",
|
||||||
|
"company.name",
|
||||||
|
"company.owners_provided",
|
||||||
|
"company.phone",
|
||||||
|
"company.tax_id",
|
||||||
|
"external_account",
|
||||||
|
"owners.address.city",
|
||||||
|
"owners.address.line1",
|
||||||
|
"owners.address.postal_code",
|
||||||
|
"owners.address.state",
|
||||||
|
"owners.dob.day",
|
||||||
|
"owners.dob.month",
|
||||||
|
"owners.dob.year",
|
||||||
|
"owners.email",
|
||||||
|
"owners.first_name",
|
||||||
|
"owners.id_number",
|
||||||
|
"owners.last_name",
|
||||||
|
"owners.phone",
|
||||||
|
"owners.relationship.title",
|
||||||
|
"owners.verification.document",
|
||||||
|
"representative.address.city",
|
||||||
|
"representative.address.line1",
|
||||||
|
"representative.address.postal_code",
|
||||||
|
"representative.address.state",
|
||||||
|
"representative.dob.day",
|
||||||
|
"representative.dob.month",
|
||||||
|
"representative.dob.year",
|
||||||
|
"representative.email",
|
||||||
|
"representative.first_name",
|
||||||
|
"representative.id_number",
|
||||||
|
"representative.last_name",
|
||||||
|
"representative.phone",
|
||||||
|
"representative.relationship.executive",
|
||||||
|
"representative.relationship.title",
|
||||||
|
"tos_acceptance.date",
|
||||||
|
"tos_acceptance.ip"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"individual": {
|
||||||
|
"additional": [
|
||||||
|
"individual.verification.document"
|
||||||
|
],
|
||||||
|
"minimum": [
|
||||||
|
"business_profile.mcc",
|
||||||
|
"business_profile.url",
|
||||||
|
"business_type",
|
||||||
|
"external_account",
|
||||||
|
"individual.address.city",
|
||||||
|
"individual.address.line1",
|
||||||
|
"individual.address.postal_code",
|
||||||
|
"individual.address.state",
|
||||||
|
"individual.dob.day",
|
||||||
|
"individual.dob.month",
|
||||||
|
"individual.dob.year",
|
||||||
|
"individual.email",
|
||||||
|
"individual.first_name",
|
||||||
|
"individual.id_number",
|
||||||
|
"individual.last_name",
|
||||||
|
"individual.phone",
|
||||||
|
"tos_acceptance.date",
|
||||||
|
"tos_acceptance.ip"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+begin_notes
|
||||||
|
Country data cannot, unfortunately, be expanded from an account:
|
||||||
|
|
||||||
|
#+begin_src sh :exports both :results output :wrap src json :eval no-export
|
||||||
|
stripe get acct_1IGnMkIoFf3wvXpR -d "expand[]=country"
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"message": "This property cannot be expanded (country).",
|
||||||
|
"type": "invalid_request_error"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
#+end_notes
|
||||||
|
|
||||||
|
* What happens when a payment is below the minimum allowed amount for a settlement currency?
|
||||||
|
- Verify the error when making a price with an unsupported currenccy
|
||||||
|
- Check that the flat fee is calculated correctly
|
||||||
|
** What happens when I create a price with an unsupported currency?
|
||||||
|
|
||||||
|
#+name: price-bhd
|
||||||
|
#+caption: Creating a price using Bahraini dinars (BHD)
|
||||||
|
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||||
|
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||||
|
#+header: :wrap src json
|
||||||
|
#+begin_src shell :var product=product :results output :exports both :eval no-export
|
||||||
|
stripe prices create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--product=$product \
|
||||||
|
--unit-amount=350 \
|
||||||
|
--currency=bhd \
|
||||||
|
-d "nickname"="Price in BHD"
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS: price-bhd
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"message": "Invalid currency: bhd. Stripe accounts in US do not support bhd. Your account currently supports these currencies: usd, aed, afn, all, amd, ang, aoa, ars, aud, awg, azn, bam, bbd, bdt, bgn, bif, bmd, bnd, bob, brl, bsd, bwp, bzd, cad, cdf, chf, clp, cny, cop, crc, cve, czk, djf, dkk, dop, dzd, egp, etb, eur, fjd, fkp, gbp, gel, gip, gmd, gnf, gtq, gyd, hkd, hnl, hrk, htg, huf, idr, ils, inr, isk, jmd, jpy, kes, kgs, khr, kmf, krw, kyd, kzt, lak, lbp, lkr, lrd, lsl, mad, mdl, mga, mkd, mmk, mnt, mop, mro, mur, mvr, mwk, mxn, myr, mzn, nad, ngn, nio, nok, npr, nzd, pab, pen, pgk, php, pkr, pln, pyg, qar, ron, rsd, rub, rwf, sar, sbd, scr, sek, sgd, shp, sll, sos, srd, std, szl, thb, tjs, top, try, ttd, twd, tzs, uah, ugx, uyu, uzs, vnd, vuv, wst, xaf, xcd, xof, xpf, yer, zar, zmw.",
|
||||||
|
"param": "currency",
|
||||||
|
"type": "invalid_request_error"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
** What do fees look like when using an alternate currency?
|
||||||
|
#+name: price-jpy
|
||||||
|
#+caption: Creating a price using Japanese yen (JPY)
|
||||||
|
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||||
|
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||||
|
#+header: :wrap src json
|
||||||
|
#+begin_src shell :var product=product :results output :exports both :eval no-export
|
||||||
|
stripe prices create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--product=$product \
|
||||||
|
--unit-amount=3500 \
|
||||||
|
--currency=jpy \
|
||||||
|
-d "nickname"="Price in JPY"
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS: price-jpy
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"id": "price_1JBOBRIoFf3wvXpR24iMvhoG",
|
||||||
|
"object": "price",
|
||||||
|
"active": true,
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1625854337,
|
||||||
|
"currency": "jpy",
|
||||||
|
"livemode": false,
|
||||||
|
"lookup_key": null,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Price in JPY",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"recurring": null,
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_quantity": null,
|
||||||
|
"type": "one_time",
|
||||||
|
"unit_amount": 3500,
|
||||||
|
"unit_amount_decimal": "3500"
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
*** Flat fee in a one-time purchase invoice
|
||||||
|
#+caption: Invoicing a purchase in JPY for a US (USD) account
|
||||||
|
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||||
|
#+header: :var price="price_1JBOBRIoFf3wvXpR24iMvhoG"
|
||||||
|
#+begin_src sh :exports both :results output :wrap src json :eval no-export
|
||||||
|
customer=$(stripe customers create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--description="JPY Invoice Customer" \
|
||||||
|
--name="Correl Roush" \
|
||||||
|
--email="correl+stripe.roam.docs@gmail.com" \
|
||||||
|
| jq -r .id)
|
||||||
|
|
||||||
|
stripe invoiceitems create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--customer=$customer \
|
||||||
|
--price=$price >/dev/null
|
||||||
|
|
||||||
|
invoice=$(stripe invoices create \
|
||||||
|
--stripe-account="$account" \
|
||||||
|
--customer="$customer" \
|
||||||
|
--application-fee-amount=500 \
|
||||||
|
| jq -r .id)
|
||||||
|
|
||||||
|
intent=$(stripe invoices finalize_invoice \
|
||||||
|
--stripe-account=$account \
|
||||||
|
$invoice \
|
||||||
|
| jq -r .payment_intent)
|
||||||
|
|
||||||
|
stripe payment_intents confirm $intent \
|
||||||
|
--stripe-account="$account" \
|
||||||
|
--payment-method=pm_card_visa
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"id": "pi_1JBOVQIoFf3wvXpRRJIbx8xf",
|
||||||
|
"object": "payment_intent",
|
||||||
|
"amount": 3500,
|
||||||
|
"amount_capturable": 0,
|
||||||
|
"amount_received": 3500,
|
||||||
|
"application": "ca_IIJmzFRoIG2rIdny5psryEvYurgcOmEP",
|
||||||
|
"application_fee_amount": 500,
|
||||||
|
"canceled_at": null,
|
||||||
|
"cancellation_reason": null,
|
||||||
|
"capture_method": "automatic",
|
||||||
|
"charges": {
|
||||||
|
"object": "list",
|
||||||
|
"data": [
|
||||||
|
{
|
||||||
|
"id": "ch_1JBOVRIoFf3wvXpRCYVnRz0y",
|
||||||
|
"object": "charge",
|
||||||
|
"amount": 3500,
|
||||||
|
"amount_captured": 3500,
|
||||||
|
"amount_refunded": 0,
|
||||||
|
"application": "ca_IIJmzFRoIG2rIdny5psryEvYurgcOmEP",
|
||||||
|
"application_fee": "fee_1JBOVSIoFf3wvXpR2DbO4Xxe",
|
||||||
|
"application_fee_amount": 500,
|
||||||
|
"balance_transaction": "txn_1JBOVSIoFf3wvXpRXaKfOsWB",
|
||||||
|
"billing_details": {
|
||||||
|
"address": {
|
||||||
|
"city": null,
|
||||||
|
"country": null,
|
||||||
|
"line1": null,
|
||||||
|
"line2": null,
|
||||||
|
"postal_code": null,
|
||||||
|
"state": null
|
||||||
|
},
|
||||||
|
"email": null,
|
||||||
|
"name": null,
|
||||||
|
"phone": null
|
||||||
|
},
|
||||||
|
"calculated_statement_descriptor": "CORRELS STUFF",
|
||||||
|
"captured": true,
|
||||||
|
"created": 1625855577,
|
||||||
|
"currency": "jpy",
|
||||||
|
"customer": "cus_Jp2a7jolV9KaII",
|
||||||
|
"description": "Payment for Invoice",
|
||||||
|
"destination": null,
|
||||||
|
"dispute": null,
|
||||||
|
"disputed": false,
|
||||||
|
"failure_code": null,
|
||||||
|
"failure_message": null,
|
||||||
|
"fraud_details": {
|
||||||
|
},
|
||||||
|
"invoice": "in_1JBOVPIoFf3wvXpRuM28s36R",
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"on_behalf_of": null,
|
||||||
|
"order": null,
|
||||||
|
"outcome": {
|
||||||
|
"network_status": "approved_by_network",
|
||||||
|
"reason": null,
|
||||||
|
"risk_level": "normal",
|
||||||
|
"risk_score": 15,
|
||||||
|
"seller_message": "Payment complete.",
|
||||||
|
"type": "authorized"
|
||||||
|
},
|
||||||
|
"paid": true,
|
||||||
|
"payment_intent": "pi_1JBOVQIoFf3wvXpRRJIbx8xf",
|
||||||
|
"payment_method": "pm_1JBOVRIoFf3wvXpR3JKo6m2a",
|
||||||
|
"payment_method_details": {
|
||||||
|
"card": {
|
||||||
|
"brand": "visa",
|
||||||
|
"checks": {
|
||||||
|
"address_line1_check": null,
|
||||||
|
"address_postal_code_check": null,
|
||||||
|
"cvc_check": null
|
||||||
|
},
|
||||||
|
"country": "US",
|
||||||
|
"exp_month": 7,
|
||||||
|
"exp_year": 2022,
|
||||||
|
"fingerprint": "C5qD8oSCGvVCbtcH",
|
||||||
|
"funding": "credit",
|
||||||
|
"installments": null,
|
||||||
|
"last4": "4242",
|
||||||
|
"network": "visa",
|
||||||
|
"three_d_secure": null,
|
||||||
|
"wallet": null
|
||||||
|
},
|
||||||
|
"type": "card"
|
||||||
|
},
|
||||||
|
"receipt_email": null,
|
||||||
|
"receipt_number": null,
|
||||||
|
"receipt_url": "https://pay.stripe.com/receipts/acct_1IGnMkIoFf3wvXpR/ch_1JBOVRIoFf3wvXpRCYVnRz0y/rcpt_Jp2aVQIwbWqn8CZ7T5uF9Wr7MX016cT",
|
||||||
|
"refunded": false,
|
||||||
|
"refunds": {
|
||||||
|
"object": "list",
|
||||||
|
"data": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"has_more": false,
|
||||||
|
"total_count": 0,
|
||||||
|
"url": "/v1/charges/ch_1JBOVRIoFf3wvXpRCYVnRz0y/refunds"
|
||||||
|
},
|
||||||
|
"review": null,
|
||||||
|
"shipping": null,
|
||||||
|
"source": null,
|
||||||
|
"source_transfer": null,
|
||||||
|
"statement_descriptor": null,
|
||||||
|
"statement_descriptor_suffix": null,
|
||||||
|
"status": "succeeded",
|
||||||
|
"transfer_data": null,
|
||||||
|
"transfer_group": null
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"has_more": false,
|
||||||
|
"total_count": 1,
|
||||||
|
"url": "/v1/charges?payment_intent=pi_1JBOVQIoFf3wvXpRRJIbx8xf"
|
||||||
|
},
|
||||||
|
"client_secret": "pi_1JBOVQIoFf3wvXpRRJIbx8xf_secret_sP4fa41NT86GIrhcyfEZEmFJE",
|
||||||
|
"confirmation_method": "automatic",
|
||||||
|
"created": 1625855576,
|
||||||
|
"currency": "jpy",
|
||||||
|
"customer": "cus_Jp2a7jolV9KaII",
|
||||||
|
"description": "Payment for Invoice",
|
||||||
|
"invoice": "in_1JBOVPIoFf3wvXpRuM28s36R",
|
||||||
|
"last_payment_error": null,
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"next_action": null,
|
||||||
|
"on_behalf_of": null,
|
||||||
|
"payment_method": "pm_1JBOVRIoFf3wvXpR3JKo6m2a",
|
||||||
|
"payment_method_options": {
|
||||||
|
"card": {
|
||||||
|
"installments": null,
|
||||||
|
"network": null,
|
||||||
|
"request_three_d_secure": "automatic"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"payment_method_types": [
|
||||||
|
"card"
|
||||||
|
],
|
||||||
|
"receipt_email": null,
|
||||||
|
"review": null,
|
||||||
|
"setup_future_usage": null,
|
||||||
|
"shipping": null,
|
||||||
|
"source": null,
|
||||||
|
"statement_descriptor": null,
|
||||||
|
"statement_descriptor_suffix": null,
|
||||||
|
"status": "succeeded",
|
||||||
|
"transfer_data": null,
|
||||||
|
"transfer_group": null
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
*** Percentage fee in a subscription
|
||||||
|
#+caption: Creating a subscription with a recurring JPY fee for a US (USD) account
|
||||||
|
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||||
|
#+header: :var product="prod_JgnmcPS2MRwDtp"
|
||||||
|
#+header: :var price="price_1JBOBRIoFf3wvXpR24iMvhoG"
|
||||||
|
#+header: :exports both :results output :wrap src json :eval no-export
|
||||||
|
#+begin_src sh
|
||||||
|
price=$(stripe prices create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--product=$product \
|
||||||
|
--unit-amount=11500 \
|
||||||
|
--currency=vnd \
|
||||||
|
-d "recurring[interval]"="month" \
|
||||||
|
-d "nickname"="Recurring Price in VND" \
|
||||||
|
| jq -r .id)
|
||||||
|
payment_method=$(stripe payment_methods create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--type=card \
|
||||||
|
-d "card[number]"=4242424242424242 \
|
||||||
|
-d "card[exp_month]"=3 \
|
||||||
|
-d "card[exp_year]"=2022 \
|
||||||
|
-d "card[cvc]"=314 \
|
||||||
|
| jq -r '.id')
|
||||||
|
customer=$(stripe customers create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--description="JPY Subscription Customer" \
|
||||||
|
--name="Correl Roush" \
|
||||||
|
--email="correl+stripe.roam.docs@gmail.com" \
|
||||||
|
| jq -r .id)
|
||||||
|
stripe payment_methods attach "$payment_method" \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--customer="$customer" >/dev/null
|
||||||
|
stripe customers update "$customer" \
|
||||||
|
--stripe-account=$account \
|
||||||
|
-d "invoice_settings[default_payment_method]=$payment_method" >/dev/null
|
||||||
|
stripe subscriptions create \
|
||||||
|
--stripe-account=$account \
|
||||||
|
--customer="$customer" \
|
||||||
|
--application-fee-percent=1 \
|
||||||
|
-d "items[0][price]"=$price
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"id": "sub_JqAatXRV9aIjfK",
|
||||||
|
"object": "subscription",
|
||||||
|
"application_fee_percent": 1.0,
|
||||||
|
"automatic_tax": {
|
||||||
|
"enabled": false
|
||||||
|
},
|
||||||
|
"billing_cycle_anchor": 1626115944,
|
||||||
|
"billing_thresholds": null,
|
||||||
|
"cancel_at": null,
|
||||||
|
"cancel_at_period_end": false,
|
||||||
|
"canceled_at": null,
|
||||||
|
"collection_method": "charge_automatically",
|
||||||
|
"created": 1626115944,
|
||||||
|
"current_period_end": 1628794344,
|
||||||
|
"current_period_start": 1626115944,
|
||||||
|
"customer": "cus_JqAaapAUdcr46Z",
|
||||||
|
"days_until_due": null,
|
||||||
|
"default_payment_method": null,
|
||||||
|
"default_source": null,
|
||||||
|
"default_tax_rates": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"discount": null,
|
||||||
|
"ended_at": null,
|
||||||
|
"items": {
|
||||||
|
"object": "list",
|
||||||
|
"data": [
|
||||||
|
{
|
||||||
|
"id": "si_JqAaoXdAhBEisj",
|
||||||
|
"object": "subscription_item",
|
||||||
|
"billing_thresholds": null,
|
||||||
|
"created": 1626115944,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"plan": {
|
||||||
|
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||||
|
"object": "plan",
|
||||||
|
"active": true,
|
||||||
|
"aggregate_usage": null,
|
||||||
|
"amount": 11500,
|
||||||
|
"amount_decimal": "11500",
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1626115939,
|
||||||
|
"currency": "vnd",
|
||||||
|
"interval": "month",
|
||||||
|
"interval_count": 1,
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Recurring Price in VND",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_usage": null,
|
||||||
|
"trial_period_days": null,
|
||||||
|
"usage_type": "licensed"
|
||||||
|
},
|
||||||
|
"price": {
|
||||||
|
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||||
|
"object": "price",
|
||||||
|
"active": true,
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1626115939,
|
||||||
|
"currency": "vnd",
|
||||||
|
"livemode": false,
|
||||||
|
"lookup_key": null,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Recurring Price in VND",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"recurring": {
|
||||||
|
"aggregate_usage": null,
|
||||||
|
"interval": "month",
|
||||||
|
"interval_count": 1,
|
||||||
|
"trial_period_days": null,
|
||||||
|
"usage_type": "licensed"
|
||||||
|
},
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_quantity": null,
|
||||||
|
"type": "recurring",
|
||||||
|
"unit_amount": 11500,
|
||||||
|
"unit_amount_decimal": "11500"
|
||||||
|
},
|
||||||
|
"quantity": 1,
|
||||||
|
"subscription": "sub_JqAatXRV9aIjfK",
|
||||||
|
"tax_rates": [
|
||||||
|
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"has_more": false,
|
||||||
|
"total_count": 1,
|
||||||
|
"url": "/v1/subscription_items?subscription=sub_JqAatXRV9aIjfK"
|
||||||
|
},
|
||||||
|
"latest_invoice": "in_1JCUEuIoFf3wvXpRl1be4Ymf",
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"next_pending_invoice_item_invoice": null,
|
||||||
|
"pause_collection": null,
|
||||||
|
"pending_invoice_item_interval": null,
|
||||||
|
"pending_setup_intent": null,
|
||||||
|
"pending_update": null,
|
||||||
|
"plan": {
|
||||||
|
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||||
|
"object": "plan",
|
||||||
|
"active": true,
|
||||||
|
"aggregate_usage": null,
|
||||||
|
"amount": 11500,
|
||||||
|
"amount_decimal": "11500",
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1626115939,
|
||||||
|
"currency": "vnd",
|
||||||
|
"interval": "month",
|
||||||
|
"interval_count": 1,
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Recurring Price in VND",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_usage": null,
|
||||||
|
"trial_period_days": null,
|
||||||
|
"usage_type": "licensed"
|
||||||
|
},
|
||||||
|
"quantity": 1,
|
||||||
|
"schedule": null,
|
||||||
|
"start_date": 1626115944,
|
||||||
|
"status": "active",
|
||||||
|
"transfer_data": null,
|
||||||
|
"trial_end": null,
|
||||||
|
"trial_start": null
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+caption: Fetching the initial invoice
|
||||||
|
#+header: :var account="acct_1IGnMkIoFf3wvXpR"
|
||||||
|
#+header: :exports both :results output :wrap src json :eval no-export
|
||||||
|
#+begin_src sh
|
||||||
|
stripe get in_1JCUEuIoFf3wvXpRl1be4Ymf \
|
||||||
|
--stripe-account=$account
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_src json
|
||||||
|
{
|
||||||
|
"id": "in_1JCUEuIoFf3wvXpRl1be4Ymf",
|
||||||
|
"object": "invoice",
|
||||||
|
"account_country": "US",
|
||||||
|
"account_name": "Correl's Stuff",
|
||||||
|
"account_tax_ids": null,
|
||||||
|
"amount_due": 11500,
|
||||||
|
"amount_paid": 11500,
|
||||||
|
"amount_remaining": 0,
|
||||||
|
"application_fee_amount": 115,
|
||||||
|
"attempt_count": 1,
|
||||||
|
"attempted": true,
|
||||||
|
"auto_advance": false,
|
||||||
|
"automatic_tax": {
|
||||||
|
"enabled": false,
|
||||||
|
"status": null
|
||||||
|
},
|
||||||
|
"billing_reason": "subscription_create",
|
||||||
|
"charge": "ch_1JCUEuIoFf3wvXpRB1M6kEQz",
|
||||||
|
"collection_method": "charge_automatically",
|
||||||
|
"created": 1626115944,
|
||||||
|
"currency": "vnd",
|
||||||
|
"custom_fields": null,
|
||||||
|
"customer": "cus_JqAaapAUdcr46Z",
|
||||||
|
"customer_address": null,
|
||||||
|
"customer_email": "correl+stripe.roam.docs@gmail.com",
|
||||||
|
"customer_name": "Correl Roush",
|
||||||
|
"customer_phone": null,
|
||||||
|
"customer_shipping": null,
|
||||||
|
"customer_tax_exempt": "none",
|
||||||
|
"customer_tax_ids": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"default_payment_method": null,
|
||||||
|
"default_source": null,
|
||||||
|
"default_tax_rates": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"description": null,
|
||||||
|
"discount": null,
|
||||||
|
"discounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"due_date": null,
|
||||||
|
"ending_balance": 0,
|
||||||
|
"footer": null,
|
||||||
|
"hosted_invoice_url": "https://invoice.stripe.com/i/acct_1IGnMkIoFf3wvXpR/invst_JqAahNswSL7xgr1R8bfwCjJA4V0UouX",
|
||||||
|
"invoice_pdf": "https://pay.stripe.com/invoice/acct_1IGnMkIoFf3wvXpR/invst_JqAahNswSL7xgr1R8bfwCjJA4V0UouX/pdf",
|
||||||
|
"last_finalization_error": null,
|
||||||
|
"lines": {
|
||||||
|
"object": "list",
|
||||||
|
"data": [
|
||||||
|
{
|
||||||
|
"id": "il_1JCUEuIoFf3wvXpRcrDHYRXJ",
|
||||||
|
"object": "line_item",
|
||||||
|
"amount": 11500,
|
||||||
|
"currency": "vnd",
|
||||||
|
"description": "1 × Example Product (Org-Roam Doc) (at ₫11,500 / month)",
|
||||||
|
"discount_amounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"discountable": true,
|
||||||
|
"discounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"period": {
|
||||||
|
"end": 1628794344,
|
||||||
|
"start": 1626115944
|
||||||
|
},
|
||||||
|
"plan": {
|
||||||
|
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||||
|
"object": "plan",
|
||||||
|
"active": true,
|
||||||
|
"aggregate_usage": null,
|
||||||
|
"amount": 11500,
|
||||||
|
"amount_decimal": "11500",
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1626115939,
|
||||||
|
"currency": "vnd",
|
||||||
|
"interval": "month",
|
||||||
|
"interval_count": 1,
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Recurring Price in VND",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_usage": null,
|
||||||
|
"trial_period_days": null,
|
||||||
|
"usage_type": "licensed"
|
||||||
|
},
|
||||||
|
"price": {
|
||||||
|
"id": "price_1JCUEpIoFf3wvXpRNX67TZZB",
|
||||||
|
"object": "price",
|
||||||
|
"active": true,
|
||||||
|
"billing_scheme": "per_unit",
|
||||||
|
"created": 1626115939,
|
||||||
|
"currency": "vnd",
|
||||||
|
"livemode": false,
|
||||||
|
"lookup_key": null,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"nickname": "Recurring Price in VND",
|
||||||
|
"product": "prod_JgnmcPS2MRwDtp",
|
||||||
|
"recurring": {
|
||||||
|
"aggregate_usage": null,
|
||||||
|
"interval": "month",
|
||||||
|
"interval_count": 1,
|
||||||
|
"trial_period_days": null,
|
||||||
|
"usage_type": "licensed"
|
||||||
|
},
|
||||||
|
"tiers_mode": null,
|
||||||
|
"transform_quantity": null,
|
||||||
|
"type": "recurring",
|
||||||
|
"unit_amount": 11500,
|
||||||
|
"unit_amount_decimal": "11500"
|
||||||
|
},
|
||||||
|
"proration": false,
|
||||||
|
"quantity": 1,
|
||||||
|
"subscription": "sub_JqAatXRV9aIjfK",
|
||||||
|
"subscription_item": "si_JqAaoXdAhBEisj",
|
||||||
|
"tax_amounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"tax_rates": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"type": "subscription"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"has_more": false,
|
||||||
|
"total_count": 1,
|
||||||
|
"url": "/v1/invoices/in_1JCUEuIoFf3wvXpRl1be4Ymf/lines"
|
||||||
|
},
|
||||||
|
"livemode": false,
|
||||||
|
"metadata": {
|
||||||
|
},
|
||||||
|
"next_payment_attempt": null,
|
||||||
|
"number": "CC4DCDC8-0001",
|
||||||
|
"on_behalf_of": null,
|
||||||
|
"paid": true,
|
||||||
|
"payment_intent": "pi_1JCUEuIoFf3wvXpRYSMrrCoL",
|
||||||
|
"payment_settings": {
|
||||||
|
"payment_method_options": null,
|
||||||
|
"payment_method_types": null
|
||||||
|
},
|
||||||
|
"period_end": 1626115944,
|
||||||
|
"period_start": 1626115944,
|
||||||
|
"post_payment_credit_notes_amount": 0,
|
||||||
|
"pre_payment_credit_notes_amount": 0,
|
||||||
|
"receipt_number": "2336-2115",
|
||||||
|
"starting_balance": 0,
|
||||||
|
"statement_descriptor": null,
|
||||||
|
"status": "paid",
|
||||||
|
"status_transitions": {
|
||||||
|
"finalized_at": 1626115944,
|
||||||
|
"marked_uncollectible_at": null,
|
||||||
|
"paid_at": 1626115944,
|
||||||
|
"voided_at": null
|
||||||
|
},
|
||||||
|
"subscription": "sub_JqAatXRV9aIjfK",
|
||||||
|
"subtotal": 11500,
|
||||||
|
"tax": null,
|
||||||
|
"total": 11500,
|
||||||
|
"total_discount_amounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"total_tax_amounts": [
|
||||||
|
|
||||||
|
],
|
||||||
|
"transfer_data": null,
|
||||||
|
"webhooks_delivered_at": 1626115947
|
||||||
|
}
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
* What do we do about sales tracking?
|
||||||
|
- Start storing currency with sales
|
||||||
|
+ Add a currency column to the analytics database
|
||||||
|
+ Add a currency field to the avro event
|
||||||
|
- +Hide non-usd transactions from reports, etc. until they are ready?+
|
||||||
|
- Beware of mixed-currency totals
|
||||||
|
|
||||||
|
* Tasks
|
||||||
|
** Add an endpoint exposing an account's supported currencies
|
||||||
|
** Remove =usd= restriction to the currency field in Stripe requests
|
||||||
|
** Add acceptance tests for multiple currencies
|
||||||
|
** Reporting
|
||||||
|
*** Add a currency column to the analytics database
|
||||||
|
*** Add a currency field to the pageview avro event
|
||||||
|
*** Update page hits consumer to store the currency field
|
||||||
|
*** Update reports
|
||||||
|
Both classic and redesigned reports need to be updated to handle multiple
|
||||||
|
currencies.
|
||||||
|
|
||||||
|
- Totals must be broken down by currency
|
||||||
|
- Values must be displayed in a manner appropriate for its currency
|
||||||
|
|
||||||
|
It'll be more straightforward to duplicate the graph/table for each currency or
|
||||||
|
otherwise filter them rather than attempt to restructure them to accomodate
|
||||||
|
another dimension (currency).
|
||||||
|
|
||||||
|
- Do we want to eliminate the old reports? Or at least the old sales over time
|
||||||
|
report? The new sales over time report will need the activity detail section.
|
||||||
|
+ /Yes, provided it's doable in a similar or shorter amount of time./
|
||||||
|
**** Revenue Over Time
|
||||||
|
https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/analytics_charts_controller.php
|
||||||
|
**** Sales over time
|
||||||
|
*** Update display of sale value in subscriber details
|
||||||
|
We'll want to either find a way to mimic the behavior of the frontend currency
|
||||||
|
display library, or pass the value through in such a way that we could use that
|
||||||
|
library.
|
||||||
199
aweber/20210630150008-stripe_poller.org
Normal file
199
aweber/20210630150008-stripe_poller.org
Normal file
|
|
@ -0,0 +1,199 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 83d61eef-0781-46e0-b959-1a739cff5ea3
|
||||||
|
:END:
|
||||||
|
#+title: Stripe Poller
|
||||||
|
|
||||||
|
Identify webhook events tracked by Stripe that have not yet been processed by
|
||||||
|
our service, and replay them against it.
|
||||||
|
|
||||||
|
* Investigation
|
||||||
|
** Fetching events on behalf of each connected account
|
||||||
|
On the afternoon of [[id:8c85f055-9d0c-4b7b-991e-1e32905d38ba][2021-07-02]] I fetched the 860 connected accounts and fetched
|
||||||
|
webhook events from the previous two hours. Each request to Stripe took, on
|
||||||
|
average, 0.38 seconds, requiring one request per account, totalling 323.77
|
||||||
|
seconds, or nearly five and a half minutes. No degradation nor failure of Stripe
|
||||||
|
functionality was noted. From all of these requests, a total of 19 events were
|
||||||
|
retrieved.
|
||||||
|
** Alternatives
|
||||||
|
*** The Stripe dashboard
|
||||||
|
Viewing a particular webhook within the Stripe dashboard allows us to view the
|
||||||
|
events sent specifically to that webhook, independent of the account to which it
|
||||||
|
relates. Regrettably, this is fed from a dashboard-specific API which does not
|
||||||
|
appear to be exposed elsewhere for consumption.
|
||||||
|
*** Email notification
|
||||||
|
We are emailed when a webhook event fails to process. It may make sense to alert
|
||||||
|
when this occurs and correct the issue manually, or find a way to automate its
|
||||||
|
replay.
|
||||||
|
** Conclusion
|
||||||
|
Barring the release of an API to fetch all events sent to a single webhook,
|
||||||
|
ideally filtered by delivery status, fetching these events from Stripe is
|
||||||
|
terribly inefficient. Though we can process recent events in a timely manner, we
|
||||||
|
will likely be further slowed as more customers connect with Stripe, and there
|
||||||
|
is no good solution at this time for dealing with that problem. Given the
|
||||||
|
infrequency with which Stripe fails to send us an event via their retry
|
||||||
|
mechanisms, I expect we will be better served by reacting to notifications of
|
||||||
|
those failures than to routinely slam their API with inefficient requests.
|
||||||
|
* Tasks
|
||||||
|
** Build Poller
|
||||||
|
*** Prepare a rundeck job for the Stripe poller
|
||||||
|
- Create the project
|
||||||
|
- Build the rundeck job
|
||||||
|
*** Add a stripe-payments endpoint to fetch logged events
|
||||||
|
- Needs only return event ids
|
||||||
|
- Events are partitioned by date and sorted by time in the time-index GSI, which projects keys only
|
||||||
|
|
||||||
|
#+begin_src yaml
|
||||||
|
paths:
|
||||||
|
/stripe/events:
|
||||||
|
get:
|
||||||
|
summary: Fetch webhook events
|
||||||
|
tags:
|
||||||
|
- Webhooks
|
||||||
|
parameters:
|
||||||
|
- name: last_id
|
||||||
|
in: query
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/EventId'
|
||||||
|
- name: limit
|
||||||
|
in: query
|
||||||
|
description: Number of events to return per page of results
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 1000
|
||||||
|
default: 100
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Event list
|
||||||
|
headers:
|
||||||
|
Link:
|
||||||
|
description: Pagination links
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
example: >-
|
||||||
|
<{{ base_url }}/stripe/events?last_id=evt_1234abcdef>; rel="next"
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: '#/components/schemas/EventId'
|
||||||
|
components:
|
||||||
|
schemas:
|
||||||
|
EventId:
|
||||||
|
type: string
|
||||||
|
description: Unique identifier for the webhook event
|
||||||
|
example: "evt_1IHAsBIoFf3wvXpR7VLvGfae"
|
||||||
|
|
||||||
|
#+end_src
|
||||||
|
*** Add integrations endpoint enumerating connected Stripe accounts
|
||||||
|
*** Reprocess unlogged Stripe events
|
||||||
|
- Iterate processed event ids into memory
|
||||||
|
- Iterate over published events, sending them to the stripe-payments webhook
|
||||||
|
endpoint if they are not in the processed set
|
||||||
|
|
||||||
|
#+begin_src plantuml :file stripe-poller.svg
|
||||||
|
loop until last page of results
|
||||||
|
Poller -> StripePayments : GET /stripe/events?since=<timestamp>[&last_id=<last_id>]
|
||||||
|
StripePayments --> Poller : Return list of event IDs
|
||||||
|
end
|
||||||
|
loop until last page of results
|
||||||
|
Poller -> Integrations : Get list of connected accounts
|
||||||
|
Integrations --> Poller : Return list of Stripe account IDs
|
||||||
|
loop for each account
|
||||||
|
loop until last page of results
|
||||||
|
Poller -> Stripe : GET /v1/events?created[gte]=<timestamp>&types[]=<...>[&starting_after=<last_id>]
|
||||||
|
StripePayments --> Poller : Return list of event IDs
|
||||||
|
alt is an AWeber Ecommerce event and event not in processed
|
||||||
|
Poller -> StripePayments : POST /stripe/webhooks
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
[[file:stripe-poller.svg]]
|
||||||
|
|
||||||
|
**** Determining how far back to look for events
|
||||||
|
Stripe events are retained for a maximum of thirty days, on their end and also
|
||||||
|
in our event database.
|
||||||
|
|
||||||
|
The job could use consul to store the time of the latest fetched event to be
|
||||||
|
referenced in subsequent runs. If it is set, it should collect all events newer
|
||||||
|
than one hour prior to that time (this window may also be configurable). If that
|
||||||
|
time is not set, it should collect all available events. This value should be
|
||||||
|
updated in consul only when the job completes successfully.
|
||||||
|
|
||||||
|
Hit the Rundeck API to get the time of the last successful execution.
|
||||||
|
|
||||||
|
**** Retrieving webhook events
|
||||||
|
https://stripe.com/docs/api/events/list
|
||||||
|
|
||||||
|
+ Events can be filtered by type, creation time, and whether they were
|
||||||
|
successfully delivered (the webhook endpoint returned a =200 OK= response)
|
||||||
|
- Wait, how does it know it was succesfully delivered to /our/ webhook
|
||||||
|
endpoint?
|
||||||
|
+ Should we set up another "client" with its own rate limiting with Stripe for
|
||||||
|
the poller's requests?
|
||||||
|
|
||||||
|
#+CAPTION: Support chat on [2021-07-01 Thu]
|
||||||
|
#+begin_quote
|
||||||
|
- My team is hoping to build an automated process to check via the Stripe API
|
||||||
|
for any events that weren't successfully delivered to our webhook endpoint and
|
||||||
|
see that they're handled appropriately. I've been trying out the events list
|
||||||
|
endpoint, and it does not seem to return all of the events that were sent to
|
||||||
|
our webhook, nor does there appear to be any way to identify a webhook to that
|
||||||
|
endpoint to find and filter events for it. We assume this is because the
|
||||||
|
events are triggered from connected accounts. We noticed that the Stripe
|
||||||
|
dashboard is able to show events sent to a webhook and their status, is there
|
||||||
|
some way to achieve this through the API?
|
||||||
|
+ /Hector Otero has joined./
|
||||||
|
+ Hi, there. I am glad to help.
|
||||||
|
- Hello. Are you able to see my previous message?
|
||||||
|
+ Yes, I am just reading through it.
|
||||||
|
+ Ok, so your main question is if there is a way to show events sent to a webhook and their status using the API, similar to how it is done on the Stripe dashboard.
|
||||||
|
+ Is that correct?
|
||||||
|
- Correct
|
||||||
|
+ Ok, my brief look into the documentation has not turned up anything regarding how to implement this. I am going to consult with my team members regarding whether any of them knows how to implement this functionality. Once I have more info I can reach out to you via email, is that alright?
|
||||||
|
- Yes, thank you.
|
||||||
|
+ Sure, no worries. Please keep an eye on your email inbox awapi@aweber.com for further correspondence regarding this issue.
|
||||||
|
+ Have a nice day! we will be in contact soon.
|
||||||
|
- Thanks!
|
||||||
|
+ /Hector Otero has left./
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
**** Signing the webhook request
|
||||||
|
Webhook events [[https://stripe.com/docs/webhooks/signatures][must be signed using a secret key]], which we store [[http://consul.service.production.consul/ui/production/kv/services/cp/services/stripe-payments/stripe_webhook_secret/edit][in consul]]. The
|
||||||
|
following /should/ result in a valid signature header:
|
||||||
|
|
||||||
|
#+CAPTION: Stripe signature generation example
|
||||||
|
#+begin_src python :results output :exports code
|
||||||
|
import hmac
|
||||||
|
import hashlib
|
||||||
|
import time
|
||||||
|
|
||||||
|
unix_timestamp = int(time.time())
|
||||||
|
json_payload = '{ ... }'
|
||||||
|
secret_key = 'secret'
|
||||||
|
|
||||||
|
message = f'{unix_timestamp}.{json_payload}'
|
||||||
|
signature = hmac.new(bytes(secret_key, 'utf-8'),
|
||||||
|
msg=bytes(message, 'utf-8'),
|
||||||
|
digestmod=hashlib.sha256).hexdigest()
|
||||||
|
|
||||||
|
print(f'Stripe-Signature: t={unix_timestamp},v1={signature}')
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
: Stripe-Signature: t=1625084385,v1=dce1ef0332969bce98fd76b5fd08d1b07af0d0fd5f9788d9f8435537e5c3cd12
|
||||||
|
*** Create playbook and dashboard for the Stripe poller
|
||||||
|
- Track how many events are resent for processing vs how many are already processed
|
||||||
|
** KILL Create an event processing endpoint
|
||||||
|
Create an endpoint in the Stripe Payments service for internal use that will,
|
||||||
|
given an event id, fetch that event from Stripe and process it as though it had
|
||||||
|
been sent to the webhook endpoint.
|
||||||
|
** TODO Document how to find and replay a Stripe event
|
||||||
|
- Use the Stripe UI to find failed events within the past 15 days
|
||||||
|
- Include steps for fetching an event from more than 15 days ago from the API
|
||||||
|
and sending that to the webhook endpoint.
|
||||||
5
aweber/20210706161607-supporting_multiple_time_zones.org
Normal file
5
aweber/20210706161607-supporting_multiple_time_zones.org
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 89c2dda6-46d7-41c9-8af7-18ce604a2daf
|
||||||
|
:END:
|
||||||
|
#+title: Supporting multiple time zones
|
||||||
|
* Initial investigation
|
||||||
31
aweber/20210714102625-deploying_projects.org
Normal file
31
aweber/20210714102625-deploying_projects.org
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: e06b26c8-9227-4fcc-8f0a-9b83c64693b4
|
||||||
|
:END:
|
||||||
|
#+title: Deploying projects
|
||||||
|
* Environments
|
||||||
|
- Testing :: Newly merged code is automatically deployed to this environment to
|
||||||
|
be tested.
|
||||||
|
- Staging :: Tagged releases are automatically deployed to this environment for
|
||||||
|
spot-checking prior to production release.
|
||||||
|
- Production :: Tagged releases are manually deployed to the live production
|
||||||
|
environment.
|
||||||
|
|
||||||
|
* Deployment methods
|
||||||
|
** Gitlab CI
|
||||||
|
Projects define pipelines in a =.gitlab-ci.yml= file to automate running tests,
|
||||||
|
building the project, and deploying it to our three platform environments.
|
||||||
|
** Jenkins :deprecated:
|
||||||
|
Pipelines are defined in Jenkins to react to pushed and tagged code in source
|
||||||
|
control to run tests and deploy projects to our platform environments.
|
||||||
|
** Chef / Puppet :deprecated:
|
||||||
|
* Deployment targets
|
||||||
|
** Buzzops (Local Kubernetes)
|
||||||
|
https://confluence.aweber.io/display/STD/Kubernetes+Application+Deployment
|
||||||
|
** Amazon Web Services
|
||||||
|
When appropriate, dockerized applications may be deployed to Amazon ECS
|
||||||
|
** Chef and Puppet managed virtual machines :deprecated:
|
||||||
|
* Further information
|
||||||
|
** Front-End Applications
|
||||||
|
https://confluence.aweber.io/display/FEBOF/Web+App+Deployment
|
||||||
|
** Control Panel (Sites)
|
||||||
|
https://confluence.aweber.io/display/AR/Control+Panel+%28Sites%29+Playbook
|
||||||
6
aweber/20210714105000-team_member_onboarding.org
Normal file
6
aweber/20210714105000-team_member_onboarding.org
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: bf5d1146-9481-4710-8143-61086f263a7a
|
||||||
|
:END:
|
||||||
|
#+title: Team Member Onboarding
|
||||||
|
|
||||||
|
- [[id:e06b26c8-9227-4fcc-8f0a-9b83c64693b4][Deploying projects]]
|
||||||
20
aweber/20210810095435-deploying_s4_to_kubernetes.org
Normal file
20
aweber/20210810095435-deploying_s4_to_kubernetes.org
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 9cfd85fd-998e-4f21-b82e-c7963576c202
|
||||||
|
:END:
|
||||||
|
#+title: Deploying S4 to Kubernetes
|
||||||
|
* DONE Deploying the service
|
||||||
|
* Migrating the Redis database
|
||||||
|
* Deploying the workers
|
||||||
|
** DONE Use environment variables for configuration rather than consul
|
||||||
|
https://gitlab.aweber.io/CP/Rundeck/s4-utils/-/commit/8d4dd3b8196bcd716a70e1c751b4743fdaa62646
|
||||||
|
** DONE Set up all the rundeck jobs in testing
|
||||||
|
- Configure the rundeck jobs
|
||||||
|
- Ensure all ACL tokens are accounted for
|
||||||
|
** TODO Update the appdb credentials
|
||||||
|
- [ ] Testing
|
||||||
|
- [ ] Staging
|
||||||
|
- [ ] Production
|
||||||
|
** TODO Set up all the rundeck jobs in staging
|
||||||
|
** TODO Set up all the rundeck jobs in production
|
||||||
|
** TODO Add missing config values
|
||||||
|
- IP ranges =lock_key= and =lock_minutes=
|
||||||
68
aweber/20210813142844-projects.org
Normal file
68
aweber/20210813142844-projects.org
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 207560cc-7700-4d06-918d-cc01ae530146
|
||||||
|
:END:
|
||||||
|
#+title: Projects
|
||||||
|
#+STARTUP: indent logdrawer
|
||||||
|
#+COLUMNS: %TAGS %JIRA_ID %50ITEM %TODO %4StoryPoints{+} %COMPONENT %BLOCKER
|
||||||
|
#+PROPERTY: StoryPoints_ALL 0 1 2 3 5 8 13 20 40 100
|
||||||
|
#+PROPERTY: Effort_ALL 0:30 1:00 0.5d 1d 2d 3d 4d 1w
|
||||||
|
#+PROPERTY: ClassificationOfWork_ALL backend frontend ops product design
|
||||||
|
#+TODO: TODO(t!) BACKLOG(b!) RE-EVALUATE(r!) | DONE(d@!) CANCELLED(c@!)
|
||||||
|
#+TAGS: { SPRINT(S) EPIC(e) STORY(s) BUG(b) TASK(t) }
|
||||||
|
#+OPTIONS: num:nil toc:t arch:nil p:t prop:t
|
||||||
|
#+LINK: jira https://jira.aweber.io/browse/
|
||||||
|
|
||||||
|
* Service Upgrades
|
||||||
|
** DONE [[id:9cfd85fd-998e-4f21-b82e-c7963576c202][Deploying S4 to Kubernetes]]
|
||||||
|
:PROPERTIES:
|
||||||
|
:JIRA_ID: CCPANEL-10549
|
||||||
|
:END:
|
||||||
|
:LOGBOOK:
|
||||||
|
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||||
|
:END:
|
||||||
|
** TODO [[id:6413d680-ee2e-43e6-b7c7-10f14e0873c2][Deploying Bulk Tagging to Kubernetes]]
|
||||||
|
:PROPERTIES:
|
||||||
|
:JIRA_ID: CCPANEL-11615
|
||||||
|
:END:
|
||||||
|
:LOGBOOK:
|
||||||
|
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||||
|
:END:
|
||||||
|
** TODO Deploying Domain Validator to Kubernetes
|
||||||
|
:PROPERTIES:
|
||||||
|
:JIRA_ID: CCPANEL-10554
|
||||||
|
:END:
|
||||||
|
:LOGBOOK:
|
||||||
|
- State "TODO" from [2021-09-01 Wed 13:42]
|
||||||
|
:END:
|
||||||
|
** TODO GeoIP
|
||||||
|
:PROPERTIES:
|
||||||
|
:JIRA_ID: CCPANEL-11592
|
||||||
|
:END:
|
||||||
|
:LOGBOOK:
|
||||||
|
- State "TODO" from [2021-09-01 Wed 13:44]
|
||||||
|
:END:
|
||||||
|
* Tracking live vs dead / removed code branches in Sites
|
||||||
|
* Search Service
|
||||||
|
* Analytics View Service
|
||||||
|
* Replace CAPI Services
|
||||||
|
** List API
|
||||||
|
*** TODO Set EOL date for awlists
|
||||||
|
- [2021-08-13 Fri 15:21] :: Discussed this. Also talked about separation of
|
||||||
|
concerns about account status vs list status. Also discussed how an
|
||||||
|
entitlements service might fit into our architecture and how we handle state
|
||||||
|
transitions and reverals (e.g. cancellations).
|
||||||
|
- [2021-08-17 Tue 16:44] :: Set a one-year time limit? Should the public list
|
||||||
|
endpoints be in the new service as well, deprecating public api lists?
|
||||||
|
** Subscribers API
|
||||||
|
* Frontend Client Upgrades
|
||||||
|
** Upgrade Dashboard to React
|
||||||
|
*** TODO Need an API for broadcasts and sent messages across lists
|
||||||
|
:PROPERTIES:
|
||||||
|
:JIRA_ID: CCPANEL-11609
|
||||||
|
:END:
|
||||||
|
:LOGBOOK:
|
||||||
|
- State "TODO" from [2021-09-01 Wed 13:33]
|
||||||
|
:END:
|
||||||
|
** Upgrade other non-React projects to React
|
||||||
|
*** Add subscriber
|
||||||
|
* New List Management Interface
|
||||||
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 6413d680-ee2e-43e6-b7c7-10f14e0873c2
|
||||||
|
:END:
|
||||||
|
#+title: Deploying Bulk Tagging to Kubernetes
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 92 KiB |
22
aweber/ecommerce-cancellation-webhooks.svg
Normal file
22
aweber/ecommerce-cancellation-webhooks.svg
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="242px" preserveAspectRatio="none" style="width:554px;height:242px;" version="1.1" viewBox="0 0 554 242" width="554px" zoomAndPan="magnify"><defs><filter height="300%" id="f1g5bqu9vn3r6s" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><rect fill="#DDDDDD" height="230.5293" style="stroke:#A80036;stroke-width:1.0;" width="85" x="395.5" y="6"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="53" x="411.5" y="18.5684">Internal</text><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="132" x2="132" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="437.5" x2="437.5" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="515.5" x2="515.5" y1="60.7988" y2="198.041"/><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="45.8457">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="217.5762">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="73" x="399.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="59" x="406.5" y="45.8457">Core API</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="73" x="399.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="59" x="406.5" y="217.5762">Core API</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="486.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="493.5" y="45.8457">Stripe</text><rect fill="#FEFECE" filter="url(#f1g5bqu9vn3r6s)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="486.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="493.5" y="217.5762">Stripe</text><polygon fill="#A80036" points="143,88.1094,133,92.1094,143,96.1094,139,92.1094" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="137" x2="514.5" y1="92.1094" y2="92.1094"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="359" x="149" y="87.3672">POST /stripe/webhooks (customer.subscription.updated)</text><polygon fill="#A80036" points="503.5,117.4199,513.5,121.4199,503.5,125.4199,507.5,121.4199" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="509.5" y1="121.4199" y2="121.4199"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="149" x="139" y="116.6777">Fetch product metadata</text><polygon fill="#A80036" points="426,146.7305,436,150.7305,426,154.7305,430,150.7305" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="432" y1="150.7305" y2="150.7305"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="282" x="139" y="145.9883">Remove tags from subscriber or unsubscribe</text><polygon fill="#A80036" points="503.5,176.041,513.5,180.041,503.5,184.041,507.5,180.041" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="509.5" y1="180.041" y2="180.041"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="46" x="139" y="175.2988">200 OK</text><!--MD5=[ad97cdfe8b8290a77b1ba215f8dc6abf]
|
||||||
|
@startuml
|
||||||
|
participant "Stripe Payments (Unauthenticated)" as sp
|
||||||
|
box "Internal"
|
||||||
|
participant "Core API" as capi
|
||||||
|
end box
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
stripe -> sp : POST /stripe/webhooks (customer.subscription.updated)
|
||||||
|
sp -> stripe : Fetch product metadata
|
||||||
|
sp -> capi : Remove tags from subscriber or unsubscribe
|
||||||
|
sp -> stripe : 200 OK
|
||||||
|
@enduml
|
||||||
|
|
||||||
|
PlantUML version 1.2021.00(Sun Jan 10 05:25:05 EST 2021)
|
||||||
|
(GPL source distribution)
|
||||||
|
Java Runtime: Java(TM) SE Runtime Environment
|
||||||
|
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||||
|
Default Encoding: UTF-8
|
||||||
|
Language: en
|
||||||
|
Country: US
|
||||||
|
--></g></svg>
|
||||||
|
After Width: | Height: | Size: 5.1 KiB |
22
aweber/ecommerce-payment-webhooks.svg
Normal file
22
aweber/ecommerce-payment-webhooks.svg
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="242px" preserveAspectRatio="none" style="width:638px;height:242px;" version="1.1" viewBox="0 0 638 242" width="638px" zoomAndPan="magnify"><defs><filter height="300%" id="fd0b0cv41yrbd" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><rect fill="#DDDDDD" height="230.5293" style="stroke:#A80036;stroke-width:1.0;" width="93" x="471.5" y="6"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="53" x="491.5" y="18.5684">Internal</text><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="132" x2="132" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="517.5" x2="517.5" y1="60.7988" y2="198.041"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="599.5" x2="599.5" y1="60.7988" y2="198.041"/><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="45.8457">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="250" x="5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="236" x="12" y="217.5762">Stripe Payments (Unauthenticated)</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="81" x="475.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="67" x="482.5" y="45.8457">RabbitMQ</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="81" x="475.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="67" x="482.5" y="217.5762">RabbitMQ</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="570.5" y="25.3105"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="577.5" y="45.8457">Stripe</text><rect fill="#FEFECE" filter="url(#fd0b0cv41yrbd)" height="30.4883" style="stroke:#A80036;stroke-width:1.5;" width="54" x="570.5" y="197.041"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="577.5" y="217.5762">Stripe</text><polygon fill="#A80036" points="143,88.1094,133,92.1094,143,96.1094,139,92.1094" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="137" x2="598.5" y1="92.1094" y2="92.1094"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="330" x="149" y="87.3672">POST /stripe/webhooks (payment_intent.succeeded)</text><polygon fill="#A80036" points="506,117.4199,516,121.4199,506,125.4199,510,121.4199" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="512" y1="121.4199" y2="121.4199"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="215" x="139" y="116.6777">Sales tracking event (pageview.v4)</text><polygon fill="#A80036" points="506,146.7305,516,150.7305,506,154.7305,510,150.7305" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="512" y1="150.7305" y2="150.7305"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="362" x="139" y="145.9883">Payment succeeded event (stripe_payment_succeeded.v1)</text><polygon fill="#A80036" points="587.5,176.041,597.5,180.041,587.5,184.041,591.5,180.041" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="132" x2="593.5" y1="180.041" y2="180.041"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="46" x="139" y="175.2988">200 OK</text><!--MD5=[407d974f4df3412fe8be153d8bb2d58c]
|
||||||
|
@startuml
|
||||||
|
participant "Stripe Payments (Unauthenticated)" as sp
|
||||||
|
box "Internal"
|
||||||
|
participant "RabbitMQ" as amqp
|
||||||
|
end box
|
||||||
|
participant "Stripe" as stripe
|
||||||
|
|
||||||
|
stripe -> sp : POST /stripe/webhooks (payment_intent.succeeded)
|
||||||
|
sp -> amqp : Sales tracking event (pageview.v4)
|
||||||
|
sp -> amqp : Payment succeeded event (stripe_payment_succeeded.v1)
|
||||||
|
sp -> stripe : 200 OK
|
||||||
|
@enduml
|
||||||
|
|
||||||
|
PlantUML version 1.2021.00(Sun Jan 10 05:25:05 EST 2021)
|
||||||
|
(GPL source distribution)
|
||||||
|
Java Runtime: Java(TM) SE Runtime Environment
|
||||||
|
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||||
|
Default Encoding: UTF-8
|
||||||
|
Language: en
|
||||||
|
Country: US
|
||||||
|
--></g></svg>
|
||||||
|
After Width: | Height: | Size: 5.1 KiB |
32
aweber/ecommerce-products.svg
Normal file
32
aweber/ecommerce-products.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 10 KiB |
34
aweber/ecommerce-subscribing.svg
Normal file
34
aweber/ecommerce-subscribing.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 9.9 KiB |
BIN
aweber/icons/details.png
Normal file
BIN
aweber/icons/details.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 744 B |
BIN
aweber/icons/flag-green.png
Normal file
BIN
aweber/icons/flag-green.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 583 B |
BIN
aweber/icons/flag-red.png
Normal file
BIN
aweber/icons/flag-red.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 578 B |
BIN
aweber/icons/flag-yellow.png
Normal file
BIN
aweber/icons/flag-yellow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 577 B |
BIN
aweber/icons/resource.png
Normal file
BIN
aweber/icons/resource.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 722 B |
BIN
aweber/icons/resourcegroup.png
Normal file
BIN
aweber/icons/resourcegroup.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 866 B |
BIN
aweber/icons/task.png
Normal file
BIN
aweber/icons/task.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 494 B |
BIN
aweber/icons/taskgroup.png
Normal file
BIN
aweber/icons/taskgroup.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 488 B |
BIN
aweber/icons/trend-down.png
Normal file
BIN
aweber/icons/trend-down.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 756 B |
BIN
aweber/icons/trend-flat.png
Normal file
BIN
aweber/icons/trend-flat.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 735 B |
BIN
aweber/icons/trend-up.png
Normal file
BIN
aweber/icons/trend-up.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 768 B |
Binary file not shown.
|
After Width: | Height: | Size: 744 B |
Binary file not shown.
|
After Width: | Height: | Size: 583 B |
Binary file not shown.
|
After Width: | Height: | Size: 578 B |
Binary file not shown.
|
After Width: | Height: | Size: 577 B |
Binary file not shown.
|
After Width: | Height: | Size: 722 B |
Binary file not shown.
|
After Width: | Height: | Size: 866 B |
Binary file not shown.
|
After Width: | Height: | Size: 494 B |
Binary file not shown.
|
After Width: | Height: | Size: 488 B |
Binary file not shown.
|
After Width: | Height: | Size: 756 B |
Binary file not shown.
|
After Width: | Height: | Size: 735 B |
Binary file not shown.
|
After Width: | Height: | Size: 768 B |
87
aweber/sites-release.org
Normal file
87
aweber/sites-release.org
Normal file
|
|
@ -0,0 +1,87 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 6c7250d0-6871-4030-98f2-2a53c6ca5eb3
|
||||||
|
:END:
|
||||||
|
#+TITLE: Sites Releases
|
||||||
|
#+STARTUP: indent
|
||||||
|
|
||||||
|
Effective Monday, March 23rd 2020, changes in the sites repository will be
|
||||||
|
released regularly on Tuesdays and Thursdays at 11AM. These releases will
|
||||||
|
contain *all code in the master branch at that time*, and will be performed by
|
||||||
|
the Control Panel teammate on call. Notifications of the release will be sent
|
||||||
|
via email and Slack out one hour prior to the repository being tagged to provide
|
||||||
|
time for removing anything that has been merged that is not ready for production
|
||||||
|
deployment.
|
||||||
|
|
||||||
|
Sites changes may still be released outside of these windows in the cases of
|
||||||
|
expedites, etc., but those should be the exception rather than the rule.
|
||||||
|
|
||||||
|
* Release procedure
|
||||||
|
|
||||||
|
The following steps are to be taken by the Control Panel person on call each
|
||||||
|
Tuesday and Thursday.
|
||||||
|
|
||||||
|
** At 10AM
|
||||||
|
*** Identify the changes going out
|
||||||
|
|
||||||
|
In your local clone of sites, check out the master branch, and update it to
|
||||||
|
match what's currently in the upstream repository. You'll then want to check the
|
||||||
|
list of changes since the last tag.
|
||||||
|
|
||||||
|
To get the most recent tag, you can run the following command:
|
||||||
|
: git describe --tags --abbrev=0
|
||||||
|
|
||||||
|
A script is available to pull information on tickets from Jira which will work
|
||||||
|
handily with git logs piped to it. You can find that script [[https://gitlab.aweber.io/correlr/jira-cli-tool][here]].
|
||||||
|
|
||||||
|
Running a git log for changes since the last tag and piping it to the jira
|
||||||
|
utility script will give you a report of the unique jira issues found and their
|
||||||
|
relevant details:
|
||||||
|
|
||||||
|
: git log 1.212.0.. | jira -i
|
||||||
|
|
||||||
|
#+begin_example
|
||||||
|
Issue Status Summary
|
||||||
|
CC-5343 In Development As Molly I want to be able to add an EXISTING custom field to my signup form on my landing page so that I can get more specific customer info
|
||||||
|
CCPANEL-10176 Closed INVESTIGATE: "OR" search on tags | Backend
|
||||||
|
CCPANEL-10294 Testing Update character limits for subscribers for consistency
|
||||||
|
INT-4298 Testing Integrations JS App: Setup pages
|
||||||
|
#+end_example
|
||||||
|
*** Notify teams via email
|
||||||
|
Send an announcement to the development mailing list:
|
||||||
|
|
||||||
|
#+begin_src message
|
||||||
|
To: dev@aweber.net
|
||||||
|
From: "Your Name" <xxxxxxx@aweber.com>
|
||||||
|
Subject: Sites Release 2020-04-07
|
||||||
|
|
||||||
|
Sites is scheduled for release in one hour. The following tickets have commits
|
||||||
|
since the last tag:
|
||||||
|
|
||||||
|
| Issue | Status | Summary |
|
||||||
|
|---------------+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
|
| CC-5343 | In Development | As Molly I want to be able to add an EXISTING custom field to my signup form on my landing page so that I can get more specific customer info |
|
||||||
|
| CCPANEL-10176 | Closed | INVESTIGATE: "OR" search on tags - Backend |
|
||||||
|
| CCPANEL-10294 | Testing | Update character limits for subscribers for consistency |
|
||||||
|
| INT-4298 | Testing | Integrations JS App: Setup pages |
|
||||||
|
|
||||||
|
Please review your tickets and ensure that only code that is ready to release is
|
||||||
|
in the sites master branch. The master branch will be tagged and released at
|
||||||
|
11am.
|
||||||
|
#+end_src
|
||||||
|
*** Notify teams via Slack
|
||||||
|
|
||||||
|
Post a notice in the =#devel= room in Slack. Tag any teams that have changes in
|
||||||
|
the release:
|
||||||
|
|
||||||
|
#+begin_quote
|
||||||
|
Sites is scheduled to be released at 11:00 today. If you have any changes please make sure they're ready for release or are removed by then.
|
||||||
|
- CC-5343
|
||||||
|
- CCPANEL-10176
|
||||||
|
- CCPANEL-10294
|
||||||
|
- INT-4298
|
||||||
|
|
||||||
|
/cc @cc-team @cp-team @integrations-team
|
||||||
|
#+end_quote
|
||||||
|
** At 11AM
|
||||||
|
Tag the master branch on the sites repo and push the tag to the upstream
|
||||||
|
repository.
|
||||||
58
aweber/stripe-api-otp.svg
Normal file
58
aweber/stripe-api-otp.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 9 KiB |
37
aweber/stripe-legacy-purchase-tracking.svg
Normal file
37
aweber/stripe-legacy-purchase-tracking.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 13 KiB |
18
aweber/stripe-payment-states.svg
Normal file
18
aweber/stripe-payment-states.svg
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="198px" preserveAspectRatio="none" style="width:269px;height:198px;" version="1.1" viewBox="0 0 269 198" width="269px" zoomAndPan="magnify"><defs><filter height="300%" id="fymc1q9d4g4fl" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><ellipse cx="44" cy="32" fill="#000000" filter="url(#fymc1q9d4g4fl)" rx="10" ry="10" style="stroke:none;stroke-width:1.0;"/><g id="New"><rect fill="#FEFECE" filter="url(#fymc1q9d4g4fl)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="50" x="89" y="7"/><line style="stroke:#A80036;stroke-width:1.5;" x1="89" x2="139" y1="33.4883" y2="33.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="29" x="99.5" y="25.5352">New</text></g><g id="Paid"><rect fill="#FEFECE" filter="url(#fymc1q9d4g4fl)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="50" x="7" y="134"/><line style="stroke:#A80036;stroke-width:1.5;" x1="7" x2="57" y1="160.4883" y2="160.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="29" x="17.5" y="152.5352">Paid</text></g><g id="Failed"><rect fill="#FEFECE" filter="url(#fymc1q9d4g4fl)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="61" x="166.5" y="134"/><line style="stroke:#A80036;stroke-width:1.5;" x1="166.5" x2="227.5" y1="160.4883" y2="160.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="176.5" y="152.5352">Failed</text></g><!--MD5=[b801211b1cf3cc2d27e45a912a639925]
|
||||||
|
link *start to New--><path d="M54.12,32 C63.94,32 73.77,32 83.6,32 " fill="none" id="*start-to-New" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="88.74,32,79.74,28,83.74,32,79.74,36,88.74,32" style="stroke:#A80036;stroke-width:1.0;"/><!--MD5=[248cfbdb1094300370a1d70a64b46375]
|
||||||
|
link New to Paid--><path d="M88.97,47.24 C65.13,61.06 32.39,80.83 29,87 C22.13,99.5 22.08,115.2 24.09,128.65 " fill="none" id="New-to-Paid" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="24.99,133.88,27.4157,124.3345,24.1472,128.9515,19.5301,125.6831,24.99,133.88" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="123" x="30" y="100.5684">Payment succeeded</text><!--MD5=[cb481be5d4f5cd6698b94d961c2951ad]
|
||||||
|
link New to Failed--><path d="M135.13,57.01 C142.72,66.12 151.11,76.77 158,87 C166.96,100.3 175.67,115.83 182.58,128.97 " fill="none" id="New-to-Failed" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="185.05,133.7,184.4397,123.8701,182.7401,129.2656,177.3446,127.5659,185.05,133.7" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="92" x="169" y="100.5684">Payment failed</text><!--MD5=[f72f4d324b90c1fdd622b8cfdc11261e]
|
||||||
|
@startuml
|
||||||
|
[*] -> New
|
||||||
|
New - -> Paid : Payment succeeded
|
||||||
|
New - -> Failed : Payment failed
|
||||||
|
@enduml
|
||||||
|
|
||||||
|
PlantUML version 1.2021.00(Sun Jan 10 05:25:05 EST 2021)
|
||||||
|
(GPL source distribution)
|
||||||
|
Java Runtime: Java(TM) SE Runtime Environment
|
||||||
|
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||||
|
Default Encoding: UTF-8
|
||||||
|
Language: en
|
||||||
|
Country: US
|
||||||
|
--></g></svg>
|
||||||
|
After Width: | Height: | Size: 3.7 KiB |
29
aweber/stripe-poller.svg
Normal file
29
aweber/stripe-poller.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 11 KiB |
16
aweber/stripe-purchase-states.svg
Normal file
16
aweber/stripe-purchase-states.svg
Normal file
|
|
@ -0,0 +1,16 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="71px" preserveAspectRatio="none" style="width:235px;height:71px;" version="1.1" viewBox="0 0 235 71" width="235px" zoomAndPan="magnify"><defs><filter height="300%" id="ft23wx4e5csov" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><ellipse cx="16" cy="32" fill="#000000" filter="url(#ft23wx4e5csov)" rx="10" ry="10" style="stroke:none;stroke-width:1.0;"/><g id="New"><rect fill="#FEFECE" filter="url(#ft23wx4e5csov)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="50" x="61" y="7"/><line style="stroke:#A80036;stroke-width:1.5;" x1="61" x2="111" y1="33.4883" y2="33.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="29" x="71.5" y="25.5352">New</text></g><g id="Fulfilled"><rect fill="#FEFECE" filter="url(#ft23wx4e5csov)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="75" x="146.5" y="7"/><line style="stroke:#A80036;stroke-width:1.5;" x1="146.5" x2="221.5" y1="33.4883" y2="33.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="55" x="156.5" y="25.5352">Fulfilled</text></g><!--MD5=[b801211b1cf3cc2d27e45a912a639925]
|
||||||
|
link *start to New--><path d="M26.12,32 C35.94,32 45.77,32 55.6,32 " fill="none" id="*start-to-New" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="60.74,32,51.74,28,55.74,32,51.74,36,60.74,32" style="stroke:#A80036;stroke-width:1.0;"/><!--MD5=[5d9ac0cc099169aa53dab4ce2ad0bcdf]
|
||||||
|
link New to Fulfilled--><path d="M111.27,32 C121.27,32 131.27,32 141.27,32 " fill="none" id="New-to-Fulfilled" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="146.5,32,137.5,28,141.5,32,137.5,36,146.5,32" style="stroke:#A80036;stroke-width:1.0;"/><!--MD5=[8337f11062fc7bb204c44793897c8040]
|
||||||
|
@startuml
|
||||||
|
[*] -> New
|
||||||
|
New -> Fulfilled
|
||||||
|
@enduml
|
||||||
|
|
||||||
|
PlantUML version 1.2021.00(Sun Jan 10 05:25:05 EST 2021)
|
||||||
|
(GPL source distribution)
|
||||||
|
Java Runtime: Java(TM) SE Runtime Environment
|
||||||
|
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||||
|
Default Encoding: UTF-8
|
||||||
|
Language: en
|
||||||
|
Country: US
|
||||||
|
--></g></svg>
|
||||||
|
After Width: | Height: | Size: 2.5 KiB |
63
aweber/stripe-purchase-subscription-events.svg
Normal file
63
aweber/stripe-purchase-subscription-events.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 20 KiB |
37
aweber/stripe-purchase-tracking-payment-events.svg
Normal file
37
aweber/stripe-purchase-tracking-payment-events.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 12 KiB |
40
aweber/stripe-purchase-tracking.svg
Normal file
40
aweber/stripe-purchase-tracking.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 15 KiB |
22
aweber/stripe-subscription-states.svg
Normal file
22
aweber/stripe-subscription-states.svg
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="198px" preserveAspectRatio="none" style="width:395px;height:198px;" version="1.1" viewBox="0 0 395 198" width="395px" zoomAndPan="magnify"><defs><filter height="300%" id="fn9yatwmacn7p" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><ellipse cx="16" cy="32" fill="#000000" filter="url(#fn9yatwmacn7p)" rx="10" ry="10" style="stroke:none;stroke-width:1.0;"/><g id="New"><rect fill="#FEFECE" filter="url(#fn9yatwmacn7p)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="50" x="61" y="7"/><line style="stroke:#A80036;stroke-width:1.5;" x1="61" x2="111" y1="33.4883" y2="33.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="29" x="71.5" y="25.5352">New</text></g><g id="Active"><rect fill="#FEFECE" filter="url(#fn9yatwmacn7p)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="61" x="169.5" y="7"/><line style="stroke:#A80036;stroke-width:1.5;" x1="169.5" x2="230.5" y1="33.4883" y2="33.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="179.5" y="25.5352">Active</text></g><g id="Terminated"><rect fill="#FEFECE" filter="url(#fn9yatwmacn7p)" height="50" rx="12.5" ry="12.5" style="stroke:#A80036;stroke-width:1.5;" width="99" x="150.5" y="134"/><line style="stroke:#A80036;stroke-width:1.5;" x1="150.5" x2="249.5" y1="160.4883" y2="160.4883"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="79" x="160.5" y="152.5352">Terminated</text></g><!--MD5=[b801211b1cf3cc2d27e45a912a639925]
|
||||||
|
link *start to New--><path d="M26.12,32 C35.94,32 45.77,32 55.6,32 " fill="none" id="*start-to-New" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="60.74,32,51.74,28,55.74,32,51.74,36,60.74,32" style="stroke:#A80036;stroke-width:1.0;"/><!--MD5=[7add0a041df65cbaf5eda635fe656769]
|
||||||
|
link New to Active--><path d="M111.38,32 C128.85,32 146.31,32 163.77,32 " fill="none" id="New-to-Active" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="169.19,32,160.19,28,164.19,32,160.19,36,169.19,32" style="stroke:#A80036;stroke-width:1.0;"/><!--MD5=[d1011ee76343bf54a89fc5ee8188cc62]
|
||||||
|
link Active to Terminated--><path d="M198.13,57.31 C197.29,71.02 196.57,88.46 197,104 C197.22,111.93 197.62,120.51 198.06,128.44 " fill="none" id="Active-to-Terminated" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="198.37,133.82,201.8485,124.6059,198.0839,128.8282,193.8616,125.0636,198.37,133.82" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="94" x="198" y="100.5684">Payment Failed</text><!--MD5=[d1011ee76343bf54a89fc5ee8188cc62]
|
||||||
|
link Active to Terminated--><path d="M230.66,41.01 C265.44,51.77 315.14,73.14 297,104 C287.45,120.25 271.04,131.97 254.4,140.24 " fill="none" id="Active-to-Terminated-1" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="249.71,142.48,259.5551,142.2087,254.2214,140.3242,256.1059,134.9905,249.71,142.48" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="86" x="301" y="100.5684">Unsubscribed</text><!--MD5=[63b3c870cfc694cf40336563ee7fed3c]
|
||||||
|
link New to Terminated--><path d="M82.85,57.01 C82.16,71.78 83.69,90.41 93,104 C105.42,122.12 125.79,134.47 145.44,142.74 " fill="none" id="New-to-Terminated" style="stroke:#A80036;stroke-width:1.0;"/><polygon fill="#A80036" points="150.41,144.76,143.5844,137.66,145.7795,142.8735,140.566,145.0687,150.41,144.76" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="94" x="94" y="100.5684">Payment Failed</text><!--MD5=[a4ca0cb7c35d5186987020697bc0a913]
|
||||||
|
@startuml
|
||||||
|
[*] -> New
|
||||||
|
New -> Active
|
||||||
|
Active - -> Terminated : Payment Failed
|
||||||
|
Active - -> Terminated : Unsubscribed
|
||||||
|
New - -> Terminated : Payment Failed
|
||||||
|
@enduml
|
||||||
|
|
||||||
|
PlantUML version 1.2021.00(Sun Jan 10 05:25:05 EST 2021)
|
||||||
|
(GPL source distribution)
|
||||||
|
Java Runtime: Java(TM) SE Runtime Environment
|
||||||
|
JVM: Java HotSpot(TM) 64-Bit Server VM
|
||||||
|
Default Encoding: UTF-8
|
||||||
|
Language: en
|
||||||
|
Country: US
|
||||||
|
--></g></svg>
|
||||||
|
After Width: | Height: | Size: 4.6 KiB |
47
daily/2020-07-13.org
Normal file
47
daily/2020-07-13.org
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 81ada51d-463a-4e2a-9e7a-af123720dde7
|
||||||
|
:END:
|
||||||
|
#+title: 2020-07-13
|
||||||
|
|
||||||
|
* Ops Initiative Workshop
|
||||||
|
- [[id:ac416861-ce45-49ac-8b60-f8ea39362135][Migration to common RabbitMQ]]
|
||||||
|
- EDELIV parent ticket: https://jira.aweber.io/browse/EDELIV-4083
|
||||||
|
- Eric to look into [[id:e4d00c11-da8a-4c91-8f38-ce939846e5cb][CoreAPI]] changes needed for common-rabbitmq migration.
|
||||||
|
- Looking at Sites RabbitMQ publishing
|
||||||
|
- Enlightener
|
||||||
|
- Billing
|
||||||
|
- Also updating with new control-panel credentials
|
||||||
|
- Sites (docker) common-rabbitmq migration changes:
|
||||||
|
https://gitlab.aweber.io/CP/applications/sites/-/merge_requests/5258
|
||||||
|
- Sites ([[id:ddeea682-c8f0-4607-8e2b-0f8ee4fd6191][Puppet]]) common-rabbitmq migration changes:
|
||||||
|
https://gitlab.aweber.io/PSE/config-management/puppet/-/merge_requests/158
|
||||||
|
* Compromised Account Credentials
|
||||||
|
#+begin_quote
|
||||||
|
Tom Kulzer Today at 1:03 PM
|
||||||
|
@MeghanN @correlr we are seeing major issues with account credentials being compromised by someone that’s sending phishing emails. @Josh Smith ID’d that they are likely using bots to test credentials from other site data compromises and catching people that have logins where they use the same email/pswd elsewhere. Our data on the login dashboard appears broken.. https://aweber.slack.com/archives/CF62W5U10/p1594645641053600
|
||||||
|
https://grafana.aweber.io/d/000000530/account-logins?orgId=1&refresh=5m
|
||||||
|
Does anyone have suggestions on how we can be preventing or catching these kind of compromises better?
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
#+begin_quote
|
||||||
|
Ian Ratti
|
||||||
|
The PayPal phishing abuser is now logging into old accounts to send phishing notices. Some recent accounts:
|
||||||
|
https://admin.aweber.io/account/index/1515621#
|
||||||
|
https://admin.aweber.io/account/index/1506549#
|
||||||
|
https://admin.aweber.io/account/index/1516301#
|
||||||
|
Now logging in from Egypt and promoting the same phishing page links (https://wlpork.co.za/ ) on these two older accounts starting 7/11/20 that were previously inactive for years:
|
||||||
|
https://admin.aweber.io/account/index/247035#
|
||||||
|
https://admin.aweber.io/account/index/304061# (sent in a request to close due to the compromise, found this via the 8 huge imports waiting in review)
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
#+begin_quote
|
||||||
|
Tom Kulzer 21 minutes ago
|
||||||
|
thoughts I’ve had:
|
||||||
|
- sift.com has an account takeover product that we’re not using and could potentially, but it’s expensive and wouldn’t have the historical data on these accounts that’d be necessary to catch these specific bad actor instances.
|
||||||
|
- email alerts when someone logs in with an IP or region different than they’ve done in the past.
|
||||||
|
- do some sort of cross match on publicly available compromised account password files to see if we have crossover and force reset pswds on those users.
|
||||||
|
- force an email verification click when someone logs in from a different region than they’ve historically logged in from.
|
||||||
|
I’m not sure on other ideas.
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
Brian H has been tackling this so far: https://jira.aweber.io/browse/CCPANEL-10593
|
||||||
98
daily/2020-07-14.org
Normal file
98
daily/2020-07-14.org
Normal file
|
|
@ -0,0 +1,98 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 94c0bb8c-f9ed-46cb-89da-3eb7cacc4c1d
|
||||||
|
:END:
|
||||||
|
#+title: 2020-07-14
|
||||||
|
|
||||||
|
* Tech Initiative Sync-Up
|
||||||
|
SCHEDULED: <2020-07-14 Tue 09:30>
|
||||||
|
|
||||||
|
** [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] Dockerization progress
|
||||||
|
Nearly done, just need to test and iron out issues sending legacy mail via the
|
||||||
|
[[id:24578fe5-6ca0-4000-a7cd-201e952e4c76][Mail Relay]] service.
|
||||||
|
** Migrating legacy emails to [[id:32c66bc8-a397-4f50-96cd-2aec70dd14c5][Corporate Notifications]]
|
||||||
|
Meghan will assist to label notifications as internal or external.
|
||||||
|
** Replace direct consul usage with templated configuration files
|
||||||
|
Tickets to be created.
|
||||||
|
** Sites deployment pipeline
|
||||||
|
The sites deployment will continue to be triggered exclusively by Jenkins, which will trigger the associated control-panel deployment in Gitlab.
|
||||||
|
** ICON support
|
||||||
|
Coordinating with Gavin.
|
||||||
|
** Ops initiatives
|
||||||
|
- [[id:ac416861-ce45-49ac-8b60-f8ea39362135][Migration to common RabbitMQ]]
|
||||||
|
- [[id:592aa825-154c-4659-8193-75b0ce1f2e5c][PGBouncer port migration]]
|
||||||
|
|
||||||
|
** COI Message Editor Preact to React
|
||||||
|
|
||||||
|
* Sites release
|
||||||
|
SCHEDULED: <2020-07-14 Tue 14:00>
|
||||||
|
- tags :: [[id:6c7250d0-6871-4030-98f2-2a53c6ca5eb3][Sites Releases]]
|
||||||
|
|
||||||
|
| Issue | Status | Summary |
|
||||||
|
|---------------+------------------+---------------------------------------------------------------------------|
|
||||||
|
| CC-5333 | Closed | UI For Adding/Customizing Landing Page Subdomains |
|
||||||
|
| CCPANEL-10555 | Awaiting Release | Add WPMU DEV to Partner Offers Pages (Both User and Public Partner pages) |
|
||||||
|
| CONV-3961 | Awaiting Release | Add package ID to AW.vars |
|
||||||
|
| CONV-3977 | Raw | Design improvements to /users/upgrade form |
|
||||||
|
| CONV-3978 | In Development | Add sift JS to /free.htm |
|
||||||
|
| CONV-3979 | Awaiting Release | Freemium account creations is sending bogus data to Sift.com. Fix that. |
|
||||||
|
|
||||||
|
Please review your tickets and ensure that only code related to Freemium that is
|
||||||
|
ready to release is in the sites master branch. The master branch will be tagged
|
||||||
|
and released at 2pm.
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
Sites is scheduled to be released at 2:00pm today. If you have any changes that are not related to Freemium, please revert them if you haven't already.
|
||||||
|
- CC-5333
|
||||||
|
- CCPANEL-10555
|
||||||
|
- CONV-3961
|
||||||
|
- CONV-3977
|
||||||
|
- CONV-3978
|
||||||
|
- CONV-3979
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
Released at [2020-07-14 Tue 14:11].
|
||||||
|
* CAPI pages
|
||||||
|
[2020-07-14 Tue 12:11]
|
||||||
|
A variety of alerts went off in [[id:ebea379a-8fa6-4e22-9275-a9fc98c02804][Pagerduty]], seemingly caused by work being done
|
||||||
|
on the old RabbitMQ nodes as part of the [[id:ac416861-ce45-49ac-8b60-f8ea39362135][Migration to common RabbitMQ]] project. A
|
||||||
|
rolling restart of [[id:e4d00c11-da8a-4c91-8f38-ce939846e5cb][CoreAPI]] successfully addressed the issues.
|
||||||
|
|
||||||
|
* Fixing mail-relay issues
|
||||||
|
Working with Ryan Steele and Eric Toner to resolve issues sending mail with the
|
||||||
|
[[id:24578fe5-6ca0-4000-a7cd-201e952e4c76][Mail Relay]] service.
|
||||||
|
|
||||||
|
The staging environment, [[id:e1b95d0e-366e-4ecf-b867-409b6b6c6ee8][Momentum]] will not send to aweber.com, only aweber.net.
|
||||||
|
Very few external domains are allowed to avoid accidentally emailing customers.
|
||||||
|
|
||||||
|
Mail seems to be working fine in production.
|
||||||
|
|
||||||
|
[2020-07-14 Tue 14:24] Working with Eric Toner and Chris Fox to test account
|
||||||
|
signup to verify that billing and invoice receipt emails are going out properly.
|
||||||
|
|
||||||
|
Documented steps for [[id:7a362881-875f-4f74-8053-55f63826da63][Refunding an Order]].
|
||||||
|
|
||||||
|
----------------------------------------------------------------------
|
||||||
|
|
||||||
|
We saw the emails succeeding, but not hitting the relay. It turned out the
|
||||||
|
X-Kube header wasn't actually being used. Getting that enabled led us to see the
|
||||||
|
NUE experience being broken.
|
||||||
|
|
||||||
|
* Compromised Account Credentials
|
||||||
|
|
||||||
|
Considering forcing password reset on next login based on login attempt
|
||||||
|
throttling by user / IP.
|
||||||
|
|
||||||
|
[2020-07-14 Tue 22:00]
|
||||||
|
|
||||||
|
Added a temporary logging change, determined that the attacker is using the same
|
||||||
|
SIFT id on all requests(=Al9h1qsyeZcUYlB6VPnE6736i-by7fG1=). Added it to the
|
||||||
|
blocked SIFT id list.
|
||||||
|
|
||||||
|
Proposed changes to the [[id:d17e934b-b340-4246-88f0-9b36527100c0][Login Throttling]] code that were hacked together tonight
|
||||||
|
include forcing password reset on next login when:
|
||||||
|
- 20 login attempts within 30 minutes for the same username
|
||||||
|
- 20 login attempts within 30 minutes from the same IP address
|
||||||
|
- Any login attempt from a GeoIP-detected IP address that does not match the
|
||||||
|
country of any attached account.
|
||||||
33
daily/2020-07-15.org
Normal file
33
daily/2020-07-15.org
Normal file
|
|
@ -0,0 +1,33 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: a1117ce1-b7ab-47ae-a06c-13b5bd9ced11
|
||||||
|
:END:
|
||||||
|
#+title: 2020-07-15
|
||||||
|
|
||||||
|
* Compromised Account Credentials
|
||||||
|
[2020-07-15 Wed 08:47]
|
||||||
|
This morning's plan for updating the [[id:d17e934b-b340-4246-88f0-9b36527100c0][Login Throttling]] code:
|
||||||
|
- [X] Tighten IP throttling to 3 requests in 12 hours
|
||||||
|
- [X]Revert last night's changes and move them to a separate branch for rework
|
||||||
|
- [X] Log additional information when a login attempt is throttled (username, IP, Sift ID)
|
||||||
|
- [X] Extend session timeout to reduce natural re-authentication to 7 days
|
||||||
|
- [X] Add dynamic throttling based on Sift ID
|
||||||
|
|
||||||
|
[2020-07-15 Wed 13:28]
|
||||||
|
- Implement CSRF on the login form by moving the form to the sites codebase
|
||||||
|
([[https://jira.aweber.io/browse/CCPANEL-10596][CCPANEL-10596]])
|
||||||
|
|
||||||
|
#+begin_quote
|
||||||
|
Gavin M Roy 28 minutes ago
|
||||||
|
Sure, my main reference for them would be to point out how Tornado does it as a built-in behavior:
|
||||||
|
https://www.tornadoweb.org/en/stable/guide/security.html#cross-site-request-forgery-protection
|
||||||
|
https://github.com/tornadoweb/tornado/blob/master/tornado/web.py#L1489
|
||||||
|
https://github.com/tornadoweb/tornado/blob/master/tornado/web.py#L1371
|
||||||
|
https://github.com/tornadoweb/tornado/blob/master/tornado/web.py#L1527
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
Pages are moved into the CP with CSRF tokens being injected into the session and
|
||||||
|
the form. The controller is updated in a separate MR to require the token and
|
||||||
|
validate it against the value in the session. We're updating the F5 to route the
|
||||||
|
login and landing pages to the staging environment to test that they load
|
||||||
|
correctly. We'll do the same in production tomorrow, and then release the login
|
||||||
|
controller changes once that's done.
|
||||||
131
daily/2020-07-16.org
Normal file
131
daily/2020-07-16.org
Normal file
|
|
@ -0,0 +1,131 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: d28b1f6c-0a13-4306-86e6-d959705d3867
|
||||||
|
:END:
|
||||||
|
#+title: 2020-07-16
|
||||||
|
#+SETUPFILE: ../worklog.setup
|
||||||
|
* Import evaluation issue
|
||||||
|
Got an alert in [[id:ebea379a-8fa6-4e22-9275-a9fc98c02804][Pagerduty]] about [[file:projects/import.org][Import Evaluation]] message age.
|
||||||
|
|
||||||
|
Saw some of this earlier, where it was getting =null= for its compromised
|
||||||
|
addresses database URL. The configuration is correct in consul, so maybe this
|
||||||
|
was a weird intermittent thing.
|
||||||
|
#+begin_example
|
||||||
|
{"name":"rejected.process","levelname":"ERROR","asctime":"2020-07-16T09:49:46.103741+0000","processName":"subscriber-import-evaluation-1","message":"Error creating the consumer \"subscriber_import_evaluation.consumer.Consumer\": pgsql_compromised_addresses: null is invalid - expected format: ^postgres(ql)?://.*","module":"process","correlation_id":"c20e024c-160c-45a1-9ea3-e3bb04d26413","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\", line 338, in validate_settings\n jsonschema.validate(settings, self.SETTINGS_SCHEMA)\n"," File \"/usr/local/lib/python3.7/site-packages/jsonschema/validators.py\", line 899, in validate\n raise error\n","jsonschema.exceptions.ValidationError: 'null' does not match '^postgres(ql)?://.*'\n\nFailed validating 'pattern' in schema['properties']['pgsql_compromised_addresses']:\n {'description': 'The RDS compromised addresses connection URI',\n 'pattern': '^postgres(ql)?://.*',\n 'type': 'string'}\n\nOn instance['pgsql_compromised_addresses']:\n 'null'\n","\nDuring handling of the above exception, another exception occurred:\n\n","Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.7/site-packages/rejected/process.py\", line 455, in get_consumer\n return handle(**kwargs)\n"," File \"/usr/local/lib/python3.7/site-packages/avroconsumer.py\", line 25, in __init__\n super(Consumer, self).__init__(*args, **kwargs)\n"," File \"/usr/local/lib/python3.7/site-packages/rejected/consumer.py\", line 208, in __init__\n self.initialize()\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\", line 291, in initialize\n self.validate_settings(self.settings.dict())\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\", line 351, in validate_settings\n raise Exception(message)\n","Exception: pgsql_compromised_addresses: null is invalid - expected format: ^postgres(ql)?://.*\n"],"log":"{\"name\":\"rejected.process\",\"levelname\":\"ERROR\",\"asctime\":\"2020-07-16T09:49:46.103741+0000\",\"processName\":\"subscriber-import-evaluation-1\",\"message\":\"Error creating the consumer \\\"subscriber_import_evaluation.consumer.Consumer\\\": pgsql_compromised_addresses: null is invalid - expected format: ^postgres(ql)?://.*\",\"module\":\"process\",\"correlation_id\":\"c20e024c-160c-45a1-9ea3-e3bb04d26413\",\"exc_info\":[\"Traceback (most recent call last):\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\\\", line 338, in validate_settings\\n jsonschema.validate(settings, self.SETTINGS_SCHEMA)\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/jsonschema/validators.py\\\", line 899, in validate\\n raise error\\n\",\"jsonschema.exceptions.ValidationError: 'null' does not match '^postgres(ql)?://.*'\\n\\nFailed validating 'pattern' in schema['properties']['pgsql_compromised_addresses']:\\n {'description': 'The RDS compromised addresses connection URI',\\n 'pattern': '^postgres(ql)?://.*',\\n 'type': 'string'}\\n\\nOn instance['pgsql_compromised_addresses']:\\n 'null'\\n\",\"\\nDuring handling of the above exception, another exception occurred:\\n\\n\",\"Traceback (most recent call last):\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/rejected/process.py\\\", line 455, in get_consumer\\n return handle(**kwargs)\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/avroconsumer.py\\\", line 25, in __init__\\n super(Consumer, self).__init__(*args, **kwargs)\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/rejected/consumer.py\\\", line 208, in __init__\\n self.initialize()\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\\\", line 291, in initialize\\n self.validate_settings(self.settings.dict())\\n\",\" File \\\"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\\\", line 351, in validate_settings\\n raise Exception(message)\\n\",\"Exception: pgsql_compromised_addresses: null is invalid - expected format: ^postgres(ql)?://.*\\n\"]}\n","stream":"stdout","docker":{"container_id":"8cee26b83b012277ad0ee740e11b260e25baed5aff912264ae195b30ace5ab21"},"kubernetes":{"container_name":"subscriber-import-evaluation","namespace_name":"cp","pod_name":"subscriber-import-evaluation-6bdd96bdcd-jx4w9","pod_id":"4e098e40-bae8-11ea-9f30-6a3b5e3f247e","labels":{"app":"subscriber-import-evaluation","component":"application","pod-template-hash":"2688526878","version":"e142756"},"host":"kube01-w02.testing.aweberint.com","master_url":"https://10.50.0.1:443/api"},"timestamp":"2020-07-16T09:49:46+00:00"}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
Here, though, we've got a message repeatedly being requeued due to a list lookup failure.
|
||||||
|
#+begin_example
|
||||||
|
{"name":"subscriber_import_evaluation.consumer","levelname":"DEBUG","asctime":"2020-07-16T12:54:18.250984+0000","processName":"subscriber-import-evaluation-1","message":"Returning {'account': '62949269-3df5-46d9-b883-74e7771504f2', 'import_uuid': 'f4cd8ddc-95c3-445d-afbe-386e93a29854', 'legacy_account_id': 1326226, 'legacy_list_id': 5202426, 'list': 'ce641169-a75d-45bc-bc48-ef8a8bda31c8', 'timestamp': '2020-07-16T09:36:37.989035+00:00', 'acquisition_method': 'Acquisition method: \"other\"\\nProvider: Did not bring from another provider\\nDescribe how you got this list of subscribers\\nIam exporting and re-importing and merging my lists.', 'automation_enabled': True, 'coi_enabled': False, 'column_mapping': {'email': 0, 'name': 1, 'note': 8, 'tags': 19}, 'existing_subscriber_action': 'Ignore', 'remote_ip': '110.144.41.250', 'tags': [], 'message_number': None}","module":"avroconsumer","correlation_id":"43b4305c-02fa-4e5a-aba1-e7db9c4bb926","exc_info":null}
|
||||||
|
{"name":"subscriber_import_evaluation.consumer","levelname":"WARNING","asctime":"2020-07-16T12:54:18.268914+0000","processName":"subscriber-import-evaluation-1","message":"get_list: GET request to http://list.service.production.consul/v1/accounts/1326226/lists/5202426 failed with message: HTTP 404: ListNotFound","module":"services","correlation_id":"43b4305c-02fa-4e5a-aba1-e7db9c4bb926","exc_info":null}
|
||||||
|
{"name":"subscriber_import_evaluation.consumer","levelname":"ERROR","asctime":"2020-07-16T12:54:18.269250+0000","processName":"subscriber-import-evaluation-1","message":"Exception processing delivery 1: HTTP 404: ListNotFound","module":"consumer","correlation_id":"43b4305c-02fa-4e5a-aba1-e7db9c4bb926","exc_info":null}
|
||||||
|
{"name":"subscriber_import_evaluation.consumer","levelname":"ERROR","asctime":"2020-07-16T12:54:18.269401+0000","processName":"subscriber-import-evaluation-1","message":"Processor handled HTTPClientError: HTTP 404: ListNotFound","module":"consumer","correlation_id":"43b4305c-02fa-4e5a-aba1-e7db9c4bb926","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.7/site-packages/rejected/consumer.py\", line 908, in execute\n yield result\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 735, in run\n value = future.result()\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 742, in run\n yielded = self.gen.throw(*exc_info) # type: ignore\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/consumer.py\", line 734, in process\n fields=[\"coi_bitmap\"],\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 735, in run\n value = future.result()\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 742, in run\n yielded = self.gen.throw(*exc_info) # type: ignore\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/services.py\", line 113, in get_list\n error_reason='Unable to fetch list')\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 735, in run\n value = future.result()\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 742, in run\n yielded = self.gen.throw(*exc_info) # type: ignore\n"," File \"/usr/local/lib/python3.7/site-packages/subscriber_import_evaluation/services.py\", line 65, in _http_fetch\n request_timeout=request_timeout, **kwargs)\n"," File \"/usr/local/lib/python3.7/site-packages/tornado/gen.py\", line 735, in run\n value = future.result()\n","tornado.httpclient.HTTPClientError: HTTP 404: ListNotFound\n"]}
|
||||||
|
{"name":"rejected.process","levelname":"WARNING","asctime":"2020-07-16T12:54:18.281276+0000","processName":"subscriber-import-evaluation-1","message":"Rejecting message 1 with requeue","module":"process","correlation_id":"7a3a8390-10de-40ef-96be-100e8fa98596","exc_info":null}
|
||||||
|
{"name":"rejected.process","levelname":"INFO","asctime":"2020-07-16T12:54:18.281544+0000","processName":"subscriber-import-evaluation-1","message":"Resetting failure window, 1594904058 seconds since last","module":"process","correlation_id":"01ab189d-9c3e-4fb1-830b-eec0049dc434","exc_info":null}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
The lookup fails with the account in the path, but succeeds without it?
|
||||||
|
|
||||||
|
#+begin_src http :pretty :exports both
|
||||||
|
GET http://list.service.production.consul/v1/accounts/1326226/lists/5202426
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[612dcf937d3cdea3381ed3836f7cedfdd3c23a3e]:
|
||||||
|
: {
|
||||||
|
: "error": {
|
||||||
|
: "status_code": 404,
|
||||||
|
: "exception": "ListNotFound",
|
||||||
|
: "message": "List with given id not found"
|
||||||
|
: }
|
||||||
|
: }
|
||||||
|
|
||||||
|
#+begin_src http :pretty :exports both
|
||||||
|
GET http://list.service.production.consul/v1/lists/5202426
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[67a16238d6aad00f1a7abc4a59736d3ed4f5aea4]:
|
||||||
|
#+begin_example
|
||||||
|
{
|
||||||
|
"friendly_list_name": "15 minute manifestation.",
|
||||||
|
"address_id": 779131,
|
||||||
|
"coi_confirmation_text": "",
|
||||||
|
"sender_name": "Geoff Stevens",
|
||||||
|
"coi_subject_text": "Response Required: Please confirm your request for information.\n",
|
||||||
|
"coi_button_text": "Confirm my subscription",
|
||||||
|
"contact_address": "Palmer Rd\nPortland vic 3305\nAUSTRALIA",
|
||||||
|
"list_name": "awlist5202426",
|
||||||
|
"custom_fields": {},
|
||||||
|
"coi_closing": "",
|
||||||
|
"city": "Portland",
|
||||||
|
"country_id": 13,
|
||||||
|
"zipcode": "3305",
|
||||||
|
"locale": "en-US",
|
||||||
|
"state": "vic",
|
||||||
|
"list_description": "0000000",
|
||||||
|
"coi_subject_id": 6,
|
||||||
|
"company_name": "Geoff Stevenson",
|
||||||
|
"contact_emails": [
|
||||||
|
{
|
||||||
|
"reply": 1,
|
||||||
|
"confirmation": 0,
|
||||||
|
"email": "gst1965@bigpond.com",
|
||||||
|
"name": "Geoff Stevens"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"list_id": 5202426,
|
||||||
|
"country_name": "AUSTRALIA",
|
||||||
|
"account_id": 1326226,
|
||||||
|
"address1": "Palmer Rd",
|
||||||
|
"address2": null,
|
||||||
|
"company_website": "",
|
||||||
|
"coi_bitmap": 10,
|
||||||
|
"confirmation_button_text_id": 1,
|
||||||
|
"sender_email": "gst1965@bigpond.com",
|
||||||
|
"signature": null
|
||||||
|
}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
Popped the offending message off the queue:
|
||||||
|
Properties:
|
||||||
|
#+begin_example
|
||||||
|
app_id: subscriberimportapi/3.0.11
|
||||||
|
type: import_requested.v1
|
||||||
|
timestamp: 1594892197
|
||||||
|
message_id: e4b0e88c-d90c-4d1f-bf22-863ca6a140a7
|
||||||
|
correlation_id: 43b4305c-02fa-4e5a-aba1-e7db9c4bb926
|
||||||
|
headers:
|
||||||
|
account_id: 62949269-3df5-46d9-b883-74e7771504f2
|
||||||
|
x-shovelled:
|
||||||
|
dest-exchange: events
|
||||||
|
dest-uri: amqp://rabbitmq.aweberprod.com:5672/%2f
|
||||||
|
shovel-name: rabbitmq-events
|
||||||
|
shovel-type: dynamic
|
||||||
|
shovel-vhost: /
|
||||||
|
shovelled-by: production
|
||||||
|
src-queue: rabbitmq-events
|
||||||
|
src-uri: amqp://
|
||||||
|
content_type: application/vnd.apache.avro.datum
|
||||||
|
#+end_example
|
||||||
|
Payload:
|
||||||
|
#+begin_example
|
||||||
|
SDYyOTQ5MjY5LTNkZjUtNDZkOS1iODgzLTc0ZTc3NzE1MDRmMkhmNGNkOGRkYy05NWMzLTQ0NWQtYWZiZS0zODZlOTNhMjk4NTSk8qEB9If7BEhjZTY0MTE2
|
||||||
|
OS1hNzVkLTQ1YmMtYmM0OC1lZjhhOGJkYTMxYzhAMjAyMC0wNy0xNlQwOTozNjozNy45ODkwMzUrMDA6MDDYAkFjcXVpc2l0aW9uIG1ldGhvZDogIm90aGVy
|
||||||
|
IgpQcm92aWRlcjogRGlkIG5vdCBicmluZyBmcm9tIGFub3RoZXIgcHJvdmlkZXIKRGVzY3JpYmUgaG93IHlvdSBnb3QgdGhpcyBsaXN0IG9mIHN1YnNjcmli
|
||||||
|
ZXJzCklhbSBleHBvcnRpbmcgYW5kIHJlLWltcG9ydGluZyBhbmQgbWVyZ2luZyBteSBsaXN0cy4BAAgKZW1haWwACG5hbWUCCG5vdGUQCHRhZ3MmAAAcMTEw
|
||||||
|
LjE0NC40MS4yNTAAAA==
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
There seem to be more messages failing the same way, it'd probably be faster to just make a code change to drop messages on 4XX errors.
|
||||||
|
* Login and Landing pages
|
||||||
|
#+begin_example
|
||||||
|
# Send logged in customers to /users
|
||||||
|
RewriteCond %{HTTP:Cookie} (^|;\ *)loggedIn=1 [NC]
|
||||||
|
RewriteRule ^/(landing|login).htm$ /users [L,R=302]
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
=loginAjax= only appears to be hit from the landing/login pages:
|
||||||
|
#+begin_example
|
||||||
|
root@web-controlpanel1:/home/aweber/logs# grep loginAjax httpd/access_log |cut -d ' ' -f 14|cut -d '?' -f 1|sed 's/"//g'|sort|uniq -c
|
||||||
|
9 -
|
||||||
|
2244 https://www.aweber.com/landing.htm
|
||||||
|
500 https://www.aweber.com/login.htm
|
||||||
|
#+end_example
|
||||||
68
daily/2020-07-17.org
Normal file
68
daily/2020-07-17.org
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: b1c6f5ac-0f96-4597-98fe-0f60329a80e6
|
||||||
|
:END:
|
||||||
|
#+title: 2020-07-17
|
||||||
|
#+setupfile: ../worklog.setup
|
||||||
|
|
||||||
|
* Tracking login attempts without CSRF tokens
|
||||||
|
|
||||||
|
#+name: login-attempts-without-csrf
|
||||||
|
#+begin_src bash :dir ~/Downloads :exports none
|
||||||
|
grep -h 'CSRF challenge:.*sent: "", session: ""' account_controller* \
|
||||||
|
| sed -e 's/.*\(2020-07-[[:digit:]]* [[:digit:]]*\).*ip: "\([[:digit:]]*\.[[:digit:]]*\.[[:digit:]]*\.[[:digit:]]*\).*/\1 \2/' \
|
||||||
|
| sort | uniq | awk '{print $1, $2}' \
|
||||||
|
| uniq -c | awk '{print $2, $3 "," $1}'
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[fa1c8ae01ac81c4b0c465f01e3cd2815081e1ede]: login-attempts-without-csrf
|
||||||
|
| 2020-07-16 17 | 6 |
|
||||||
|
| 2020-07-16 18 | 38 |
|
||||||
|
| 2020-07-16 19 | 48 |
|
||||||
|
| 2020-07-16 20 | 31 |
|
||||||
|
| 2020-07-16 21 | 27 |
|
||||||
|
| 2020-07-16 22 | 31 |
|
||||||
|
| 2020-07-16 23 | 24 |
|
||||||
|
| 2020-07-17 00 | 26 |
|
||||||
|
| 2020-07-17 01 | 20 |
|
||||||
|
| 2020-07-17 02 | 26 |
|
||||||
|
| 2020-07-17 03 | 27 |
|
||||||
|
| 2020-07-17 04 | 21 |
|
||||||
|
| 2020-07-17 05 | 26 |
|
||||||
|
| 2020-07-17 06 | 34 |
|
||||||
|
| 2020-07-17 07 | 34 |
|
||||||
|
| 2020-07-17 08 | 34 |
|
||||||
|
| 2020-07-17 09 | 36 |
|
||||||
|
| 2020-07-17 10 | 49 |
|
||||||
|
| 2020-07-17 11 | 34 |
|
||||||
|
| 2020-07-17 12 | 53 |
|
||||||
|
| 2020-07-17 13 | 36 |
|
||||||
|
|
||||||
|
#+HEADER: :var data=login-attempts-without-csrf
|
||||||
|
#+BEGIN_SRC python :var filename="2020-07-17-login-attempts-without-csrf.png" :exports results :results file
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
|
|
||||||
|
x = [a[0] for a in data]
|
||||||
|
y = [a[1] for a in data]
|
||||||
|
a, = plt.plot(x, y, marker='o')
|
||||||
|
plt.title('Login attempts without CSRF tokens by IP')
|
||||||
|
plt.ylabel('Attempts per IP')
|
||||||
|
plt.xlabel('Hour')
|
||||||
|
plt.grid(True)
|
||||||
|
plt.xticks(rotation=70)
|
||||||
|
plt.savefig(filename, transparent=True)
|
||||||
|
return filename
|
||||||
|
#+END_SRC
|
||||||
|
|
||||||
|
#+RESULTS[66dd9d9ba4cfd43c058d2aac4b5a3cbd8772b099]:
|
||||||
|
[[file:2020-07-17-login-attempts-without-csrf.png]]
|
||||||
|
|
||||||
|
Login attempts without CSRF tokens appear to be fairly stable, without much
|
||||||
|
drop-off. Once we're comfortable with the frequency with which this occurs, we
|
||||||
|
can apply [[https://gitlab.aweber.io/CP/applications/sites/-/merge_requests/5283/diffs][this change]] to the [[id:d17e934b-b340-4246-88f0-9b36527100c0][Login Throttling]] code to mark login attempts
|
||||||
|
without a token as invalid, rather than presenting the end-user with a CAPTCHA
|
||||||
|
as we're doing now.
|
||||||
|
|
||||||
|
* Add captcha to login attempts without customer cookie
|
||||||
|
* Sift Account Takeover product
|
||||||
|
https://docs.google.com/document/d/15PhnBOLPIlRnRal-hz2dliA4Pmzf_bkFy253femOzqE/edit
|
||||||
25
daily/2021-04-14.org
Normal file
25
daily/2021-04-14.org
Normal file
|
|
@ -0,0 +1,25 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: c3b441db-7ab0-40cd-bc82-0d7b67c183d5
|
||||||
|
:END:
|
||||||
|
#+title: 2021-04-14
|
||||||
|
* CS Shadowing (Q2 2021)
|
||||||
|
SCHEDULED: <2021-04-14 Wed 10:00-12:00>
|
||||||
|
:PROPERTIES:
|
||||||
|
:Shadowing: Katherine Storm
|
||||||
|
:END:
|
||||||
|
** Lists and landing pages
|
||||||
|
A customer was unsure whether subscribers from their landing page would go into
|
||||||
|
a particular list.
|
||||||
|
|
||||||
|
- Is there an easier way for a customer to see that a landing page is associated
|
||||||
|
with a particular list?
|
||||||
|
+ The customer must check the active list in the upper lefthand corner.
|
||||||
|
- The customer uses [[https://linklyhq.com/][Linkly]] to track clicks, it's unclear how that works with the
|
||||||
|
landing page.
|
||||||
|
+ It does a redirect, and appears to work normally.
|
||||||
|
** Feature request: follow-up archive link
|
||||||
|
- Maybe for campaign messages?
|
||||||
|
- Katherine submitted the feature request on behalf of the customer.
|
||||||
|
** Removing unsubscribed / inactive subscribers
|
||||||
|
- Customer created segments on each list, which Katherine then assisted them
|
||||||
|
with deleting.
|
||||||
67
daily/2021-04-21.org
Normal file
67
daily/2021-04-21.org
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 9d7476de-5b15-4f36-bad8-c35868f54f73
|
||||||
|
:END:
|
||||||
|
#+title: 2021-04-21
|
||||||
|
* CS Shadowing (Q2 2021)
|
||||||
|
SCHEDULED: <2021-04-21 Wed 10:00-12:00>
|
||||||
|
:PROPERTIES:
|
||||||
|
:Shadowing: Kayla Haack
|
||||||
|
:END:
|
||||||
|
|
||||||
|
** Explaining free vs paid plans
|
||||||
|
- IP :: =2.37.116.178=
|
||||||
|
Customer appears to be from Italy, and only sees an option for AWeber Pro.
|
||||||
|
|
||||||
|
#+begin_src http :pretty :cache yes :exports both
|
||||||
|
GET http://geoip.aweberprod.com/2.37.116.178
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[e453245ee586e029f0e38a36c336a1992bf34637]:
|
||||||
|
#+begin_example
|
||||||
|
{
|
||||||
|
"country_code": "IT",
|
||||||
|
"country_name": "Italy",
|
||||||
|
"region_code": "MI",
|
||||||
|
"region_name": "Milan",
|
||||||
|
"city": "Milan",
|
||||||
|
"zip_code": "20124",
|
||||||
|
"postal_code": "20124",
|
||||||
|
"latitude": 45.4643,
|
||||||
|
"longitude": 9.1895,
|
||||||
|
"time_zone": "Europe/Rome",
|
||||||
|
"metro_code": null
|
||||||
|
}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
An account was created on the customer's behalf.
|
||||||
|
** Campaign messages are repeating
|
||||||
|
Customer has a welcome campaign that sends three emails that appears to end up
|
||||||
|
in a loop.
|
||||||
|
|
||||||
|
The campaign is triggered on new subscriptions. It seems that the customer added
|
||||||
|
themselves to test the campaign, deleted, then re-added themselves. Their email
|
||||||
|
address reports as being added today, though the customer claims to have only
|
||||||
|
logged in after seeing the email arrive in their inbox. They are not the owner
|
||||||
|
on the account, perhaps the owner added them.
|
||||||
|
** Customer is receiving unwanted email
|
||||||
|
They've been subscribed to a list since 2019 and are confused as to why they are
|
||||||
|
recieving email from us. Clarified that they were subscribed to a customer's
|
||||||
|
list and do not have a relationship with AWeber itself.
|
||||||
|
** Request to remove the mailing address on the bottom of emails
|
||||||
|
The address is required per the CAN-SPAM act. Recommended using a P.O. box.
|
||||||
|
** Customing landing page behavior
|
||||||
|
Customer needed info on how to set up their "thank you" redirect, and how to
|
||||||
|
customize the URL of their landing page.
|
||||||
|
** Customer needs to update their CC info
|
||||||
|
Hadn't logged in for three years. Successfully logged in using their email
|
||||||
|
address rather than legacy login.
|
||||||
|
** Updating campaign content
|
||||||
|
Customer wanted to know how updating content and adding messages to campaigns
|
||||||
|
works for subscribers in the campaign.
|
||||||
|
** Call to action landing page button
|
||||||
|
The customer would like a call to action button above the fold in the landing
|
||||||
|
page that scrolls to the form below.
|
||||||
|
|
||||||
|
There is an outstanding feature request to add anchor links to landing pages.
|
||||||
|
** Admin data tables warning
|
||||||
|
After searching for an AID, "Unknown parameter a_pkg_id for row 1, column 1"
|
||||||
29
daily/2021-06-24.org
Normal file
29
daily/2021-06-24.org
Normal file
|
|
@ -0,0 +1,29 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 9c81ff15-f1e3-423f-bb76-4bbbc12516a9
|
||||||
|
:END:
|
||||||
|
#+title: 2021-06-24
|
||||||
|
* ECommerce sync-up
|
||||||
|
Sync-up on the upcoming [[id:a9835afc-e0be-4436-8274-c3898fdf119c][Milestone 2 release]].
|
||||||
|
** Backend tasks
|
||||||
|
*** DONE Log webhook events
|
||||||
|
- Some events we treat as successful aren't logged.
|
||||||
|
*** DONE Adapt rate limits
|
||||||
|
https://jira.aweber.io/browse/INT-5283
|
||||||
|
|
||||||
|
Limit calls to the purchase endpoint per IP address to prevent [[https://stripe.com/docs/card-testing][card-testing]].
|
||||||
|
|
||||||
|
- Request limit per minute is configured in [[https://consul.service.production.consul/ui/production/kv/services/conv/infrastructure/unauthenticated-kong/services/stripe-payments/purchase-endpoint/rate-limiting/minute/edit][consul]].
|
||||||
|
- Can we alert on kong rate limiting?
|
||||||
|
+ =applications.unauthenticated-kong.$environment.stripe-payments.request.status.429.count=
|
||||||
|
*** DONE Release unauthenticated Kong purchase endpoint changes
|
||||||
|
https://jira.aweber.io/browse/INT-5257
|
||||||
|
|
||||||
|
- Have conversions release =1.10.8= to production.
|
||||||
|
*** DONE Remove dry-run
|
||||||
|
https://jira.aweber.io/browse/INT-5250
|
||||||
|
|
||||||
|
In process purchase in the webhook handler, switch to an =addsub_blocklist=.
|
||||||
|
** Monday Release
|
||||||
|
- Release will occur at 10AM
|
||||||
|
- Scott will perform the front-end release.
|
||||||
|
- Ihar will test scenarios using the product credit card, provided by Chris V.
|
||||||
17
daily/2021-06-29.org
Normal file
17
daily/2021-06-29.org
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 41a7a068-d0a6-4602-9540-ee473f6d561a
|
||||||
|
:END:
|
||||||
|
#+title: 2021-06-29
|
||||||
|
* Ecommerce customer requests
|
||||||
|
https://confluence.aweber.io/pages/viewpage.action?spaceKey=PD&title=2021-07-12+AWeber+Ecommerce+Customer+Requests
|
||||||
|
|
||||||
|
- [[id:022406d2-0480-4470-90d0-9533f6b9fa32][Supporting multiple currencies in Stripe]]
|
||||||
|
+ Multiple currency support will require investigation as to which currencies
|
||||||
|
are available for a given customer, how fees get collected, etc.
|
||||||
|
- Which currencies are supported for a customer?
|
||||||
|
- How do fees get collected?
|
||||||
|
- What happens when supported currencies change?
|
||||||
|
- How do we handle different minimum charge amounts?
|
||||||
|
+ We'll return an error to the buyer. The minimum charge is based on the
|
||||||
|
customer's settlement currency, and will fluctuate with foreign exchange
|
||||||
|
rates.
|
||||||
20
daily/2021-06-30.org
Normal file
20
daily/2021-06-30.org
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 8f824b4a-65df-44a6-a9f9-d500e90cd70e
|
||||||
|
:END:
|
||||||
|
#+title: 2021-06-30
|
||||||
|
* CP Outage retro
|
||||||
|
- CP experienced a login DDOS resulting in an outage on [2021-06-25 Fri]
|
||||||
|
+ [[id:d17e934b-b340-4246-88f0-9b36527100c0][Login Throttling]] flagged most via Sift ID
|
||||||
|
- Ops BOF discussed Apache possibly permitting PHP processes more memory than
|
||||||
|
the pod allows, resulting in them getting OOM-killed
|
||||||
|
- How much memory is the login endpoint using?
|
||||||
|
- ini set request body limit per path
|
||||||
|
- [ ] look into pod memory limits
|
||||||
|
- why was there so much cpu usage for a login attack?
|
||||||
|
- is there an opportunity to short circuit login attacks by IP?
|
||||||
|
+ could it trigger something in the F5?
|
||||||
|
+ could it be enhanced to look at CIDR blocks?
|
||||||
|
- assume everything is a =/24=?
|
||||||
|
- Add an intermediary tool or service to handle throttling?
|
||||||
|
+ Put login behind Kong?
|
||||||
|
- Separate the login page and give it its own scaling rules?
|
||||||
4
daily/2021-07-01.org
Normal file
4
daily/2021-07-01.org
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 40a20f58-1fc5-434d-b9da-18a99aeb665c
|
||||||
|
:END:
|
||||||
|
#+TITLE: 2021 07 01
|
||||||
1792
daily/2021-07-02.org
Normal file
1792
daily/2021-07-02.org
Normal file
File diff suppressed because it is too large
Load diff
92
daily/2021-08-20.org
Normal file
92
daily/2021-08-20.org
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 1a08da33-7079-4a99-998e-2f622ab1cc54
|
||||||
|
:END:
|
||||||
|
#+title: 2021-08-20
|
||||||
|
#+PROPERTY: header-args :cache yes :eval no-export
|
||||||
|
|
||||||
|
* Reports not working for an account in staging
|
||||||
|
#+begin_src http :exports both
|
||||||
|
GET https://anabroker.aweberstage.com/assignment/14313
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[047ea3da469346d5fb1944afd17b4308a8ef384a]:
|
||||||
|
: HTTP/2 404
|
||||||
|
: date: Fri, 20 Aug 2021 19:45:27 GMT
|
||||||
|
: content-type: application/json; charset="utf-8"
|
||||||
|
: content-length: 92
|
||||||
|
: server: anabroker/2.5.1
|
||||||
|
: correlation-id: d7bf3fc1-2598-4431-8d04-cdb7c7210487
|
||||||
|
: vary: Accept
|
||||||
|
:
|
||||||
|
: {"type":"HTTPError","traceback":null,"message":"HTTP 404: Not Found (Assignment Not Found)"}
|
||||||
|
|
||||||
|
#+begin_src http :exports both
|
||||||
|
GET https://anabroker.aweberstage.com/assignment-request/14313
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[bd2a44623b256561164c9142feffc700ab85e2b9]:
|
||||||
|
#+begin_example
|
||||||
|
HTTP/2 200
|
||||||
|
date: Fri, 20 Aug 2021 19:45:30 GMT
|
||||||
|
content-type: application/json; charset="utf-8"
|
||||||
|
content-length: 37
|
||||||
|
server: anabroker/2.5.1
|
||||||
|
correlation-id: 4b5f9f69-f136-481f-8115-253c805049d5
|
||||||
|
last-modified: Fri, 22 May 2020 18:11:47 +0000
|
||||||
|
cache-control: public, max-age=120
|
||||||
|
link: <https://anabroker.aweberstage.com/assignment-request/14313>; rel="self";
|
||||||
|
vary: Accept
|
||||||
|
|
||||||
|
{"account_id":14313,"processed":true}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
#+begin_src http
|
||||||
|
DELETE https://anabroker.aweberstage.com/assignment-request/14313
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[494ee16e8de0fb7b24360a9ff55ce1b8a9b5c665]:
|
||||||
|
: HTTP/2 204
|
||||||
|
: date: Fri, 20 Aug 2021 19:46:04 GMT
|
||||||
|
: server: anabroker/2.5.1
|
||||||
|
: correlation-id: 336a7484-bbc8-4c12-aa43-7960005db825
|
||||||
|
:
|
||||||
|
|
||||||
|
#+begin_src http
|
||||||
|
POST https://anabroker.aweberstage.com/assignment-request/14313
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[273c4c7fae94e95dc11daaf5294fc41e3d486e63]:
|
||||||
|
#+begin_example
|
||||||
|
HTTP/2 200
|
||||||
|
date: Fri, 20 Aug 2021 19:46:14 GMT
|
||||||
|
content-type: application/json; charset="utf-8"
|
||||||
|
content-length: 38
|
||||||
|
server: anabroker/2.5.1
|
||||||
|
correlation-id: 484fde6a-8b56-450b-980a-d95d2071a54a
|
||||||
|
last-modified: Fri, 20 Aug 2021 19:46:14 +0000
|
||||||
|
cache-control: public, max-age=120
|
||||||
|
link: <https://anabroker.aweberstage.com/assignment-request/14313>; rel="self";
|
||||||
|
vary: Accept
|
||||||
|
|
||||||
|
{"account_id":14313,"processed":false}
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
#+begin_src http :exports both
|
||||||
|
GET https://anabroker.aweberstage.com/assignment-request/14313
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS[bd2a44623b256561164c9142feffc700ab85e2b9]:
|
||||||
|
#+begin_example
|
||||||
|
HTTP/2 200
|
||||||
|
date: Fri, 20 Aug 2021 19:52:36 GMT
|
||||||
|
content-type: application/json; charset="utf-8"
|
||||||
|
content-length: 37
|
||||||
|
server: anabroker/2.5.1
|
||||||
|
correlation-id: 91583dea-315d-4655-82df-e3778b6fae48
|
||||||
|
last-modified: Fri, 20 Aug 2021 19:46:14 +0000
|
||||||
|
cache-control: public, max-age=120
|
||||||
|
link: <https://anabroker.aweberstage.com/assignment-request/14313>; rel="self";
|
||||||
|
vary: Accept
|
||||||
|
|
||||||
|
{"account_id":14313,"processed":true}
|
||||||
|
#+end_example
|
||||||
38
daily/2021-08-26.org
Normal file
38
daily/2021-08-26.org
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 7cbf528d-8f3c-496d-a8c0-3845129baaea
|
||||||
|
:END:
|
||||||
|
#+title: 2021-08-26
|
||||||
|
|
||||||
|
* Bulk-tagging
|
||||||
|
Working on [[id:6413d680-ee2e-43e6-b7c7-10f14e0873c2][Deploying Bulk Tagging to Kubernetes]].
|
||||||
|
|
||||||
|
The last change on this project was to update the setup file and gitlab
|
||||||
|
pipeline:
|
||||||
|
https://gitlab.aweber.io/CP/bulk-tagging/-/commit/fecc53cd8b8e34d826d06fbdfa581aa54e508bc7.
|
||||||
|
The pipeline succeeded, but the service is crash-looping in testing. Looks like
|
||||||
|
a whole mess of configuration issues:
|
||||||
|
|
||||||
|
#+begin_example {"asctime":"2021-08-26T19:31:40.964622+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to ::1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.870100+0000","correlation_id":"-","levelname":"INFO","message":"sentry DSN not found, not installing client","module":"__init__","name":"sprockets.mixins.sentry","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.870435+0000","correlation_id":"-","levelname":"INFO","message":"starting processes on port 8000","module":"runner","name":"Runner","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.873323+0000","correlation_id":"-","levelname":"INFO","message":"sprockets_influxdb v2.2.1 installed; 5000 measurements or 60.00 seconds will trigger batch submission","module":"sprockets_influxdb","name":"sprockets_influxdb","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.874454+0000","correlation_id":"-","levelname":"WARNING","message":"CONSUL_HTTP_ADDR should be a well formed URL","module":"consul","name":"aiomappinglib.consul","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.888571+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to ::1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.888805+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to ::1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.889003+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to 127.0.0.1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.889294+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to 127.0.0.1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.889457+0000","correlation_id":"-","levelname":"WARNING","message":"Could not connect, 2 attempts left","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:42.889581+0000","correlation_id":"-","levelname":"INFO","message":"Retrying in 2 seconds","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.891861+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to ::1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.892210+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to ::1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.892408+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to 127.0.0.1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.892620+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to 127.0.0.1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.892793+0000","correlation_id":"-","levelname":"WARNING","message":"Could not connect, 1 attempts left","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.892961+0000","correlation_id":"-","levelname":"INFO","message":"Retrying in 2 seconds","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.893576+0000","correlation_id":"-","levelname":"ERROR","message":"before_run callback <function before_run at 0x7f114e9b5040> cancelled start","module":"app","name":"Application","service":"bulk-tagging","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/app.py\", line 104, in start\n callback(self.tornado_application, io_loop)\n"," File \"/usr/local/lib/python3.9/site-packages/bulk_tagging/app.py\", line 151, in before_run\n raise RuntimeError('AWS credentials not found')\n","RuntimeError: AWS credentials not found\n"]}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.897295+0000","correlation_id":"-","levelname":"INFO","message":"starting IOLoop shutdown process","module":"app","name":"_ShutdownHandler","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.897470+0000","correlation_id":"-","levelname":"INFO","message":"stopped IOLoop","module":"app","name":"_ShutdownHandler","service":"bulk-tagging","exc_info":null}
|
||||||
|
{"asctime":"2021-08-26T19:37:44.897635+0000","correlation_id":"-","levelname":"ERROR","message":"application terminated during start, exiting","module":"runner","name":"Runner","service":"bulk-tagging","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/runner.py\", line 129, in run\n self.application.start(iol)\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/app.py\", line 104, in start\n callback(self.tornado_application, io_loop)\n"," File \"/usr/local/lib/python3.9/site-packages/bulk_tagging/app.py\", line 151, in before_run\n raise RuntimeError('AWS credentials not found')\n","RuntimeError: AWS credentials not found\n"]
|
||||||
|
#+end_example
|
||||||
|
|
||||||
|
Nothing is set in consul yet, and it's concerning that it's also complaining about =CONSUL_HTTP_ADDR=.
|
||||||
103
daily/2021-08-30.org
Normal file
103
daily/2021-08-30.org
Normal file
|
|
@ -0,0 +1,103 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: 6109f700-5c45-4d09-9c58-6f57f6002a3d
|
||||||
|
:END:
|
||||||
|
#+title: 2021-08-30
|
||||||
|
|
||||||
|
* Researching recent sites outages
|
||||||
|
|
||||||
|
Researching recent [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] outages due to OOM-killed pods.
|
||||||
|
|
||||||
|
Checking the log configuration, we don't appear to be capturing request
|
||||||
|
durations nor request size.
|
||||||
|
|
||||||
|
- Retrieve server status
|
||||||
|
+ http://control-panel.service.production.consul/server-status
|
||||||
|
+ [[http://control-panel.service.production.consul/server-status?auto]]
|
||||||
|
+ Retrieve per-pod with a job
|
||||||
|
+ Also grab memory usage by PID from =/proc=
|
||||||
|
|
||||||
|
#+begin_src bash :results output
|
||||||
|
ENVIRONMENT=${ENVIRONMENT:-development}
|
||||||
|
SERVICE=${SERVICE:-aweber-classic}
|
||||||
|
STATSD_HOST=statsd
|
||||||
|
STATSD_PORT=8125
|
||||||
|
|
||||||
|
TMPFILE=$(mktemp)
|
||||||
|
|
||||||
|
capture() {
|
||||||
|
local stat="$1"
|
||||||
|
local path="$2"
|
||||||
|
local type="$3"
|
||||||
|
local fullpath="applications.${SERVICE}.${ENVIRONMENT}.${path}"
|
||||||
|
local value=$(sed -n "s/^${stat}: //p" $TMPFILE)
|
||||||
|
local metric="${fullpath}:${value}|${type}"
|
||||||
|
echo "Capturing metric ${metric}"
|
||||||
|
# echo "${metric}" | nc -w 1 -c ${STATSD_HOST} ${STATSD_PORT}
|
||||||
|
}
|
||||||
|
|
||||||
|
counter() {
|
||||||
|
local stat="$1"
|
||||||
|
local path="$2"
|
||||||
|
capture "$stat" "counters.apache.$path" "c"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
CONTROL_PANEL_PODS=$(kubectl get pods -n cp -l app=control-panel | grep -v 'feature-branch\|sensu' | awk '{ print $1}')
|
||||||
|
|
||||||
|
for pod in $CONTROL_PANEL_PODS
|
||||||
|
do
|
||||||
|
POD_IP=$(kubectl get pod $pod -oyaml | grep " podIP: " | awk '{print $2}')
|
||||||
|
echo "Fetching status from ${pod} (${POD_IP})"
|
||||||
|
curl http://${POD_IP}/server-status?auto > "$TMPFILE"
|
||||||
|
counter "Load1" "load_1"
|
||||||
|
counter "Load5" "load_5"
|
||||||
|
counter "Load15" "load_15"
|
||||||
|
counter "CPUUser" "cpu_user"
|
||||||
|
counter "CPUSystem" "cpu_system"
|
||||||
|
counter "CPUChildrenUser" "cpu_children_user"
|
||||||
|
counter "CPUChildrenSystem" "cpu_children_system"
|
||||||
|
counter "CPULoad" "cpu_load"
|
||||||
|
counter "ReqPerSec" "requests_per_second"
|
||||||
|
counter "BytesPerSec" "bytes_per_second"
|
||||||
|
counter "BytesPerReq" "bytes_per_request"
|
||||||
|
counter "DurationPerReq" "duration_per_request"
|
||||||
|
counter "BusyWorkers" "busy_workers"
|
||||||
|
counter "IdleWorkers" "idle_workers"
|
||||||
|
done
|
||||||
|
|
||||||
|
rm "$TMPFILE"
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+RESULTS:
|
||||||
|
#+begin_example
|
||||||
|
Pod: {}
|
||||||
|
10.51.12.43
|
||||||
|
Pod: {}
|
||||||
|
10.51.27.62
|
||||||
|
Pod: {}
|
||||||
|
10.51.20.19
|
||||||
|
Pod: {}
|
||||||
|
10.51.23.32
|
||||||
|
Pod: {}
|
||||||
|
10.51.13.57
|
||||||
|
Pod: {}
|
||||||
|
10.51.19.22
|
||||||
|
Pod: {}
|
||||||
|
10.51.21.18
|
||||||
|
Pod: {}
|
||||||
|
10.51.15.47
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.load_1:1.96|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.load_5:2.02|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.load_15:1.91|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.cpu_user:42.35|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.cpu_system:55.37|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.cpu_children_user:42393.1|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.cpu_children_system:9040.76|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.cpu_load:27.1413|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.requests_per_second:4.08449|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.bytes_per_second:18589.1|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.bytes_per_request:4551.15|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.duration_per_request:367.236|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.busy_workers:13|c
|
||||||
|
Capturing metric applications.aweber-classic.development.counters.apache.idle_workers:3|c
|
||||||
|
#+end_example
|
||||||
Loading…
Reference in a new issue