updates

2022-06-09 15:46:25 -04:00 · 2022-06-09 15:46:25 -04:00 · d13b06a454
parent 3a64ad410d
commit d13b06a454
7 changed files with 81 additions and 1 deletions
--- a/20220609135721-http_analytics_view_service_production_consul_static_index_html_tag_reports_paths_1reports_1new_subscribers_daily_get.org
+++ b/20220609135721-http_analytics_view_service_production_consul_static_index_html_tag_reports_paths_1reports_1new_subscribers_daily_get.org
--- a/aweber/20210813142844-projects.org
+++ b/aweber/20210813142844-projects.org
@ -54,7 +54,7 @@
 :LOGBOOK:
 - State "BACKLOG"    from              [2021-10-26 Tue 14:02]
 :END:
-** BACKLOG Deploying Recipient Service to Kubernetes
+** DONE Deploying Recipient Service to Kubernetes
 :LOGBOOK:
 - State "BACKLOG"    from "TODO"       [2021-10-20 Wed 15:53]
 - State "TODO"       from              [2021-10-13 Wed 16:26]
@ -75,6 +75,7 @@
 :LOGBOOK:
 - State "TODO"       from              [2021-10-20 Wed 15:57]
 :END:
+* [[id:ee5b8d5f-e3d4-45c2-9ce6-bcd8c7a63376][Retire Redcache]]
 * [[id:4df15f2f-d2e1-40f4-8acd-dbfb78fe304f][Deploy CoreAPI to Kubernetes]]
 * Replacing CAPI Services
 ** TODO [[id:619b6c78-7be9-4ee4-a0b7-9d1a4d7536e2][Migrating services to use the new List service]]
@ -90,6 +91,7 @@
  endpoints be in the new service as well, deprecating public api lists?
 - [2021-10-18 Mon] :: The expectation is set to be migrated to the new list service exclusively by the end of Q2 2022
 ** Subscribers API
+*** [[id:2c1a7b1d-8726-4b88-9534-2f5abfec35f0][Use AppDB as the source of truth for subscriber data in Recipient]]
 * Frontend Client Upgrades
 ** Upgrade Dashboard to React
 *** BACKLOG Create an API for broadcasts and sent messages across lists
--- a/aweber/20210915133101-analytics_view_service.org
+++ b/aweber/20210915133101-analytics_view_service.org
@ -2,6 +2,7 @@
 :ID:       c45881de-46f2-4f76-9579-063626c5956c
 :END:
 #+title: Analytics View Service
+#+TODO: WAITING(w) READY(r) | DONE(d)

 The Analytics View Service provides a collection of report endpoints. These
 endpoints handle querying the analytics databases or, in the longer term,
@ -702,3 +703,35 @@ Uses the =public.leads_stats_day= table in AppDB.
 #+end_src

 ** Migrate reports to the Analytics View Service
+*** READY Use the new analytics view endpoints for the opens over time reports
+Update the opens over time report for [[https://www.aweber.com/users/report/opens_all][all lists]] and the [[https://www.aweber.com/users/report/opens][current list]] to use the
+new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1opens/get][daily opens endpoint]] in the Analytics View service.
+*** READY Use the new analytics view endpoints for the clicks over time reports
+Update the clicks over time report for [[https://www.aweber.com/users/report/clicks_all][all lists]] and the [[https://www.aweber.com/users/report/clicks][current list]] to use the
+new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1clicks/get][daily clicks endpoint]] in the Analytics View service.
+*** WAITING Use the new analytics view endpoints for the sales over time report
+Update the [[https://www.aweber.com/users/report/sales_tracked_all][sales over time report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1sales-by-day/get][daily sales endpoint]] for graph
+summary data and the sale events endpoint for the sales table.
+
+#+begin_notes
+The sale events endpoint present, but is not yet documented!
+#+end_notes
+*** DONE New subscribers
+Update the new subscribers report to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-daily/get][daily]], [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-weekly/get][weekly]], and [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1new-subscribers-monthly/get][monthly]] new
+subscriber endpoints in the Analytics View service.
+*** DONE Subscriber totals
+Update the subscriber totals report to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-daily/get][daily]], [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-weekly/get][weekly]], and [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1total-subscribers-monthly/get][monthly]]
+total subscriber endpoints in the Analytics View service.
+*** WAITING Use the new analytics view endpoints for the broadcast totals report
+Update the [[https://www.aweber.com/users/report/broadcast_totals][broadcast totals report]] to use the new completed broadcasts endpoint
+in the Analytics View service.
+
+#+begin_notes
+The completed broadcasts endpoint is [[https://jira.aweber.io/browse/CCPANEL-11788][not yet complete]].
+#+end_notes
+*** Use the new analytics view endpoints for the follow-up totals report
+Update the [[https://www.aweber.com/users/report/followup_totals][followup totals report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1followup-totals/get][followup totals]] endpoint in the
+Analytics View service.
+*** Use the new analytics view endpoints for the location totals report
+Update the [[https://www.aweber.com/users/report/subscribers_by_location][location totals report]] to use the new [[http://analytics-view.service.production.consul/static/index.html#tag/Reports/paths/~1reports~1subscribers-by-location/get][subscribers by location]]
+endpoint in the Analytics View service.
--- a/aweber/20220520154427-recipient_service.org
+++ b/aweber/20220520154427-recipient_service.org
@ -1,4 +1,5 @@
 :PROPERTIES:
 :ID:       1ff6586e-2dba-41a2-a887-753cc5ac27c9
+:ROAM_REFS: https://gitlab.aweber.io/CP/Services/recipient
 :END:
 #+title: Recipient Service
--- a/aweber/20220609145347-retire_redcache.org
+++ b/aweber/20220609145347-retire_redcache.org
@ -0,0 +1,6 @@
+:PROPERTIES:
+:ID:       ee5b8d5f-e3d4-45c2-9ce6-bcd8c7a63376
+:END:
+#+title: Retire Redcache
+
+https://jira.aweber.io/browse/AWEB-378
--- a/aweber/20220609150836-use_appdb_as_the_source_of_truth_for_subscriber_data_in_recipient.org
+++ b/aweber/20220609150836-use_appdb_as_the_source_of_truth_for_subscriber_data_in_recipient.org
@ -0,0 +1,33 @@
+:PROPERTIES:
+:ID:       2c1a7b1d-8726-4b88-9534-2f5abfec35f0
+:END:
+#+title: Use AppDB as the source of truth for subscriber data in Recipient
+
+The [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]] shall replace [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] as the API for mailing list
+subscribers. Currently, [[id:aa9b1fdc-d766-41bb-ab7b-11c35bca54fa][AWSubscribers]] is responsible for all subscriber creation
+and updating and uses [[id:dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea][AppDB]] as its data store, which is the current source of
+truth for subscriber data. The [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]] maintains its own copy of
+subscriber data in DynamoDB along with its additional recipient data, which is
+kept up to date via the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync Consumer]]. Having both services sharing
+the same data store will allow us to migrate calls towards the [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient Service]]
+without risking data becoming desynchronized between them.
+
+
+
+* Read subscriber data from AppDB by default
+Campaigns may fail as they race to retrieve information from the [[id:1ff6586e-2dba-41a2-a887-753cc5ac27c9][Recipient
+Service]] for a newly created subscriber, which may not have synchronized over
+from [[id:dd113e53-6144-4cb2-a4aa-da3dc2e3e6ea][AppDB]]. Reads are also the least impactful change to make, as it will only
+affect the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync Consumer]] as it checks whether to create or update a
+record. This makes this change the ideal starting point for this project.
+
+The current DynamoDB behavior will be kept intact so that the [[id:b285adee-2dab-48ed-b2d8-2df5594f9d30][Subscriber Sync
+Consumer]] can continue to keep the data stored in DynamoDB up to date.
+** DONE Add support for reading subscriber data from AppDB to Recipient
+** DONE Update subscriber-sync to use flag to read from DynamoDB
+
+** TODO Switch recipient to read subscriber data from AppDB by default
+
+* Write subscriber data to AppDB
+
+* Retire subscriber data in DynamoDB
--- a/aweber/20220609151255-subscriber_sync_consumer.org
+++ b/aweber/20220609151255-subscriber_sync_consumer.org
@ -0,0 +1,5 @@
+:PROPERTIES:
+:ID:       b285adee-2dab-48ed-b2d8-2df5594f9d30
+:ROAM_REFS: https://gitlab.aweber.io/CP/Consumers/subscriber-sync
+:END:
+#+title: Subscriber Sync Consumer