correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
roam/aweber/20210915133101-analytics_view_service.org

705 lines
17 KiB
Org Mode
Raw Normal View History

updates
2021-09-15 21:01:02 +00:00
:PROPERTIES:
:ID: c45881de-46f2-4f76-9579-063626c5956c
:END:
#+title: Analytics View Service
updates
2021-12-01 22:28:09 +00:00
The Analytics View Service provides a collection of report endpoints. These
endpoints handle querying the analytics databases or, in the longer term,
exposing efficient materialized data views.
updates
2021-11-03 03:49:10 +00:00
* Plan
updates
2021-12-01 22:28:09 +00:00
- Parent ticket :: [[https://jira.aweber.io/browse/CCPANEL-11781][CCPANEL-11781]]
updates
2021-11-03 03:49:10 +00:00
** Create the analytics view service
- New project using cookie cutter
- Deployed to kubernetes
- Grafana dashboard created
** Create the analytics view service playbook
** Plan API structure
- Pathing (=/reports/*=)?
notes
2022-02-18 19:34:35 +00:00
- Report versioning? (=/reports/$NAME.v$VERSION=)?
updates
2021-11-03 03:49:10 +00:00
** Create endpoints for existing reports
- Based on the endpoints provided in the [[id:3ddc4e32-932f-4748-bfe9-7025d4d6b352][Report API Controller]]
- Are all of these report endpoints in use?
updates
2021-12-01 22:28:09 +00:00
*** Opens over time
#+attr_confluence: :as-table t
- Name :: daily-opens
- Parameters ::
- List (default: all lists)
- Date range (default: last 30 days)
- Report API controller endpoints ::
- opens_all_range
- opens_list_range
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
updates
2021-11-04 20:52:09 +00:00
{
"2021-11-02T00:00:00Z": {
"broadcasts": 2499,
"followups": 2547,
"unique": 2923,
"total": 5046
},
"2021-11-03T00:00:00Z": {
"broadcasts": 25808,
"followups": 2430,
"unique": 24876,
"total": 28238
},
"2021-11-04T00:00:00Z": {
"broadcasts": 16733,
"followups": 1437,
"unique": 14780,
"total": 18170
}
}
updates
2021-12-01 22:28:09 +00:00
#+end_src
*** Clicks over time
#+attr_confluence: :as-table t
- Name :: daily-clicks
- Parameters ::
- List (default: all lists)
- Date range (default: last 30 days)
- Report API controller endpoints ::
- clicks_all_range
- clicks_list_range
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
updates
2021-11-04 20:52:09 +00:00
{
"2021-11-02T00:00:00Z": {
"broadcasts": 105,
"followups": 137,
"unique": 130,
"total": 242
},
"2021-11-03T00:00:00Z": {
"broadcasts": 636,
"followups": 185,
"unique": 622,
"total": 821
},
"2021-11-04T00:00:00Z": {
"broadcasts": 480,
"followups": 109,
"unique": 426,
"total": 589
}
}
updates
2021-12-01 22:28:09 +00:00
#+end_src
*** Sales over time (events)
#+attr_confluence: :as-table t
- Name :: sale-events
- Parameters ::
- Date range (default: last 30 days)
- Currency (default: USD)
- Report API controller endpoints ::
- sales_tracked_events
updates
2021-11-04 20:52:09 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
updates
2021-11-04 20:52:09 +00:00
[
{
"time": "2021-11-02 09:37:36-04",
"type": "followup",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "team@harmoniamedia.com"
},
{
"time": "2021-11-02 09:37:37-04",
"type": "followup",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "team@harmoniamedia.com"
},
{
"time": "2021-11-02 12:01:17-04",
"type": "followup",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "giuliagiardino12@gmail.com"
},
{
"time": "2021-11-02 12:01:19-04",
"type": "followup",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "giuliagiardino12@gmail.com"
},
{
"time": "2021-11-04 05:21:35-04",
"type": "broadcast",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "jeremy@jeremy-quick.com"
},
{
"time": "2021-11-04 05:21:36-04",
"type": "broadcast",
"currency": "USD",
"revenue": "19.00",
"note": "",
"description": "Upgraded to Pro",
"source_url": "https://www.aweber.com/users/#upgraded",
"email": "jeremy@jeremy-quick.com"
}
]
updates
2021-12-01 22:28:09 +00:00
#+end_src
*** Sales over time (summary)
#+attr_confluence: :as-table t
- Name :: daily-sales
- Parameters ::
- Date range (default: last 60 days)
- Currency (default: USD)
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
updates
2021-11-04 20:52:09 +00:00
{
"2021-11-02T00:00:00Z": {
"broadcast": 0,
"followup": 76,
"pageview": 76,
"ecommerce": 0,
"total": 76
},
"2021-11-03T00:00:00Z": {
"broadcast": 0,
"followup": 0,
"pageview": 0,
"ecommerce": 0,
"total": 0
},
"2021-11-04T00:00:00Z": {
"broadcast": 38,
"followup": 0,
"pageview": 38,
"ecommerce": 0,
"total": 38
}
}
updates
2021-12-01 22:28:09 +00:00
#+end_src
*** Lifetime Sale Totals
#+attr_confluence: :as-table t
- Name :: sale-totals
- Parameters ::
- Currency (default: USD)
- Report API controller endpoints ::
- sales_tracked_total
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
{
"count": 94924,
"pagehits": 1820122.79,
"revenue": 2181498.96
}
#+end_src
*** Sale Currencies
#+attr_confluence: :as-table t
- Name :: sale-currencies
- Parameters ::
- Date range (default: last 60 days)
- Report API controller endpoints ::
- sales_tracked_currencies
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
["USD", "CAD"]
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Pending Broadcasts
updates
2021-12-01 22:28:09 +00:00
#+attr_confluence: :as-table t
- Name :: pending-broadcasts
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- broadcasts_pending
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
[
{
"unit_id": "3854",
"send_date": "01/01/25 12:00am",
"broadcast_id": "1800243",
"subject": "Scheduled Broadcast Test",
"status": "Queue",
"percent_done": "0",
"mesg_encoding": "utf-8",
"campaign_id": "1802217",
"unit": "awlist3854",
"orig_send_date": "2025-01-01 00:00:00-05",
"friendly_list_name": "Fluff Cafe",
"id": "1802217",
"list_id": "3854",
"account_id": "778",
"campaign_type_id": "b",
"uses_block_editor": "t"
}
]
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Completed Broadcasts
updates
2021-12-01 22:28:09 +00:00
#+attr_confluence: :as-table t
- Name :: completed-broadcasts
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- broadcasts_completed
- broadcasts_completed_all
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
[
{
"a_id": "778",
"broadcast_id": "1258855",
"for_sent_date": "08/14/20 03:41 PM",
"track_click_rate": "1",
"unit_id": "3854",
"assassin_pts": "0",
"mesg_type": "HTML",
"mesg_encoding": "utf-8",
"sent_date": "2020-08-14 15:41:05.24086-04",
"num_emailed": "1",
"num_undeliv": 0,
"num_opened": 0,
"num_attachments": 0,
"num_complaints": 0,
"subject": "Testing a bad segment",
"created_date": "2020-08-14 15:38:57.59072-04",
"status": "Sent Composer",
"lead_view_id": "42564",
"unit": "awlist3854",
"pct_opened": "0",
"show_opens_warning": true,
"pct_undeliv": "0",
"total_clicks": 0,
"clicks_analytics_type": "premium",
"pct_click": "0",
"pct_complaints": "0",
"segment_id": "42564",
"extra_lists": "",
"excluded_lists": "",
"extra_lists_count": 0,
"excluded_lists_count": 0,
"friendly_list_name": "Fluff Cafe"
},
{
"a_id": "778",
"broadcast_id": "1243650",
"for_sent_date": "08/05/20 06:33 PM",
"track_click_rate": "1",
"unit_id": "3854",
"assassin_pts": "0",
"mesg_type": "Text/HTML",
"mesg_encoding": "utf-8",
"sent_date": "2020-08-05 18:33:48.638875-04",
"num_emailed": "27",
"num_undeliv": 0,
"num_opened": 0,
"num_attachments": 0,
"num_complaints": 0,
"subject": "Buggssss 🐛🐛🐛🐛🐛🐛🐛",
"created_date": "2020-08-05 18:31:10.567418-04",
"status": "Sent Composer",
"lead_view_id": "8",
"unit": "awlist3854",
"pct_opened": "0",
"show_opens_warning": true,
"pct_undeliv": "0",
"total_clicks": 0,
"clicks_analytics_type": "premium",
"pct_click": "0",
"pct_complaints": "0",
"segment_id": "8",
"extra_lists": "",
"excluded_lists": "",
"extra_lists_count": 0,
"excluded_lists_count": 0,
"friendly_list_name": "Fluff Cafe"
}
]
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Cities, States, and Countries
updates
2021-12-01 22:28:09 +00:00
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- city_state_countries
updates
2021-11-03 03:49:10 +00:00
updates
2021-12-01 22:28:09 +00:00
#+caption: Sample response
#+begin_src json
[
{
"country": null,
"state": null,
"city": null,
"pending": 0,
"unsubscribed": 457262,
"subscribed": 276245,
"total": 733507
},
{
"country": "US",
"state": "PA",
"city": "Newtown",
"pending": 0,
"unsubscribed": 279304,
"subscribed": 3236,
"total": 282540
},
{
"country": "US",
"state": null,
"city": null,
"pending": 0,
"unsubscribed": 109074,
"subscribed": 2456,
"total": 111530
},
{
"country": "US",
"state": "PA",
"city": "Philadelphia",
"pending": 0,
"unsubscribed": 48485,
"subscribed": 657,
"total": 49142
},
{
"country": "US",
"state": "PA",
"city": "Huntingdon Valley",
"pending": 0,
"unsubscribed": 4839,
"subscribed": 90,
"total": 4929
},
{
"country": "VN",
"state": "64",
"city": "Hanoi",
"pending": 0,
"unsubscribed": 204,
"subscribed": 2,
"total": 206
},
{
"country": "VN",
"state": "65",
"city": "Ho Chi Minh City",
"pending": 0,
"unsubscribed": 176,
"subscribed": 1,
"total": 177
}
]
#+end_src
notes
2022-02-18 19:34:35 +00:00
*** Campaign Statistics
Add the campaign start endpoint into analytics-view including
- DynamoDB fixtures
- Dynamo dbhelpers (currently there is no dynamo connectivity in analytics-view)
- CampaignStarted Handler
https://gitlab.aweber.io/CP/Services/campaignstats/-/blob/master/campaignstats/handlers.py
- Grafana dashboard updated
- Alerts configured
- Confluence docs updated
updates
2021-11-03 03:49:10 +00:00
*** Followups
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: followup-totals
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- followups
Uses the =freq_mesg= and =freq_mesg_stats= tables in AppDB combined with data
from the =messages= table in Analytics.
=editor_path= is hard-coded in the current report endpoint.
#+caption: Sample Response (Production AID 91)
#+begin_src json
[
{
"followup_message": "1",
"num_emailed": 83024,
"num_opened": 89950,
"open_percentage": 23.659423781075,
"clicks": "13198",
"message_id": "28613652",
"click_percentage": 0,
"unique_opens": "19643",
"unique_clicks": "1584",
"clicks_percentage": 23.659423781075,
"editor_path": "messages#/active"
},
{
"followup_message": "2",
"num_emailed": 73714,
"num_opened": 49979,
"open_percentage": 13.675828200884,
"clicks": "6169",
"message_id": "28613655",
"click_percentage": 0,
"unique_opens": "10081",
"unique_clicks": "649",
"clicks_percentage": 13.675828200884,
"editor_path": "messages#/active"
},
{
"followup_message": "3",
"num_emailed": 70880,
"num_opened": 38472,
"open_percentage": 10.272291196388,
"clicks": "4322",
"message_id": "28613659",
"click_percentage": 0,
"unique_opens": "7281",
"unique_clicks": "409",
"clicks_percentage": 10.272291196388,
"editor_path": "messages#/active"
},
{
"followup_message": "4",
"num_emailed": 69375,
"num_opened": 37496,
"open_percentage": 10.105945945946,
"clicks": "4419",
"message_id": "28613661",
"click_percentage": 0,
"unique_opens": "7011",
"unique_clicks": "447",
"clicks_percentage": 10.105945945946,
"editor_path": "messages#/active"
},
{
"followup_message": "5",
"num_emailed": 67478,
"num_opened": 30996,
"open_percentage": 8.2752897240582,
"clicks": "3236",
"message_id": "28613662",
"click_percentage": 0,
"unique_opens": "5584",
"unique_clicks": "257",
"clicks_percentage": 8.2752897240582,
"editor_path": "messages#/active"
},
{
"followup_message": "6",
"num_emailed": 65414,
"num_opened": 32483,
"open_percentage": 9.2334974164552,
"clicks": "3127",
"message_id": "28613664",
"click_percentage": 0,
"unique_opens": "6040",
"unique_clicks": "323",
"clicks_percentage": 9.2334974164552,
"editor_path": "messages#/active"
},
{
"followup_message": "7",
"num_emailed": 63871,
"num_opened": 28756,
"open_percentage": 8.1883797028385,
"clicks": "1964",
"message_id": "28613665",
"click_percentage": 0,
"unique_opens": "5230",
"unique_clicks": "166",
"clicks_percentage": 8.1883797028385,
"editor_path": "messages#/active"
}
]
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** New Subscribers Daily
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: daily-new-subscribers
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- new_subscribers_daily
updates
2021-11-03 03:49:10 +00:00
notes
2022-02-18 19:34:35 +00:00
Uses the =public.leads_stats_day= table in AppDB.
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2022-01-09T00:00:00Z": {
"subscribed": 324,
"unsubscribed": 83
},
"2022-01-10T00:00:00Z": {
"subscribed": 417,
"unsubscribed": 80
},
"2022-01-11T00:00:00Z": {
"subscribed": 433,
"unsubscribed": 92
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** New Subscribers Weekly
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: weekly-new-subscribers
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- new_subscribers_weekly
Uses the =public.leads_stats_day= table in AppDB.
updates
2021-11-03 03:49:10 +00:00
notes
2022-02-18 19:34:35 +00:00
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2021-03-26T00:00:00Z": {
"subscribed": 3182,
"unsubscribed": 1249
},
"2021-04-02T00:00:00Z": {
"subscribed": 3423,
"unsubscribed": 1497
},
"2021-04-09T00:00:00Z": {
"subscribed": 3052,
"unsubscribed": 1217
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** New Subscribers Monthly
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: monthly-new-subscribers
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- new_subscribers_monthly
updates
2021-11-03 03:49:10 +00:00
notes
2022-02-18 19:34:35 +00:00
Uses the =public.leads_stats_day= table in AppDB.
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2021-03-01T00:00:00Z": {
"subscribed": 14972,
"unsubscribed": 4770
},
"2021-04-01T00:00:00Z": {
"subscribed": 14973,
"unsubscribed": 7181
},
"2021-05-01T00:00:00Z": {
"subscribed": 12652,
"unsubscribed": 5243
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Subscriber Totals Daily
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- subscriber_totals_daily
updates
2021-11-03 03:49:10 +00:00
notes
2022-02-18 19:34:35 +00:00
Uses the =public.leads_stats_day= table in AppDB.
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2022-01-09T00:00:00Z": {
"subscribed": 289192,
"unsubscribed": 920812
},
"2022-01-10T00:00:00Z": {
"subscribed": 289609,
"unsubscribed": 920892
},
"2022-01-11T00:00:00Z": {
"subscribed": 290042,
"unsubscribed": 920984
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Subscriber Totals Weekly
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- subscriber_totals_weekly
Uses the =public.leads_stats_day= table in AppDB.
updates
2021-11-03 03:49:10 +00:00
notes
2022-02-18 19:34:35 +00:00
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2021-02-08T00:00:00Z": {
"subscribed": 191745,
"unsubscribed": 887713
},
"2021-02-15T00:00:00Z": {
"subscribed": 193972,
"unsubscribed": 888733
},
"2021-02-22T00:00:00Z": {
"subscribed": 196200,
"unsubscribed": 889717
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
*** Subscriber Totals Monthly
notes
2022-02-18 19:34:35 +00:00
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
- List (default: all lists)
- Report API controller endpoints ::
- subscriber_totals_monthly
Uses the =public.leads_stats_day= table in AppDB.
#+caption: Sample Response (Production AID 91)
#+begin_src json
{
"2021-02-01T00:00:00Z": {
"subscribed": 196200,
"unsubscribed": 889717
},
"2021-03-01T00:00:00Z": {
"subscribed": 206569,
"unsubscribed": 894298
},
"2021-04-01T00:00:00Z": {
"subscribed": 215813,
"unsubscribed": 900020
},
...
}
#+end_src
updates
2021-11-03 03:49:10 +00:00
** Migrate reports to the Analytics View Service
Reference in a new issue Copy permalink