:PROPERTIES:
:ID:       c45881de-46f2-4f76-9579-063626c5956c
:END:
#+title: Analytics View Service

The Analytics View Service provides a collection of report endpoints. These
endpoints handle querying the analytics databases or, in the longer term,
exposing efficient materialized data views.

* Plan
- Parent ticket :: [[https://jira.aweber.io/browse/CCPANEL-11781][CCPANEL-11781]]
** Create the analytics view service
- New project using cookie cutter
- Deployed to kubernetes
- Grafana dashboard created
** Create the analytics view service playbook
** Plan API structure
- Pathing (=/reports/*=)?
- Report versioning? (=/reports/$NAME.v$VERSION=)?
** Create endpoints for existing reports
- Based on the endpoints provided in the [[id:3ddc4e32-932f-4748-bfe9-7025d4d6b352][Report API Controller]]
- Are all of these report endpoints in use?
*** Opens over time
#+attr_confluence: :as-table t
- Name :: daily-opens
- Parameters ::
  - List (default: all lists)
  - Date range (default: last 30 days)
- Report API controller endpoints ::
  - opens_all_range
  - opens_list_range

#+caption: Sample response
#+begin_src json
  {
    "2021-11-02T00:00:00Z": {
      "broadcasts": 2499,
      "followups": 2547,
      "unique": 2923,
      "total": 5046
    },
    "2021-11-03T00:00:00Z": {
      "broadcasts": 25808,
      "followups": 2430,
      "unique": 24876,
      "total": 28238
    },
    "2021-11-04T00:00:00Z": {
      "broadcasts": 16733,
      "followups": 1437,
      "unique": 14780,
      "total": 18170
    }
  }
#+end_src

*** Clicks over time
#+attr_confluence: :as-table t
- Name :: daily-clicks
- Parameters ::
  - List (default: all lists)
  - Date range (default: last 30 days)
- Report API controller endpoints ::
  - clicks_all_range
  - clicks_list_range

#+caption: Sample response
#+begin_src json
  {
    "2021-11-02T00:00:00Z": {
      "broadcasts": 105,
      "followups": 137,
      "unique": 130,
      "total": 242
    },
    "2021-11-03T00:00:00Z": {
      "broadcasts": 636,
      "followups": 185,
      "unique": 622,
      "total": 821
    },
    "2021-11-04T00:00:00Z": {
      "broadcasts": 480,
      "followups": 109,
      "unique": 426,
      "total": 589
    }
  }
#+end_src

*** Sales over time (events)
#+attr_confluence: :as-table t
- Name :: sale-events
- Parameters ::
  - Date range (default: last 30 days)
  - Currency (default: USD)
- Report API controller endpoints ::
  - sales_tracked_events

#+caption: Sample response
#+begin_src json
  [
    {
      "time": "2021-11-02 09:37:36-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "team@harmoniamedia.com"
    },
    {
      "time": "2021-11-02 09:37:37-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "team@harmoniamedia.com"
    },
    {
      "time": "2021-11-02 12:01:17-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "giuliagiardino12@gmail.com"
    },
    {
      "time": "2021-11-02 12:01:19-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "giuliagiardino12@gmail.com"
    },
    {
      "time": "2021-11-04 05:21:35-04",
      "type": "broadcast",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "jeremy@jeremy-quick.com"
    },
    {
      "time": "2021-11-04 05:21:36-04",
      "type": "broadcast",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "jeremy@jeremy-quick.com"
    }
  ]
#+end_src
*** Sales over time (summary)
#+attr_confluence: :as-table t
- Name :: daily-sales
- Parameters ::
  - Date range (default: last 60 days)
  - Currency (default: USD)

#+caption: Sample response
#+begin_src json
  {
    "2021-11-02T00:00:00Z": {
      "broadcast": 0,
      "followup": 76,
      "pageview": 76,
      "ecommerce": 0,
      "total": 76
    },
    "2021-11-03T00:00:00Z": {
      "broadcast": 0,
      "followup": 0,
      "pageview": 0,
      "ecommerce": 0,
      "total": 0
    },
    "2021-11-04T00:00:00Z": {
      "broadcast": 38,
      "followup": 0,
      "pageview": 38,
      "ecommerce": 0,
      "total": 38
    }
  }
#+end_src
*** Lifetime Sale Totals
#+attr_confluence: :as-table t
- Name :: sale-totals
- Parameters ::
  - Currency (default: USD)
- Report API controller endpoints ::
  - sales_tracked_total

#+caption: Sample response
#+begin_src json
  {
    "count": 94924,
    "pagehits": 1820122.79,
    "revenue": 2181498.96
  }
#+end_src
*** Sale Currencies
#+attr_confluence: :as-table t
- Name :: sale-currencies
- Parameters ::
  - Date range (default: last 60 days)
- Report API controller endpoints ::
  - sales_tracked_currencies

#+caption: Sample response
#+begin_src json
["USD", "CAD"]
#+end_src
*** Pending Broadcasts
#+attr_confluence: :as-table t
- Name :: pending-broadcasts
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - broadcasts_pending

#+caption: Sample response
#+begin_src json
  [
    {
      "unit_id": "3854",
      "send_date": "01/01/25 12:00am",
      "broadcast_id": "1800243",
      "subject": "Scheduled Broadcast Test",
      "status": "Queue",
      "percent_done": "0",
      "mesg_encoding": "utf-8",
      "campaign_id": "1802217",
      "unit": "awlist3854",
      "orig_send_date": "2025-01-01 00:00:00-05",
      "friendly_list_name": "Fluff Cafe",
      "id": "1802217",
      "list_id": "3854",
      "account_id": "778",
      "campaign_type_id": "b",
      "uses_block_editor": "t"
    }
  ]
#+end_src
*** Completed Broadcasts
#+attr_confluence: :as-table t
- Name :: completed-broadcasts
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - broadcasts_completed
  - broadcasts_completed_all

#+caption: Sample response
#+begin_src json
  [
    {
      "a_id": "778",
      "broadcast_id": "1258855",
      "for_sent_date": "08/14/20 03:41 PM",
      "track_click_rate": "1",
      "unit_id": "3854",
      "assassin_pts": "0",
      "mesg_type": "HTML",
      "mesg_encoding": "utf-8",
      "sent_date": "2020-08-14 15:41:05.24086-04",
      "num_emailed": "1",
      "num_undeliv": 0,
      "num_opened": 0,
      "num_attachments": 0,
      "num_complaints": 0,
      "subject": "Testing a bad segment",
      "created_date": "2020-08-14 15:38:57.59072-04",
      "status": "Sent Composer",
      "lead_view_id": "42564",
      "unit": "awlist3854",
      "pct_opened": "0",
      "show_opens_warning": true,
      "pct_undeliv": "0",
      "total_clicks": 0,
      "clicks_analytics_type": "premium",
      "pct_click": "0",
      "pct_complaints": "0",
      "segment_id": "42564",
      "extra_lists": "",
      "excluded_lists": "",
      "extra_lists_count": 0,
      "excluded_lists_count": 0,
      "friendly_list_name": "Fluff Cafe"
    },
    {
      "a_id": "778",
      "broadcast_id": "1243650",
      "for_sent_date": "08/05/20 06:33 PM",
      "track_click_rate": "1",
      "unit_id": "3854",
      "assassin_pts": "0",
      "mesg_type": "Text/HTML",
      "mesg_encoding": "utf-8",
      "sent_date": "2020-08-05 18:33:48.638875-04",
      "num_emailed": "27",
      "num_undeliv": 0,
      "num_opened": 0,
      "num_attachments": 0,
      "num_complaints": 0,
      "subject": "Buggssss 🐛🐛🐛🐛🐛🐛🐛",
      "created_date": "2020-08-05 18:31:10.567418-04",
      "status": "Sent Composer",
      "lead_view_id": "8",
      "unit": "awlist3854",
      "pct_opened": "0",
      "show_opens_warning": true,
      "pct_undeliv": "0",
      "total_clicks": 0,
      "clicks_analytics_type": "premium",
      "pct_click": "0",
      "pct_complaints": "0",
      "segment_id": "8",
      "extra_lists": "",
      "excluded_lists": "",
      "extra_lists_count": 0,
      "excluded_lists_count": 0,
      "friendly_list_name": "Fluff Cafe"
    }
  ]
#+end_src
*** Cities, States, and Countries
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - city_state_countries

#+caption: Sample response
#+begin_src json
  [
    {
      "country": null,
      "state": null,
      "city": null,
      "pending": 0,
      "unsubscribed": 457262,
      "subscribed": 276245,
      "total": 733507
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Newtown",
      "pending": 0,
      "unsubscribed": 279304,
      "subscribed": 3236,
      "total": 282540
    },
    {
      "country": "US",
      "state": null,
      "city": null,
      "pending": 0,
      "unsubscribed": 109074,
      "subscribed": 2456,
      "total": 111530
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Philadelphia",
      "pending": 0,
      "unsubscribed": 48485,
      "subscribed": 657,
      "total": 49142
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Huntingdon Valley",
      "pending": 0,
      "unsubscribed": 4839,
      "subscribed": 90,
      "total": 4929
    },
    {
      "country": "VN",
      "state": "64",
      "city": "Hanoi",
      "pending": 0,
      "unsubscribed": 204,
      "subscribed": 2,
      "total": 206
    },
    {
      "country": "VN",
      "state": "65",
      "city": "Ho Chi Minh City",
      "pending": 0,
      "unsubscribed": 176,
      "subscribed": 1,
      "total": 177
    }
  ]
#+end_src
*** Campaign Statistics
Add the campaign start endpoint into analytics-view including

- DynamoDB fixtures
- Dynamo dbhelpers (currently there is no dynamo connectivity in analytics-view)
- CampaignStarted Handler
  https://gitlab.aweber.io/CP/Services/campaignstats/-/blob/master/campaignstats/handlers.py
- Grafana dashboard updated
- Alerts configured
- Confluence docs updated

*** Followups
#+attr_confluence: :as-table t
- Name :: followup-totals
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - followups

Uses the =freq_mesg= and =freq_mesg_stats= tables in AppDB combined with data
from the =messages= table in Analytics.

=editor_path= is hard-coded in the current report endpoint.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  [
    {
      "followup_message": "1",
      "num_emailed": 83024,
      "num_opened": 89950,
      "open_percentage": 23.659423781075,
      "clicks": "13198",
      "message_id": "28613652",
      "click_percentage": 0,
      "unique_opens": "19643",
      "unique_clicks": "1584",
      "clicks_percentage": 23.659423781075,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "2",
      "num_emailed": 73714,
      "num_opened": 49979,
      "open_percentage": 13.675828200884,
      "clicks": "6169",
      "message_id": "28613655",
      "click_percentage": 0,
      "unique_opens": "10081",
      "unique_clicks": "649",
      "clicks_percentage": 13.675828200884,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "3",
      "num_emailed": 70880,
      "num_opened": 38472,
      "open_percentage": 10.272291196388,
      "clicks": "4322",
      "message_id": "28613659",
      "click_percentage": 0,
      "unique_opens": "7281",
      "unique_clicks": "409",
      "clicks_percentage": 10.272291196388,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "4",
      "num_emailed": 69375,
      "num_opened": 37496,
      "open_percentage": 10.105945945946,
      "clicks": "4419",
      "message_id": "28613661",
      "click_percentage": 0,
      "unique_opens": "7011",
      "unique_clicks": "447",
      "clicks_percentage": 10.105945945946,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "5",
      "num_emailed": 67478,
      "num_opened": 30996,
      "open_percentage": 8.2752897240582,
      "clicks": "3236",
      "message_id": "28613662",
      "click_percentage": 0,
      "unique_opens": "5584",
      "unique_clicks": "257",
      "clicks_percentage": 8.2752897240582,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "6",
      "num_emailed": 65414,
      "num_opened": 32483,
      "open_percentage": 9.2334974164552,
      "clicks": "3127",
      "message_id": "28613664",
      "click_percentage": 0,
      "unique_opens": "6040",
      "unique_clicks": "323",
      "clicks_percentage": 9.2334974164552,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "7",
      "num_emailed": 63871,
      "num_opened": 28756,
      "open_percentage": 8.1883797028385,
      "clicks": "1964",
      "message_id": "28613665",
      "click_percentage": 0,
      "unique_opens": "5230",
      "unique_clicks": "166",
      "clicks_percentage": 8.1883797028385,
      "editor_path": "messages#/active"
    }
  ]
#+end_src
*** New Subscribers Daily
#+attr_confluence: :as-table t
- Name :: daily-new-subscribers
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - new_subscribers_daily

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2022-01-09T00:00:00Z": {
      "subscribed": 324,
      "unsubscribed": 83
    },
    "2022-01-10T00:00:00Z": {
      "subscribed": 417,
      "unsubscribed": 80
    },
    "2022-01-11T00:00:00Z": {
      "subscribed": 433,
      "unsubscribed": 92
    },
    ...
  }
#+end_src
*** New Subscribers Weekly
#+attr_confluence: :as-table t
- Name :: weekly-new-subscribers
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - new_subscribers_weekly

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2021-03-26T00:00:00Z": {
      "subscribed": 3182,
      "unsubscribed": 1249
    },
    "2021-04-02T00:00:00Z": {
      "subscribed": 3423,
      "unsubscribed": 1497
    },
    "2021-04-09T00:00:00Z": {
      "subscribed": 3052,
      "unsubscribed": 1217
    },
    ...
  }
#+end_src
*** New Subscribers Monthly
#+attr_confluence: :as-table t
- Name :: monthly-new-subscribers
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - new_subscribers_monthly

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2021-03-01T00:00:00Z": {
      "subscribed": 14972,
      "unsubscribed": 4770
    },
    "2021-04-01T00:00:00Z": {
      "subscribed": 14973,
      "unsubscribed": 7181
    },
    "2021-05-01T00:00:00Z": {
      "subscribed": 12652,
      "unsubscribed": 5243
    },
    ...
  }
#+end_src
*** Subscriber Totals Daily
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - subscriber_totals_daily

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2022-01-09T00:00:00Z": {
      "subscribed": 289192,
      "unsubscribed": 920812
    },
    "2022-01-10T00:00:00Z": {
      "subscribed": 289609,
      "unsubscribed": 920892
    },
    "2022-01-11T00:00:00Z": {
      "subscribed": 290042,
      "unsubscribed": 920984
    },
    ...
  }
#+end_src
*** Subscriber Totals Weekly
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - subscriber_totals_weekly

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2021-02-08T00:00:00Z": {
      "subscribed": 191745,
      "unsubscribed": 887713
    },
    "2021-02-15T00:00:00Z": {
      "subscribed": 193972,
      "unsubscribed": 888733
    },
    "2021-02-22T00:00:00Z": {
      "subscribed": 196200,
      "unsubscribed": 889717
    },
    ...
  }
#+end_src
*** Subscriber Totals Monthly
#+attr_confluence: :as-table t
- Name :: subscribers-by-location
- Parameters ::
  - List (default: all lists)
- Report API controller endpoints ::
  - subscriber_totals_monthly

Uses the =public.leads_stats_day= table in AppDB.

#+caption: Sample Response (Production AID 91)
#+begin_src json
  {
    "2021-02-01T00:00:00Z": {
      "subscribed": 196200,
      "unsubscribed": 889717
    },
    "2021-03-01T00:00:00Z": {
      "subscribed": 206569,
      "unsubscribed": 894298
    },
    "2021-04-01T00:00:00Z": {
      "subscribed": 215813,
      "unsubscribed": 900020
    },
    ...
  }
#+end_src

** Migrate reports to the Analytics View Service