correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
roam/aweber/20210915133101-analytics_view_service.org
Correl Roush d13b06a454 updates
2022-06-09 15:46:25 -04:00

20 KiB
Raw Blame History

Analytics View Service

The Analytics View Service provides a collection of report endpoints. These endpoints handle querying the analytics databases or, in the longer term, exposing efficient materialized data views.

Plan

Parent ticket
CCPANEL-11781

Create the analytics view service

  • New project using cookie cutter
  • Deployed to kubernetes
  • Grafana dashboard created

Create the analytics view service playbook

Plan API structure

  • Pathing (/reports/*)?
  • Report versioning? (/reports/$NAME.v$VERSION)?

Create endpoints for existing reports

Opens over time

Name
daily-opens
Parameters
  • List (default: all lists)
  • Date range (default: last 30 days)
Report API controller endpoints
  • opens_all_range
  • opens_list_range
  {
    "2021-11-02T00:00:00Z": {
      "broadcasts": 2499,
      "followups": 2547,
      "unique": 2923,
      "total": 5046
    },
    "2021-11-03T00:00:00Z": {
      "broadcasts": 25808,
      "followups": 2430,
      "unique": 24876,
      "total": 28238
    },
    "2021-11-04T00:00:00Z": {
      "broadcasts": 16733,
      "followups": 1437,
      "unique": 14780,
      "total": 18170
    }
  }
Sample response

Clicks over time

Name
daily-clicks
Parameters
  • List (default: all lists)
  • Date range (default: last 30 days)
Report API controller endpoints
  • clicks_all_range
  • clicks_list_range
  {
    "2021-11-02T00:00:00Z": {
      "broadcasts": 105,
      "followups": 137,
      "unique": 130,
      "total": 242
    },
    "2021-11-03T00:00:00Z": {
      "broadcasts": 636,
      "followups": 185,
      "unique": 622,
      "total": 821
    },
    "2021-11-04T00:00:00Z": {
      "broadcasts": 480,
      "followups": 109,
      "unique": 426,
      "total": 589
    }
  }
Sample response

Sales over time (events)

Name
sale-events
Parameters
  • Date range (default: last 30 days)
  • Currency (default: USD)
Report API controller endpoints
  • sales_tracked_events
  [
    {
      "time": "2021-11-02 09:37:36-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "team@harmoniamedia.com"
    },
    {
      "time": "2021-11-02 09:37:37-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "team@harmoniamedia.com"
    },
    {
      "time": "2021-11-02 12:01:17-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "giuliagiardino12@gmail.com"
    },
    {
      "time": "2021-11-02 12:01:19-04",
      "type": "followup",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "giuliagiardino12@gmail.com"
    },
    {
      "time": "2021-11-04 05:21:35-04",
      "type": "broadcast",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "jeremy@jeremy-quick.com"
    },
    {
      "time": "2021-11-04 05:21:36-04",
      "type": "broadcast",
      "currency": "USD",
      "revenue": "19.00",
      "note": "",
      "description": "Upgraded to Pro",
      "source_url": "https://www.aweber.com/users/#upgraded",
      "email": "jeremy@jeremy-quick.com"
    }
  ]
Sample response

Sales over time (summary)

Name
daily-sales
Parameters
  • Date range (default: last 60 days)
  • Currency (default: USD)
  {
    "2021-11-02T00:00:00Z": {
      "broadcast": 0,
      "followup": 76,
      "pageview": 76,
      "ecommerce": 0,
      "total": 76
    },
    "2021-11-03T00:00:00Z": {
      "broadcast": 0,
      "followup": 0,
      "pageview": 0,
      "ecommerce": 0,
      "total": 0
    },
    "2021-11-04T00:00:00Z": {
      "broadcast": 38,
      "followup": 0,
      "pageview": 38,
      "ecommerce": 0,
      "total": 38
    }
  }
Sample response

Lifetime Sale Totals

Name
sale-totals
Parameters
  • Currency (default: USD)
Report API controller endpoints
  • sales_tracked_total
  {
    "count": 94924,
    "pagehits": 1820122.79,
    "revenue": 2181498.96
  }
Sample response

Sale Currencies

Name
sale-currencies
Parameters
  • Date range (default: last 60 days)
Report API controller endpoints
  • sales_tracked_currencies
["USD", "CAD"]
Sample response

Pending Broadcasts

Name
pending-broadcasts
Parameters
  • List (default: all lists)
Report API controller endpoints
  • broadcasts_pending
  [
    {
      "unit_id": "3854",
      "send_date": "01/01/25 12:00am",
      "broadcast_id": "1800243",
      "subject": "Scheduled Broadcast Test",
      "status": "Queue",
      "percent_done": "0",
      "mesg_encoding": "utf-8",
      "campaign_id": "1802217",
      "unit": "awlist3854",
      "orig_send_date": "2025-01-01 00:00:00-05",
      "friendly_list_name": "Fluff Cafe",
      "id": "1802217",
      "list_id": "3854",
      "account_id": "778",
      "campaign_type_id": "b",
      "uses_block_editor": "t"
    }
  ]
Sample response

Completed Broadcasts

Name
completed-broadcasts
Parameters
  • List (default: all lists)
Report API controller endpoints
  • broadcasts_completed
  • broadcasts_completed_all
  [
    {
      "a_id": "778",
      "broadcast_id": "1258855",
      "for_sent_date": "08/14/20 03:41 PM",
      "track_click_rate": "1",
      "unit_id": "3854",
      "assassin_pts": "0",
      "mesg_type": "HTML",
      "mesg_encoding": "utf-8",
      "sent_date": "2020-08-14 15:41:05.24086-04",
      "num_emailed": "1",
      "num_undeliv": 0,
      "num_opened": 0,
      "num_attachments": 0,
      "num_complaints": 0,
      "subject": "Testing a bad segment",
      "created_date": "2020-08-14 15:38:57.59072-04",
      "status": "Sent Composer",
      "lead_view_id": "42564",
      "unit": "awlist3854",
      "pct_opened": "0",
      "show_opens_warning": true,
      "pct_undeliv": "0",
      "total_clicks": 0,
      "clicks_analytics_type": "premium",
      "pct_click": "0",
      "pct_complaints": "0",
      "segment_id": "42564",
      "extra_lists": "",
      "excluded_lists": "",
      "extra_lists_count": 0,
      "excluded_lists_count": 0,
      "friendly_list_name": "Fluff Cafe"
    },
    {
      "a_id": "778",
      "broadcast_id": "1243650",
      "for_sent_date": "08/05/20 06:33 PM",
      "track_click_rate": "1",
      "unit_id": "3854",
      "assassin_pts": "0",
      "mesg_type": "Text/HTML",
      "mesg_encoding": "utf-8",
      "sent_date": "2020-08-05 18:33:48.638875-04",
      "num_emailed": "27",
      "num_undeliv": 0,
      "num_opened": 0,
      "num_attachments": 0,
      "num_complaints": 0,
      "subject": "Buggssss 🐛🐛🐛🐛🐛🐛🐛",
      "created_date": "2020-08-05 18:31:10.567418-04",
      "status": "Sent Composer",
      "lead_view_id": "8",
      "unit": "awlist3854",
      "pct_opened": "0",
      "show_opens_warning": true,
      "pct_undeliv": "0",
      "total_clicks": 0,
      "clicks_analytics_type": "premium",
      "pct_click": "0",
      "pct_complaints": "0",
      "segment_id": "8",
      "extra_lists": "",
      "excluded_lists": "",
      "extra_lists_count": 0,
      "excluded_lists_count": 0,
      "friendly_list_name": "Fluff Cafe"
    }
  ]
Sample response

Cities, States, and Countries

Name
subscribers-by-location
Parameters
  • List (default: all lists)
Report API controller endpoints
  • city_state_countries
  [
    {
      "country": null,
      "state": null,
      "city": null,
      "pending": 0,
      "unsubscribed": 457262,
      "subscribed": 276245,
      "total": 733507
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Newtown",
      "pending": 0,
      "unsubscribed": 279304,
      "subscribed": 3236,
      "total": 282540
    },
    {
      "country": "US",
      "state": null,
      "city": null,
      "pending": 0,
      "unsubscribed": 109074,
      "subscribed": 2456,
      "total": 111530
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Philadelphia",
      "pending": 0,
      "unsubscribed": 48485,
      "subscribed": 657,
      "total": 49142
    },
    {
      "country": "US",
      "state": "PA",
      "city": "Huntingdon Valley",
      "pending": 0,
      "unsubscribed": 4839,
      "subscribed": 90,
      "total": 4929
    },
    {
      "country": "VN",
      "state": "64",
      "city": "Hanoi",
      "pending": 0,
      "unsubscribed": 204,
      "subscribed": 2,
      "total": 206
    },
    {
      "country": "VN",
      "state": "65",
      "city": "Ho Chi Minh City",
      "pending": 0,
      "unsubscribed": 176,
      "subscribed": 1,
      "total": 177
    }
  ]
Sample response

Campaign Statistics

Add the campaign start endpoint into analytics-view including

Followups

Name
followup-totals
Parameters
  • List (default: all lists)
Report API controller endpoints
  • followups

Uses the freq_mesg and freq_mesg_stats tables in AppDB combined with data from the messages table in Analytics.

editor_path is hard-coded in the current report endpoint.

  [
    {
      "followup_message": "1",
      "num_emailed": 83024,
      "num_opened": 89950,
      "open_percentage": 23.659423781075,
      "clicks": "13198",
      "message_id": "28613652",
      "click_percentage": 0,
      "unique_opens": "19643",
      "unique_clicks": "1584",
      "clicks_percentage": 23.659423781075,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "2",
      "num_emailed": 73714,
      "num_opened": 49979,
      "open_percentage": 13.675828200884,
      "clicks": "6169",
      "message_id": "28613655",
      "click_percentage": 0,
      "unique_opens": "10081",
      "unique_clicks": "649",
      "clicks_percentage": 13.675828200884,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "3",
      "num_emailed": 70880,
      "num_opened": 38472,
      "open_percentage": 10.272291196388,
      "clicks": "4322",
      "message_id": "28613659",
      "click_percentage": 0,
      "unique_opens": "7281",
      "unique_clicks": "409",
      "clicks_percentage": 10.272291196388,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "4",
      "num_emailed": 69375,
      "num_opened": 37496,
      "open_percentage": 10.105945945946,
      "clicks": "4419",
      "message_id": "28613661",
      "click_percentage": 0,
      "unique_opens": "7011",
      "unique_clicks": "447",
      "clicks_percentage": 10.105945945946,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "5",
      "num_emailed": 67478,
      "num_opened": 30996,
      "open_percentage": 8.2752897240582,
      "clicks": "3236",
      "message_id": "28613662",
      "click_percentage": 0,
      "unique_opens": "5584",
      "unique_clicks": "257",
      "clicks_percentage": 8.2752897240582,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "6",
      "num_emailed": 65414,
      "num_opened": 32483,
      "open_percentage": 9.2334974164552,
      "clicks": "3127",
      "message_id": "28613664",
      "click_percentage": 0,
      "unique_opens": "6040",
      "unique_clicks": "323",
      "clicks_percentage": 9.2334974164552,
      "editor_path": "messages#/active"
    },
    {
      "followup_message": "7",
      "num_emailed": 63871,
      "num_opened": 28756,
      "open_percentage": 8.1883797028385,
      "clicks": "1964",
      "message_id": "28613665",
      "click_percentage": 0,
      "unique_opens": "5230",
      "unique_clicks": "166",
      "clicks_percentage": 8.1883797028385,
      "editor_path": "messages#/active"
    }
  ]
Sample Response (Production AID 91)

New Subscribers Daily

Name
daily-new-subscribers
Parameters
  • List (default: all lists)
Report API controller endpoints
  • new_subscribers_daily

Uses the public.leads_stats_day table in AppDB.

  {
    "2022-01-09T00:00:00Z": {
      "subscribed": 324,
      "unsubscribed": 83
    },
    "2022-01-10T00:00:00Z": {
      "subscribed": 417,
      "unsubscribed": 80
    },
    "2022-01-11T00:00:00Z": {
      "subscribed": 433,
      "unsubscribed": 92
    },
    ...
  }
Sample Response (Production AID 91)

New Subscribers Weekly

Name
weekly-new-subscribers
Parameters
  • List (default: all lists)
Report API controller endpoints
  • new_subscribers_weekly

Uses the public.leads_stats_day table in AppDB.

  {
    "2021-03-26T00:00:00Z": {
      "subscribed": 3182,
      "unsubscribed": 1249
    },
    "2021-04-02T00:00:00Z": {
      "subscribed": 3423,
      "unsubscribed": 1497
    },
    "2021-04-09T00:00:00Z": {
      "subscribed": 3052,
      "unsubscribed": 1217
    },
    ...
  }
Sample Response (Production AID 91)

New Subscribers Monthly

Name
monthly-new-subscribers
Parameters
  • List (default: all lists)
Report API controller endpoints
  • new_subscribers_monthly

Uses the public.leads_stats_day table in AppDB.

  {
    "2021-03-01T00:00:00Z": {
      "subscribed": 14972,
      "unsubscribed": 4770
    },
    "2021-04-01T00:00:00Z": {
      "subscribed": 14973,
      "unsubscribed": 7181
    },
    "2021-05-01T00:00:00Z": {
      "subscribed": 12652,
      "unsubscribed": 5243
    },
    ...
  }
Sample Response (Production AID 91)

Subscriber Totals Daily

Name
subscribers-by-location
Parameters
  • List (default: all lists)
Report API controller endpoints
  • subscriber_totals_daily

Uses the public.leads_stats_day table in AppDB.

  {
    "2022-01-09T00:00:00Z": {
      "subscribed": 289192,
      "unsubscribed": 920812
    },
    "2022-01-10T00:00:00Z": {
      "subscribed": 289609,
      "unsubscribed": 920892
    },
    "2022-01-11T00:00:00Z": {
      "subscribed": 290042,
      "unsubscribed": 920984
    },
    ...
  }
Sample Response (Production AID 91)

Subscriber Totals Weekly

Name
subscribers-by-location
Parameters
  • List (default: all lists)
Report API controller endpoints
  • subscriber_totals_weekly

Uses the public.leads_stats_day table in AppDB.

  {
    "2021-02-08T00:00:00Z": {
      "subscribed": 191745,
      "unsubscribed": 887713
    },
    "2021-02-15T00:00:00Z": {
      "subscribed": 193972,
      "unsubscribed": 888733
    },
    "2021-02-22T00:00:00Z": {
      "subscribed": 196200,
      "unsubscribed": 889717
    },
    ...
  }
Sample Response (Production AID 91)

Subscriber Totals Monthly

Name
subscribers-by-location
Parameters
  • List (default: all lists)
Report API controller endpoints
  • subscriber_totals_monthly

Uses the public.leads_stats_day table in AppDB.

  {
    "2021-02-01T00:00:00Z": {
      "subscribed": 196200,
      "unsubscribed": 889717
    },
    "2021-03-01T00:00:00Z": {
      "subscribed": 206569,
      "unsubscribed": 894298
    },
    "2021-04-01T00:00:00Z": {
      "subscribed": 215813,
      "unsubscribed": 900020
    },
    ...
  }
Sample Response (Production AID 91)

Migrate reports to the Analytics View Service

READY Use the new analytics view endpoints for the opens over time reports

Update the opens over time report for all lists and the current list to use the new daily opens endpoint in the Analytics View service.

READY Use the new analytics view endpoints for the clicks over time reports

Update the clicks over time report for all lists and the current list to use the new daily clicks endpoint in the Analytics View service.

WAITING Use the new analytics view endpoints for the sales over time report

Update the sales over time report to use the new daily sales endpoint for graph summary data and the sale events endpoint for the sales table.

The sale events endpoint present, but is not yet documented!

DONE New subscribers

Update the new subscribers report to use the new daily, weekly, and monthly new subscriber endpoints in the Analytics View service.

DONE Subscriber totals

Update the subscriber totals report to use the new daily, weekly, and monthly total subscriber endpoints in the Analytics View service.

WAITING Use the new analytics view endpoints for the broadcast totals report

Update the broadcast totals report to use the new completed broadcasts endpoint in the Analytics View service.

The completed broadcasts endpoint is not yet complete.

Use the new analytics view endpoints for the follow-up totals report

Update the followup totals report to use the new followup totals endpoint in the Analytics View service.

Use the new analytics view endpoints for the location totals report

Update the location totals report to use the new subscribers by location endpoint in the Analytics View service.