correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
roam/aweber/20210914142000-subscriber_search_service.org
Correl c6ec0dd4b0 updates
2022-01-31 20:55:32 -05:00

6.9 KiB
Raw Blame History

Subscriber Search Service

A replacement for the current Sites Subscriber Search and Broadcast Segment Search implementations.

Architecture Notes

Added: [2019-08-06 Tue 10:34]

  • Broadcast sending manipulated queries before performing them to suit its needs

    • Handle list exclusion in the search service? (CCPANEL-9557)
    • Blocked emails
    • De-duplication
  • Lead view ids -> segment ids -> search service segments? (CCPANEL-9556)

  • Side-by-side comparison

    • Use broadcast-segment to compare its results to the search api results
  • Recipient-style representation or leads?

  • Define API (CCPANEL-9554)

    • Include saved searches (segments)

  • Iterate release (CCPANEL-9555)

    • Current UI

    • New UI with same capabilities of old UI

      • Still writing old-style segments

  • Research getting off the leads table (aurora?)

  • Deal with subscriber_tags table bloat in AppDB

Component Diagram 

  database Analytics {
    database Ana as Ana01
    database Ana as Ana02
    database Ana as Ana03
  }
  database App
  database "Results Cache" as ResultsCache

  component "Search Service" as Service {

    component Search
    component Results

    Search -- Analytics
    Search -- App
    Search --> ResultsCache

    Results <-- ResultsCache

  }

/correlr/roam/media/commit/8490be9a68a44beca97c07c4a6820a8ba1b5c08f/aweber/search-components.svg

Concerns

Performance

Fitness to Purpose

This service will need to fulfill the needs of both end-user subscriber searches and segment emailing.

Plan

ITEM Effort
Plan 18d 0:00
Search Centralization 18d 0:00
Expose search inputs backed with the existing database 2d
Enable dblink on the search master database 2d
Create new unlogged search results table 2d
Define the search result format 1d
Perform search using new search DSL 5d
Perform search using legacy segment ID 3d
Manage segments using the existing database 3d
Migrate to an updated schema
Migrate to new search service
Create new subscriber management React application
Update broadcast-segment to use new search service
Milestone 3: Add new search features

A Dedicated Service for Subscriber Search

Create a dedicated, publicly exposed service for performing searches on subscribers using subscriber and analytics criteria. The goal of this project is to replace the current implementation from sites and in the broadcast segment service.

Expose search inputs backed with the existing database

  • Include IDs required to build existing POST format

Enable dblink on the search master database

CCPANEL-7147

NO Create new unlogged search results table

https://jira.aweber.io/browse/CCPANEL-7077

Define the search result format

https://xd.adobe.com/view/ae8fb2b2-c039-4e88-8ade-ff2562a8c8cf-fbdc/screen/c03f09c7-187e-4a6f-8b8c-571d131daee1/ (ignore engagement column)

  • name
  • email
  • source
  • status
  • date added
  • last updated

Perform search using new search DSL

DONE Perform search using text comparisons
  • Is / Is Not
  • Contains / Does not contain
  • Starts with / Does not start with
  • Ends with / Does not end with
DONE Perform search using numeric comparisons
DONE Perform search using tag sets
DONE Perform search using enumerated values
Add support for remaining static AppDB filters
  • Date range fields
  • Remaining string / numeric fields
Add support for custom field filters
  • Add all custom field columns as supported filters
  • Fetch the custom fields for the list and use them when building the list of available filters.
Add support for Analytics filters
  • Clicks
  • Opens
  • Messages
  • Web pages (links)

Perform search using legacy segment ID

Provided with a legacy segment ID, execute a search using its stored parameters.

Manage segments using the existing database

Create, retrieve, update, and delete legacy segments using the new search DSL.

Manage segments stored using the new DSL

Migrate legacy segments to the new DSL

Migrate to an updated schema

Support filtering options used by broadcast-segment

Centralizing Subscriber Search

Applications making use of subscriber search will be updated to use the new dedicated service, eliminating multiple search implementations.

Create new subscriber management React application

Update broadcast-segment to use new search service

Milestone 3: Add new search features

Implementation

Modeling the new search DSL

Constructing SQL queries programmatically

Translating legacy segments

Gathering results

Reaching into Analytics

Resources