correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updates

Browse source
This commit is contained in:
Correl Roush 2021-12-13 17:29:02 -05:00
parent e0dc8620f2
commit 1f8fc7cde3
11 changed files with 762 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
aweber/20210914142000-subscriber_search_service.org
View file

@ -110,7 +110,7 @@ service.
:END:
[[jira:CCPANEL-7147][CCPANEL-7147]]
*** Create new unlogged search results table
*** NO Create new unlogged search results table
:PROPERTIES:
:Effort: 2d
:END:
@ -133,26 +133,41 @@ https://xd.adobe.com/view/ae8fb2b2-c039-4e88-8ade-ff2562a8c8cf-fbdc/screen/c03f0
:PROPERTIES:
:Effort: 5d
:END:
**** Perform search using text comparisons
**** DONE Perform search using text comparisons
- Is / Is Not
- Contains / Does not contain
- Starts with / Does not start with
- Ends with / Does not end with
**** Perform search using numeric comparisons
**** Perform search using date ranges
**** Perform search using message and link IDs
**** Perform search using tag sets
**** DONE Perform search using numeric comparisons
**** DONE Perform search using tag sets
**** DONE Perform search using enumerated values
**** Add support for remaining static AppDB filters
- Date range fields
- Remaining string / numeric fields
**** Add support for custom field filters
- Add all custom field columns as supported filters
- Fetch the custom fields for the list and use them when building the list of
available filters.
**** Add support for Analytics filters
- Clicks
- Opens
- Messages
- Web pages (links)
*** Perform search using legacy segment ID
:PROPERTIES:
:Effort: 3d
:END:
- Include flags for broadcast segment-required filtering?
Provided with a legacy segment ID, execute a search using its stored parameters.
*** Manage segments using the existing database
:PROPERTIES:
:Effort: 3d
:END:
- Translate a DSL search into a legacy stored segment
Create, retrieve, update, and delete legacy segments using the new search DSL.
*** Manage segments stored using the new DSL
*** Migrate legacy segments to the new DSL
*** Migrate to an updated schema
*** Support filtering options used by broadcast-segment
** Centralizing Subscriber Search
Applications making use of subscriber search will be updated to use the new
dedicated service, eliminating multiple search implementations.

517
aweber/20211208140356-legacy_search_filters.org Normal file
View file

@ -0,0 +1,517 @@
:PROPERTIES:
:ID: 2f42c362-ae96-41d5-988b-329f8c162e45
:END:
#+title: Legacy Search Filters
Search filters provided by [[id:d9cb2b55-3b0e-4ab3-8369-f71ebc3cd882][Sites Subscriber Search]] and supported in the new
[[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]].
#+name: search-boxes
#+header: :engine postgresql
#+header: :dbhost 127.0.0.1 :dbport 56893
#+header: :dbuser postgres
#+begin_src sql :cache yes :eval no-export
select box.id, input.description, input.column, term.description
from search_boxes as box
join search_inputs as input on (box.search_input_id = input.id)
join search_terms as term on (box.search_term_id = term.id)
#+end_src
#+RESULTS[50d2c2ce018c9fd37f4093af3646725bcd98131f]: search-boxes
| id | description | column | description |
|-----+-----------------------+----------------------+-----------------------------|
| 1 | Email | email | is |
| 2 | Email | email | is not |
| 3 | Email | email | contains |
| 4 | Email | email | does not contain |
| 5 | Email | email | starts with |
| 6 | Email | email | ends with |
| 7 | Email | email | does not start with |
| 8 | Email | email | does not end with |
| 9 | Name | name | is |
| 10 | Name | name | is not |
| 11 | Name | name | contains |
| 12 | Name | name | does not contain |
| 13 | Name | name | starts with |
| 14 | Name | name | ends with |
| 15 | Name | name | does not start with |
| 16 | Name | name | does not end with |
| 17 | Ad Tracking | note | is |
| 18 | Ad Tracking | note | is not |
| 19 | Ad Tracking | note | contains |
| 20 | Ad Tracking | note | does not contain |
| 21 | Ad Tracking | note | starts with |
| 22 | Ad Tracking | note | ends with |
| 23 | Ad Tracking | note | does not start with |
| 24 | Ad Tracking | note | does not end with |
| 25 | Additional Notes | name2 | is |
| 26 | Additional Notes | name2 | is not |
| 27 | Additional Notes | name2 | contains |
| 28 | Additional Notes | name2 | does not contain |
| 29 | Additional Notes | name2 | starts with |
| 30 | Additional Notes | name2 | ends with |
| 31 | Additional Notes | name2 | does not start with |
| 32 | Additional Notes | name2 | does not end with |
| 33 | Last Message # | message | is |
| 34 | Last Message # | message | is not |
| 35 | Last Message # | message | contains |
| 36 | Last Message # | message | does not contain |
| 37 | Last Message # | message | starts with |
| 38 | Last Message # | message | ends with |
| 39 | Last Message # | message | does not start with |
| 40 | Last Message # | message | does not end with |
| 41 | Last Message # | message | is less than |
| 42 | Last Message # | message | is less than or equal to |
| 43 | Last Message # | message | is greater than |
| 44 | Last Message # | message | is greater than or equal to |
| 45 | Stop Status | stop_status | is |
| 46 | Stop Status | stop_status | is not |
| 47 | Stop Method | stop_method | is |
| 48 | Stop Method | stop_method | is not |
| 49 | Confirmed? | verified | is |
| 50 | Confirmed? | verified | is not |
| 51 | Add Method | add_method | is |
| 52 | Add Method | add_method | is not |
| 53 | Add URL | add_url | is |
| 54 | Add URL | add_url | is not |
| 55 | Add URL | add_url | contains |
| 56 | Add URL | add_url | does not contain |
| 57 | Add URL | add_url | starts with |
| 58 | Add URL | add_url | ends with |
| 59 | Add URL | add_url | does not start with |
| 60 | Add URL | add_url | does not end with |
| 86 | Date Stopped | stop_time | date is on or before |
| 61 | Add IP | add_notes | is |
| 62 | Add IP | add_notes | is not |
| 63 | Add IP | add_notes | contains |
| 64 | Add IP | add_notes | does not contain |
| 65 | Add IP | add_notes | starts with |
| 66 | Add IP | add_notes | ends with |
| 67 | Add IP | add_notes | does not start with |
| 68 | Add IP | add_notes | does not end with |
| 69 | Confirmation IP | verification_notes | is |
| 70 | Confirmation IP | verification_notes | is not |
| 71 | Confirmation IP | verification_notes | contains |
| 72 | Confirmation IP | verification_notes | does not contain |
| 73 | Confirmation IP | verification_notes | starts with |
| 74 | Confirmation IP | verification_notes | ends with |
| 75 | Confirmation IP | verification_notes | does not start with |
| 76 | Confirmation IP | verification_notes | does not end with |
| 87 | Date Stopped | stop_time | date is after |
| 88 | Date Stopped | stop_time | date is on or after |
| 89 | Date Confirmed | verification_time | date is before |
| 90 | Date Confirmed | verification_time | date is on or before |
| 91 | Date Confirmed | verification_time | date is after |
| 92 | Date Confirmed | verification_time | date is on or after |
| 127 | Latitude (from IP) | geog_lat | is |
| 128 | Latitude (from IP) | geog_lat | is not |
| 129 | Latitude (from IP) | geog_lat | contains |
| 130 | Latitude (from IP) | geog_lat | does not contain |
| 131 | Latitude (from IP) | geog_lat | starts with |
| 132 | Latitude (from IP) | geog_lat | ends with |
| 133 | Latitude (from IP) | geog_lat | does not start with |
| 134 | Latitude (from IP) | geog_lat | does not end with |
| 135 | Latitude (from IP) | geog_lat | is less than |
| 136 | Latitude (from IP) | geog_lat | is less than or equal to |
| 93 | Country (from IP) | geog_country | is |
| 94 | Country (from IP) | geog_country | is not |
| 95 | Country (from IP) | geog_country | contains |
| 96 | Country (from IP) | geog_country | does not contain |
| 97 | Country (from IP) | geog_country | starts with |
| 98 | Country (from IP) | geog_country | ends with |
| 99 | Country (from IP) | geog_country | does not start with |
| 100 | Country (from IP) | geog_country | does not end with |
| 101 | Region (from IP) | geog_region | is |
| 102 | Region (from IP) | geog_region | is not |
| 103 | Region (from IP) | geog_region | contains |
| 104 | Region (from IP) | geog_region | does not contain |
| 105 | Region (from IP) | geog_region | starts with |
| 106 | Region (from IP) | geog_region | ends with |
| 107 | Region (from IP) | geog_region | does not start with |
| 108 | Region (from IP) | geog_region | does not end with |
| 137 | Latitude (from IP) | geog_lat | is greater than |
| 138 | Latitude (from IP) | geog_lat | is greater than or equal to |
| 111 | City (from IP) | geog_city | is |
| 112 | City (from IP) | geog_city | is not |
| 113 | City (from IP) | geog_city | contains |
| 114 | City (from IP) | geog_city | does not contain |
| 115 | City (from IP) | geog_city | starts with |
| 116 | City (from IP) | geog_city | ends with |
| 117 | City (from IP) | geog_city | does not start with |
| 118 | City (from IP) | geog_city | does not end with |
| 119 | Postal Code (from IP) | geog_postal | is |
| 120 | Postal Code (from IP) | geog_postal | is not |
| 121 | Postal Code (from IP) | geog_postal | contains |
| 122 | Postal Code (from IP) | geog_postal | does not contain |
| 123 | Postal Code (from IP) | geog_postal | starts with |
| 124 | Postal Code (from IP) | geog_postal | ends with |
| 125 | Postal Code (from IP) | geog_postal | does not start with |
| 126 | Postal Code (from IP) | geog_postal | does not end with |
| 77 | Date Added | timehit | date is before |
| 78 | Date Added | timehit | date is on or before |
| 79 | Date Added | timehit | date is after |
| 80 | Date Added | timehit | date is on or after |
| 81 | Date Last Follow Up | followuptime | date is before |
| 82 | Date Last Follow Up | followuptime | date is on or before |
| 83 | Date Last Follow Up | followuptime | date is after |
| 84 | Date Last Follow Up | followuptime | date is on or after |
| 85 | Date Stopped | stop_time | date is before |
| 139 | Longitude (from IP) | geog_lon | is |
| 144 | Longitude (from IP) | geog_lon | starts with |
| 140 | Longitude (from IP) | geog_lon | is not |
| 141 | Longitude (from IP) | geog_lon | contains |
| 142 | Longitude (from IP) | geog_lon | does not contain |
| 145 | Longitude (from IP) | geog_lon | ends with |
| 146 | Longitude (from IP) | geog_lon | does not start with |
| 147 | Longitude (from IP) | geog_lon | does not end with |
| 148 | Longitude (from IP) | geog_lon | is less than |
| 149 | Longitude (from IP) | geog_lon | is less than or equal to |
| 150 | Longitude (from IP) | geog_lon | is greater than |
| 151 | Longitude (from IP) | geog_lon | is greater than or equal to |
| 152 | Area Code (from IP) | geog_area_code | is |
| 153 | Area Code (from IP) | geog_area_code | is not |
| 154 | Area Code (from IP) | geog_area_code | contains |
| 155 | Area Code (from IP) | geog_area_code | does not contain |
| 156 | Area Code (from IP) | geog_area_code | starts with |
| 157 | Area Code (from IP) | geog_area_code | ends with |
| 158 | Area Code (from IP) | geog_area_code | does not start with |
| 159 | Area Code (from IP) | geog_area_code | does not end with |
| 160 | Area Code (from IP) | geog_area_code | is less than |
| 161 | Area Code (from IP) | geog_area_code | is less than or equal to |
| 162 | Area Code (from IP) | geog_area_code | is greater than |
| 163 | Area Code (from IP) | geog_area_code | is greater than or equal to |
| 164 | DMA Code (from IP) | geog_dma_code | is |
| 165 | DMA Code (from IP) | geog_dma_code | is not |
| 166 | DMA Code (from IP) | geog_dma_code | contains |
| 167 | DMA Code (from IP) | geog_dma_code | does not contain |
| 168 | DMA Code (from IP) | geog_dma_code | starts with |
| 169 | DMA Code (from IP) | geog_dma_code | ends with |
| 170 | DMA Code (from IP) | geog_dma_code | does not start with |
| 171 | DMA Code (from IP) | geog_dma_code | does not end with |
| 172 | DMA Code (from IP) | geog_dma_code | is less than |
| 173 | DMA Code (from IP) | geog_dma_code | is less than or equal to |
| 174 | DMA Code (from IP) | geog_dma_code | is greater than |
| 175 | DMA Code (from IP) | geog_dma_code | is greater than or equal to |
| 176 | Message not opened | app_message_id | is |
| 177 | Message opened | app_message_id | is |
| 226 | datum1 | datum1 | is |
| 227 | datum1 | datum1 | is not |
| 228 | datum1 | datum1 | contains |
| 229 | datum1 | datum1 | does not contain |
| 230 | datum1 | datum1 | starts with |
| 231 | datum1 | datum1 | ends with |
| 232 | datum1 | datum1 | does not start with |
| 233 | datum1 | datum1 | does not end with |
| 234 | datum2 | datum2 | is |
| 235 | datum2 | datum2 | is not |
| 236 | datum2 | datum2 | contains |
| 237 | datum2 | datum2 | does not contain |
| 238 | datum2 | datum2 | starts with |
| 239 | datum2 | datum2 | ends with |
| 240 | datum2 | datum2 | does not start with |
| 241 | datum2 | datum2 | does not end with |
| 242 | datum3 | datum3 | is |
| 243 | datum3 | datum3 | is not |
| 244 | datum3 | datum3 | contains |
| 245 | datum3 | datum3 | does not contain |
| 246 | datum3 | datum3 | starts with |
| 247 | datum3 | datum3 | ends with |
| 248 | datum3 | datum3 | does not start with |
| 249 | datum3 | datum3 | does not end with |
| 250 | datum4 | datum4 | is |
| 251 | datum4 | datum4 | is not |
| 252 | datum4 | datum4 | contains |
| 253 | datum4 | datum4 | does not contain |
| 254 | datum4 | datum4 | starts with |
| 255 | datum4 | datum4 | ends with |
| 256 | datum4 | datum4 | does not start with |
| 257 | datum4 | datum4 | does not end with |
| 258 | datum5 | datum5 | is |
| 259 | datum5 | datum5 | is not |
| 260 | datum5 | datum5 | contains |
| 261 | datum5 | datum5 | does not contain |
| 262 | datum5 | datum5 | starts with |
| 263 | datum5 | datum5 | ends with |
| 264 | datum5 | datum5 | does not start with |
| 265 | datum5 | datum5 | does not end with |
| 266 | datum6 | datum6 | is |
| 267 | datum6 | datum6 | is not |
| 268 | datum6 | datum6 | contains |
| 269 | datum6 | datum6 | does not contain |
| 270 | datum6 | datum6 | starts with |
| 271 | datum6 | datum6 | ends with |
| 272 | datum6 | datum6 | does not start with |
| 273 | datum6 | datum6 | does not end with |
| 274 | datum7 | datum7 | is |
| 275 | datum7 | datum7 | is not |
| 276 | datum7 | datum7 | contains |
| 277 | datum7 | datum7 | does not contain |
| 278 | datum7 | datum7 | starts with |
| 279 | datum7 | datum7 | ends with |
| 280 | datum7 | datum7 | does not start with |
| 281 | datum7 | datum7 | does not end with |
| 282 | datum8 | datum8 | is |
| 283 | datum8 | datum8 | is not |
| 284 | datum8 | datum8 | contains |
| 285 | datum8 | datum8 | does not contain |
| 286 | datum8 | datum8 | starts with |
| 287 | datum8 | datum8 | ends with |
| 288 | datum8 | datum8 | does not start with |
| 289 | datum8 | datum8 | does not end with |
| 290 | datum9 | datum9 | is |
| 291 | datum9 | datum9 | is not |
| 292 | datum9 | datum9 | contains |
| 293 | datum9 | datum9 | does not contain |
| 294 | datum9 | datum9 | starts with |
| 295 | datum9 | datum9 | ends with |
| 296 | datum9 | datum9 | does not start with |
| 297 | datum9 | datum9 | does not end with |
| 298 | datum10 | datum10 | is |
| 299 | datum10 | datum10 | is not |
| 300 | datum10 | datum10 | contains |
| 301 | datum10 | datum10 | does not contain |
| 302 | datum10 | datum10 | starts with |
| 303 | datum10 | datum10 | ends with |
| 304 | datum10 | datum10 | does not start with |
| 305 | datum10 | datum10 | does not end with |
| 306 | datum11 | datum11 | is |
| 307 | datum11 | datum11 | is not |
| 308 | datum11 | datum11 | contains |
| 309 | datum11 | datum11 | does not contain |
| 310 | datum11 | datum11 | starts with |
| 311 | datum11 | datum11 | ends with |
| 312 | datum11 | datum11 | does not start with |
| 313 | datum11 | datum11 | does not end with |
| 314 | datum12 | datum12 | is |
| 315 | datum12 | datum12 | is not |
| 316 | datum12 | datum12 | contains |
| 317 | datum12 | datum12 | does not contain |
| 318 | datum12 | datum12 | starts with |
| 319 | datum12 | datum12 | ends with |
| 320 | datum12 | datum12 | does not start with |
| 321 | datum12 | datum12 | does not end with |
| 322 | datum13 | datum13 | is |
| 323 | datum13 | datum13 | is not |
| 324 | datum13 | datum13 | contains |
| 325 | datum13 | datum13 | does not contain |
| 326 | datum13 | datum13 | starts with |
| 327 | datum13 | datum13 | ends with |
| 328 | datum13 | datum13 | does not start with |
| 329 | datum13 | datum13 | does not end with |
| 330 | datum14 | datum14 | is |
| 331 | datum14 | datum14 | is not |
| 332 | datum14 | datum14 | contains |
| 333 | datum14 | datum14 | does not contain |
| 334 | datum14 | datum14 | starts with |
| 335 | datum14 | datum14 | ends with |
| 336 | datum14 | datum14 | does not start with |
| 337 | datum14 | datum14 | does not end with |
| 338 | datum15 | datum15 | is |
| 339 | datum15 | datum15 | is not |
| 340 | datum15 | datum15 | contains |
| 341 | datum15 | datum15 | does not contain |
| 342 | datum15 | datum15 | starts with |
| 343 | datum15 | datum15 | ends with |
| 344 | datum15 | datum15 | does not start with |
| 345 | datum15 | datum15 | does not end with |
| 346 | datum16 | datum16 | is |
| 347 | datum16 | datum16 | is not |
| 348 | datum16 | datum16 | contains |
| 349 | datum16 | datum16 | does not contain |
| 350 | datum16 | datum16 | starts with |
| 351 | datum16 | datum16 | ends with |
| 352 | datum16 | datum16 | does not start with |
| 353 | datum16 | datum16 | does not end with |
| 354 | datum17 | datum17 | is |
| 355 | datum17 | datum17 | is not |
| 356 | datum17 | datum17 | contains |
| 357 | datum17 | datum17 | does not contain |
| 358 | datum17 | datum17 | starts with |
| 359 | datum17 | datum17 | ends with |
| 360 | datum17 | datum17 | does not start with |
| 361 | datum17 | datum17 | does not end with |
| 362 | datum18 | datum18 | is |
| 363 | datum18 | datum18 | is not |
| 364 | datum18 | datum18 | contains |
| 365 | datum18 | datum18 | does not contain |
| 366 | datum18 | datum18 | starts with |
| 367 | datum18 | datum18 | ends with |
| 368 | datum18 | datum18 | does not start with |
| 369 | datum18 | datum18 | does not end with |
| 370 | datum19 | datum19 | is |
| 371 | datum19 | datum19 | is not |
| 372 | datum19 | datum19 | contains |
| 373 | datum19 | datum19 | does not contain |
| 374 | datum19 | datum19 | starts with |
| 375 | datum19 | datum19 | ends with |
| 376 | datum19 | datum19 | does not start with |
| 377 | datum19 | datum19 | does not end with |
| 534 | Link clicked | link_id | is |
| 535 | Link not clicked | link_id | is |
| 429 | datum1 | datum1 | is greater than |
| 433 | datum2 | datum2 | is greater than |
| 437 | datum3 | datum3 | is greater than |
| 441 | datum4 | datum4 | is greater than |
| 445 | datum5 | datum5 | is greater than |
| 449 | datum6 | datum6 | is greater than |
| 453 | datum7 | datum7 | is greater than |
| 457 | datum8 | datum8 | is greater than |
| 461 | datum9 | datum9 | is greater than |
| 465 | datum10 | datum10 | is greater than |
| 469 | datum11 | datum11 | is greater than |
| 473 | datum12 | datum12 | is greater than |
| 477 | datum13 | datum13 | is greater than |
| 430 | datum1 | datum1 | is greater than or equal to |
| 434 | datum2 | datum2 | is greater than or equal to |
| 438 | datum3 | datum3 | is greater than or equal to |
| 442 | datum4 | datum4 | is greater than or equal to |
| 446 | datum5 | datum5 | is greater than or equal to |
| 450 | datum6 | datum6 | is greater than or equal to |
| 454 | datum7 | datum7 | is greater than or equal to |
| 458 | datum8 | datum8 | is greater than or equal to |
| 462 | datum9 | datum9 | is greater than or equal to |
| 466 | datum10 | datum10 | is greater than or equal to |
| 470 | datum11 | datum11 | is greater than or equal to |
| 474 | datum12 | datum12 | is greater than or equal to |
| 478 | datum13 | datum13 | is greater than or equal to |
| 482 | datum14 | datum14 | is greater than or equal to |
| 486 | datum15 | datum15 | is greater than or equal to |
| 490 | datum16 | datum16 | is greater than or equal to |
| 494 | datum17 | datum17 | is greater than or equal to |
| 498 | datum18 | datum18 | is greater than or equal to |
| 502 | datum19 | datum19 | is greater than or equal to |
| 432 | datum1 | datum1 | is less than or equal to |
| 436 | datum2 | datum2 | is less than or equal to |
| 440 | datum3 | datum3 | is less than or equal to |
| 444 | datum4 | datum4 | is less than or equal to |
| 448 | datum5 | datum5 | is less than or equal to |
| 452 | datum6 | datum6 | is less than or equal to |
| 456 | datum7 | datum7 | is less than or equal to |
| 460 | datum8 | datum8 | is less than or equal to |
| 464 | datum9 | datum9 | is less than or equal to |
| 468 | datum10 | datum10 | is less than or equal to |
| 472 | datum11 | datum11 | is less than or equal to |
| 476 | datum12 | datum12 | is less than or equal to |
| 480 | datum13 | datum13 | is less than or equal to |
| 484 | datum14 | datum14 | is less than or equal to |
| 488 | datum15 | datum15 | is less than or equal to |
| 492 | datum16 | datum16 | is less than or equal to |
| 496 | datum17 | datum17 | is less than or equal to |
| 500 | datum18 | datum18 | is less than or equal to |
| 504 | datum19 | datum19 | is less than or equal to |
| 431 | datum1 | datum1 | is less than |
| 435 | datum2 | datum2 | is less than |
| 439 | datum3 | datum3 | is less than |
| 443 | datum4 | datum4 | is less than |
| 447 | datum5 | datum5 | is less than |
| 451 | datum6 | datum6 | is less than |
| 455 | datum7 | datum7 | is less than |
| 459 | datum8 | datum8 | is less than |
| 463 | datum9 | datum9 | is less than |
| 467 | datum10 | datum10 | is less than |
| 471 | datum11 | datum11 | is less than |
| 475 | datum12 | datum12 | is less than |
| 479 | datum13 | datum13 | is less than |
| 483 | datum14 | datum14 | is less than |
| 487 | datum15 | datum15 | is less than |
| 491 | datum16 | datum16 | is less than |
| 495 | datum17 | datum17 | is less than |
| 499 | datum18 | datum18 | is less than |
| 503 | datum19 | datum19 | is less than |
| 529 | Sale Amount | monetary_value | is |
| 530 | Sale Amount | monetary_value | is less than |
| 531 | Sale Amount | monetary_value | is less than or equal to |
| 532 | Sale Amount | monetary_value | is greater than |
| 533 | Sale Amount | monetary_value | is greater than or equal to |
| 481 | datum14 | datum14 | is greater than |
| 485 | datum15 | datum15 | is greater than |
| 489 | datum16 | datum16 | is greater than |
| 493 | datum17 | datum17 | is greater than |
| 497 | datum18 | datum18 | is greater than |
| 501 | datum19 | datum19 | is greater than |
| 536 | Undeliverable | unit_id | is |
| 537 | Web Page Visited | web_page_id | is |
| 538 | No Opens | created | since |
| 587 | datum20 | datum20 | is |
| 588 | datum20 | datum20 | is not |
| 589 | datum20 | datum20 | contains |
| 590 | datum20 | datum20 | does not contain |
| 591 | datum20 | datum20 | starts with |
| 592 | datum20 | datum20 | ends with |
| 593 | datum20 | datum20 | does not start with |
| 594 | datum20 | datum20 | does not end with |
| 595 | datum20 | datum20 | is less than |
| 596 | datum20 | datum20 | is less than or equal to |
| 597 | datum20 | datum20 | is greater than |
| 598 | datum20 | datum20 | is greater than or equal to |
| 599 | datum21 | datum21 | is |
| 600 | datum21 | datum21 | is not |
| 601 | datum21 | datum21 | contains |
| 602 | datum21 | datum21 | does not contain |
| 603 | datum21 | datum21 | starts with |
| 604 | datum21 | datum21 | ends with |
| 605 | datum21 | datum21 | does not start with |
| 606 | datum21 | datum21 | does not end with |
| 607 | datum21 | datum21 | is less than |
| 608 | datum21 | datum21 | is less than or equal to |
| 609 | datum21 | datum21 | is greater than |
| 610 | datum21 | datum21 | is greater than or equal to |
| 611 | datum22 | datum22 | is |
| 612 | datum22 | datum22 | is not |
| 613 | datum22 | datum22 | contains |
| 614 | datum22 | datum22 | does not contain |
| 615 | datum22 | datum22 | starts with |
| 616 | datum22 | datum22 | ends with |
| 617 | datum22 | datum22 | does not start with |
| 618 | datum22 | datum22 | does not end with |
| 619 | datum22 | datum22 | is less than |
| 620 | datum22 | datum22 | is less than or equal to |
| 621 | datum22 | datum22 | is greater than |
| 622 | datum22 | datum22 | is greater than or equal to |
| 623 | datum23 | datum23 | is |
| 624 | datum23 | datum23 | is not |
| 625 | datum23 | datum23 | contains |
| 626 | datum23 | datum23 | does not contain |
| 627 | datum23 | datum23 | starts with |
| 628 | datum23 | datum23 | ends with |
| 629 | datum23 | datum23 | does not start with |
| 630 | datum23 | datum23 | does not end with |
| 631 | datum23 | datum23 | is less than |
| 632 | datum23 | datum23 | is less than or equal to |
| 633 | datum23 | datum23 | is greater than |
| 634 | datum23 | datum23 | is greater than or equal to |
| 635 | datum24 | datum24 | is |
| 636 | datum24 | datum24 | is not |
| 637 | datum24 | datum24 | contains |
| 638 | datum24 | datum24 | does not contain |
| 639 | datum24 | datum24 | starts with |
| 640 | datum24 | datum24 | ends with |
| 641 | datum24 | datum24 | does not start with |
| 642 | datum24 | datum24 | does not end with |
| 643 | datum24 | datum24 | is less than |
| 644 | datum24 | datum24 | is less than or equal to |
| 645 | datum24 | datum24 | is greater than |
| 646 | datum24 | datum24 | is greater than or equal to |
| 647 | datum25 | datum25 | is |
| 648 | datum25 | datum25 | is not |
| 649 | datum25 | datum25 | contains |
| 650 | datum25 | datum25 | does not contain |
| 651 | datum25 | datum25 | starts with |
| 652 | datum25 | datum25 | ends with |
| 653 | datum25 | datum25 | does not start with |
| 654 | datum25 | datum25 | does not end with |
| 655 | datum25 | datum25 | is less than |
| 656 | datum25 | datum25 | is less than or equal to |
| 657 | datum25 | datum25 | is greater than |
| 658 | datum25 | datum25 | is greater than or equal to |
| 659 | Tag | subscriber_tags.tags | is |
| 660 | Tag | subscriber_tags.tags | is not |
| 661 | Any Opens | opens | since |
| 662 | Any Clicks | clicks | since |
| 663 | Any Clicks | clicks | before |
| 664 | Any Opens | opens | before |
| 665 | No Opens | created | before |
| 666 | Tag | subscriber_tags.tags | is any of these |
| 667 | Tag | subscriber_tags.tags | includes all of these |

8
aweber/20211209115433-admin_restore_deleted_controller.org Normal file
View file

@ -0,0 +1,8 @@
:PROPERTIES:
:ID: 4bf81e33-8020-40f2-b6ed-ce4e1eae2234
:ROAM_ALIASES: "Restoring deleted subscribers"
:END:
#+title: Admin restore deleted controller
Lists have a deleted subscriber history view, from which deleted subscribers can
be restored.

74
aweber/20211210114117-legacy_search_segment_construction.org Normal file
View file

@ -0,0 +1,74 @@
:PROPERTIES:
:ID: e6a2c650-ff59-4b72-b073-970731796888
:END:
#+title: Legacy search segment construction
Thoughts on constructing a segment representation from legacy data in the
[[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]], referencing a [[https://gitlab.aweber.io/CP/Services/subscriber-search/-/merge_requests/15/diffs?commit_id=4d203dfec800cf49825d32a7ea663c4b862693bb][WIP commit]] loading legacy segments from
the database.
- The legacy segment stores field and operator information encoded in a search
box ID, and a match value in a list view criteria field.
- Legacy segments only support a top-level AND grouping of criteria
Therefore, given a list of criteria on a segment, a Search can be formed as
follows:
#+begin_src python
import dataclasses
from subscribersearch import legacy, search
@dataclasses.dataclass
class Segment:
"""A saved search on a list representing a segment of subscribers.
Likely belongs in the search module.
"""
id: typing.Optional[int]
name: str
search: search.Search
@staticmethod
def from_dict(value: dict) -> Search:
return Segment(
id=value.get("id"),
name=value["name"],
search=search.from_dict(value["search"]),
)
def to_dict(self) -> typing.Dict[str, typing.Any]:
return {
"id": self.id,
"name": self.name,
"search": self.search.to_dict(),
}
# segment = fetch legacy segment info
# rows = fetch legacy segment criteria
segment = Segment(
id=segment["id"],
name=segment["name"],
search=search.Search(
list_id=segment["list_id"],
group=search.Group(
group_type=search.GroupType.AND,
conditions=[
search.Condition(
field=filter.field,
operator=filter.operator,
match=match,
)
for filter, match in [
(legacy.filter_from_id(row["sb_id"]), row["lvc_criteria"]) in rows
]
],
),
),
)
#+end_src
By having functions that can retrieve a Segment object from the database and
save one back to it, we have an interface that we can then copy later for
storing and retrieving segments in an updated schema, and migrate between the
two.

BIN
aweber/controllers-migrated-in-sites.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

6
aweber/index.org
View file

@ -1,8 +1,11 @@
#+TITLE: Correl Roush's Org Roam Notes
- [[file:20211013162152-addlead.org][Addlead]]
- [[file:20211209115433-admin_restore_deleted_controller.org][Admin restore deleted controller]]
- [[file:20210915133101-analytics_view_service.org][Analytics View Service]]
- [[file:20211202143853-appdb.org][AppDB]]
- [[file:20210914144833-broadcast_segment_search.org][Broadcast Segment Search]]
- [[file:20211108150155-bulk_job_service.org][Bulk Job Service]]
- [[file:20210921090818-bulk_tagging_service.org][Bulk Tagging Service]]
- [[file:20211019140007-cp_leads_and_product_sync_up.org][CP Leads and Product Sync-Up]]
- [[file:20210323162555-cleaning_up_public_endpoints_in_proxy_services.org][Cleaning up public endpoints in proxy services]]
@ -23,6 +26,8 @@
- [[file:20201028130030-easy_commerce_mvp_brainstorm_notes.org][Easy Commerce MVP Brainstorm Notes]]
- [[file:20210318110509-finding_number_of_subscribers_with_a_tag.org][Finding number of subscribers with a tag]]
- [[file:20210324133247-imbi.org][Imbi]]
- [[file:20211208140356-legacy_search_filters.org][Legacy Search Filters]]
- [[file:20211210114117-legacy_search_segment_construction.org][Legacy search segment construction]]
- [[file:20200714212153-login_throttling.org][Login Throttling]]
- [[file:20200714130151-mail_relay.org][Mail Relay]]
- [[file:20211019142241-manager_one_on_one.org][Manager one-on-one]]
@ -64,5 +69,6 @@
- [[file:20210323162325-technical_initiative.org][Technical Initiative]]
- [[file:20211001095858-tracking_progress_of_moving_pages_out_of_sites.org][Tracking progress of moving pages out of Sites]]
- [[file:20210406120229-troubleshooting_an_unresolvable_kubernetes_service_hostname.org][Troubleshooting an unresolvable kubernetes service hostname]]
- [[file:20211103100927-updating_projects_using_tagbox.org][Updating projects using Tagbox]]
- [[file:20211013161201-validating_and_sanitizing_tags.org][Validating and sanitizing tags]]
- [[file:20211022094829-validation_and_sanitization_guidelines.org][Validation and Sanitization Guidelines]]

6
daily/2021-10-19.org
View file

@ -7,12 +7,10 @@
- Finding a more cohesive strategy for FE work
- Different work between David G and David R
- Need a better understanding of dependencies between projects
- Can I get more involved with the planning around frontend projects? ([[file:~/Nextcloud/org/aweber.org::*Can I get more involved with the planning around frontend projects?][TODO]]
)
- Can I get more involved with the planning around frontend projects?
- Defining deadlines for BE service work
- Set team goals on when we want to have things done
- Define dates for [[id:c45881de-46f2-4f76-9579-063626c5956c][Analytics View Service]] and [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]] ([[file:~/Nextcloud/org/aweber.org::*Define dates for analytics view and search
service][TODO]])
- Define dates for [[id:c45881de-46f2-4f76-9579-063626c5956c][Analytics View Service]] and [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]]
- Plan an order of attack on larger KTLO [[id:207560cc-7700-4d06-918d-cc01ae530146][Projects]] (goals for end of this year
and next year)
- [[id:e4d00c11-da8a-4c91-8f38-ce939846e5cb][CAPI]]

16
daily/2021-12-07.org Normal file
View file

@ -0,0 +1,16 @@
:PROPERTIES:
:ID: 7f299b76-333d-4282-873d-1f26fadbe07a
:END:
#+title: 2021-12-07
* AWS issues
#+begin_example
core-api5 [07-Dec-2021 16:12:04] ERROR core_api api awsubscribers.mixins.services - HTTP Error 409 getting mapping for subscriber 3778909271 (CID: ce9cfbc1-3261-42a9-83b4-ddceacf4266e)
core-api5 [07-Dec-2021 16:12:04] INFO core_api api awsubscribers.views.subscribers - Writing safestore for error: HTTP 500: MAPPING ERROR 409 (CID ce9cfbc1-3261-42a9-83b4-ddceacf4266e)
#+end_example
#+begin_example
core-api5 [07-Dec-2021 16:13:56] ERROR core_api api awsubscribers.mixins.services - HTTP Error 409 getting mapping for subscriber 3778909741 (CID: 581f82dd-4cf8-4e53-86e7-7e952d8b3fb5)
core-api5 [07-Dec-2021 16:13:56] INFO core_api api awsubscribers.views.subscribers - Writing safestore for error: HTTP 500: MAPPING ERROR 409 (CID 581f82dd-4cf8-4e53-86e7-7e952d8b3fb5)
core-api5 [07-Dec-2021 16:13:56] INFO core_api api awsubscribers.mixins.correlation - Saving suspenders retry file /var/apisuspenders/retry_queue/awsubscribers_POST_581f82dd-4cf8-4e53-86e7-7e952d8b3fb5.json (CID 581f82dd-4cf8-4e53-86e7-7e952d8b3fb5)
#+end_example

86
daily/2021-12-09.org Normal file
View file

@ -0,0 +1,86 @@
:PROPERTIES:
:ID: 07210cf2-cca0-401e-8492-fb9bd4b0eba1
:END:
#+title: 2021-12-09
* [[id:bdea0611-e377-4378-a118-aef6d4a70bdf][CREASE]]
Session with Steve Eisner.
Reviewing the [[https://confluence.aweber.io/pages/viewpage.action?pageId=155890548.][notes]] on what we wanted to achieve in our [[id:29e51b04-ce89-4934-b17f-1f64bffc2069][last session]], which was
cut short.
Steve's current queue is a bit sparse, so we're looking through the Top ASE
Customer Requests and Upcoming Product Team Backlogs - CSLead SLAs sections of
the [[https://jira.aweber.io/secure/StructureBoard.jspa?s=156#][ASE structure board]].
** Etsy imports
Requires less babysitting now. Better than before, but not quite all the way
there.
** [[https://jira.aweber.io/browse/ASE-8301][Campaign messages]]
- Possibly related to [[https://jira.aweber.io/browse/CC-1759][CC-1759]]
Messages attached to a campaign and subsequently removed aren't fully untethered
from campaigns, causing them to not show up in drafts. Steve has Jupyter
notebooks for correcting the issue.
** [[https://jira.aweber.io/browse/CCPANEL-11147][Export broadcast quickstats in bulk]]
Should be much less relevant now that clicks and opens are in the account
export.
** [[https://jira.aweber.io/browse/CCPANEL-8608][Create a way for customers to automatically merge lists]]
Steve would discard some data, including custom fields, to simplify the manual
process.
** [[https://jira.aweber.io/browse/CCPANEL-10695][Ability to bulk tag over 50,000 subscribers]]
Steve wonders whether this could be worked around by using update-on-import
(does that have a row limit? 20k?).
** [[https://jira.aweber.io/browse/CCPANEL-8844][View duplicate subscribers across lists]]
Steve wonders if the customer is aiming to lower their bill?
** [[https://jira.aweber.io/browse/CCPANEL-11140][Remove and edit previous contact addresses from account]]
I suggested that it is probably best to add a flag to retire it / keep it from
being displayed to the user.
** [[https://jira.aweber.io/browse/PSE-4468][Bulk delete inactive subscribers across many lists]]
Steve believes this makes sense to keep this in ASE, warns that in the cases of
mistakes things have to be restored, which can be difficult and miss things
([[id:4bf81e33-8020-40f2-b6ed-ce4e1eae2234][Admin restore deleted controller]]). The subscriber ID /should/ be restored now,
but are the mappings?
** [[https://jira.aweber.io/browse/CCPANEL-11219][Delete duplicate subscribers across lists]]
** [[https://jira.aweber.io/browse/CCPANEL-11864][Customer export file size limited to 100MB (now 200MB)]]
Due to a cap in file upload sizes in Tornado.
** [[https://jira.aweber.io/browse/CCPANEL-11345][As an AWeber customer, I would like a report of tags applied on a given day]]
Steve managed this once using Athena queries, doesn't recall requests from any
other customers.
** [[https://jira.aweber.io/browse/CCPANEL-11680][Broadcast Automations are not Applying/Removing Tags]]
Steve looked into this and was able to replicate the issue on one of his own
subscribers.
[[id:4bf81e33-8020-40f2-b6ed-ce4e1eae2234][Restoring a subscriber]] doesn't restore the recipient record (remains in a
deleted state).
* Average Response Time alert - Subscriber Proxy
[[id:ebea379a-8fa6-4e22-9275-a9fc98c02804][Pagerduty]] alert.
- [[https://grafana.aweber.io/d/CTPUDEjMk/subscriber-proxy-kubernetes-service?tab=alert&viewPanel=4&orgId=1&from=1639058400000&to=1639069200000&var-summarize=1m&var-environment=production][Subscriber proxy graph]]
- [[https://grafana.aweber.io/d/000000031/awsubscribers-dashboard?orgId=1&viewPanel=3&from=1639058400000&to=1639069200000&var-summarize=1m][AWSubscribers graph]]
Seems related to bulk tag job retrieval (=JobHandler.GET=).
This aligns with requests in the bulk tagging logs.
#+begin_src json
{
"asctime": "2021-12-09T18:20:45.538764+0000",
"correlation_id": "-",
"levelname": "INFO",
"message": "10.51.23.1 - - [09/Dec/2021:18:20:44 +0000] \"GET /jobs?account_id=1044937 HTTP/1.1\" 200 - \"-\" \"subscriberproxy/1.2.5\" (secs:1.281)",
"module": "helpers",
"name": "tornado.access",
"service": "bulk-tagging",
"exc_info": null
}
#+end_src
This account in particular is a test account with a large number of jobs, and it
appears to be regularly requested. It looks like bulk-tagging isn't successfully
applying limits and ends up taking a while to collect a large number of jobs.
#+begin_src http :cache yes :pretty :select "length" :exports both
GET http://bulk-tagging.service.production.consul/jobs?account=1044937&limit=3
#+end_src
#+RESULTS[0b736d0b171d5cb83cafebbfdfc9ebaea9ee7eae]:
: 4530

5
daily/2021-12-10.org Normal file
View file

@ -0,0 +1,5 @@
:PROPERTIES:
:ID: a2a17671-ff79-4aaf-ada4-d50e748004d4
:END:
#+title: 2021-12-10
* [[id:e6a2c650-ff59-4b72-b073-970731796888][Legacy search segment construction]]

25
daily/2021-12-13.org Normal file
View file

@ -0,0 +1,25 @@
:PROPERTIES:
:ID: c3f7d0ad-9fc4-454b-81c8-22f85c6bf35b
:END:
#+title: 2021-12-13
* [[id:ac416861-ce45-49ac-8b60-f8ea39362135][Migration to common RabbitMQ]]
- Shovel events to common-rabbitmq
- Move consumers
- Move publishers
- Disable shovel
- Remove old queue
- Update dashboards
Separately, there is a new pattern for handling retries to allow us to use a
single header-based (=X-Original-Queue= / =X-Original-Exchange=) exchange.
(available in rejected version 3.21 and higher).
Only recipient events are currently being shoveled on the events exchange to
common-rabbitmq. =import.approved= and =account.payment.{failure,success}= are
currently being shoveled to AWS from common-rabbitmq.
** [[id:321075e7-db53-4676-b785-7c77ed9d1150][Bulk Tagging Service]]
[[https://jira.aweber.io/browse/CCPANEL-11859][CCPANEL-11859]]
- [X] Testing
- [X] Staging
- [ ] Production