correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updates

Browse source
This commit is contained in:
Correl Roush 2022-06-21 14:30:39 -04:00
parent 3b2f25715a
commit 26fa2f8117
2 changed files with 78 additions and 62 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
aweber/20211013162152-addlead.org
View file

@ -4,3 +4,7 @@
#+title: Addlead #+title: Addlead
A business-critical nightmare in Perl. A business-critical nightmare in Perl.
https://confluence.aweber.io/display/~erict/Addlead+Notes
* Create ACP to rewrite Addlead as a Python service
* Break down tickets

136
aweber/20220609145347-retire_redcache.org
View file

@ -5,125 +5,137 @@
#+title: Retire RedCache #+title: Retire RedCache
The goal of this initiative is to identify uses of the shared RedCache servers The goal of this initiative is to identify uses of the shared RedCache servers
and eliminate them. If a key/value store is required, use a sidecar redis or and eliminate them. If a key/value store is required, use a sidecar Redis or
memcached instance as appropriate. memcached instance as appropriate.
Any keys that will require cross-team coordination to remove should be Any keys that will require cross-team coordination to remove should be
documented in the [[https://confluence.aweber.io/display/AWD/RedCache+Inventory][RedCache Inventory]] page in Confluence. documented in the [[https://confluence.aweber.io/display/AWD/RedCache+Inventory][RedCache Inventory]] page in Confluence.
* Usage in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] * Usage in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]]
** Cake Cache ** Caching
These use cases treat Redis as a temporary cache. They could be safely and
seamlessly switched over to a new Redis instance.
*** Cake Cache
The default Cake framework cache engine for the application is [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/config/core.php#L236-243][configured to use The default Cake framework cache engine for the application is [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/config/core.php#L236-243][configured to use
RedCache]]. RedCache]].
** Mapping *** Mapping
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/mapping.php][Mapping component]] caches mapping lookups. The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/mapping.php][Mapping component]] caches mapping lookups.
** Avro Schemas *** Avro Schemas
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/avro.php][Avro component]] caches schema documents. The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/avro.php][Avro component]] caches schema documents.
** Private Labeling *** Private Labeling
A [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/purge_pl_memcache.php][script to purge the private label cache]] exists, though it does not appear to A [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/purge_pl_memcache.php][script to purge the private label cache]] exists, though it does not appear to
be used. be used.
#+begin_notes #+begin_notes
Private labeling is no longer used in the CP. Private labeling is no longer used in the CP.
#+end_notes #+end_notes
** Click Tracking *** Click Tracking
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/click_tracker.php][Click tracking component]] caches tracking url lookups in AppDB. The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/click_tracker.php][Click tracking component]] caches tracking url lookups in AppDB.
** Throttling *** Showcased Applications
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/application.php][Application model]] caches the number of customers using each application from
queries to AppDB for six hours.
*** Web Form Templates
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/web_form_template_category.php][Web Form Template Category model]] caches the top ten web form template
families using the =template_directory_web_form_popular= key.
*** Active Lists
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/auto_responder.php][Auto Responder model]] caches the active lists for an account using the key
format =aweber_app_lists_{$a_id}_{$aServId}=, where =$a_id= is the integer
account ID and =$aServId= is an integer list ID or =false=.
*** Web Form Chicklet Images
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/form/ci/index.php][Web form chicklet image handler]] caches image data stored in AppDB using the
key format =aweber_app_chicklet_$id= where =$id= is the integer ID of the
chicklet image.
*** Template Gallery
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/views/helpers/template_gallery.php][Template Gallery view helper]] caches email templates from the [[http://template-directory.service.production.consul][Template
Directory Service]] using the =template_directory_{$type}_family_data= key format,
where =$type= is =web_form= or =block=.
*** List Settings
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/settings_controller.php][Settings controller]] clears cached list settings using the key pattern
=aweber_app_{$namespace}_remove_options_{$list_id}=, where =$namespace= is the
value retrieved from the key =aweber_app_remove_options_namespace=.
*** Web Form Serving
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_server.php][Web Form Server component]] caches web form split tests loaded from AppDB
using the key format =aweber_app_{$namespace}_web-form_split_{$split_id}= where
=$namespace= is the value retrieved from the key
=aweber_app_web_form_namespace=.
** Other
These use cases treat Redis as a key/value store with specific expectations
around if/when the key is cleared. Data migration may be necessary for a move to
a separate Redis instance.
*** Throttling
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/throttler.php][Throttler component]] uses cache keys with a TTL to rate-limit various actions The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/throttler.php][Throttler component]] uses cache keys with a TTL to rate-limit various actions
in the CP including logins. An older [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/throttle.php][Throttle component]] also exists with in the CP including logins. An older [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/throttle.php][Throttle component]] also exists with
references to redcache, but appears unused. references to redcache, but appears unused.
** Verify-Optin Processing *** Verify-Optin Processing
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/php5-vendors/vendors/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine
whether the control panel is under scheduled maintenance. whether the control panel is under scheduled maintenance.
This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][Verify-Optin]] project, doing the This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][Verify-Optin]] project, doing the
same thing, and should be removed from the sites repository. same thing, and should be removed from the sites repository.
** Unsubscribe / Manage Subscriptions *** Unsubscribe / Manage Subscriptions
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/index.htm][Manage Subscriptions page]] uses the =aweber_app_db_down= cache key to The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/index.htm][Manage Subscriptions page]] uses the =aweber_app_db_down= cache key to
determine whether the control panel is under scheduled maintenance. determine whether the control panel is under scheduled maintenance.
This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/unsubscribe][Unsubscribe]] project, doing the This component also resides separately in the [[https://gitlab.aweber.io/CP/applications/unsubscribe][Unsubscribe]] project, doing the
same thing, and should be removed from the sites repository. same thing, and should be removed from the sites repository.
** Refer a Friend *** Refer a Friend
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/refer_a_friend.php][Refer a Friend component]] uses the =refer_a_friend:access_token= key to store The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/components/refer_a_friend.php][Refer a Friend component]] uses the =refer_a_friend:access_token= key to store
and verify an access token. and verify an access token.
** Showcased Applications *** One-Click Unsubscribe
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/application.php][Application model]] caches the number of customers using each application from
queries to AppDB for six hours.
** Web Form Templates
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/web_form_template_category.php][Web Form Template Category model]] caches the top ten web form template
families using the =template_directory_web_form_popular= key.
** Active Lists
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/auto_responder.php][Auto Responder model]] caches the active lists for an account using the key
format =aweber_app_lists_{$a_id}_{$aServId}=, where =$a_id= is the integer
account ID and =$aServId= is an integer list ID or =false=.
** List Twitter Account
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/vendor_account_list.php][Vendor Account List model]] clears the twitter account on a list using the key
format =orm.list.by_id.$listId.twitter_account=. It is unclear what sets that
key.
** One-Click Unsubscribe
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/one_click_remove.php][One-click remove handler]] uses the =aweber_app_db_down= cache key to The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/z/r/one_click_remove.php][One-click remove handler]] uses the =aweber_app_db_down= cache key to
determine whether the control panel is under scheduled maintenance. determine whether the control panel is under scheduled maintenance.
** Web Form Chicklet Images *** Blocked Orders
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/webroot/form/ci/index.php][Web form chicklet image handler]] caches image data stored in AppDB using the
key format =aweber_app_chicklet_$id= where =$id= is the integer ID of the
chicklet image.
** Template Gallery
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/views/helpers/template_gallery.php][Template Gallery view helper]] caches email templates from the [[http://template-directory.service.production.consul][Template
Directory Service]] using the =template_directory_{$type}_family_data= key format,
where =$type= is =web_form= or =block=.
** Blocked Orders
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] checks and stores IP addresses for blocking orders with the The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] checks and stores IP addresses for blocking orders with the
key pattern =orders_blocked_for_{$_SERVER['REMOTE_ADDR']}= key pattern =orders_blocked_for_{$_SERVER['REMOTE_ADDR']}=
** Flagged Credit Card Numbers *** Feed Broadcasts
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] retrieves flagged credit card numbers using the key
=aweber_app_flagged_cc_bins=. Used to check the first six digits of a card for
flagging.
** Message Templates
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/message_templates_controller.php][Message Templates controller]] clears the key
=template_directory_block_family_data= when a message template is saved.
** Feed Broadcasts
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/feedbroadcaster_controller.php][Feed Broadcasts controller]] tracks feed gearman job status by setting and The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/feedbroadcaster_controller.php][Feed Broadcasts controller]] tracks feed gearman job status by setting and
retrieving the redis key patterns =aweber_app_process_feed_{$sid}_complete= and retrieving the Redis key patterns =aweber_app_process_feed_{$sid}_complete= and
=aweber_app_process_feed_{$sid}_status=, where =$sid= is the current session ID. =aweber_app_process_feed_{$sid}_status=, where =$sid= is the current session ID.
** Preferences *** Preferences
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/preferences.php][Preferences controller component]] stores and retrieves account-level The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/preferences.php][Preferences controller component]] stores and retrieves account-level
preferences using the key pattern preferences using the key pattern
=aweber_app_{$namespace}_cp_preference_{$aId}=. =aweber_app_{$namespace}_cp_preference_{$aId}=.
** Subscriber Search Locking *** Subscriber Search Locking
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/search_mutex.php][Search Mutex controller component]] sets and clears a lock preventing The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/components/search_mutex.php][Search Mutex controller component]] sets and clears a lock preventing
concurrent searches within an account using the key pattern concurrent searches within an account using the key pattern
=subscriber_search_lock_{$accountId}=. =subscriber_search_lock_{$accountId}=.
** Lead Editing *** Login Email Verification
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/leads_controller.php][Leads controller]] checks for blocked emails and email domains using the key
patterns =0-$email= and =0-$domain=.
** List Settings
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/settings_controller.php][Settings controller]] clears cached list settings using the key pattern
=aweber_app_{$namespace}_remove_options_{$list_id}=, where =$namespace= is the
value retrieved from the key =aweber_app_remove_options_namespace=.
** Login Email Verification
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores an email verification token using the key pattern The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores an email verification token using the key pattern
=login_email:verification_token:{$user['id']}= =login_email:verification_token:{$user['id']}=
** Sift Login Verification *** Sift Login Verification
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores a flag allowing previously verified logins to The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/account_controller.php][Account controller]] stores a flag allowing previously verified logins to
bypass the Sift score check using the key pattern bypass the Sift score check using the key pattern
=login_verification:verified_bypass:{$login}:{$ipAddress}=. =login_verification:verified_bypass:{$login}:{$ipAddress}=.
** Application maintenance *** Application maintenance
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/app_controller.php][App controller]] uses the =aweber_app_db_down= cache key to determine whether The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/app_controller.php][App controller]] uses the =aweber_app_db_down= cache key to determine whether
the control panel is under scheduled maintenance. the control panel is under scheduled maintenance.
** Web Form Generation *** Web Form Generation
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_generator.php][Web Form Generator component]] stores and retrieves web form settings using the key pattern ={$namespace}_web_form_{$webFormId}{$type}= where =$namespace= is the value retrieved from the key =aweber_app_web_form_namespace=, and type is one of the following: The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_generator.php][Web Form Generator component]] stores and retrieves web form settings using the key pattern ={$namespace}_web_form_{$webFormId}{$type}= where =$namespace= is the value retrieved from the key =aweber_app_web_form_namespace=, and type is one of the following:
- =_js= (JavaScript) - =_js= (JavaScript)
- =s_js= (split JavaScript) - =s_js= (split JavaScript)
- =_htm= - =_htm=
- =_html= - =_html=
** Web Form Serving ** Unclear
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/vendors/web_form_server.php][Web Form Server component]] caches web form split tests loaded from AppDB These use cases read or clear keys in the key/value store, but the keys may be
using the key format =aweber_app_{$namespace}_web-form_split_{$split_id}= where managed elsewhere. It is unsafe to migrate these to another Redis instance until
=$namespace= is the value retrieved from the key they are fully understood.
=aweber_app_web_form_namespace=. *** List Twitter Account
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/models/vendor_account_list.php][Vendor Account List model]] clears the twitter account on a list using the key
format =orm.list.by_id.$listId.twitter_account=. It is unclear what sets that
key.
*** Flagged Credit Card Numbers
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/order_controller.php][Order controller]] retrieves flagged credit card numbers using the key
=aweber_app_flagged_cc_bins=. Used to check the first six digits of a card for
flagging.
*** Message Templates
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/message_templates_controller.php][Message Templates controller]] clears the key
=template_directory_block_family_data= when a message template is saved.
*** Lead Editing
The [[https://gitlab.aweber.io/CP/applications/sites/-/blob/master/aweber_app/controllers/leads_controller.php][Leads controller]] checks for blocked emails and email domains using the key
patterns =0-$email= and =0-$domain=.
* Usage in Verify-Optin * Usage in Verify-Optin
** Control Panel Maintenance ** Control Panel Maintenance
The [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine The [[https://gitlab.aweber.io/CP/applications/verify-optin/-/blob/master/verify-optin/include/vo_processor.php][VO Processor component]] uses the =aweber_app_db_down= cache key to determine