updates
This commit is contained in:
parent
51dc784f84
commit
457de828f7
8 changed files with 402 additions and 12 deletions
18
20211104164651-input_data_properties.org
Normal file
18
20211104164651-input_data_properties.org
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: d6d1f003-5aaa-4a36-867c-02d43cdf2475
|
||||||
|
:END:
|
||||||
|
#+title: Input data properties
|
||||||
|
|
||||||
|
How could we describe data that we wish to [[id:9914d09e-99fe-46a6-95be-676c5b78ed90][validate]] and possibly [[id:2ba04972-f498-41c2-970e-a64c7f3f1c3b][sanitize]]
|
||||||
|
differently, and present different experiences to end users for providing that
|
||||||
|
data?
|
||||||
|
|
||||||
|
- What causes us to want to trim or prevent leading or trailing whitespace from
|
||||||
|
some inputs?
|
||||||
|
- What causes us to limit the characters a field accepts?
|
||||||
|
- What sort of data would we want to sanitize or otherwise alter /as/ we accept
|
||||||
|
it (e.g. accepting but also sanitizing HTML to be stored as trusted HTML)?
|
||||||
|
|
||||||
|
Is there a good language for describing these categories of data, can we better
|
||||||
|
understand what differentiates them, and use that knowledge to build more
|
||||||
|
consistent guidelines for UX and input handling?
|
||||||
|
|
@ -7,6 +7,9 @@
|
||||||
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to rewrite pages in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] still built in PHP.
|
A [[id:db322997-ff5e-416a-8dc8-f29e6a4928c8][Technical Initiative]] to rewrite pages in the [[id:57ee2f00-9bcd-4e0f-8a77-ae1f2d4cda89][Control Panel]] still built in PHP.
|
||||||
|
|
||||||
* Pages in Sites
|
* Pages in Sites
|
||||||
|
|
||||||
|
- Tracking document :: https://confluence.aweber.io/display/BETL/PHP+to+React+Page+Needs
|
||||||
|
|
||||||
** TODO Dashboard
|
** TODO Dashboard
|
||||||
*** Broadcasts
|
*** Broadcasts
|
||||||
*** Lists
|
*** Lists
|
||||||
|
|
|
||||||
|
|
@ -64,7 +64,7 @@
|
||||||
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:53]
|
- State "BACKLOG" from "TODO" [2021-10-20 Wed 15:53]
|
||||||
- State "TODO" from [2021-10-13 Wed 16:26]
|
- State "TODO" from [2021-10-13 Wed 16:26]
|
||||||
:END:
|
:END:
|
||||||
* Eliminating Sites
|
* [[id:f633f967-11d2-432c-b5ff-ad842c88a51c][Decommissioning Sites]]
|
||||||
** [[id:3cc8bd09-dd02-4950-8c89-a737f92809fd][Tracking progress of moving pages out of Sites]]
|
** [[id:3cc8bd09-dd02-4950-8c89-a737f92809fd][Tracking progress of moving pages out of Sites]]
|
||||||
** Creating a new Control Panel shell application
|
** Creating a new Control Panel shell application
|
||||||
* TODO Create the [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]]
|
* TODO Create the [[id:11edd6c9-b976-403b-a419-b5542ddedaae][Subscriber Search Service]]
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,29 @@
|
||||||
- Last 30 days
|
- Last 30 days
|
||||||
- 30 day range
|
- 30 day range
|
||||||
|
|
||||||
|
#+caption: opens_all
|
||||||
|
#+begin_example
|
||||||
|
{
|
||||||
|
"2021-11-02T00:00:00Z": {
|
||||||
|
"broadcasts": 2499,
|
||||||
|
"followups": 2547,
|
||||||
|
"unique": 2923,
|
||||||
|
"total": 5046
|
||||||
|
},
|
||||||
|
"2021-11-03T00:00:00Z": {
|
||||||
|
"broadcasts": 25808,
|
||||||
|
"followups": 2430,
|
||||||
|
"unique": 24876,
|
||||||
|
"total": 28238
|
||||||
|
},
|
||||||
|
"2021-11-04T00:00:00Z": {
|
||||||
|
"broadcasts": 16733,
|
||||||
|
"followups": 1437,
|
||||||
|
"unique": 14780,
|
||||||
|
"total": 18170
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_example
|
||||||
*** Clicks
|
*** Clicks
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:Effort: 1d
|
:Effort: 1d
|
||||||
|
|
@ -72,6 +95,29 @@
|
||||||
- Last 30 days
|
- Last 30 days
|
||||||
- 30 day range
|
- 30 day range
|
||||||
|
|
||||||
|
#+caption: clicks_all
|
||||||
|
#+begin_example
|
||||||
|
{
|
||||||
|
"2021-11-02T00:00:00Z": {
|
||||||
|
"broadcasts": 105,
|
||||||
|
"followups": 137,
|
||||||
|
"unique": 130,
|
||||||
|
"total": 242
|
||||||
|
},
|
||||||
|
"2021-11-03T00:00:00Z": {
|
||||||
|
"broadcasts": 636,
|
||||||
|
"followups": 185,
|
||||||
|
"unique": 622,
|
||||||
|
"total": 821
|
||||||
|
},
|
||||||
|
"2021-11-04T00:00:00Z": {
|
||||||
|
"broadcasts": 480,
|
||||||
|
"followups": 109,
|
||||||
|
"unique": 426,
|
||||||
|
"total": 589
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_example
|
||||||
*** Sales
|
*** Sales
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:Effort: 1d
|
:Effort: 1d
|
||||||
|
|
@ -80,6 +126,72 @@
|
||||||
- Last 30 days
|
- Last 30 days
|
||||||
- 30 day range
|
- 30 day range
|
||||||
- Currency
|
- Currency
|
||||||
|
|
||||||
|
#+caption: sales_tracked_events
|
||||||
|
#+begin_example
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"time": "2021-11-02 09:37:36-04",
|
||||||
|
"type": "followup",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "team@harmoniamedia.com"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"time": "2021-11-02 09:37:37-04",
|
||||||
|
"type": "followup",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "team@harmoniamedia.com"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"time": "2021-11-02 12:01:17-04",
|
||||||
|
"type": "followup",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "giuliagiardino12@gmail.com"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"time": "2021-11-02 12:01:19-04",
|
||||||
|
"type": "followup",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "giuliagiardino12@gmail.com"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"time": "2021-11-04 05:21:35-04",
|
||||||
|
"type": "broadcast",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "jeremy@jeremy-quick.com"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"time": "2021-11-04 05:21:36-04",
|
||||||
|
"type": "broadcast",
|
||||||
|
"currency": "USD",
|
||||||
|
"revenue": "19.00",
|
||||||
|
"note": "",
|
||||||
|
"description": "Upgraded to Pro",
|
||||||
|
"source_url": "https://www.aweber.com/users/#upgraded",
|
||||||
|
"email": "jeremy@jeremy-quick.com"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
#+end_example
|
||||||
*** Sales Totals
|
*** Sales Totals
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:Effort: 1d
|
:Effort: 1d
|
||||||
|
|
@ -88,6 +200,32 @@
|
||||||
- Last 60 days
|
- Last 60 days
|
||||||
- 60 day range
|
- 60 day range
|
||||||
|
|
||||||
|
#+caption: sales_tracked_summary
|
||||||
|
#+begin_example
|
||||||
|
{
|
||||||
|
"2021-11-02T00:00:00Z": {
|
||||||
|
"broadcast": 0,
|
||||||
|
"followup": 76,
|
||||||
|
"pageview": 76,
|
||||||
|
"ecommerce": 0,
|
||||||
|
"total": 76
|
||||||
|
},
|
||||||
|
"2021-11-03T00:00:00Z": {
|
||||||
|
"broadcast": 0,
|
||||||
|
"followup": 0,
|
||||||
|
"pageview": 0,
|
||||||
|
"ecommerce": 0,
|
||||||
|
"total": 0
|
||||||
|
},
|
||||||
|
"2021-11-04T00:00:00Z": {
|
||||||
|
"broadcast": 38,
|
||||||
|
"followup": 0,
|
||||||
|
"pageview": 38,
|
||||||
|
"ecommerce": 0,
|
||||||
|
"total": 38
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#+end_example
|
||||||
*** Sale Currencies
|
*** Sale Currencies
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:Effort: 1d
|
:Effort: 1d
|
||||||
|
|
@ -95,7 +233,10 @@
|
||||||
- Time
|
- Time
|
||||||
- 60 day range
|
- 60 day range
|
||||||
- All time
|
- All time
|
||||||
|
#+caption: sales_tracked_currencies
|
||||||
|
#+begin_example
|
||||||
|
["USD"]
|
||||||
|
#+end_example
|
||||||
*** Sale Events
|
*** Sale Events
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:Effort: 1d
|
:Effort: 1d
|
||||||
|
|
|
||||||
|
|
@ -2,3 +2,87 @@
|
||||||
:ID: 321075e7-db53-4676-b785-7c77ed9d1150
|
:ID: 321075e7-db53-4676-b785-7c77ed9d1150
|
||||||
:END:
|
:END:
|
||||||
#+title: Bulk Tagging Service
|
#+title: Bulk Tagging Service
|
||||||
|
|
||||||
|
The Bulk Tagging service queues the application of tags to a set of subscribers
|
||||||
|
and tracks the overall progress of the operation. This allows customers to
|
||||||
|
select a large number of subscribers and apply tags to all of them in a single
|
||||||
|
operation, while also giving them visibility into how their tagging request is
|
||||||
|
progressing.
|
||||||
|
|
||||||
|
* Problems Solved
|
||||||
|
|
||||||
|
** Background processing of tagging operations
|
||||||
|
Tagging operations are assigned a bulk-tagging job with one task per subscriber
|
||||||
|
to add or remove the specified tags. These operations are performed
|
||||||
|
asynchronously by consumers such that the end-user need not wait for all
|
||||||
|
operations to be completed before moving on.
|
||||||
|
** Rate-limited application of tags
|
||||||
|
Consumers are configured to consume tasks no faster than a configured maximum
|
||||||
|
rate to control the load placed upon downstream services (e.g. rules engine,
|
||||||
|
etc.). The ideal rate is divided amongst the number of consumers available.
|
||||||
|
** Serial application of operations within an account
|
||||||
|
There is an expectation that operations a customer applies to subscribers will
|
||||||
|
be performed sequentially. To allow this while still allowing jobs for /other/
|
||||||
|
accounts to be worked upon simultaneously, jobs are divided into queues
|
||||||
|
deterministically using a consistent hash of their account IDs.
|
||||||
|
** Tracking overall job progress
|
||||||
|
The job itself may be in one of the following states, which is updated as tasks
|
||||||
|
are acted upon:
|
||||||
|
- Pending :: No tasks have yet been acted upon
|
||||||
|
- Processing :: Some, but not all, tasks have been acted upon
|
||||||
|
- Succeeded :: All tasks have completed successfully
|
||||||
|
- Failed :: All tasks have completed, but at least one task was not successful
|
||||||
|
** Tracking of individual task status
|
||||||
|
Any particular task, once acted upon, will be updated as having succeeded or
|
||||||
|
failed with a message explaining the issue.
|
||||||
|
** Jobs must survive queue failures
|
||||||
|
A guarantee of the service is such that if a job is successfully submitted to
|
||||||
|
the Bulk Tagging service, we will not lose it, and can take steps manually if
|
||||||
|
necessary to ensure its completion. To account for unexpected failures when
|
||||||
|
submitting tasks to a queue or consuming a task from the queue, the job request
|
||||||
|
is archived to S3. This archive contains sufficient information to requeue the
|
||||||
|
job for processing as it was requested without further input.
|
||||||
|
** Customer visibility into job progress
|
||||||
|
End users are able to, via the service API, fetch any and all jobs stored for
|
||||||
|
their account, as well as their associated tasks.
|
||||||
|
** Administrative visibility into job progress
|
||||||
|
Administrative (internal) users are able to, via the service API, fetch any and
|
||||||
|
all jobs stored for any or all accounts. This is used to populate dashboards for
|
||||||
|
insight into the progress of Bulk Tagging as a whole, and whether jobs from
|
||||||
|
different accounts are holding each other up.
|
||||||
|
|
||||||
|
Administrative users also have access to delete or requeue jobs.
|
||||||
|
** Prioritizing smaller jobs over larger jobs
|
||||||
|
Tasks for jobs affecting a number of subscribers lower than a defined threshold
|
||||||
|
are assigned higher priority, causing them to be processed /ahead/ of any other
|
||||||
|
ongoing jobs in their respective queues. This avoids leaving a customer waiting
|
||||||
|
long periods of time for quick operations, which they may not expect to be held
|
||||||
|
up by other jobs outside their control.
|
||||||
|
* Known Problems
|
||||||
|
** Jobs appearing "stuck"
|
||||||
|
Jobs may appear stuck (either failing to start processing, or failing to
|
||||||
|
complete processing).
|
||||||
|
*** Job has not yet begun processing
|
||||||
|
This is caused by one of two things. Either a job is stuck behind other jobs
|
||||||
|
(this or another account's job could be in front of it in the same queue), or
|
||||||
|
the tasks failed to be written successfully to the queue. In the former case,
|
||||||
|
the job will complete normally once the tasks ahead of it in the queue are
|
||||||
|
processed. In the latter case, the job will need to be requeued.
|
||||||
|
*** Job is stuck in an in-progress state
|
||||||
|
This is typically due to the overall job progress being out of sync with the
|
||||||
|
actual state of its consituent tasks ([[https://jira.aweber.io/browse/CCPANEL-8660][CCPANEL-8660]]).
|
||||||
|
|
||||||
|
The overall status of a job and the status of its individual tasks are stored in
|
||||||
|
separate Dynamo tables. Because Dynamo tables are independent, there is no way
|
||||||
|
to update an individual task and the job's overall status in a single, atomic
|
||||||
|
operation. This means they may (and do) at times get out of sync, if
|
||||||
|
infrequently. The indexes on the tables are not designed to make reconciliation
|
||||||
|
easy, nor does an automated reconciliation process exist.
|
||||||
|
|
||||||
|
This could be solved by adding indexes to allow rapid computation of the actual
|
||||||
|
state of all tasks in a job and creating a scheduled task to synchronize these
|
||||||
|
counts, or by changing the underlying storage to a relational database which
|
||||||
|
would make it easier to compute these states.
|
||||||
|
* Resources
|
||||||
|
- ACP :: https://confluence.aweber.io/display/AR/ACP+Bulk+Tagging
|
||||||
|
- Playbook :: https://confluence.aweber.io/display/AR/Bulk+Tagging+Service+Playbook
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,7 @@ modernization of our public-facing application stack.
|
||||||
The project will engage multiple teams to coordinate the following three
|
The project will engage multiple teams to coordinate the following three
|
||||||
efforts:
|
efforts:
|
||||||
|
|
||||||
* Moving applications out of Sites
|
* [[id:193f7c04-0a03-4870-90c8-2b5e3c4c92ce][Moving applications out of Sites]]
|
||||||
- Individual pages will be replaced with React applications using public APIs
|
- Individual pages will be replaced with React applications using public APIs
|
||||||
- Static content will be moved to CDN hosting
|
- Static content will be moved to CDN hosting
|
||||||
- Independent applications will be broken out into separate services
|
- Independent applications will be broken out into separate services
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@
|
||||||
:END:
|
:END:
|
||||||
#+title: Updating projects using Tagbox
|
#+title: Updating projects using Tagbox
|
||||||
#+OPTIONS: prop:("JIRA_ID")
|
#+OPTIONS: prop:("JIRA_ID")
|
||||||
#+todo: TODO(t) INVESTIGATE(i) AWAITING-RELEASE(a) | DONE(d) NO-ACTION(n)
|
#+todo: TODO(t) INVESTIGATE(i) TESTING(s) AWAITING-RELEASE(a) | DONE(d) NO-ACTION(n)
|
||||||
|
|
||||||
The Tagbox component has been updated with a fix addressing an XSS security
|
The Tagbox component has been updated with a fix addressing an XSS security
|
||||||
vulnerability in its tag label auto-completion which needs to be propogated out
|
vulnerability in its tag label auto-completion which needs to be propogated out
|
||||||
|
|
@ -21,17 +21,19 @@ Target versions:
|
||||||
* Projects
|
* Projects
|
||||||
Parent ticket: [[https://jira.aweber.io/browse/CCPANEL-11762][CCPANEL-11762]]
|
Parent ticket: [[https://jira.aweber.io/browse/CCPANEL-11762][CCPANEL-11762]]
|
||||||
|
|
||||||
** TODO Campaign Builder (Direct)
|
** TESTING Campaign Builder (Direct)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CC-7550
|
:JIRA_ID: CC-7550
|
||||||
:END:
|
:END:
|
||||||
- Tagbox :: 5.0.3
|
- Tagbox :: 5.0.3
|
||||||
** TODO GoToWebinar Client (Direct)
|
** NO-ACTION GoToWebinar Client (Direct)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: INT-5508
|
:JIRA_ID: INT-5508
|
||||||
:END:
|
:END:
|
||||||
- Tagbox :: 3.0.0
|
- Tagbox :: 3.0.0
|
||||||
** TODO Subscriber Import (Direct)
|
|
||||||
|
This project does not use the Tagbox component.
|
||||||
|
** TESTING Subscriber Import (Direct)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CCPANEL-11768
|
:JIRA_ID: CCPANEL-11768
|
||||||
:END:
|
:END:
|
||||||
|
|
@ -39,28 +41,28 @@ Parent ticket: [[https://jira.aweber.io/browse/CCPANEL-11762][CCPANEL-11762]]
|
||||||
- AWeberUI :: 2
|
- AWeberUI :: 2
|
||||||
|
|
||||||
Uses ramda, etc.
|
Uses ramda, etc.
|
||||||
** AWAITING-RELEASE List Automation Client (AWeberUI)
|
** TESTING List Automation Client (AWeberUI)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CCPANEL-11769
|
:JIRA_ID: CCPANEL-11769
|
||||||
:END:
|
:END:
|
||||||
- AWeberUI :: 8.3
|
- AWeberUI :: 8.3
|
||||||
** NO-ACTION User Management Client (AWeberUI)
|
** NO-ACTION User Management Client (AWeberUI)
|
||||||
This project does not use the Tagbox component.
|
This project does not use the Tagbox component.
|
||||||
** INVESTIGATE Landing Page Editor (AWeberUI)
|
** TESTING Landing Page Editor (AWeberUI)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CC-7551
|
:JIRA_ID: CC-7551
|
||||||
:END:
|
:END:
|
||||||
** TODO Draft Bin (Direct)
|
** TESTING Draft Bin (Direct)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CC-7552
|
:JIRA_ID: CC-7552
|
||||||
:END:
|
:END:
|
||||||
- Tagbox :: 5.0.0
|
- Tagbox :: 5.0.0
|
||||||
** INVESTIGATE Subscribers Client (AWeber UI)
|
** TESTING Subscribers Client (AWeber UI)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CCPANEL-11770
|
:JIRA_ID: CCPANEL-11770
|
||||||
:END:
|
:END:
|
||||||
- AWeberUI :: 8.6.1
|
- AWeberUI :: 8.6.1
|
||||||
** TODO Webform Generator (Standalone file)
|
** TESTING Webform Generator (Standalone file)
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:JIRA_ID: CC-7553
|
:JIRA_ID: CC-7553
|
||||||
:END:
|
:END:
|
||||||
|
|
|
||||||
142
daily/2021-11-04.org
Normal file
142
daily/2021-11-04.org
Normal file
|
|
@ -0,0 +1,142 @@
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: a20323e3-fc41-496c-8acb-cf62cdb3ba27
|
||||||
|
:END:
|
||||||
|
#+title: 2021-11-04
|
||||||
|
* Backend BOF
|
||||||
|
- Scott M
|
||||||
|
- Kevin V
|
||||||
|
- Josh E
|
||||||
|
- Pycares was failing to install due to a dependency (safety virtual
|
||||||
|
environment did not have wheel)
|
||||||
|
- Josh B
|
||||||
|
- Redis issue in sessions resuilting in rescheduling kubenetes node workers
|
||||||
|
- May need to change how we're running certain Redises in k8s; the old
|
||||||
|
workload took time to shut down, the new instance read old data, failures
|
||||||
|
ensued.
|
||||||
|
- We need some measure of fault tolerance / HA
|
||||||
|
- Reasonable way to run a 3-pod redis that would be durable to that kind of
|
||||||
|
failure?
|
||||||
|
- Be really clear about the use cases and failure cases of Redis or any
|
||||||
|
persistent store in k8s.
|
||||||
|
- Redis usually used as a cache or database. Applications should be able to
|
||||||
|
work without a cache. Databases should have durability.
|
||||||
|
- (Dave S) We're treating redis like a cache, but failing to consider
|
||||||
|
connection timeouts and failures (blocking connect). Lack of data didn't
|
||||||
|
affect the service, but lack of connectivity did.
|
||||||
|
- (Amber H) We have existing VMs, is that an option?
|
||||||
|
- (Josh B) It is. There are challenges with client behaviors using HA Redis
|
||||||
|
and handling failover. VMs aren't my first choice.
|
||||||
|
- (Gavin R) We should focus on figuring out a workable k8s solution first.
|
||||||
|
The underlying storage solution is in essence the same thing.
|
||||||
|
- (Dave S) Aioredis doesn't have direct support for sharding or replication
|
||||||
|
(needs an exception handling wrapper). Simliar with TRedis.
|
||||||
|
- (Gavin R) A fork of redis implements transparent clustering, which directs
|
||||||
|
the client to the correct instance. (https://keydb.dev/)
|
||||||
|
- (Josh B) Also has multi-master
|
||||||
|
- (Dave S) The trouble is writes result in a denial response, leaving it to
|
||||||
|
the client to find the correct instance it can write to
|
||||||
|
- (Josh B) HAProxy could be told to talk the Redis protocol to find the
|
||||||
|
primary and send traffic there for clients that can't handle the read-only
|
||||||
|
response well.
|
||||||
|
- (Dave S) In k8s even when running as a cache, Redis will occasionally (4-5
|
||||||
|
times past year) fail to write to disk when there's some confusion between
|
||||||
|
CEPH and the underlying mount
|
||||||
|
- (Josh B) This was a lower-level CEPH issue; a race condition based on our
|
||||||
|
specific configuration.
|
||||||
|
- I will take time to put a Redis pattern together
|
||||||
|
- Ihar H
|
||||||
|
- Recently got feedback updating one of the service pipelines to meet
|
||||||
|
standards, want to clarify expectations. We'd decided to split the stages
|
||||||
|
- (Gavin R) I believe you can accomplish what you want to accomplish without
|
||||||
|
creating so many explicit stages
|
||||||
|
- (Ihar H) You have to click on a pipeline stage to see whether one of the nested pipeline tasks failed, and further stages could be run without the prior one passing
|
||||||
|
- (Amber H / Gavin R) We do need to sometimes get deploys out even when
|
||||||
|
acceptance tests fail
|
||||||
|
- (Ihar H) Even if code is deployed to production, the pipeline will be marked
|
||||||
|
as failed in some cases
|
||||||
|
- (Gavin R) Your proposal is fine, we need to clarify best practices
|
||||||
|
documentation to cover all cases.
|
||||||
|
- (Dave S) Gitlab visualization has changed over time, we can see if
|
||||||
|
anything can be improved to make things visually clearer
|
||||||
|
- (Gavin R) Since we are using immutable image building, there's little
|
||||||
|
value in running unit and integration tests in staging
|
||||||
|
- (Dave S) ... they /shouldn't/
|
||||||
|
- (Gavin R) The prior method was tying stages to environments, which we
|
||||||
|
don't want to do, they are different things.
|
||||||
|
- (Ihar H) I will update the confluence page and follow up with Dave S
|
||||||
|
- (Amber H) I want to see more of the /why/ in that document
|
||||||
|
- Gavin R
|
||||||
|
- Update on experiences with psycopg3, fixed the bug I found, I've been using
|
||||||
|
it pretty heavily. Seems to work well. API is backwards compatible and has
|
||||||
|
also evolved. Idiomatic row factory usage is different. One of the cool
|
||||||
|
things is if you use dataclasses, you can use a dataclass row factory.
|
||||||
|
- (Amber H) We just need them and Pydantic to have a baby
|
||||||
|
- (Gavin R) That probably wouldn't be too hard to do
|
||||||
|
- (Andrew R) I really don't like what has to be done to pass a value to the
|
||||||
|
=IN= keyword. In Psycopg, you have to format the comma-separated value yourself.
|
||||||
|
- (Gavin R) You can specify an array
|
||||||
|
- (Andrew R) That works with ~= ANY (...)~, but is much less performant than
|
||||||
|
using =IN=.
|
||||||
|
- (Alex C) A newer version of Postgres may fix the performance issue
|
||||||
|
- Eric T
|
||||||
|
- David R
|
||||||
|
- Been doing a lot of front-end experimentation with the concept of module
|
||||||
|
federation to power out micro-frontend architecture for our react clients to
|
||||||
|
build an app shell to compose them into a single MFA. We should discuss soon
|
||||||
|
what it'd look like to deploy something like that. Grossly simplifying,
|
||||||
|
every MFE exposes as part of its build a =remoteEntry.js= file that the
|
||||||
|
shell application needs to be told as part of its configuration where that
|
||||||
|
lives so it can load the module when necessary. The route willi then use the
|
||||||
|
remote to on-demand load the module and its dependencies to load that
|
||||||
|
application. I believe we just need that remote to be built and available as
|
||||||
|
part of our deployment strategy. Will want to get the POC I've been working
|
||||||
|
on into staging to start getting it working outside of local development.
|
||||||
|
- (Gavin R) This is the effort to replace Sites as our application shell? (Yes)
|
||||||
|
- https://confluence.aweber.io/display/BETL/Decommissioning+Sites
|
||||||
|
- (Gavin R) The larger implication to backend is sessions and sites going
|
||||||
|
away and the front-end application using the Public API going forward.
|
||||||
|
- (Scott M) Something will need to fetch AWVars data to provide to the
|
||||||
|
applications, from the session service for now, as a dynamic service
|
||||||
|
hosting the shell application?
|
||||||
|
- (David R) It would just be static content / JavaScript, provided it is
|
||||||
|
able to discover the contained applications.
|
||||||
|
- (Gavin R) JS should be on aweber-static, the shell application should be
|
||||||
|
hosted on something TBD, and we'll need to bridge the transition between
|
||||||
|
using sessions for stored data to something else.
|
||||||
|
- (David R) I see the session service as a middle state (stepping stone)
|
||||||
|
to work towards no longer needing it.
|
||||||
|
- (Scott M) We should look into how it can get direct access to the session service
|
||||||
|
- (Gavin R) I don't think we want that to be a dependency. We need a
|
||||||
|
global state managed in a better way
|
||||||
|
- (Dave S) We should start putting thought into how we're using local
|
||||||
|
storage, etc. in a consistent way so data can be cached and not
|
||||||
|
re-fetched, avoiding having each service have its own data model.
|
||||||
|
- (Josh B) The shell app could provide a common interface to access
|
||||||
|
shared cache data.
|
||||||
|
- (David R) Let's make sure there isn't an assumption that the scout file and the
|
||||||
|
remote entry file are the same thing.
|
||||||
|
- Dave S
|
||||||
|
- Had something, will post it; Updating docs on aweber API and what's behind it.
|
||||||
|
- Updated endpoint map on
|
||||||
|
https://confluence.aweber.io/display/STD/api.aweber.com+Endpoint+Management
|
||||||
|
- If anybody really understands CORS, we need someone in the company that
|
||||||
|
does. We're having CORS failures with companies like FaceBook, etc. If
|
||||||
|
anyone has experience, please speak up in BoF or elsewhere.
|
||||||
|
- Cedric W
|
||||||
|
- Alex and I drafted an ACP for bulk actions, will post a link for review.
|
||||||
|
Will probably be more after tomorrow's meeting.
|
||||||
|
- https://confluence.aweber.io/display/AR/Bulk-Action+Consumers+ACP
|
||||||
|
- Arnela M
|
||||||
|
- Andrew R
|
||||||
|
- Pydantic's awesome!
|
||||||
|
- Amber H
|
||||||
|
- Working with the analytics ingestion service, it's pretty awesome for opens
|
||||||
|
and clicks reporting. Currently in the development staging
|
||||||
|
- Talking with Scott the other day about how to do some cross-team code
|
||||||
|
reviews and wanted to float the idea here on having other people on other
|
||||||
|
teams to commit to some time during a sprint to review code from other
|
||||||
|
teams.
|
||||||
|
- Alex C
|
||||||
|
- Correl R
|
||||||
|
- Perl stinks. Derefencing data structures from scalars with weird symbols
|
||||||
|
stinks.
|
||||||
Loading…
Reference in a new issue