correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
roam/daily/2022-06-30.org
Correl Roush 74f9662c68 updates
2022-07-06 16:46:00 -04:00

1.5 KiB
Raw Blame History

2022-06-30

Chatting with Gavin

Addlead storage

Discussing queuing options for the Addlead Rewrite.

  • Needs to be HIGHLY available
  • Could it function Lambda?

  • Similar availablity constraints as Analytics ingestion, which must function if

    • AppDB is down
    • RabbitMQ is down
    • Could use redis as a persistent cache for url codes (on a standalone VM?)

  • Disk becomes more complicated in K8s (shared storage, locking, etc.)

    • Issues with the CEPH filesystem may impede our ability to write (would affect CEPH-backed VMs as well)
  • Examining KeyDB as a highly-available alternative to Redis, still a lot of unknowns
  • Considering multiple layers of storage options (rabbitmq, fail over to SQS, …)
  • Kafka could be an alternative to RabbitMQ for do-it-later queues
  • Consider the simplicity in disaster recovery of the current implementation (take files on the server, put them in the right spot)
  • Amazon EFS file systems with ECS?

  • Note that signup forms / thank you pages are included in a future state URL layout

    • Routing is handled by a PSE-maintained Kong instance running in Kubernetes (so it would have to route to a service addressable in K8s)
  • RabbitMQ Streams https://wrobell.dcmod.org/rbfly/index.html?