Increase default queued metrics limit from 1,000 to 1,000,000

Understanding that asyncio.Queue stores items in a collections.deque
instance, this is the memory usage difference between the current
default of 1,000 and the new default of 1,000,000.

1,000 metrics is roughly 8.9KB
>>> d = collections.deque()
>>> for _ in range(1000):
...   d.append(b'applications.this-is-my-service.development.errors.rabbitmq.unroutable:1|c')
...
>>> sys.getsizeof(d)
9072

1,000,000 metrics is roughly 7.9MB
>>> d = collections.deque()
>>> for _ in range(1000000):
...   d.append(b'applications.this-is-my-service.development.errors.rabbitmq.unroutable:1|c')
...
>>> sys.getsizeof(d)
8250624
This commit is contained in:
Andrew Rabert 2022-03-30 14:49:28 -04:00
parent 76399b9a02
commit c60d37b425

View file

@ -496,7 +496,7 @@ class Processor:
host: str,
port: int = 8125,
ip_protocol: int = socket.IPPROTO_TCP,
max_queue_size: int = 1000,
max_queue_size: int = 1000000,
reconnect_sleep: float = 1.0,
wait_timeout: float = 0.1) -> None:
super().__init__()