Commit graph

69 commits

Author SHA1 Message Date
Andrew Rabert
d9b2ed5c69
Merge pull request #10 from gmr/master
Attempt to fix the reconnection race condition
2021-04-15 11:22:05 -04:00
Gavin M. Roy
661a5986c7 Attempt to fix the reconnection race condition
1. Add new property function in the ApplicationMixin to indicate if Postgres is connected
2. Add a new guard in the RequestHandlerMixin to check that Postgres is connected prior to executing a query, raising a 503 if it is not
3. Catch OperationalError in RequestHandlerMixin and return 503 for it
4. Timeout when waiting on the connection when attempting to reconnect
5. Log when we're creating a new pool
6. Add debug logging to trace when connectios open
7. Add tests that ensure reconnect logic works as expected
2021-04-14 15:59:13 -04:00
Gavin M. Roy
d0740754c9
Revert the version bump 2021-01-20 16:53:26 -05:00
Gavin M. Roy
57c41365f1
Bump the version 2021-01-20 16:51:25 -05:00
Gavin M. Roy
3d7940ee9c
Merge pull request #8 from nvllsvm/fix_before_connect
Fix postgres_status before first connection attempt
2021-01-20 16:46:41 -05:00
Andrew Rabert
afb6ea99a5 Fix postgres_status before first connection attempt
It's possible to call postgres_status before the first connection is
established. Before this fix, an exception was raised due to
`self._postgres_connected` being `None`.

```
  File "/home/ar/code/github/sprockets/sprockets-postgres/sprockets_postgres.py", line 411, in postgres_status
    if not self._postgres_connected.is_set():
AttributeError: 'NoneType' object has no attribute 'is_set'
```
2021-01-20 16:04:21 -05:00
Gavin M. Roy
16b12cf470 Explicitly raise a FK error 2021-01-11 22:31:54 -05:00
Gavin M. Roy
849cd5bb21 Update supported versions for next time 2021-01-08 18:29:11 -05:00
Gavin M. Roy
230983e803 Melt a few flake8 flakes 2021-01-08 18:19:46 -05:00
Gavin M. Roy
3d6345a882 Missed some abstractions from envvars in 1.6 2021-01-08 18:14:39 -05:00
Gavin M. Roy
ca362e2bc5 Allow for settings based configuration 2021-01-08 14:55:25 -05:00
Andrew Rabert
6c2e26ef0f
Merge pull request #7 from gmr/master
Add tornado-problem-details support and fix an issue when omitting an on_error callback
2020-09-17 12:06:44 -04:00
Gavin M. Roy
499972a7e8 Remove detail due to potential info disclosure issues 2020-09-17 12:05:18 -04:00
Gavin M. Roy
839fb82034 Fix a missing edge case for no error handler, add tests
The error handler behavior descended down through the connection management part of the code and that was missed, this covers that.
Add test coverage for the new branches
2020-09-17 11:40:34 -04:00
Gavin M. Roy
e2e80eea2f Add tornado-problem-details support 2020-09-16 19:57:01 -04:00
Gavin M. Roy
b7ddd137b6
Merge pull request #5 from nvllsvm/on_error
Make on_error optional for PostgresConnector
2020-09-16 19:28:52 -04:00
Gavin M. Roy
e63396be2c
Merge pull request #6 from nvllsvm/override
Add on_postgres_error and on_postgres_timing to public API of RequestHanderMixin
2020-09-16 19:28:21 -04:00
Andrew Rabert
fbb7b4dd9a Add on_postgres_error and on_postgres_timing to public API of RequestHandlerMixin
The methods are intended to be overrideable, but the leading underscore
convention conflicts with Python practice.

See https://docs.python.org/3.8/tutorial/classes.html#private-variables
2020-09-16 17:28:22 -04:00
Andrew Rabert
e57646df00 Make on_error optional for PostgresConnector 2020-09-16 17:20:50 -04:00
Gavin M. Roy
69b806f9b3
Merge pull request #4 from bkorty/obscure-password
Obscure password
2020-09-01 14:47:35 -04:00
Brian Korty
4bbcf51132 Bump version to 1.4.1
Bump the release version to 1.4.1
2020-09-01 10:57:50 -04:00
Brian Korty
893308e340 Obscure password in debug logs.
Obscure password in the postgres connection DSN when logging in debug mode.
Will also not print the DSN with obscured password when failing to connect
at start up.
2020-09-01 10:55:06 -04:00
Gavin M. Roy
ce6e2ecee7 Bump the version for the next release 2020-08-11 19:41:31 -04:00
Gavin M. Roy
e880017fc8 Add a little context of why this code exists 2020-08-11 19:40:56 -04:00
Gavin M. Roy
9a49d769df Include a StatusRequestHandler to make things easy 2020-08-11 19:40:14 -04:00
Gavin M. Roy
507704a6e7 Merge branch 'master' of github.com:sprockets/sprockets-postgres 2020-08-11 19:31:56 -04:00
Gavin M. Roy
664bedbb24 Don't blindly swallow ProgrammingError
ProgrammingError is raised when you try and fetch data from a cursor and there is no data to fetch. When this happens ProgrammingError.pgcode is None.

It is also raised when your query has errors in it. Now if that's the case, they will be caught on L249 and not inside the function at L259.

This new branch in the code will ensure that should we unexpectedly encounter a "real" programming error from Postgres, it is not blindly swallowed and a warning is issued. This should NEVER happen based upon my understanding of the psycopg2 internals.

Unfortunately I couldn't come up with a good test case using mocks to make it happen, as ProgrammingError() takes no keyword arguments and pgcode is a read-only attribute on a ProgrammingError instance. I also couldn't figure out a way to raise ProgrammingError from psycopg2.errors.lookup/1.

Thus, the # pragma: nocover
2020-08-11 19:25:34 -04:00
Gavin M. Roy
54a92667d1 Don't catch every psycopg2 exception in connector
This was swallowing exceptions downstream in the RequestHandler, like psycopg2.DataError
2020-08-11 18:30:30 -04:00
Gavin M. Roy
f95e40ef7d
Merge pull request #2 from cknave/fix-record-timing-args
Fix argument order for record_timing()
2020-07-30 15:11:27 -04:00
Kevin Vance
1e7279916c Bump version to 1.3.3 2020-07-30 15:09:55 -04:00
Kevin Vance
c53cf4f94b Fix argument order for record_timing() 2020-07-30 15:00:09 -04:00
Gavin M. Roy
3dc8030c3e Fix tests 2020-07-08 14:44:10 -04:00
Gavin M. Roy
440def6f3c Dont create objects if we do not need to 2020-07-08 14:34:41 -04:00
Gavin M. Roy
4f60c7bf0a Delay the initialization of asyncio objects to on_start 2020-07-08 14:33:04 -04:00
Gavin M. Roy
70cf5c005b Show the actual URL that is being connected to 2020-07-07 18:37:06 -04:00
Gavin M. Roy
aaf0548bdf Fix indentation, duplicate test name 2020-07-07 17:31:16 -04:00
Gavin M. Roy
a1736bccae Actually wait while the first in reconnects 2020-07-07 16:21:01 -04:00
Gavin M. Roy
ecc070e974 Add reconfiguration for SRV based connections
- Attempt to add graceful reconfiguration for SRV based connections. On connection failure, the pool will be closed and reopened after fetching to get new SRV records.
- When using SRV, use all return hosts in the PostgreSQL URL
- If multiple requests hit a disconnect error, the reconnect logic will allow the first one in to reconnect and the others to wait
2020-07-07 16:17:56 -04:00
Gavin M. Roy
9c448c11e3 Include libffi-dev for testing 2020-06-02 11:34:11 -04:00
Gavin M. Roy
b64a2ee204 Include aiodns in the requirements list 2020-06-02 11:32:44 -04:00
Gavin M. Roy
3e36210c91 Add support for SRV based configuration
Supports both postgresql+srv and aws+srv where aws+srv supports the ECS service discovery Route53 SRV record lookup behavior.
2020-06-02 11:26:07 -04:00
Gavin M. Roy
10b98cba10 Change sprockets_postgres.QueryResult to a class
After use in a couple APIs, I found having to check against QueryResult.row and QueryResult.rows too difficult, so this change will always have content in QueryResult.rows, even if it's one row.

In addition, it turns the object into an iterator and adds __repr__ and __len__ magic methods to make interacting with it easier
2020-04-27 18:16:18 -04:00
Gavin M. Roy
86b6c022f0 Update DEFAULT_POSTGRES_MAX_POOL_SIZE
Even though ``0`` is documented as the max pool size in the aiopg docs, setting it to 0 causes an exception in `aiopg.pool.Pool._fill_free_pool`:

```
  File "/Users/gavinr/Source/PSE/anabroker/env/lib/python3.7/site-packages/aiopg/pool.py", line 208, in _fill_free_pool
    if override_min and self.size < self.maxsize:
TypeError: '<' not supported between instances of 'int' and 'NoneType'
```

Set to the same default as aiopg, @ 10.
2020-04-13 14:20:09 -04:00
Gavin M. Roy
3ba6c8c069 Remove the extra backtick 2020-04-10 16:42:29 -04:00
Gavin M. Roy
4e70836ddd More documentaiton updates 2020-04-10 16:37:23 -04:00
Gavin M. Roy
d7406ac1d3 docstring update for class linking 2020-04-10 16:34:39 -04:00
Gavin M. Roy
8aee0fec5e Fix README badges 2020-04-10 16:31:21 -04:00
Gavin M. Roy
58100b3a54 pymodules -> py_modules, fix packaging 2020-04-10 16:26:18 -04:00
Gavin M. Roy
9056d9b8b1
Merge pull request #1 from gmr/master
Initial Version
2020-04-10 15:38:28 -04:00
Gavin M. Roy
a7b15254f4 Bump the version 2020-04-08 18:48:10 -04:00