6.5 KiB
6.5 KiB
2021-08-26
Bulk-tagging
Working on Deploying Bulk Tagging to Kubernetes.
The last change on this project was to update the setup file and gitlab pipeline: https://gitlab.aweber.io/CP/bulk-tagging/-/commit/fecc53cd8b8e34d826d06fbdfa581aa54e508bc7. The pipeline succeeded, but the service is crash-looping in testing. Looks like a whole mess of configuration issues:
{"asctime":"2021-08-26T19:37:42.870100+0000","correlation_id":"-","levelname":"INFO","message":"sentry DSN not found, not installing client","module":"__init__","name":"sprockets.mixins.sentry","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.870435+0000","correlation_id":"-","levelname":"INFO","message":"starting processes on port 8000","module":"runner","name":"Runner","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.873323+0000","correlation_id":"-","levelname":"INFO","message":"sprockets_influxdb v2.2.1 installed; 5000 measurements or 60.00 seconds will trigger batch submission","module":"sprockets_influxdb","name":"sprockets_influxdb","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.874454+0000","correlation_id":"-","levelname":"WARNING","message":"CONSUL_HTTP_ADDR should be a well formed URL","module":"consul","name":"aiomappinglib.consul","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.888571+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to ::1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.888805+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to ::1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.889003+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to 127.0.0.1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.889294+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to 127.0.0.1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.889457+0000","correlation_id":"-","levelname":"WARNING","message":"Could not connect, 2 attempts left","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:42.889581+0000","correlation_id":"-","levelname":"INFO","message":"Retrying in 2 seconds","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.891861+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to ::1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.892210+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to ::1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.892408+0000","correlation_id":"-","levelname":"INFO","message":"Pika version 0.13.1 connecting to 127.0.0.1:5672","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.892620+0000","correlation_id":"-","levelname":"ERROR","message":"Connection to 127.0.0.1:5672 failed: [Errno 111] Connection refused","module":"base_connection","name":"pika.adapters.base_connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.892793+0000","correlation_id":"-","levelname":"WARNING","message":"Could not connect, 1 attempts left","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.892961+0000","correlation_id":"-","levelname":"INFO","message":"Retrying in 2 seconds","module":"connection","name":"pika.connection","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.893576+0000","correlation_id":"-","levelname":"ERROR","message":"before_run callback <function before_run at 0x7f114e9b5040> cancelled start","module":"app","name":"Application","service":"bulk-tagging","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/app.py\", line 104, in start\n callback(self.tornado_application, io_loop)\n"," File \"/usr/local/lib/python3.9/site-packages/bulk_tagging/app.py\", line 151, in before_run\n raise RuntimeError('AWS credentials not found')\n","RuntimeError: AWS credentials not found\n"]} {"asctime":"2021-08-26T19:37:44.897295+0000","correlation_id":"-","levelname":"INFO","message":"starting IOLoop shutdown process","module":"app","name":"_ShutdownHandler","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.897470+0000","correlation_id":"-","levelname":"INFO","message":"stopped IOLoop","module":"app","name":"_ShutdownHandler","service":"bulk-tagging","exc_info":null} {"asctime":"2021-08-26T19:37:44.897635+0000","correlation_id":"-","levelname":"ERROR","message":"application terminated during start, exiting","module":"runner","name":"Runner","service":"bulk-tagging","exc_info":["Traceback (most recent call last):\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/runner.py\", line 129, in run\n self.application.start(iol)\n"," File \"/usr/local/lib/python3.9/site-packages/sprockets/http/app.py\", line 104, in start\n callback(self.tornado_application, io_loop)\n"," File \"/usr/local/lib/python3.9/site-packages/bulk_tagging/app.py\", line 151, in before_run\n raise RuntimeError('AWS credentials not found')\n","RuntimeError: AWS credentials not found\n"]
Nothing is set in consul yet, and it's concerning that it's also complaining about CONSUL_HTTP_ADDR
.