Merge pull request #1 from sprockets/initial-version

Initial version
This commit is contained in:
Gavin M. Roy 2015-06-12 15:57:59 -04:00
commit df926bab64
17 changed files with 725 additions and 23 deletions

24
.travis.yml Normal file
View File

@ -0,0 +1,24 @@
language: python
python:
- 2.6
- 2.7
- pypy
- 3.2
- 3.3
- 3.4
install:
- if [[ $TRAVIS_PYTHON_VERSION == '2.6' ]]; then pip install -r requirements-2.6.txt; fi
- pip install -r requirements.txt
script: nosetests
after_success:
- coveralls
deploy:
distributions: sdist bdist_wheel
provider: pypi
on:
python: 2.7
tags: true
all_branches: true
user: sprockets
password:
secure: VjpiEYdgOh+FXy0Xh7hoz15OQaViHo0JwkQAZh41EC5CGtLz4hUT3sUxxR0hsBLdFALv7j8yf68UyzyvKho1Rc1apAydDyzGduFBk9jsc9U3ZDjQ2rF+ROWjdjB0PN+yx1BJEui4YeK5W2hCzP9iPs/KRPuce8+UESTDyfZB3Kc=

41
LICENSE
View File

@ -1,22 +1,25 @@
The MIT License (MIT)
Copyright (c) 2014 AWeber Communications
All rights reserved.
Copyright (c) 2014 Sprockets
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Sprockets nor the names of its contributors may be used to
endorse or promote products derived from this software without specific
prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

2
MANIFEST.in Normal file
View File

@ -0,0 +1,2 @@
include LICENSE
include README.rst

View File

@ -1,4 +0,0 @@
sprockets.cli
=============
Command line tool for starting a Sprockets application

105
README.rst Normal file
View File

@ -0,0 +1,105 @@
sprockets.cli
=============
The sprockets CLI interface for running applications. Applications are meant
to be run by a controller that is managed by the sprockets CLI interface.
The sprockets CLI interface loads controller applications that are registered
using setuptools entry points.
|Version| |Downloads| |Status| |Coverage| |License|
Example CLI Usage
-----------------
Help:
.. code::
# sprockets --help
usage: sprockets [-h] [--apps] [--plugins] [-e [PLUGIN]] [-s] [-v] [--version]
CONTROLLER ... [APP]
Command line tool for starting a Sprockets application
positional arguments:
CONTROLLER Available sprockets application controllers
http HTTP Application Controller
amqp RabbitMQ Worker Controller
APP Application to run
optional arguments:
-h, --help show this help message and exit
--apps List installed applications
--plugins List installed plugins
-e [PLUGIN], --enable [PLUGIN]
Enable a plugin
-s, --syslog Log to syslog
-v, --verbose Verbose logging output, use -vv for DEBUG level
logging
--version show program's version number and exit
Find more Sprockets controllers and plugins at
https://sprockets.readthedocs.org
Starting a Web App with the NewRelic plugin:
.. code::
# sprockets -e newrelic http my_web_app
Controllers
-----------
Each controller is expected to expose at least a ``main(application, args)``
method that would be invoked when starting the application. Additional, a
controller can implement a ``add_cli_arguments(parser)`` method that will be
invoked when setting up the command line parameters. This allows controllers
to inject configuration directives into the cli.
Controller API Summary:
.. code:: python
module.add_cli_arguments(ArgumentParser) # optional
module.main(app_module, argparse.Namespace)
Plugins
-------
Plugins are able to inject themselves at multiple points in the application
lifecycle. Plugins that implement a ``initialization(controller)`` method will
see that method invoked before a controller is started. In addition, if a
``on_startup(controller)`` method is defined, it will be invoked after a
Controller has started a application. Finally if a ``on_shutdown(controller)``
method is defined, it will be invoked when a controller has shutdown.
Plugin API Summary:
.. code:: python
plugin.initialize(controller_module) # optional
plugin.on_start(controller_module) # optional
plugin.on_shutdown(controller_module) # optional
Applications
------------
Applications can be a python package or module and if they are registered
to a specific controller, can be referenced by an alias. Application contracts
vary by controller.
.. |Version| image:: https://badge.fury.io/py/sprockets.cli.svg?
:target: http://badge.fury.io/py/sprockets.cli
.. |Status| image:: https://travis-ci.org/sprockets/sprockets.cli.svg?branch=master
:target: https://travis-ci.org/sprockets/sprockets.cli
.. |Coverage| image:: https://coveralls.io/repos/sprockets/sprockets.cli/badge.png
:target: https://coveralls.io/r/sprockets/sprockets.cli
.. |Downloads| image:: https://pypip.in/d/sprockets.cli/badge.svg?
:target: https://pypi.python.org/pypi/sprockets.cli
.. |License| image:: https://pypip.in/license/sprockets.cli/badge.svg?
:target: https://sprockets.readthedocs.org

2
nose.cfg Normal file
View File

@ -0,0 +1,2 @@
[nosetests]
verbosity=3

4
requirements-2.6.txt Normal file
View File

@ -0,0 +1,4 @@
argparse
importlib
logutils
unittest2

5
requirements.txt Normal file
View File

@ -0,0 +1,5 @@
tornado>=4.0
coverage
mock
nose
python-coveralls

2
setup.cfg Normal file
View File

@ -0,0 +1,2 @@
[bdist_wheel]
universal = 1

49
setup.py Normal file
View File

@ -0,0 +1,49 @@
from setuptools import setup
import sys
requirements = []
tests_require = ['coverage', 'coveralls', 'nose']
# Requirements for Python 2.6
if sys.version_info < (2, 7):
requirements.append('argparse')
requirements.append('importlib')
requirements.append('logutils')
tests_require.append('unittest2')
if sys.version_info < (3, 0):
tests_require.append('mock')
setup(name='sprockets.cli',
version='0.1.0',
description='Sprockets command line application runner',
entry_points={'console_scripts': ['sprockets=sprockets.cli:main']},
author='AWeber Communications',
url='https://github.com/sprockets/sprockets.cli',
install_requires=requirements,
license=open('LICENSE').read(),
namespace_packages=['sprockets'],
package_data={'': ['LICENSE', 'README.rst']},
packages=['sprockets', 'sprockets.cli'],
classifiers=[
'Development Status :: 3 - Alpha',
'Environment :: No Input/Output (Daemon)',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Natural Language :: English',
'Operating System :: OS Independent',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy',
'Topic :: Software Development :: Libraries',
'Topic :: Software Development :: Libraries :: Python Modules'
],
test_suite='nose.collector',
tests_require=tests_require,
zip_safe=False)

1
sprockets/__init__.py Normal file
View File

@ -0,0 +1 @@
__import__('pkg_resources').declare_namespace(__name__)

407
sprockets/cli/__init__.py Normal file
View File

@ -0,0 +1,407 @@
"""
Sprockets CLI
=============
The sprockets CLI interface for running applications. Applications are meant
to be run by a controller that is managed by the sprockets CLI interface.
The sprockets CLI interface loads controller applications that are registered
using setuptools entry points.
Controllers
-----------
Each controller is expected to expose at least a ``main(application, args)``
method that would be invoked when starting the application. Additional, a
controller can implement a ``add_cli_arguments(parser)`` method that will be
invoked when setting up the command line parameters. This allows controllers
to inject configuration directives into the cli.
Controller API Summary:
.. code: python
module.add_cli_arguments(ArgumentParser) # optional
module.main(app_module, argparse.Namespace)
Plugins
-------
Plugins are able to inject themselves at multiple points in the application
lifecycle. Plugins that implement a ``initialization(controller)`` method will
see that method invoked before a controller is started. In addition, if a
``on_startup(controller)`` method is defined, it will be invoked after a
Controller has started a application. Finally if a ``on_shutdown(controller)``
method is defined, it will be invoked when a controller has shutdown.
Plugin API Summary:
.. code: python
plugin.initialize(controller_module) # optional
plugin.on_start(controller_module) # optional
plugin.on_shutdown(controller_module) # optional
Applications
------------
Applications can be a python package or module and if they are registered
to a specific controller, can be referenced by an alias. Application contracts
vary by controller.
"""
version_info = (0, 1, 0)
__version__ = '.'.join(str(v) for v in version_info)
import argparse
import importlib
import json
import logging
import string
import sys
# import logutils for Python 2.6 or logging.config for later versions
if sys.version_info < (2, 7):
import logutils.dictconfig as logging_config
else:
from logging import config as logging_config
import pkg_resources
APP_DESC = 'Command line tool for starting a Sprockets application'
DESCRIPTION = 'Available sprockets application controllers'
EPILOG = ('Find more Sprockets controllers and plugins at '
'https://sprockets.readthedocs.org')
# Logging formatters
SYSLOG_FORMAT = ('%(levelname)s <PID %(process)d:%(processName)s> '
'%(name)s.%(funcName)s(): %(message)s')
VERBOSE_FORMAT = ('%(levelname) -10s %(asctime)s %(process)-6d '
'%(processName) -20s %(name) -20s '
'%(funcName) -20s L%(lineno)-6d: %(message)s')
# Base logging configuration
LOGGING = {'disable_existing_loggers': True,
'filters': {},
'formatters': {'syslog': {'format': SYSLOG_FORMAT},
'verbose': {'datefmt': '%Y-%m-%d %H:%M:%S',
'format': VERBOSE_FORMAT}},
'handlers': {'console': {'class': 'logging.StreamHandler',
'formatter': 'verbose'},
'syslog': {'class': 'logging.handlers.SysLogHandler',
'formatter': 'syslog'}},
'incremental': False,
'loggers': {'sprockets': {'handlers': ['console'],
'level': logging.WARNING,
'propagate': True}},
'root': {'handlers': [],
'level': logging.CRITICAL,
'propagate': True},
'version': 1}
LOGGER = logging.getLogger(__name__)
class CLI(object):
"""The core Sprockets CLI application providing argument parsing and
logic for starting a controller.
"""
CONTROLLERS = 'sprockets.controller'
LOGGER = 'sprockets'
PLUGINS = 'sprockets.plugin'
def __init__(self):
self._controllers = self._get_controllers()
self._plugins = self._get_plugins()
self.arg_parser = argparse.ArgumentParser(description=APP_DESC,
epilog=EPILOG)
self._add_cli_args()
self._args = self.arg_parser.parse_args()
def run(self):
"""Evaluate the command line arguments, performing the appropriate
actions so the application can be started.
"""
# The apps command prevents any other processing of args
if self._args.apps:
if not self._controllers:
print('ERROR: No application controllers installed\n')
sys.exit(1)
self._print_installed_apps(self._args.controller,
self._args.as_json)
sys.exit(0)
# The plugins command prevents any other processing of args
if self._args.plugins:
self._print_installed_plugins(self._args.as_json)
sys.exit(0)
# Make sure the specified controller is valid
if self._args.controller not in self._controllers:
sys.stderr.write('ERROR: Controller "%s" not found\n' %
self._args.controller)
sys.exit(-1)
# Make sure the specified plugins are valid
for plugin in self._args.enable:
if plugin not in self._plugins:
sys.stderr.write('ERROR: Plugin "%s" not found\n' % plugin)
sys.exit(-1)
# If app is not specified at this point, raise an error
if not self._args.application:
sys.stderr.write('ERROR: Application not specified\n\n')
self.arg_parser.print_help()
sys.exit(-1)
# If it's a registered app reference by name, get the module name
app_module = self._get_application_module(self._args.controller,
self._args.application)
# Configure logging based upon the flags
self._configure_logging(app_module,
self._args.verbose,
self._args.syslog)
# Shortcut to the controller module
controller = self._controllers[self._args.controller]
# Try and run plugin initialization
for plugin in self._args.enable:
try:
self._plugins[plugin].initialize(controller)
except AttributeError:
LOGGER.debug('Plugin %s.initialize() undefined', plugin)
# Try and run the controller
try:
self._controllers[self._args.controller].main(app_module,
self._args)
except TypeError as error:
sys.stderr.write('ERROR: could not start the %s controller for %s'
': %s\n' % (self._args.controller,
app_module,
str(error)))
sys.exit(-1)
# Try and run plugin initialization
for plugin in self._args.enable:
try:
self._plugins[plugin].on_shutdown(controller)
except AttributeError:
LOGGER.debug('Plugin %s.on_shutdown() undefined', plugin)
def _add_cli_args(self):
"""Add the cli arguments to the argument parser."""
self.arg_parser.add_argument('--apps',
action='store_true',
help='List installed applications')
self.arg_parser.add_argument('--plugins',
action='store_true',
help='List installed plugins')
self.arg_parser.add_argument('--machine-readable',
action='store_true',
dest='as_json',
help='Output application or plugin list '
'as JSON')
self.arg_parser.add_argument('-e', '--enable',
action='append',
metavar='PLUGIN',
default=[],
nargs="?",
help='Enable a plugin')
self.arg_parser.add_argument('-s', '--syslog',
action='store_true',
help='Log to syslog')
self.arg_parser.add_argument('-v', '--verbose',
action='count',
help=('Verbose logging output, use -vv '
'for DEBUG level logging'))
self.arg_parser.add_argument('--version',
action='version',
version='sprockets v%s ' % __version__)
# Controller sub-parser
if '--plugins' not in sys.argv:
subparsers = self.arg_parser.add_subparsers(dest='controller',
help=DESCRIPTION)
# Iterate through the controllers and add their cli arguments
for key in self._controllers:
help_text = self._get_controller_help(key)
sub_parser = subparsers.add_parser(key, help=help_text)
try:
self._controllers[key].add_cli_arguments(sub_parser)
except AttributeError:
LOGGER.debug('Controller %s.add_cli_arguments() undefined',
key)
# The application argument
if '--apps' not in sys.argv:
self.arg_parser.add_argument('application',
metavar='APP',
help='Application to run')
def _configure_logging(self, application, verbosity=0, syslog=False):
"""Configure logging for the application, setting the appropriate
verbosity and adding syslog if it's enabled.
:param str application: The application module/package name
:param int verbosity: 1 == INFO, 2 == DEBUG
:param bool syslog: Enable the syslog handler
"""
# Create a new copy of the logging config that will be modified
config = dict(LOGGING)
# Increase the logging verbosity
if verbosity == 1:
config['loggers'][self.LOGGER]['level'] = logging.INFO
elif verbosity == 2:
config['loggers'][self.LOGGER]['level'] = logging.DEBUG
# Add syslog if it's enabled
if syslog:
config['loggers'][self.LOGGER]['handlers'].append('syslog')
# Copy the sprockets logger to the application
config['loggers'][application] = dict(config['loggers']['sprockets'])
# Configure logging
logging_config.dictConfig(config)
def _get_application_module(self, controller, application):
"""Return the module for an application. If it's a entry-point
registered application name, return the module name from the entry
points data. If not, the passed in application name is returned.
:param str controller: The controller type
:param str application: The application name or module
:rtype: str
"""
for pkg in self._get_applications(controller):
if pkg.name == application:
return pkg.module_name
return importlib.import_module(application)
@staticmethod
def _get_applications(controller):
"""Return a list of application names for the given controller type
that have registered themselves as sprockets applications.
:param str controller: The type of controller for the applications
:rtype: list
"""
group_name = 'sprockets.%s.app' % controller
return pkg_resources.iter_entry_points(group=group_name)
@staticmethod
def _get_argument_parser():
"""Return an instance of the argument parser.
:return: argparse.ArgumentParser
"""
return argparse.ArgumentParser()
def _get_controllers(self):
"""Iterate through the installed controller entry points and import
the modules, returning the dict to be assigned to the CLI._controllers
dict.
:return: dict
"""
return self._get_package_resources(self.CONTROLLERS)
def _get_controller_help(self, controller):
"""Return the value of the HELP attribute for a controller that should
describe the functionality of the controller.
:rtype: str|None
"""
if hasattr(self._controllers[controller], 'HELP'):
return self._controllers[controller].HELP
return None
@staticmethod
def _get_package_resources(group):
"""Iterate through the installed entry points for the specified group,
importing each package, returning a dict of handles by package name.
:return: dict
"""
packages = dict()
for pkg in pkg_resources.iter_entry_points(group=group):
packages[pkg.name] = importlib.import_module(pkg.module_name)
return packages
def _get_plugins(self):
"""Iterate through the installed plugin entry points and import
the modules, returning the dict to be assigned to the CLI._plugins
dict.
:return: dict
"""
return self._get_package_resources(self.PLUGINS)
def _print_installed_apps(self, controller, as_json):
"""Print out a list of installed sprockets applications
:param str controller: The name of the controller to get apps for
:param bool as_json: Output the data as json
"""
if as_json:
apps = [app.name for app in self._get_applications(controller)]
print(json.dumps({controller: apps}))
return
print('Installed Sprockets {0} Apps\n'.format(controller.upper()))
print("{0:<25} {1:>25}".format('Name', 'Module'))
print(string.ljust('', 51, '-'))
for app in self._get_applications(controller):
print('{0:<25} {1:>25}'.format(app.name,
'({0})'.format(app.module_name)))
print('')
def _print_installed_plugins(self, as_json):
"""Print out a list of installed plugin packages
:param bool as_json: Output the data as json
"""
if as_json:
print(json.dumps({'plugins': [p.name for p in self._plugins]}))
return
if not self._plugins:
print('There are no plugins installed\n')
return
print('Installed Sprockets Plugins\n')
print("{0:<25} {1:>25}".format('Name', 'Module'))
print(string.ljust('', 51, '-'))
for loader in self._plugins:
print('{0:<25} {1:>25}'.format(loader.name,
'({0})'.format(loader.module_name)))
print('')
def main():
"""Main application runner"""
cli = CLI()
cli.run()

1
sstubs/__init__.py Normal file
View File

@ -0,0 +1 @@
__author__ = 'gavinr'

8
sstubs/app.py Normal file
View File

@ -0,0 +1,8 @@
"""
Stub Controller for testing
"""
def run():
pass

12
sstubs/controller.py Normal file
View File

@ -0,0 +1,12 @@
"""
Stub Controller for testing
"""
def add_cli_arguments(parser):
pass
def main(app, args):
pass

21
sstubs/plugin.py Normal file
View File

@ -0,0 +1,21 @@
"""
Stub Plugin Module
"""
PRIORITY = 50
def add_cli_arguments(parser):
pass
def initialize(controller):
pass
def on_start(controller):
pass
def on_shutdown(controller):
pass

60
tests.py Normal file
View File

@ -0,0 +1,60 @@
"""
Test the Sprockets Command Line Interface
"""
try:
import unittest2 as unittest
except ImportError:
import unittest
import mock
from sprockets import cli
class Package(object):
def __init__(self, name, module_name):
self.name = name
self.module_name = module_name
class InitializationTests(unittest.TestCase):
@mock.patch('argparse.ArgumentParser.parse_args')
@mock.patch('pkg_resources.iter_entry_points')
def setUp(self, iter_entry_points, parse_args):
self.iter_entry_points = iter_entry_points
self.parse_args = parse_args
self.app_points = [Package('tapp', 'sstubs.app')]
self.ctrl_points = [Package('tcontroller', 'sstubs.controller')]
self.plugin_points = [Package('tplugin', 'sstubs.plugin')]
def entry_point_side_effect(*args, **kwargs):
if kwargs.get('group') == 'sprockets.controller':
return iter(self.ctrl_points)
elif kwargs.get('group') == 'sprockets.plugin':
return iter(self.plugin_points)
elif kwargs.get('group') == 'sprockets.test_http.app':
return iter(self.app_points)
self.iter_entry_points.side_effect = entry_point_side_effect
self.obj = cli.CLI()
def test_pkg_resources_iterated(self):
calls = [mock.call(group='sprockets.controller'),
mock.call(group='sprockets.plugin')]
self.iter_entry_points.assert_has_calls(calls)
def test_controller_imported(self):
for attr in ['add_cli_arguments', 'main']:
self.assertTrue(hasattr(self.obj._controllers.get('tcontroller'),
attr))
def test_plugin_is_imported(self):
for attr in ['initialize', 'on_start', 'on_shutdown']:
self.assertTrue(hasattr(self.obj._plugins.get('tplugin'), attr))