Skip to content

Commit

Permalink
New version
Browse files Browse the repository at this point in the history
  • Loading branch information
@alfred82santa
alfred82santa committed May 23, 2016
1 parent 8b65a71 commit 1642abe
Show file tree
Hide file tree
Showing 12 changed files with 1,303 additions and 390 deletions.
185 changes: 184 additions & 1 deletion README.rst
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,190 @@
|travis-master| |coverall-master| |doc-master| |pypi-downloads| |pypi-lastrelease| |python-versions|
|project-status| |project-license| |project-format| |project-implementation|

.. |travis-master| image:: https://travis-ci.org/alfred82santa/aio-service-client.svg?branch=master
:target: https://travis-ci.org/alfred82santa/aio-service-client

.. |coverall-master| image:: https://coveralls.io/repos/alfred82santa/aio-service-client/badge.svg?branch=master&service=github
:target: https://coveralls.io/r/alfred82santa/aio-service-client?branch=master

.. |doc-master| image:: https://readthedocs.org/projects/aio-service-client/badge/?version=latest
:target: http://aio-service-client.readthedocs.io/?badge=latest
:alt: Documentation Status

.. |pypi-downloads| image:: https://img.shields.io/pypi/dm/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Downloads

.. |pypi-lastrelease| image:: https://img.shields.io/pypi/v/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Latest Version

.. |python-versions| image:: https://img.shields.io/pypi/pyversions/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Supported Python versions

.. |project-status| image:: https://img.shields.io/pypi/status/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Development Status

.. |project-license| image:: https://img.shields.io/pypi/l/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: License

.. |project-format| image:: https://img.shields.io/pypi/format/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Download format

.. |project-implementation| image:: https://img.shields.io/pypi/implementation/aio-service-client.svg
:target: https://pypi.python.org/pypi/aio-service-client/
:alt: Supported Python implementations


========================
Service Client Framework
========================

Service Client Framework powered by Python asyncio.

The easiest way to implement a client to work with a service API REST.
The easiest way to implement a client to work with a service REST API.

Features
========

- Easy way to make request to service.
- AsyncIO implementation using aiohttp.
- Powerful plugin system.
- Useful plugins.
- Mock plugin in order to make tests.
- Opensource license: GNU LGPLv3

Installation
============

.. code-block:: bash
$ pip install aio-service-client
Getting started
===============

Service client framework is used to call HTTP service API's. So, you must define how to
communicate with this service API defining its endpoint:

.. code-block:: python
spec = {"get_users": {"path": "/user",
"method": "get"},
"get_user_detail": {"path": "/user/{user_id}",
"method": "get"},
"create_user": {"path": "/user",
"method": "post"},
"update_user": {"path": "/user/{user_id}",
"method": "put"}}
Imagine you are using a Rest JSON API in order to manage users. So, you data must be sent
as a JSON and response must be a JSON string. It mean you must serialize every request payload
to a JSON, and parse every response as JSON. So, you only need to define JSON parser and serializer
for your service:

.. code-block:: python
service = ServiceClient(spec=spec,
plugins=[PathToken()],
base_path="http://example.com",
parser=json_decoder,
serializer=json_encoder)
So, you are ready to make request to service API:

.. code-block:: python
resp = yield from service.call("get_users")
# it could be called directly
# resp = yield from service.get_users()
# if response is like:
# {"users": {"item": [{"userId": "12", "username": "foo"}, {"userId": "13", "username": "bar"}], "count": 2}
print("Count: %d" % resp.data['users']['count'])
for user in resp.data['users']['items']:
print("User `%s`: %s" % (user['userId'], user['username']))
In order to send a payload you must use ``payload`` keyword on call:

.. code-block:: python
resp = yield from service.call("create_user", payload={"username": "foobar"})
# it could be called directly
# resp = yield from service.create_user(payload={"username": "foobar"})
# it will make a request like:
# POST http://example.com/user
#
# {"username": "foobar"}
Plugins
=======

PathTokens
----------

It allows to fill placeholders on path in order to build uri.

.. code-block:: python
service = ServiceClient(spec={"endpoint1": {"method": "get",
"path": "/endpoint/{placeholder1}/{placeholder2}"}},
plugins=[PathToken()],
base_path="http://example.com")
resp = yield from service.call("endpoint1", placeholder1=21, placeholder1="foo")
# It will make request:
# GET http://example.com/endpoint/21/foo
Headers
-------

It allows to define default headers, endpoint headers and request headers.


.. code-block:: python
service = ServiceClient(spec={"endpoint1": {"method": "get",
"path": "/endpoint/{placeholder1}/{placeholder2}",
"headers": {"X-fake-header": "header; data"}}},
plugins=[Headers(headers={"content-type": "application/json"})],
base_path="http://example.com")
resp = yield from service.call("endpoint1", headers={"X-other-fake-header": "foo"})
# It will make request:
# GET http://example.com/endpoint/21/foo
# X-fake-header: header; data
# content-type: application/json
# X-other-fake-header: foo
Timeout
-------

It allows to define default timeout for service request, endpoint or request.

Elapsed
-------

It adds elapsed time to response.

QueryParams
-----------

It allows to use query parameters on request. They could be defined at service client, endpoint or request.

InnerLogger
-----------

It allows to log request after serialize and response before parse.

OuterLogger
-----------

It allows to log request before serialize and response after parse.
94 changes: 47 additions & 47 deletions service_client/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@ def set_custom_session(self, session):

class ServiceClient:

def __init__(self, rest_service_name='GenericService', spec=None, plugins=None, config=None,
def __init__(self, name='GenericService', spec=None, plugins=None, config=None,
parser=None, serializer=None, base_path='', loop=None, logger=None):
self._plugins = []

self.logger = logger or logging.getLogger('serviceClient.{}'.format(rest_service_name))
self.rest_service_name = rest_service_name
self.logger = logger or logging.getLogger('serviceClient.{}'.format(name))
self.name = name
self.spec = spec or {}
self.add_plugins(plugins or [])
self.config = config or {}
Expand All @@ -52,77 +52,77 @@ def __init__(self, rest_service_name='GenericService', spec=None, plugins=None,
self.session = ClientSession(connector=self.connector, loop=self.loop)

@coroutine
def call(self, service_name, payload=None, **kwargs):
self.logger.debug("Calling service_client {0}...".format(service_name))
service_desc = self.spec[service_name].copy()
service_desc['service_name'] = service_name
def call(self, endpoint, payload=None, **kwargs):
self.logger.debug("Calling service {0}...".format(endpoint))
endpoint_desc = self.spec[endpoint].copy()
endpoint_desc['endpoint'] = endpoint

request_params = kwargs
session = yield from self.prepare_session(service_desc, request_params)
session = yield from self.prepare_session(endpoint_desc, request_params)

request_params['url'] = yield from self.generate_path(service_desc, session, request_params)
request_params['method'] = service_desc.get('method', 'GET').upper()
request_params['url'] = yield from self.generate_path(endpoint_desc, session, request_params)
request_params['method'] = endpoint_desc.get('method', 'GET').upper()

yield from self.prepare_request_params(service_desc, session, request_params)
yield from self.prepare_request_params(endpoint_desc, session, request_params)

self.logger.info("Calling service_client {0} using {1} {2}".format(service_name,
request_params['method'],
request_params['url']))
self.logger.info("Calling service {0} using {1} {2}".format(endpoint,
request_params['method'],
request_params['url']))

payload = yield from self.prepare_payload(service_desc, session, request_params, payload)
payload = yield from self.prepare_payload(endpoint_desc, session, request_params, payload)
try:
if request_params['method'] not in ['GET', 'DELETE']:
try:
stream_request = service_desc['stream_request']
stream_request = endpoint_desc['stream_request']
except KeyError:
stream_request = False
if payload and not stream_request:
request_params['data'] = self.serializer(payload, session=session,
service_desc=service_desc,
endpoint_desc=endpoint_desc,
request_params=request_params)

yield from self.before_request(service_desc, session, request_params)
yield from self.before_request(endpoint_desc, session, request_params)

response = yield from session.request(**request_params)
except Exception as e:
self.logger.warn("Exception calling service_client {0}: {1}".format(service_name, e))
yield from self.on_exception(service_desc, session, request_params, e)
self.logger.warn("Exception calling service {0}: {1}".format(endpoint, e))
yield from self.on_exception(endpoint_desc, session, request_params, e)
raise e

yield from self.on_response(service_desc, session, request_params, response)
yield from self.on_response(endpoint_desc, session, request_params, response)

try:
if service_desc['stream_response']:
if endpoint_desc['stream_response']:
return response
except KeyError:
pass

try:
self.logger.info("Parsing response from {0}...".format(service_name))
self.logger.info("Parsing response from {0}...".format(endpoint))
response.data = self.parser((yield from response.read()),
session=session,
service_desc=service_desc,
endpoint_desc=endpoint_desc,
response=response)
yield from self.on_parsed_response(service_desc, session, request_params, response)
yield from self.on_parsed_response(endpoint_desc, session, request_params, response)
except Exception as e:
self.logger.warn("[Response code: {0}] Exception parsing response from service_client "
"{1}: {2}".format(response.status, service_name, e))
yield from self.on_parse_exception(service_desc, session, request_params, response, e)
self.logger.warn("[Response code: {0}] Exception parsing response from service "
"{1}: {2}".format(response.status, endpoint, e))
yield from self.on_parse_exception(endpoint_desc, session, request_params, response, e)
e.response = response
raise e

return response

@coroutine
def prepare_session(self, service_desc, request_params):
def prepare_session(self, endpoint_desc, request_params):
session = SessionWrapper(self.session)
yield from self._execute_plugin_hooks('prepare_session', service_desc=service_desc, session=session,
yield from self._execute_plugin_hooks('prepare_session', endpoint_desc=endpoint_desc, session=session,
request_params=request_params)
return session

@coroutine
def generate_path(self, service_desc, session, request_params):
path = service_desc.get('path', '')
def generate_path(self, endpoint_desc, session, request_params):
path = endpoint_desc.get('path', '')
url = list(urlparse(self.base_path))
url[2] = '/'.join([url[2].rstrip('/'), path.lstrip('/')])
url.pop()
Expand All @@ -131,49 +131,49 @@ def generate_path(self, service_desc, session, request_params):
if hasattr(plugin, 'prepare_path')]
self.logger.debug("Calling {0} plugin hooks...".format('prepare_path'))
for func in hooks:
path = yield from func(service_desc=service_desc, session=session,
path = yield from func(endpoint_desc=endpoint_desc, session=session,
request_params=request_params, path=path)

return path

@coroutine
def prepare_request_params(self, service_desc, session, request_params):
yield from self._execute_plugin_hooks('prepare_request_params', service_desc=service_desc,
def prepare_request_params(self, endpoint_desc, session, request_params):
yield from self._execute_plugin_hooks('prepare_request_params', endpoint_desc=endpoint_desc,
session=session, request_params=request_params)

@coroutine
def prepare_payload(self, service_desc, session, request_params, payload):
def prepare_payload(self, endpoint_desc, session, request_params, payload):
hooks = [getattr(plugin, 'prepare_payload') for plugin in self._plugins
if hasattr(plugin, 'prepare_payload')]
self.logger.debug("Calling {0} plugin hooks...".format('prepare_payload'))
for func in hooks:
payload = yield from func(service_desc=service_desc, session=session,
payload = yield from func(endpoint_desc=endpoint_desc, session=session,
request_params=request_params, payload=payload)
return payload

@coroutine
def before_request(self, service_desc, session, request_params):
yield from self._execute_plugin_hooks('before_request', service_desc=service_desc,
def before_request(self, endpoint_desc, session, request_params):
yield from self._execute_plugin_hooks('before_request', endpoint_desc=endpoint_desc,
session=session, request_params=request_params)

@coroutine
def on_exception(self, service_desc, session, request_params, ex):
yield from self._execute_plugin_hooks('on_exception', service_desc=service_desc,
def on_exception(self, endpoint_desc, session, request_params, ex):
yield from self._execute_plugin_hooks('on_exception', endpoint_desc=endpoint_desc,
session=session, request_params=request_params, ex=ex)

@coroutine
def on_response(self, service_desc, session, request_params, response):
yield from self._execute_plugin_hooks('on_response', service_desc=service_desc,
def on_response(self, endpoint_desc, session, request_params, response):
yield from self._execute_plugin_hooks('on_response', endpoint_desc=endpoint_desc,
session=session, request_params=request_params, response=response)

@coroutine
def on_parse_exception(self, service_desc, session, request_params, response, ex):
yield from self._execute_plugin_hooks('on_parse_exception', service_desc=service_desc,
def on_parse_exception(self, endpoint_desc, session, request_params, response, ex):
yield from self._execute_plugin_hooks('on_parse_exception', endpoint_desc=endpoint_desc,
session=session, request_params=request_params, response=response, ex=ex)

@coroutine
def on_parsed_response(self, service_desc, session, request_params, response):
yield from self._execute_plugin_hooks('on_parsed_response', service_desc=service_desc, session=session,
def on_parsed_response(self, endpoint_desc, session, request_params, response):
yield from self._execute_plugin_hooks('on_parsed_response', endpoint_desc=endpoint_desc, session=session,
request_params=request_params, response=response)

@coroutine
Expand Down
Loading

0 comments on commit 1642abe

Please sign in to comment.