Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put python-lambda under poetry #6

Merged
merged 13 commits into from
Mar 12, 2020
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ __pycache__/

# Distribution / packaging
.Python
env/
*env/
build/
develop-eggs/
dist/
Expand Down Expand Up @@ -57,3 +57,10 @@ docs/_build/

# PyBuilder
target/

# PyCharm
.idea/

# Emacs backup files
*~

5 changes: 3 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ sudo: false
python:
- '3.6'
install:
- pip install -r requirements.txt
- pip install -r dev-requirements.txt
- pip install --upgrade pip==19.0.3
- pip install poetry
- poetry install
- pip install coveralls
script:
- pytest --cov aws_lambda -v tests
Expand Down
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,24 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

================================================================================
Portions of this software are based in part on the work of Nick Ficano.
Use that earlier work requires inclusion of this additional notice,
which continues to apply to those portions:

ISC License

Copyright (c) 2016, Nick Ficano

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
5 changes: 2 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ lint:
flake8 python-lambda tests

release: clean
python setup.py sdist upload
python setup.py bdist_wheel upload
poetry publish

install: clean
python setup.py install
poetry install
123 changes: 101 additions & 22 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
========
python-λ
========
=================
python-lambda-4dn
=================

(python-λ forked for 4DN-DCIC projects)
---------------------------------------

.. image:: https://img.shields.io/pypi/v/python-lambda.svg

.. image:: https://img.shields.io/pypi/v/python-lambda-4dn.svg
:alt: Pypi
:target: https://pypi.python.org/pypi/python-lambda/
:target: https://pypi.python.org/pypi/python-lambda-4dn/

.. image:: https://travis-ci.org/4dn-dcic/python-lambda.svg?branch=master
:alt: Build Status
Expand All @@ -15,34 +18,88 @@ python-λ
:alt: Coverage
:target: https://coveralls.io/github/4dn-dcic/python-lambda?branch=master

Python-lambda is a toolset for developing and deploying *serverless* Python code in AWS Lambda.
This is a toolset for developing and deploying *serverless* Python code in AWS Lambda.

.. Important::

Important
=======================
This is a FORK of the original Python-lambda package by Nick Ficano.
It will NOT be updated regularly and is frozen per our projects needs.
This is a FORK of Nick Ficano's `Python-lambda <https://pypi.python.org/pypi/python-lambda>`_
package. It will NOT be updated regularly and is frozen for the needs of projects at the
`4D Nucleome Data Coordination and Integration Center (4DN-DCIC)
<https://github.com/4dn-dcic>`_.

Description
===========

AWS Lambda is a service that allows you to write Python, Java, or Node.js code that gets executed in response to events like http requests or files uploaded to S3.
AWS Lambda is a service that allows you to write Python, Java, or Node.js code that
gets executed in response to events like http requests or files uploaded to S3.

Working with Lambda is relatively easy, but the process of bundling and deploying your code
is not as simple as it could be.

The *Python-Lambda* library takes away the guess work of developing your Python-Lambda
services by providing you a toolset to streamline the annoying parts.

Important Legal Notice
======================

Working with Lambda is relatively easy, but the process of bundling and deploying your code is not as simple as it could be.
The original `Python-lambda <https://pypi.python.org/pypi/python-lambda>`_ is licensed under
an ISCL license. `The version of that license active at time of the fork is here
<https://github.com/nficano/python-lambda/blob/01f1b8c3651de4e772618851b2117277ca95b1b4/LICENSE>`_.
Github's summary of that license describes it as:

The *Python-Lambda* library takes away the guess work of developing your Python-Lambda services by providing you a toolset to streamline the annoying parts.
A permissive license lets people do anything with your code with proper attribution
and without warranty. The ISC license is functionally equivalent to the BSD 2-Clause
and MIT licenses, removing some language that is no longer necessary.

Requirements
============
Since our derivative work is covered under the MIT license, and on a theory
that the underlying license is equivalent to the MIT license,
we shorthand our licensing requirements as just "MIT" because that's more consistent
with how we describe licensing for other 4DN-DCIC software.
However, for the properly formal legal detail,
please refer to our actual `LICENSE <LICENSE>`_.

System Requirements
===================

* Python 3.6
* Pip (Any should work)
* Virtualenv (>=15.0.0)
* Virtualenvwrapper (>=4.7.1)

Setting Up a Virtual Environment (OPTIONAL)
===========================================

This is optional.
If you do not create a virtual environment, Poetry will make one for you.
But there are still good reasons you might want to make your own, so here
are three ways to do it:

* If you have virtualenvwrapper that knows to use Python 3.6::

mkvirtualenv myenv

* If you have virtualenv but not virtualenvwrapper, and you have python3.6 in your ``PATH``::

virtualenv myenv -p python3.6

* If you are using ``pyenv`` to control what environment you use::

pyenv exec python -m venv myenv


Installing Poetry in a Virtual Environment
==========================================

Once you have created a virtual environment, or have decided to just let Poetry handle that,
install with poetry::

poetry install


Getting Started
===============

Using this library is intended to be as straightforward as possible. Code for a very simple lambda used in the tests is reproduced below.
Using this library is intended to be as straightforward as possible.
Code for a very simple lambda used in the tests is reproduced below.

.. code:: python

Expand All @@ -57,11 +114,13 @@ Using this library is intended to be as straightforward as possible. Code for a
'description': 'Test lambda'
}


def handler(event, context):
return 'Hello! My input event is %s' % event

This code illustrates the two things required to create a lambda. The first is ``config``, which specifies metadata for AWS. One important thing to note in here is the ``role`` field. This must be a IAM role with Lambda permissions - the one in this example is ours. The second is the ``handler`` function. This is the actual code that is executed.
This code illustrates the two things required to create a lambda. The first is ``config``,
which specifies metadata for AWS. One important thing to note in here is the ``role`` field.
This must be a IAM role with Lambda permissions - the one in this example is ours.
The second is the ``handler`` function. This is the actual code that is executed.

Given this code in ``example_function.py`` you would deploy this function like so:

Expand All @@ -75,14 +134,34 @@ Given this code in ``example_function.py`` you would deploy this function like s
requirements_fpath='path/to/requirements',
extra_config={'optional_arguments_for': 'boto3'})

And that's it! You've deployed a simple lambda function. You can navigate to the AWS console to create a test event to trigger it or you can invoke it directly using Boto3.
And that's it! You've deployed a simple lambda function. You can navigate to the AWS
console to create a test event to trigger it or you can invoke it directly using Boto3.

Advanced Usage
==============

Many of the options specified in the above code block when it came to actually deploying the function are not used. These become more useful as you want to make more complicated lambda functions. The ideal way to incorporate dependencies into lambda functions is by providing a ``requirements.txt`` file. We rely on ``pip`` to install these packages and have found it to be very reliable. While it is also possible to specify local modules as well through ``package_objects``, doing so is not recommended because those modules must be specified at the top level of the repository in order to work out of the box. There is a comment on this topic in ``example_function_package.py`` with code on how to handle it.
Many of the options specified in the above code block when it came to actually
deploying the function are not used. These become more useful as you want to make more
complicated lambda functions. The ideal way to incorporate dependencies into lambda functions
is by providing a ``requirements.txt`` file. We rely on ``pip`` to install these packages
and have found it to be very reliable. While it is also possible to specify local modules
as well through ``package_objects``, doing so is not recommended because those modules
must be specified at the top level of the repository in order to work out of the box.
There is a comment on this topic in ``example_function_package.py``
with code on how to handle it.

Tests
========

Tests can be found in the ``test_aws_lambda.py``. Using the tests as a guide to develop your lambdas is probably a good idea. You can also see how to invoke the lambdas directly from Python (and interpret the response).
Tests can be found in the ``test_aws_lambda.py``. Using the tests as a guide to develop
your lambdas is probably a good idea. You can also see how to invoke the lambdas directly
from Python (and interpret the response). You can invoke all of this by just doing::

pytest

The usual ``pytest`` arguments are permited. For example, to invoke an individual test,
mention its name. To see verbose output, use ``-v``; or use ``-vv`` for extra-verbose output,
as in::

pytest -vv -k test_deploy_lambda_with_package_and_requirements

4 changes: 3 additions & 1 deletion aws_lambda/_version.py → aws_lambda/_version.py.DISABLED
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""Version information."""

# Information in this file is now superseded by the pyproject.toml file.

# The following line *must* be the last in the module, exactly as formatted:
__version__ = "1.0.1"
__version__ = "2.0.0b1"
2 changes: 0 additions & 2 deletions dev-requirements.txt

This file was deleted.

4 changes: 4 additions & 0 deletions dev-requirements.txt.DISABLED
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pytest>=3.0.5,<4
pytest-cov>=2.3.1,<3
# pygments is only needed by PyCharm to render the README
pygments>=2