Merge branch 'master' into improve_update

.github/ISSUE_TEMPLATE.md

@@ -1,7 +1,7 @@
 <!--- Provide a general summary of the issue in the Title above -->
 ## Context
 <!--- Provide a more detailed introduction to the issue itself, and why you consider it to be a bug -->
-<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 2.7/3.6 -->
+<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 3.6/3.7/3.8 -->
 
 ## Expected Behavior
 <!--- Tell us what should happen -->
@@ -25,4 +25,4 @@
 * Operating System and Python version:
 * The output of `pip freeze`:
 * Link to your project (optional):
-* Your `zappa_settings.py`: 
+* Your `zappa_settings.json`: 

.github/PULL_REQUEST_TEMPLATE.md

@@ -20,7 +20,7 @@ Before you submit this PR, please make sure that you meet these criteria:
 
 * Did you **make sure this code actually works on Lambda**, as well as locally?
 
-* Did you test this code with both **Python 2.7** and **Python 3.6**? 
+* Did you test this code with all of **Python 3.6**, **Python 3.7** and **Python 3.8** ? 
 
 * Does this commit ONLY relate to the issue at hand and have your linter shit all over the code?
 

.gitignore

@@ -87,3 +87,7 @@ zappa_settings.json
 *.sublime-workspace
 
 README.test.md
+.vscode
+
+.mypy_cache
+.pytest_cache

.travis.yml

@@ -1,7 +1,9 @@
 language: python
 python:
-  - "2.7"
   - "3.6"
+  - "3.7"
+  - "3.8"
+dist: xenial
 # command to install dependencies
 cache:
   - pip

CHANGELOG.md

@@ -1,5 +1,46 @@
 # Zappa Changelog
 
+## 0.50.0
+* Updated manylinux to support manylinux1/manylinux2010/manylinux2014 packages
+* Fixed issue with duplicated cookie headers in API Gateway
+* Various maintenance updates to the README file
+* Remove old python 2.7 code (not all yet)
+* Use pip-tools to manage dependencies (and remove pinned dependencies from the package)
+* Add option for concurrency limit
+* Use safe_load with YAML
+* Support for ABI3 binary packages (particularly cryptography)
+* Remove dependency on lambda-packages
+
+## 0.49.0
+* Added support for Python 3.8
+* Fixed boto3 reporting of long names
+* Added support for new AWS Regions
+* Minor bug and README fixes
+
+## 0.48.2
+* Last version with Python 2.7 support.
+* Fix for invalid values of HTTP_HOST and others (introduced in 0.48.0)
+
+## 0.48.1
+* Fix for an issue where you couldn't update a function if it had been deployed with a version prior to 0.48.0
+* Support for private (VPC-only) API Gateway endpoints
+
+## 0.48.0
+* Deprecation of zappa.async because async is a reserved word in 3.7, please use zappa.asynchronous
+* Support for Application Load Balancer as an alternative to API Gateway
+* Improvements in manylinux wheel downloads
+* Support for multi-value query strings
+* Python 3.7 support
+* Support for multi-value headers
+* Support for the new 256kB async limit
+* Bunch of small typos, fixes and documentation improvements
+* Use Django 2 on tests with python 3
+* Stop applying tags on buckets not created by zappa
+* Be more forgiving on 504 errors during deployment
+* Fix cleanup of DynamoDB table for async tasks
+* Detection of invalid bucket names
+* Flake8 checks added to tests
+
 ## 0.47.1
 * Handle accounts with more than 100 domains in route 53
 * Force requests upgrade to 2.20.0 for compatibility with urllib3 1.24

MANIFEST.in

@@ -1,2 +1,2 @@
-include README.md LICENSE requirements.txt test_requirements.txt
+include README.md LICENSE requirements.txt test_requirements.txt requirements.in test_requirements.in
 recursive-include zappa *.py

docs/domain_with_free_ssl_dns.md

@@ -82,4 +82,4 @@ If you want your certificates to renew automatically, use simply need to define
 }
 ```
 
-The only caveat with this is that your functions `timeout_seconds` must be greater than `60`, as that's how long it takes the DNS to propagate. The auto-renewer will be installed whenvever you next `update` or `schedule`, though you may need to re`deploy` to up your `timeout_seconds`.
+The only caveat with this is that your functions `timeout_seconds` must be greater than `60`, as that's how long it takes the DNS to propagate. The auto-renewer will be installed whenever you next `update` or `schedule`, though you may need to re`deploy` to up your `timeout_seconds`.

example/authmodule.py

@@ -4,8 +4,6 @@
      http://aws.amazon.com/apache2.0/
 or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 """
-from __future__ import print_function
-
 import re
 import time
 import pprint
@@ -70,7 +68,7 @@ class HttpVerb:
     OPTIONS = "OPTIONS"
     ALL     = "*"
 
-class AuthPolicy(object):
+class AuthPolicy:
     awsAccountId = ""
     """The AWS account id the policy will be generated for. This is used to create the method ARNs."""
     principalId = ""
@@ -215,4 +213,4 @@ def build(self):
         policy['policyDocument']['Statement'].extend(self._getStatementForEffect("Allow", self.allowMethods))
         policy['policyDocument']['Statement'].extend(self._getStatementForEffect("Deny", self.denyMethods))
 
-        return policy
+        return policy

example/mymodule.py

@@ -1,6 +1,3 @@
-from __future__ import print_function
-
-
 def myfunc():
     print('Running my function in a schedule!')
 

requirements.in

@@ -0,0 +1,21 @@
+argcomplete
+boto3
+durationpy
+hjson
+jmespath
+kappa==0.6.0
+pip>=9.0.1
+python-dateutil<2.7.0
+python-slugify
+PyYAML
+future
+# previous version don't work with urllib3 1.24
+requests>=2.20.0
+six
+toml
+tqdm
+troposphere
+Werkzeug
+wheel
+wsgi-request-logger
+pip-tools

requirements.sh

@@ -0,0 +1,7 @@
+#! /bin/bash
+
+set -e
+
+pip-compile -o test_requirements.txt requirements.in test_requirements.in
+cp test_requirements.txt requirements.txt
+pip-compile -o requirements.txt requirements.in

requirements.txt

@@ -1,24 +1,41 @@
-argcomplete==1.9.3
-botocore>=1.7.19
-boto3>=1.4.7
-docutils>=0.12
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --output-file=requirements.txt requirements.in
+#
+argcomplete==1.11.1
+boto3==1.12.1
+botocore==1.15.1          # via boto3, s3transfer
+certifi==2019.11.28       # via requests
+cfn-flip==1.2.2           # via troposphere
+chardet==3.0.4            # via requests
+click==7.0                # via cfn-flip, kappa, pip-tools
+docutils==0.15.2          # via botocore
 durationpy==0.5
-future==0.16.0
-futures==3.2.0; python_version < '3'
+future==0.18.2
 hjson==3.0.1
-jmespath==0.9.3
+idna==2.8                 # via requests
+importlib-metadata==1.5.0  # via argcomplete
+jmespath==0.9.4
 kappa==0.6.0
-lambda-packages==0.20.0
-pip>=9.0.1
-python-dateutil>=2.6.1, <2.7.0
-python-slugify==1.2.4
-PyYAML>=3.13
-# previous version don't work with urllib3 1.24
-requests>=2.20.0
-six>=1.11.0
-toml>=0.9.4
-tqdm==4.19.1
-troposphere>=1.9.0
-Werkzeug>=0.14
-wheel>=0.30.0
+pip-tools==4.4.1
+placebo==0.9.0            # via kappa
+python-dateutil==2.6.1
+python-slugify==4.0.0
+pyyaml==5.3
+requests==2.22.0
+s3transfer==0.3.3         # via boto3
+six==1.14.0
+text-unidecode==1.3       # via python-slugify
+toml==0.10.0
+tqdm==4.42.1
+troposphere==2.5.3
+urllib3==1.25.8           # via botocore, requests
+werkzeug==1.0.0
+wheel==0.34.2
 wsgi-request-logger==0.4.6
+zipp==3.0.0               # via importlib-metadata
+
+# The following packages are considered to be unsafe in a requirements file:
+# pip

setup.py

@@ -7,20 +7,10 @@
 with open('README.md') as readme_file:
     long_description = readme_file.read()
 
-with open(os.path.join(os.path.dirname(__file__), 'requirements.txt')) as f:
-    if sys.version_info[0] == 2:
-        required = f.read().splitlines()
-    else:
-        # This logic is intended to prevent the futures package from being installed in python 3 environments
-        # as it can cause unexpected syntax errors in other packages. Futures is in the standard library in python 3
-        # and is should never be installed in these environments.
-        # Related: https://github.com/Miserlou/Zappa/issues/1179
-        required = []
-        for package in f.read().splitlines():
-            if 'futures' not in package:
-                required.append(package)
+with open(os.path.join(os.path.dirname(__file__), 'requirements.in')) as f:
+    required = f.read().splitlines()
 
-with open(os.path.join(os.path.dirname(__file__), 'test_requirements.txt')) as f:
+with open(os.path.join(os.path.dirname(__file__), 'test_requirements.in')) as f:
     test_required = f.read().splitlines()
 
 setup(
@@ -49,11 +39,13 @@
         'License :: OSI Approved :: Apache Software License',
         'Operating System :: OS Independent',
         'Programming Language :: Python',
-        'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Framework :: Django',
         'Framework :: Django :: 1.11',
         'Framework :: Django :: 2.0',
+        'Framework :: Django :: 3.0',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
     ],

test_requirements.in

@@ -0,0 +1,9 @@
+coveralls
+Django
+flake8
+Flask
+mock
+nose
+nose-timer
+placebo
+

test_requirements.txt

@@ -1,11 +1,60 @@
-coveralls>=1.1
-coverage>=4.3.1
-Django>=1.10.5, <2.0; python_version < '3'
-Django>=2.0; python_version >= '3'
-flake8>=3.6.0
-Flask>=0.12
-mock>=2.0.0
-nose>=1.3.7
-nose-timer==0.6.0
-placebo>=0.8.1
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --output-file=test_requirements.txt requirements.in test_requirements.in
+#
+argcomplete==1.11.1
+asgiref==3.2.3            # via django
+boto3==1.12.1
+botocore==1.15.1          # via boto3, s3transfer
+certifi==2019.11.28       # via requests
+cfn-flip==1.2.2           # via troposphere
+chardet==3.0.4            # via requests
+click==7.0                # via cfn-flip, flask, kappa, pip-tools
+coverage==5.0.3           # via coveralls
+coveralls==1.11.1
+django==3.0.3
+docopt==0.6.2             # via coveralls
+docutils==0.15.2          # via botocore
+durationpy==0.5
+entrypoints==0.3          # via flake8
+flake8==3.7.9
+flask==1.1.1
+future==0.18.2
+hjson==3.0.1
+idna==2.8                 # via requests
+importlib-metadata==1.5.0  # via argcomplete
+itsdangerous==1.1.0       # via flask
+jinja2==2.11.1            # via flask
+jmespath==0.9.4
+kappa==0.6.0
+markupsafe==1.1.1         # via jinja2
+mccabe==0.6.1             # via flake8
+mock==4.0.1
+nose-timer==0.7.5
+nose==1.3.7
+pip-tools==4.4.1
+placebo==0.9.0
+pycodestyle==2.5.0        # via flake8
+pyflakes==2.1.1           # via flake8
+python-dateutil==2.6.1
+python-slugify==4.0.0
+pytz==2019.3              # via django
+pyyaml==5.3
+requests==2.22.0
+s3transfer==0.3.3         # via boto3
+six==1.14.0
+sqlparse==0.3.0           # via django
+text-unidecode==1.3       # via python-slugify
+toml==0.10.0
+tqdm==4.42.1
+troposphere==2.5.3
+urllib3==1.25.8           # via botocore, requests
+werkzeug==1.0.0
+wheel==0.34.2
+wsgi-request-logger==0.4.6
+zipp==3.0.0               # via importlib-metadata
 
+# The following packages are considered to be unsafe in a requirements file:
+# pip

test_settings.json

@@ -116,5 +116,9 @@
       "delete_local_zip": true,
       "use_precompiled_packages": false,
       "delete_s3_zip": false
-    }
+    },
+    "lambda_concurrency_enabled": {
+        "extends": "ttt888",
+        "lambda_concurrency": 6
+     }
 }

test_settings.py

@@ -1,6 +1,6 @@
 APP_MODULE = 'tests.test_app'
 APP_FUNCTION = 'hello_world'
-DJANGO_SETTINGS = None 
+DJANGO_SETTINGS = None
 DEBUG = 'True'
 LOG_LEVEL = 'DEBUG'
 SCRIPT_NAME = 'hello_world'
@@ -11,7 +11,7 @@
 REMOTE_ENV='s3://lmbda/test_env.json'
 ## test_env.json
 #{
-#	"hello": "world"
+#    "hello": "world"
 #}
 #
 
@@ -65,4 +65,3 @@ def authorizer_event(event, content):
 
 def command():
     print("command")
-

tests/placebo/TestZappa.test_cli_aws/lambda.DeleteFunctionConcurrency_1.json

@@ -0,0 +1,9 @@
+{
+    "status_code": 200, 
+    "data": {
+        "ResponseMetadata": {
+            "HTTPStatusCode": 200, 
+            "RequestId": "aff3a3f9-28f4-11e6-9dbb-5dd116b9ddf1"
+        }
+    }
+}

tests/placebo/TestZappa.test_cli_aws/lambda.GetAlias_1.json

@@ -0,0 +1,9 @@
+{
+    "status_code": 200,
+    "data": {
+        "AliasArn": "arn:aws:lambda:us-east-1:12345:function:test_lmbda_function55:current-alb-version",
+        "Description": "Zappa Deployment",
+        "FunctionVersion": "1",
+        "Name": "current-alb-version"
+    }
+}