Add support for Py3.12; drop support for Py3.7 (#1292)

* Add support for Py3.12; drop support for Py3.7

* Attempt to fix coveralls

.github/ISSUE_TEMPLATE.md

@@ -1,7 +1,7 @@
 <!--- Provide a general summary of the issue in the Title above -->
 ## Context
 <!--- Provide a more detailed introduction to the issue itself, and why you consider it to be a bug -->
-<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 3.7/3.8/3.9/3.10/3.11 -->
+<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 3.8/3.9/3.10/3.11/3.12 -->
 
 ## Expected Behavior
 <!--- Tell us what should happen -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -20,7 +20,7 @@ Before you submit this PR, please make sure that you meet these criteria:
 
 * Did you **make sure this code actually works on Lambda**, as well as locally?
 
-* Did you test this code with all of **Python 3.7**, **Python 3.8**, **Python 3.9**, **Python 3.10** and **Python 3.11** ?
+* Did you test this code with all of: **Python 3.8**, **Python 3.9**, **Python 3.10**, **Python 3.11**, and **Python 3.12**?
 
 * Does this commit ONLY relate to the issue at hand and have your linter shit all over the code?
 

.github/workflows/cd.yaml

@@ -11,14 +11,14 @@ on:  # yamllint disable-line rule:truthy
 
 jobs:
   publish:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     steps:
       - name: Checkout Code Repository
-        uses: actions/checkout@v3
-      - name: Set up Python 3.11
-        uses: actions/setup-python@v4
+        uses: actions/checkout@v4
+      - name: Set up Python 3.12
+        uses: actions/setup-python@v5
         with:
-          python-version: "3.11"
+          python-version: "3.12"
       - name: Install `pypa/build`
         run: python -m pip install build
       - name: Build sdist and wheel

.github/workflows/ci.yaml

@@ -8,15 +8,15 @@ on:  # yamllint disable-line rule:truthy
 
 jobs:
   test:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     strategy:
       matrix:
-        python: [3.7, 3.8, 3.9, "3.10", "3.11"]
+        python: ["3.8", "3.9", "3.10", "3.11", "3.12"]
     steps:
       - name: Checkout Code Repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Set up Python ${{ matrix.python }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
       - uses: actions/cache@v3
@@ -40,13 +40,14 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           COVERALLS_FLAG_NAME: py${{ matrix.python }}
           COVERALLS_PARALLEL: true
+          COVERALLS_SERVICE_JOB_ID: ${{ github.run_id }}
 
   coverage:
     needs: test
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     steps:
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
       - name: Finalise Coverage
         run: pip3 install --upgrade coveralls && coveralls --service=github --finish
         env:

README.md

@@ -133,7 +133,7 @@ __Awesome!__
 
 ## Installation and Configuration
 
-_Before you begin, make sure you are running Python 3.7/3.8/3.9/3.10/3.11 and you have a valid AWS account and your [AWS credentials file](https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs) is properly installed._
+_Before you begin, make sure you are running Python 3.8/3.9/3.10/3.11/3.12 and you have a valid AWS account and your [AWS credentials file](https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs) is properly installed._
 
 **Zappa** can easily be installed through pip, like so:
 
@@ -443,7 +443,7 @@ For instance, suppose you have a basic application in a file called "my_app.py",
 
 Any remote print statements made and the value the function returned will then be printed to your local console. **Nifty!**
 
-You can also invoke interpretable Python 3.7/3.8/3.9/3.10/3.11 strings directly by using `--raw`, like so:
+You can also invoke interpretable Python 3.8/3.9/3.10/3.11/3.12 strings directly by using `--raw`, like so:
 
     $ zappa invoke production "print(1 + 2 + 3)" --raw
 
@@ -985,7 +985,7 @@ to change Zappa's behavior. Use these at your own risk!
         "role_name": "MyLambdaRole", // Name of Zappa execution role. Default <project_name>-<env>-ZappaExecutionRole. To use a different, pre-existing policy, you must also set manage_roles to false.
         "role_arn": "arn:aws:iam::12345:role/app-ZappaLambdaExecutionRole", // ARN of Zappa execution role. Default to None. To use a different, pre-existing policy, you must also set manage_roles to false. This overrides role_name. Use with temporary credentials via GetFederationToken.
         "route53_enabled": true, // Have Zappa update your Route53 Hosted Zones when certifying with a custom domain. Default true.
-        "runtime": "python3.11", // Python runtime to use on Lambda. Can be one of "python3.7", "python3.8", "python3.9", or "python3.10", or "python3.11". Defaults to whatever the current Python being used is.
+        "runtime": "python3.12", // Python runtime to use on Lambda. Can be one of: "python3.8", "python3.9", "python3.10", "python3.11", or "python3.12". Defaults to whatever the current Python being used is.
         "s3_bucket": "dev-bucket", // Zappa zip bucket,
         "slim_handler": false, // Useful if project >50M. Set true to just upload a small handler to Lambda and load actual project from S3 at runtime. Default false.
         "settings_file": "~/Projects/MyApp/settings/dev_settings.py", // Server side settings file location,

setup.py

@@ -44,15 +44,15 @@
         "License :: OSI Approved :: Apache Software License",
         "Operating System :: OS Independent",
         "Programming Language :: Python",
-        "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",
+        "Programming Language :: Python :: 3.12",
         "Framework :: Django",
-        "Framework :: Django :: 1.11",
-        "Framework :: Django :: 2.0",
-        "Framework :: Django :: 3.0",
+        "Framework :: Django :: 3.2",
+        "Framework :: Django :: 4.2",
+        "Framework :: Django :: 5.0",
         "Topic :: Internet :: WWW/HTTP",
         "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
     ],

tests/placebo/TestZappa.test_cli_aws/lambda.GetFunctionConfiguration_2.json

@@ -15,7 +15,7 @@
         },
         "FunctionName": "zappa-ttt888",
         "FunctionArn": "arn:aws:lambda:us-east-1:004396165043:function:zappa-ttt888",
-        "Runtime": "python3.6",
+        "Runtime": "python3.8",
         "Role": "arn:aws:iam::004396165043:role/zappa-ttt888-ZappaLambdaExecutionRole",
         "Handler": "handler.lambda_handler",
         "CodeSize": 16975308,

tests/tests.py

@@ -92,28 +92,12 @@ def test_disable_click_colors(self):
     def test_create_lambda_package(self):
         # mock the pkg_resources.WorkingSet() to include a known package in lambda_packages so that the code
         # for zipping pre-compiled packages gets called
-        mock_installed_packages = {"psycopg2": "2.6.1"}
+        mock_installed_packages = {"psycopg": "3.1.17"}
         with mock.patch(
             "zappa.core.Zappa.get_installed_packages",
             return_value=mock_installed_packages,
         ):
-            z = Zappa(runtime="python3.7")
-            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
-            self.assertTrue(os.path.isfile(path))
-            os.remove(path)
-
-    def test_get_manylinux_python37(self):
-        z = Zappa(runtime="python3.7")
-        self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2", "2.7.6"))
-        self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))
-
-        # mock with a known manylinux wheel package so that code for downloading them gets invoked
-        mock_installed_packages = {"psycopg2": "2.7.6"}
-        with mock.patch(
-            "zappa.core.Zappa.get_installed_packages",
-            return_value=mock_installed_packages,
-        ):
-            z = Zappa(runtime="python3.7")
+            z = Zappa(runtime="python3.8")
             path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
             self.assertTrue(os.path.isfile(path))
             os.remove(path)
@@ -199,17 +183,59 @@ def test_get_manylinux_python310(self):
             self.assertTrue(os.path.isfile(path))
             os.remove(path)
 
-    def test_verify_python37_does_not_download_2_24_manylinux_wheel(self):
-        z = Zappa(runtime="python3.7")
-        cached_wheels_dir = os.path.join(tempfile.gettempdir(), "cached_wheels")
-        expected_wheel_path = os.path.join(
-            cached_wheels_dir, "cryptography-35.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl"
-        )
+    def test_get_manylinux_python311(self):
+        z = Zappa(runtime="python3.11")
+        self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
+        self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))
 
-        # Check with known manylinux wheel package
-        actual_wheel_path = z.get_cached_manylinux_wheel("cryptography", "35.0.0")
-        self.assertEqual(actual_wheel_path, expected_wheel_path)
-        os.remove(actual_wheel_path)
+        # mock with a known manylinux wheel package so that code for downloading them gets invoked
+        mock_installed_packages = {"psycopg2-binary": "2.9.7"}
+        with mock.patch(
+            "zappa.core.Zappa.get_installed_packages",
+            return_value=mock_installed_packages,
+        ):
+            z = Zappa(runtime="python3.11")
+            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
+            self.assertTrue(os.path.isfile(path))
+            os.remove(path)
+
+        # same, but with an ABI3 package
+        mock_installed_packages = {"cryptography": "2.8"}
+        with mock.patch(
+            "zappa.core.Zappa.get_installed_packages",
+            return_value=mock_installed_packages,
+        ):
+            z = Zappa(runtime="python3.11")
+            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
+            self.assertTrue(os.path.isfile(path))
+            os.remove(path)
+
+    def test_get_manylinux_python312(self):
+        z = Zappa(runtime="python3.12")
+        self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg-binary", "3.1.17"))
+        self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))
+
+        # mock with a known manylinux wheel package so that code for downloading them gets invoked
+        mock_installed_packages = {"psycopg-binary": "3.1.17"}
+        with mock.patch(
+            "zappa.core.Zappa.get_installed_packages",
+            return_value=mock_installed_packages,
+        ):
+            z = Zappa(runtime="python3.12")
+            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
+            self.assertTrue(os.path.isfile(path))
+            os.remove(path)
+
+        # same, but with an ABI3 package
+        mock_installed_packages = {"cryptography": "41.0.7"}
+        with mock.patch(
+            "zappa.core.Zappa.get_installed_packages",
+            return_value=mock_installed_packages,
+        ):
+            z = Zappa(runtime="python3.12")
+            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
+            self.assertTrue(os.path.isfile(path))
+            os.remove(path)
 
     def test_verify_downloaded_manylinux_wheel(self):
         z = Zappa(runtime="python3.10")
@@ -260,33 +286,6 @@ def test_verify_manylinux_filename_is_lowered(self):
         cached_pypi_info_dir = os.path.join(tempfile.gettempdir(), "cached_pypi_info")
         os.remove(os.path.join(cached_pypi_info_dir, "markupsafe-2.1.3.json"))
 
-    def test_get_manylinux_python311(self):
-        z = Zappa(runtime="python3.11")
-        self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
-        self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))
-
-        # mock with a known manylinux wheel package so that code for downloading them gets invoked
-        mock_installed_packages = {"psycopg2-binary": "2.9.7"}
-        with mock.patch(
-            "zappa.core.Zappa.get_installed_packages",
-            return_value=mock_installed_packages,
-        ):
-            z = Zappa(runtime="python3.11")
-            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
-            self.assertTrue(os.path.isfile(path))
-            os.remove(path)
-
-        # same, but with an ABI3 package
-        mock_installed_packages = {"cryptography": "2.8"}
-        with mock.patch(
-            "zappa.core.Zappa.get_installed_packages",
-            return_value=mock_installed_packages,
-        ):
-            z = Zappa(runtime="python3.11")
-            path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
-            self.assertTrue(os.path.isfile(path))
-            os.remove(path)
-
     def test_get_exclude_glob__file_not_deleted(self):
         z = Zappa(runtime="python3.11")
         self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
@@ -309,7 +308,7 @@ def test_get_exclude_glob__file_not_deleted(self):
                 os.remove(path)
 
     def test_getting_installed_packages(self, *args):
-        z = Zappa(runtime="python3.7")
+        z = Zappa(runtime="python3.8")
 
         # mock pkg_resources call to be same as what our mocked site packages dir has
         mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
@@ -344,7 +343,7 @@ def test_get_current_venv(self, *args):
                 self.assertEqual(current_venv, None)
 
     def test_getting_installed_packages_mixed_case_location(self, *args):
-        z = Zappa(runtime="python3.7")
+        z = Zappa(runtime="python3.8")
 
         # mock pip packages call to be same as what our mocked site packages dir has
         mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
@@ -367,7 +366,7 @@ def test_getting_installed_packages_mixed_case_location(self, *args):
                     )
 
     def test_getting_installed_packages_mixed_case(self, *args):
-        z = Zappa(runtime="python3.7")
+        z = Zappa(runtime="python3.8")
 
         # mock pkg_resources call to be same as what our mocked site packages dir has
         mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
@@ -2581,7 +2580,7 @@ def test_minor_version_only_check_when_in_docker(self, *_):
             reload(zappa)
 
     @mock.patch("os.getenv", return_value="True")
-    @mock.patch("sys.version_info", new_callable=partial(get_sys_versioninfo, 7))
+    @mock.patch("sys.version_info", new_callable=partial(get_sys_versioninfo, 8))
     def test_no_runtimeerror_when_in_docker(self, *_):
         from importlib import reload
 

zappa/__init__.py

@@ -12,8 +12,8 @@ def running_in_docker() -> bool:
     return running_in_docker_flag
 
 
-SUPPORTED_VERSIONS = [(3, 7), (3, 8), (3, 9), (3, 10), (3, 11)]
-MINIMUM_SUPPORTED_MINOR_VERSION = 7
+SUPPORTED_VERSIONS = [(3, 8), (3, 9), (3, 10), (3, 11), (3, 12)]
+MINIMUM_SUPPORTED_MINOR_VERSION = 8
 
 if not running_in_docker() and sys.version_info[:2] not in SUPPORTED_VERSIONS:
     print(running_in_docker())

zappa/core.py

@@ -279,7 +279,7 @@ def __init__(
         load_credentials=True,
         desired_role_name=None,
         desired_role_arn=None,
-        runtime="python3.7",  # Detected at runtime in CLI
+        runtime="python3.8",  # Detected at runtime in CLI
         tags=(),
         endpoint_urls={},
         xray_tracing=False,
@@ -304,27 +304,10 @@ def __init__(
 
         self.runtime = runtime
 
-        if self.runtime == "python3.7":
-            self.manylinux_suffix_start = "cp37m"
-        elif self.runtime == "python3.8":
-            # The 'm' has been dropped in python 3.8+ since builds with and without pymalloc are ABI compatible
-            # See https://github.com/pypa/manylinux for a more detailed explanation
-            self.manylinux_suffix_start = "cp38"
-        elif self.runtime == "python3.9":
-            self.manylinux_suffix_start = "cp39"
-        elif self.runtime == "python3.10":
-            self.manylinux_suffix_start = "cp310"
-        else:
-            self.manylinux_suffix_start = "cp311"
-
-        # AWS Lambda supports manylinux1/2010, manylinux2014, and manylinux_2_24
-        # Currently python3.7 lambda runtime does not support manylinux_2_24
-        # See https://github.com/zappa/Zappa/issues/1249 for more details
-        if self.runtime == "python3.7":
-            self.manylinux_suffixes = ("2014", "2010", "1")
-        else:
-            self.manylinux_suffixes = ("_2_24", "2014", "2010", "1")
-
+        # TODO: Support PEP600 properly (https://peps.python.org/pep-0600/)
+        self.manylinux_suffix_start = f"cp{self.runtime[6:].replace('.', '')}"
+        self.manylinux_suffixes = ("_2_24", "2014", "2010", "1")
+        # TODO: Support aarch64 architecture
         self.manylinux_wheel_file_match = re.compile(
             rf'^.*{self.manylinux_suffix_start}-(manylinux_\d+_\d+_x86_64[.])?manylinux({"|".join(self.manylinux_suffixes)})_x86_64[.]whl$'  # noqa: E501
         )
@@ -1100,7 +1083,7 @@ def create_lambda_function(
         publish=True,
         vpc_config=None,
         dead_letter_config=None,
-        runtime="python3.7",
+        runtime="python3.8",
         aws_environment_variables=None,
         aws_kms_key_arn=None,
         xray_tracing=False,
@@ -1285,7 +1268,7 @@ def update_lambda_configuration(
         ephemeral_storage={"Size": 512},
         publish=True,
         vpc_config=None,
-        runtime="python3.7",
+        runtime="python3.8",
         aws_environment_variables=None,
         aws_kms_key_arn=None,
         layers=None,

zappa/utilities.py

@@ -209,15 +209,19 @@ def get_runtime_from_python_version():
         raise ValueError("Python 2.x is no longer supported.")
     else:
         if sys.version_info[1] <= 7:
-            return "python3.7"
-        elif sys.version_info[1] <= 8:
+            raise ValueError("Python 3.7 and below are no longer supported.")
+        elif sys.version_info[1] == 8:
             return "python3.8"
-        elif sys.version_info[1] <= 9:
+        elif sys.version_info[1] == 9:
             return "python3.9"
-        elif sys.version_info[1] <= 10:
+        elif sys.version_info[1] == 10:
             return "python3.10"
-        else:
+        elif sys.version_info[1] == 11:
             return "python3.11"
+        elif sys.version_info[1] == 12:
+            return "python3.12"
+        else:
+            raise ValueError(f"Python f{'.'.join(str(v) for v in sys.version_info[:2])} is not yet supported.")
 
 
 ##