Skip to content

Commit

Permalink
chore(sdk): drop Python 3.7 for kfp sdk (#10750)
Browse files Browse the repository at this point in the history
* chore(components): Drop GCPC Python 3.7
PiperOrigin-RevId: 627500444

**Description of your changes:**

**Checklist:**
- [ ] The title for your pull request (PR) should follow our title convention. [Learn more about the pull request title convention used in this repository](https://github.com/kubeflow/pipelines/blob/master/CONTRIBUTING.md#pull-request-title-convention).
<!--
   PR titles examples:
    * `fix(frontend): fixes empty page. Fixes #1234`
       Use `fix` to indicate that this PR fixes a bug.
    * `feat(backend): configurable service account. Fixes #1234, fixes #1235`
       Use `feat` to indicate that this PR adds a new feature.
    * `chore: set up changelog generation tools`
       Use `chore` to indicate that this PR makes some changes that users don't need to know.
    * `test: fix CI failure. Part of #1234`
        Use `part of` to indicate that a PR is working on an issue, but shouldn't close the issue when merged.
-->

* drop support python3.7

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix tes

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* fix test

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* remove kfp related

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* remove kfp related

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* just kfp change

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* only change in kubernetes_platform/python/setup.py

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* only change in init.py

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* change

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* change

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* Update release

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* Update base image

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* change test python 3.7 to python 3.8

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* update golden snapshot

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* update yaml to python3.8 under kubernetes_platform/python/test/snapshot/data/toleration.yaml

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* update yaml python 3.8 for sdk/python/test_data/pipelines/parallelfor_fan_in

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* update yaml to fix tests

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* Update Readme

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* Update kubernetes_setup python version

Signed-off-by: rickyxie0929 <rickyxie@google.com>

* Update readme

Signed-off-by: rickyxie0929 <rickyxie@google.com>

---------

Signed-off-by: rickyxie0929 <rickyxie@google.com>
  • Loading branch information
rickyxie0929 authored May 3, 2024
1 parent cd16a33 commit 2f3dd3d
Show file tree
Hide file tree
Showing 99 changed files with 491 additions and 491 deletions.
2 changes: 1 addition & 1 deletion kubernetes_platform/python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def read_readme() -> str:
'https://github.com/kubeflow/pipelines/tree/master/kubernetes_platform/python',
},
packages=setuptools.find_namespace_packages(include=['kfp.*']),
python_requires='>=3.7.0,<3.13.0',
python_requires='>=3.8.0,<3.13.0',
install_requires=REQUIREMENTS,
include_package_data=True,
extras_require={
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ deploymentSpec:
\ *\n\ndef consumer() -> str:\n with open('/data/file.txt', 'r') as file:\n\
\ content = file.read()\n print(content)\n return content\n\
\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down Expand Up @@ -150,7 +150,7 @@ deploymentSpec:
\ file.write('Hello world')\n with open('/data/file.txt', 'r')\
\ as file:\n content = file.read()\n print(content)\n return\
\ content\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down Expand Up @@ -141,7 +141,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef get_pvc_name() -> str:\n return 'static-pvc-name'\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
2 changes: 1 addition & 1 deletion kubernetes_platform/python/test/snapshot/data/timeout.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
1 change: 1 addition & 0 deletions sdk/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Features

## Breaking changes
* Drop support for Python 3.7 since it has reached end-of-life. [\#10750](https://github.com/kubeflow/pipelines/pull/10750)

## Deprecations

Expand Down
4 changes: 2 additions & 2 deletions sdk/python/kfp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
import sys
import warnings

if sys.version_info < (3, 8):
if sys.version_info < (3, 9):
warnings.warn(
('Python 3.7 has reached end-of-life. KFP will drop support for Python 3.7 on April 23, 2024. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3.8. See https://devguide.python.org/versions/ for more details.'
('KFP will drop support for Python 3.8 on Oct 1, 2024. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3.9. See https://devguide.python.org/versions/ for more details.'
),
FutureWarning,
stacklevel=2,
Expand Down
10 changes: 5 additions & 5 deletions sdk/python/kfp/cli/component_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ def test_docker_file_is_created_correctly(self):
textwrap.dedent('''\
# Generated by KFP.
FROM python:3.7
FROM python:3.8
WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -492,7 +492,7 @@ def test_docker_file_is_created_correctly_with_one_url(self):
textwrap.dedent('''\
# Generated by KFP.
FROM python:3.7
FROM python:3.8
WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -523,7 +523,7 @@ def test_docker_file_is_created_correctly_with_two_urls(self):
textwrap.dedent('''\
# Generated by KFP.
FROM python:3.7
FROM python:3.8
WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -566,7 +566,7 @@ def test_existing_dockerfile_can_be_overwritten(self):
textwrap.dedent('''\
# Generated by KFP.
FROM python:3.7
FROM python:3.8
WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -602,7 +602,7 @@ def test_dockerfile_can_contain_custom_kfp_package(self):
file_start = textwrap.dedent('''\
# Generated by KFP.
FROM python:3.7
FROM python:3.8
WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down
4 changes: 2 additions & 2 deletions sdk/python/kfp/dsl/component_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
from kfp.dsl.types import type_annotations
from kfp.dsl.types import type_utils

_DEFAULT_BASE_IMAGE = 'python:3.7'
_DEFAULT_BASE_IMAGE = 'python:3.8'
SINGLE_OUTPUT_NAME = 'Output'


Expand Down Expand Up @@ -537,7 +537,7 @@ def create_component_from_func(
if base_image is None:
base_image = _DEFAULT_BASE_IMAGE
warnings.warn(
("Python 3.7 has reached end-of-life. The default base_image used by the @dsl.component decorator will switch from 'python:3.7' to 'python:3.8' on April 23, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.8."
("The default base_image used by the @dsl.component decorator will switch from 'python:3.8' to 'python:3.9' on Oct 1, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.9."
),
FutureWarning,
stacklevel=2,
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/kfp/dsl/component_factory_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ def test_default_base_image(self):

with self.assertWarnsRegex(
FutureWarning,
r"Python 3\.7 has reached end-of-life\. The default base_image used by the @dsl\.component decorator will switch from 'python:3\.7' to 'python:3\.8' on April 23, 2024\. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3\.8\."
r"The default base_image used by the @dsl\.component decorator will switch from 'python:3\.8' to 'python:3\.9' on Oct 1, 2024\. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3\.9\."
):

@dsl.component
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/kfp/init_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def test(self):

with self.assertWarnsRegex(
FutureWarning,
r'Python 3\.7 has reached end-of-life\. KFP will drop support for Python 3\.7 on April 23, 2024\. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3\.8\. See https:\/\/devguide\.python\.org\/versions\/ for more details\.'
r'KFP will drop support for Python 3\.8 on Oct 1, 2024\. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3\.9\. See https:\/\/devguide\.python\.org\/versions\/ for more details\.'
):
# simulate first import from kfp
importlib.reload(mod)
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/kfp/local/docker_task_handler_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ def artifact_maker(x: str, a: Output[Artifact]):
kwargs = run_mock.call_args[1]
self.assertEqual(
kwargs['image'],
'python:3.7',
'python:3.8',
)
self.assertTrue(
any('def artifact_maker' in c for c in kwargs['command']))
Expand Down
3 changes: 1 addition & 2 deletions sdk/python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ def read_readme() -> str:
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
Expand All @@ -99,7 +98,7 @@ def read_readme() -> str:
'Topic :: Software Development :: Libraries',
'Topic :: Software Development :: Libraries :: Python Modules',
],
python_requires='>=3.7.0,<3.13.0',
python_requires='>=3.8.0,<3.13.0',
include_package_data=True,
entry_points={
'console_scripts': [
Expand Down
6 changes: 3 additions & 3 deletions sdk/python/test_data/components/add_numbers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -47,7 +47,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef add_numbers(a: int, b: int) -> int:\n return a + b\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: add-numbers
root:
Expand Down Expand Up @@ -83,4 +83,4 @@ root:
Output:
parameterType: NUMBER_INTEGER
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -72,7 +72,7 @@ deploymentSpec:
\ as f:\n content_b = f.read()\n\n concatenated_string = content_a\
\ + content_b\n with open(out_dataset.path, 'w') as f:\n f.write(concatenated_string)\n\
\n return concatenated_string\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: dataset-joiner
root:
Expand Down Expand Up @@ -126,4 +126,4 @@ root:
description: The concatenated string.
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ deploymentSpec:
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location --index-url https://pypi.org/simple\
\ --trusted-host https://pypi.org/simple 'kfp==2.1.3' '--no-deps' 'typing-extensions>=3.7.4,<5;\
\ --trusted-host https://pypi.org/simple 'kfp==2.7.0' '--no-deps' 'typing-extensions>=3.7.4,<5;\
\ python_version<\"3.9\"' && python3 -m pip install --quiet --no-warn-script-location\
\ --index-url https://pypi.org/simple --trusted-host https://pypi.org/simple\
\ 'yapf' && \"$0\" \"$@\"\n"
Expand All @@ -35,7 +35,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef component_with_pip_install():\n import yapf\n print(dir(yapf))\n\
\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: component-with-pip-install
root:
Expand All @@ -49,4 +49,4 @@ root:
taskInfo:
name: component-with-pip-install
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -40,7 +40,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef exit_comp(status: dsl.PipelineTaskFinalStatus):\n print(status)\n\
\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: exit-comp
root:
Expand All @@ -63,4 +63,4 @@ root:
isOptional: true
parameterType: TASK_FINAL_STATUS
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
6 changes: 3 additions & 3 deletions sdk/python/test_data/components/concat_message.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -48,7 +48,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef concat_message(message1: str, message2: str) -> str:\n return\
\ message1 + message2\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: concat-message
root:
Expand Down Expand Up @@ -84,4 +84,4 @@ root:
Output:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
2 changes: 1 addition & 1 deletion sdk/python/test_data/components/container_io.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ root:
output_path:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Loading

0 comments on commit 2f3dd3d

Please sign in to comment.