Skip to content

Commit

Permalink
Merge 756241d into 538df59
Browse files Browse the repository at this point in the history
  • Loading branch information
nolar committed Oct 8, 2023
2 parents 538df59 + 756241d commit c4925f8
Show file tree
Hide file tree
Showing 45 changed files with 70 additions and 119 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug-report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ body:
id: python-version
attributes:
label: Python version
placeholder: e.g. 3.9 or pypy-3.7-7.3.3
placeholder: e.g. 3.10 or pypy-3.10-7.3.13

- type: textarea
id: code
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
fail-fast: false
matrix:
install-extras: [ "", "full-auth" ]
python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11" ]
python-version: [ "3.8", "3.9", "3.10", "3.11" ]
include:
- install-extras: "uvloop"
python-version: "3.11"
Expand Down Expand Up @@ -81,7 +81,7 @@ jobs:
fail-fast: false
matrix:
install-extras: [ "", "full-auth" ]
python-version: [ "pypy-3.7", "pypy-3.8", "pypy-3.9" ]
python-version: [ "pypy-3.8", "pypy-3.9" ]
name: Python ${{ matrix.python-version }} ${{ matrix.install-extras }}
runs-on: ubuntu-22.04
timeout-minutes: 10
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/thorough.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
fail-fast: false
matrix:
install-extras: [ "", "full-auth" ]
python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11" ]
python-version: [ "3.8", "3.9", "3.10", "3.11" ]
include:
- install-extras: "uvloop"
python-version: "3.11"
Expand Down Expand Up @@ -85,7 +85,7 @@ jobs:
fail-fast: false
matrix:
install-extras: [ "", "full-auth" ]
python-version: [ "pypy-3.7", "pypy-3.8", "pypy-3.9" ]
python-version: [ "pypy-3.8", "pypy-3.9" ]
name: Python ${{ matrix.python-version }} ${{ matrix.install-extras }}
runs-on: ubuntu-22.04
timeout-minutes: 10
Expand Down
2 changes: 1 addition & 1 deletion DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ If you use PyCharm, create a Run/Debug Configuration as follows:
* Mode: `module name`
* Module name: `kopf`
* Arguments: `run examples/01-minimal/example.py --verbose`
* Python Interpreter: anything with Python>=3.7
* Python Interpreter: anything with Python>=3.8

Stop the console operator, and start the IDE debug session.
Put a breakpoint in the used operator script on the first line of the function.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ That easy! For more features, see the [documentation](https://kopf.readthedocs.i

## Usage

Python 3.7+ is required:
Python 3.8+ is required:
[CPython](https://www.python.org/) and [PyPy](https://www.pypy.org/)
are officially supported and tested; other Python implementations can work too.

Expand Down
2 changes: 1 addition & 1 deletion docs/deployment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ But normally, the operators are usually deployed directly to the clusters.
Docker image
============

First of all, the operator must be packaged as a docker image with Python 3.7 or newer:
First of all, the operator must be packaged as a docker image with Python 3.8 or newer:

.. code-block:: dockerfile
:caption: Dockerfile
Expand Down
2 changes: 1 addition & 1 deletion docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Installation

Prerequisites:

* Python >= 3.7 (CPython and PyPy are officially tested and supported).
* Python >= 3.8 (CPython and PyPy are officially tested and supported).

To install Kopf::

Expand Down
2 changes: 1 addition & 1 deletion docs/walkthrough/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ We need a running Kubernetes cluster and some tools for our experiments.
If you have a cluster already preconfigured, you can skip this section.
Otherwise, let's install minikube locally (e.g. for MacOS):

* Python >= 3.7 (running in a venv is recommended, though is not necessary).
* Python >= 3.8 (running in a venv is recommended, though is not necessary).
* `Install kubectl <https://kubernetes.io/docs/tasks/tools/install-kubectl/>`_
* :doc:`Install minikube </minikube>` (a local Kubernetes cluster)
* :doc:`Install Kopf </install>`
Expand Down
4 changes: 1 addition & 3 deletions kopf/_cogs/aiokits/aioenums.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
import time
from typing import Awaitable, Generator, Generic, Optional, TypeVar

from kopf._cogs.aiokits import aiotasks

FlagReasonT = TypeVar('FlagReasonT', bound=enum.Flag)


Expand Down Expand Up @@ -168,7 +166,7 @@ def __init__(self, waiter: AsyncFlagWaiter[FlagReasonT], *, timeout: Optional[fl
def __await__(self) -> Generator[None, None, AsyncFlagWaiter[FlagReasonT]]:
name = f"time-limited waiting for the daemon stopper {self._setter!r}"
coro = asyncio.wait_for(self._setter.async_event.wait(), timeout=self._timeout)
task = aiotasks.create_task(coro, name=name)
task = asyncio.create_task(coro, name=name)
try:
yield from task
except asyncio.TimeoutError:
Expand Down
26 changes: 7 additions & 19 deletions kopf/_cogs/aiokits/aiotasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@
so there is no added overhead; instead, the implicit overhead is made explicit.
"""
import asyncio
import sys
from typing import TYPE_CHECKING, Any, Awaitable, Callable, Collection, Coroutine, \
Generator, NamedTuple, Optional, Set, Tuple, TypeVar, Union
from typing import TYPE_CHECKING, Any, Callable, Collection, Coroutine, \
NamedTuple, Optional, Set, Tuple, TypeVar

from kopf._cogs.helpers import typedefs

Expand All @@ -27,17 +26,6 @@
Future = asyncio.Future
Task = asyncio.Task

# Accept `name=` always, but simulate it for Python 3.7 to do nothing.
if sys.version_info >= (3, 8):
create_task = asyncio.create_task
else:
def create_task(
coro: Union[Generator[Any, None, _T], Awaitable[_T]],
*,
name: Optional[str] = None, # noqa: W613 # pylint: disable=unused-argument
) -> Task:
return asyncio.create_task(coro)


async def cancel_coro(
coro: Coroutine[Any, Any, Any],
Expand Down Expand Up @@ -65,7 +53,7 @@ async def cancel_coro(
coro.close() # OR: coro.throw(asyncio.CancelledError())
except AttributeError:
# The official way is to create an extra task object, thus to waste some memory.
corotask = create_task(coro=coro, name=name)
corotask = asyncio.create_task(coro=coro, name=name)
corotask.cancel()
try:
await corotask
Expand Down Expand Up @@ -133,7 +121,7 @@ def create_guarded_task(
This is only a shortcut for named task creation (name is used in 2 places).
"""
return create_task(
return asyncio.create_task(
name=name,
coro=guard(
name=name,
Expand Down Expand Up @@ -303,8 +291,8 @@ def __init__(
self._pending_coros: asyncio.Queue[SchedulerJob] = asyncio.Queue()
self._running_tasks: Set[Task] = set()
self._cleaning_queue: asyncio.Queue[Task] = asyncio.Queue()
self._cleaning_task = create_task(self._task_cleaner(), name=f"task cleaner of {self!r}")
self._spawning_task = create_task(self._task_spawner(), name=f"task spawner of {self!r}")
self._cleaning_task = asyncio.create_task(self._task_cleaner(), name=f"cleaner of {self!r}")
self._spawning_task = asyncio.create_task(self._task_spawner(), name=f"spawner of {self!r}")

def empty(self) -> bool:
""" Check if the scheduler has nothing to do. """
Expand Down Expand Up @@ -371,7 +359,7 @@ async def _task_spawner(self) -> None:
# when they are finished --- to be awaited and released "passively".
while self._can_spawn():
coro, name = self._pending_coros.get_nowait() # guaranteed by the predicate
task = create_task(coro=coro, name=name)
task = asyncio.create_task(coro=coro, name=name)
task.add_done_callback(self._task_done_callback)
self._running_tasks.add(task)
if self._closed:
Expand Down
2 changes: 1 addition & 1 deletion kopf/_cogs/clients/watching.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ async def streaming_block(
# Create the signalling future for when paused again.
operator_pause_waiter: aiotasks.Future
if operator_paused is not None:
operator_pause_waiter = aiotasks.create_task(
operator_pause_waiter = asyncio.create_task(
operator_paused.wait_for(True),
name=f"pause-waiter for {resource}")
else:
Expand Down
2 changes: 1 addition & 1 deletion kopf/_cogs/helpers/typedefs.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Rudimentary type [re-]definitions for cross-versioned Python & mypy.
The problem is that new mypy versions often bring type-sheds with StdLib types
defined as generics, while the old Python runtime (down to 3.7)
defined as generics, while the old Python runtime (down to 3.8 & 3.9 & 3.10)
does not support the usual syntax.
Examples: asyncio.Task, asyncio.Future, logging.LoggerAdapter, and others.
Expand Down
5 changes: 0 additions & 5 deletions kopf/_core/actions/throttlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,6 @@ async def throttled(
try:
yield should_run

except asyncio.CancelledError:
# CancelledError is a BaseException in 3.8 & 3.9, but a regular Exception in 3.7.
# Behave as if it is a BaseException -- to enabled tests with async-timeout.
raise

except Exception as e:

# If it is not an error-of-interest, escalate normally. BaseExceptions are escalated always.
Expand Down
2 changes: 1 addition & 1 deletion kopf/_core/engines/daemons.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ async def spawn_daemons(
stopper=stopper, # for stopping (outside of causes)
handler=handler,
logger=loggers.LocalObjectLogger(body=cause.body, settings=settings),
task=aiotasks.create_task(_runner(
task=asyncio.create_task(_runner(
settings=settings,
daemons=daemons, # for self-garbage-collection
handler=handler,
Expand Down
9 changes: 4 additions & 5 deletions kopf/_core/reactor/running.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,17 +219,17 @@ async def spawn_tasks(
posting.settings_var.set(settings)

# A few common background forever-running infrastructural tasks (irregular root tasks).
tasks.append(aiotasks.create_task(
tasks.append(asyncio.create_task(
name="stop-flag checker",
coro=_stop_flag_checker(
signal_flag=signal_flag,
stop_flag=stop_flag)))
tasks.append(aiotasks.create_task(
tasks.append(asyncio.create_task(
name="ultimate termination",
coro=_ultimate_termination(
settings=settings,
stop_flag=stop_flag)))
tasks.append(aiotasks.create_task(
tasks.append(asyncio.create_task(
name="startup/cleanup activities",
coro=_startup_cleanup_activities(
root_tasks=tasks, # used as a "live" view, populated later.
Expand Down Expand Up @@ -433,8 +433,7 @@ async def _stop_flag_checker(
if signal_flag is not None:
flags.append(signal_flag)
if stop_flag is not None:
flags.append(aiotasks.create_task(aioadapters.wait_flag(stop_flag),
name="stop-flag waiter"))
flags.append(asyncio.create_task(aioadapters.wait_flag(stop_flag), name="stop-flag waiter"))

# Wait until one of the stoppers is set/raised.
try:
Expand Down
2 changes: 0 additions & 2 deletions pytest.ini
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
[pytest]
; The standalone `mock` instead of stdlib `unittest.mock` is only for AsyncMock in Python 3.7.
mock_use_standalone_module = true
asyncio_mode = auto
addopts =
--strict-markers
3 changes: 0 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ freezegun
import-linter
isort
lxml
# Generally, `unittest.mock` is enough, but it lacks `AsyncMock` for Py 3.7.
# TODO: Once 3.7 is removed (Jun 2023), roll back to unittest.mock.
mock
# Mypy requires typed-ast, which is broken on PyPy 3.7 (could work in PyPy 3.8).
mypy==1.2.0; implementation_name == "cpython"
pre-commit
Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
Expand All @@ -53,7 +52,7 @@
],
},

python_requires='>=3.7',
python_requires='>=3.8',
setup_requires=[
'setuptools_scm',
],
Expand Down
2 changes: 1 addition & 1 deletion tests/admission/test_admission_server.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import contextlib
from unittest.mock import Mock

import pytest
from mock import Mock

import kopf
from kopf._cogs.aiokits.aiovalues import Container
Expand Down
3 changes: 2 additions & 1 deletion tests/admission/test_serving_handler_selection.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from unittest.mock import Mock

import pytest
from mock import Mock

import kopf
from kopf._cogs.structs.ids import HandlerId
Expand Down
3 changes: 2 additions & 1 deletion tests/admission/test_serving_kwargs_passthrough.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from unittest.mock import Mock

import pytest
from mock import Mock

import kopf
from kopf._core.engines.admission import serve_admission_request
Expand Down
2 changes: 1 addition & 1 deletion tests/apis/test_iterjsonlines.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from mock import Mock
from unittest.mock import Mock

from kopf._cogs.clients.api import iter_jsonlines

Expand Down
2 changes: 1 addition & 1 deletion tests/basic-structs/test_memories.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from mock import Mock
from unittest.mock import Mock

from kopf._cogs.structs.bodies import Body
from kopf._cogs.structs.ephemera import Memo
Expand Down
2 changes: 1 addition & 1 deletion tests/causation/test_kwargs.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import dataclasses
import logging
from typing import Type
from unittest.mock import Mock

import pytest
from mock import Mock

from kopf._cogs.configs.configuration import OperatorSettings
from kopf._cogs.structs import diffs
Expand Down
2 changes: 1 addition & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
import sys
import time
from typing import Set
from unittest.mock import AsyncMock, Mock

import aiohttp.web
import pytest
from mock import AsyncMock, Mock

import kopf
from kopf._cogs.clients.auth import APIContext
Expand Down
2 changes: 1 addition & 1 deletion tests/handling/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@
"""
import dataclasses
from typing import Callable
from unittest.mock import Mock

import pytest
from mock import Mock

import kopf
from kopf._core.intents.causes import ChangingCause
Expand Down
2 changes: 1 addition & 1 deletion tests/handling/daemons/conftest.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import asyncio
import contextlib
import time
from unittest.mock import MagicMock, patch

import freezegun
import pytest
from mock import MagicMock, patch

import kopf
from kopf._cogs.aiokits.aiotoggles import ToggleSet
Expand Down
2 changes: 1 addition & 1 deletion tests/handling/subhandling/test_subhandling.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import asyncio
import logging
from unittest.mock import Mock

import pytest
from mock import Mock

import kopf
from kopf._cogs.structs.ephemera import Memo
Expand Down
3 changes: 1 addition & 2 deletions tests/handling/test_parametrization.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import asyncio

from mock import Mock
from unittest.mock import Mock

import kopf
from kopf._cogs.structs.ephemera import Memo
Expand Down
3 changes: 2 additions & 1 deletion tests/hierarchies/conftest.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from unittest.mock import Mock

import pytest
from mock import Mock


class CustomIterable:
Expand Down
2 changes: 1 addition & 1 deletion tests/hierarchies/test_owner_referencing.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import copy
from unittest.mock import call

import pytest
from mock import call

import kopf
from kopf._cogs.structs.bodies import Body, RawBody, RawMeta
Expand Down
Loading

0 comments on commit c4925f8

Please sign in to comment.