[PR] Refactor runtime to full async/await; make officially embeddable

[kopf-archiver]

A pull request by nolar at 2019-07-24 16:28:57+00:00
Original URL: zalando-incubator/kopf#156
Merged by nolar at 2019-08-07 09:59:47+00:00

Make operator termination even more graceful (resource-cleanable), the task orchestration — async, and the operator core — embeddable (and officially documented).

Issue : #142.

Description

Previously in #147, there was an attempt to rework the asyncio task management with the "should-stop" event and asyncio task cancellations. It made the operator to react to the existing signals, but still had some problems with proper cleanups.

Specifically, when a aiohttp.ClientSession is added (a PR based on this one — to be created), there is no place to properly create it and close it, since all the tasks are orchestrated in a synchronous function.

This led to few issues with some aiohttp connections not being closed properly, issuing the warnings/prints on the process exiting (and a lot of them in tests).

Fully async coroutines

With this PR, and operator's body is extracted into a self-contained coroutine (operator()), which does everything: task spawning, task orchestration, resource creation and cleanups, etc.

The synchronous functions run() and create_tasks() are left as wrappers for operator() and spawn_tasks() respectively. (create_tasks() is also deprecated.)

Graceful termination

With this PR, the operator termination is now even more graceful both in case of normal exit ("should-stop" event set) and in case of error. In the latter case, all tasks & sub-tasks are cancelled without a tolerable graceful timeout, but they are still given time to cleanup their resources (e.g. in try-finally).

Embeddability

These function were already exposed via the public interface (accessible via kopf.run/kopf.create_tasks).

With this PR, all the new functions are also exposed, and this is made a documented feature of embeddability — i.e. ability to embed Kopf operators into third-party applications by explicit invocation/orchestration of the Kopf tasks.

(As an example of potential usage (not in the plans): an application that listens for the resources updates and keeps their state in-memory, while showing that state in a UI/WebUI.)

Types of Changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor/improvements
• Documentation / non-code

Review

List of tasks the reviewer must do to review the PR

• Tests
• Documentation

---

Commented by nolar at 2019-08-06 11:26:55+00:00
 

samurang87 Thanks. All is fixed now.

---

Commented by psycho-ir at 2019-08-06 15:19:31+00:00
 

👍