Allow to pass pymemcache socket keepalive and allow to instantiate a RetryingClient #205

4383 · 2021-07-15T12:50:42Z

This is the combinaison of two commits to implement and reflect new pymemcache features on the dogpile.cache side.

The first one is, the socket keepalive. These capabilities have been introduced [1][2][3] with pymemcache 3.5.0 [4][5]. These changes allow to pass keepalive configuration to the pymemcache client.

The second one is the retry mechanisms. Retry mechanisms have been implemented [6][7] with pymemcache 3.5.0 [8]. These changes allow to instantiate a RetryingClient by using dogpile.cache.

[1] pinterest/pymemcache@b289c87
[2] pinterest/pymemcache@c782de1
[3] pinterest/pymemcache@4d46f5a
[4] pinterest/pymemcache@07b5ecc
[5] https://pypi.org/project/pymemcache/3.5.0/
[6] pinterest/pymemcache@75fe5c8
[7] https://pymemcache.readthedocs.io/en/latest/getting_started.html#using-the-built-in-retrying-mechanism
[8] pinterest/pymemcache@07b5ecc

Change-Id: Ia2e76475943bc8ec86b5974217d1c92110be5b43

4383 · 2021-07-15T13:15:55Z

These changes implement two new features:

socket keepalive
retry mechanisms

I made two separated commits to allow isolated reviews.

sqla-tester

OK, this is sqla-tester setting up my work on behalf of zzzeek to try to get revision 8a7ea5d of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-07-15T16:03:16Z

New Gerrit review created for change 8a7ea5d: https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

zzzeek · 2021-07-15T16:07:24Z

dogpile/cache/backends/memcached.py

        )
+        if self.retry_client:


do we want to raise an argument error if they passed any of the "Retry" options without passing "retry_client" ?

Yes from a UX point of view that's make sense.

zzzeek · 2021-07-15T16:07:24Z

dogpile/cache/backends/memcached.py

@@ -509,15 +555,32 @@ def __init__(self, arguments):
        self.serde = arguments.get("serde", pymemcache.serde.pickle_serde)
        self.default_noreply = arguments.get("default_noreply", False)
        self.tls_context = arguments.get("tls_context", None)
+        self.socket_keepalive = arguments.get("socket_keepalive", None)
+        self.retry_client = arguments.get("retry_client", False)


how about "enable_retry_client" ?

dogpile/cache/backends/memcached.py

4383 · 2021-07-16T12:23:38Z

I was thinking that your gerrit will saw my changes as separated commits (keepalive and retries), unfortunately they have been merged into a single one commit on the gerrit side, so, I squashed them here too.

zzzeek · 2021-07-16T13:57:40Z

I was thinking that your gerrit will saw my changes as separated commits (keepalive and retries), unfortunately they have been merged into a single one commit on the gerrit side, so, I squashed them here too.

you dont need to squash them here, the transfer to gerrit does the squash regardless. either way. the two changes are not that big so it's fine w/ squash.

sqla-tester

OK, this is sqla-tester setting up my work on behalf of zzzeek to try to get revision 637c562 of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-07-16T13:57:56Z

Patchset 637c562 added to existing Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

4383 · 2021-07-16T13:58:53Z

I was thinking that your gerrit will saw my changes as separated commits (keepalive and retries), unfortunately they have been merged into a single one commit on the gerrit side, so, I squashed them here too.

you dont need to squash them here, the transfer to gerrit does the squash regardless. either way. the two changes are not that big so it's fine w/ squash.

Ok thanks for the details.

sqla-tester

OK, this is sqla-tester setting up my work on behalf of zzzeek to try to get revision ca14bf7 of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-07-16T15:04:14Z

Patchset ca14bf7 added to existing Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

zzzeek · 2021-07-16T15:14:29Z

is this dependent on an unreleased version of pymemcache?

Traceback (most recent call last):
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/tests/cache/_fixtures.py", line 32, in setup_class
    cls._check_backend_available(backend)
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/tests/cache/test_memcached_backend.py", line 35, in _check_backend_available
    client = backend._create_client()
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/dogpile/cache/backends/memcached.py", line 591, in _create_client
    return pymemcache.client.retry.RetryingClient(
AttributeError: module 'pymemcache.client' has no attribute 'retry'

4383 · 2021-07-16T15:18:43Z

is this dependent on an unreleased version of pymemcache?

Traceback (most recent call last):
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/tests/cache/_fixtures.py", line 32, in setup_class
    cls._check_backend_available(backend)
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/tests/cache/test_memcached_backend.py", line 35, in _check_backend_available
    client = backend._create_client()
  File "/home/jenkins/workspace/dogpile_gerrit/6216be66/dogpile/cache/backends/memcached.py", line 591, in _create_client
    return pymemcache.client.retry.RetryingClient(
AttributeError: module 'pymemcache.client' has no attribute 'retry'

My bad. It should be pymemcache.client.retrying

4383 · 2021-07-16T15:31:13Z

I'll be on PTO and mostly off-line during the next two weeks so I'll take long time replying to you.
Excuse me by advance.

zzzeek · 2021-07-16T15:37:37Z

I'll be on PTO and mostly off-line during the next two weeks so I'll take long time replying to you.
Excuse me by advance.

enjoy your PTO, no rush on this end

sqla-tester

OK, this is sqla-tester setting up my work on behalf of zzzeek to try to get revision 914f215 of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-07-16T15:37:56Z

Patchset 914f215 added to existing Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

jvanasco · 2021-07-16T16:25:22Z

Since this is passing in the kwargs directly to the pymemcache constructors, there should probably be a docs note about the minimum compatible pymemcache version as this could (probably?) cause exceptions on incompatible versions.

4383 · 2021-07-16T16:42:44Z

Since this is passing in the kwargs directly to the pymemcache constructors, there should probably be a docs note about the minimum compatible pymemcache version as this could (probably?) cause exceptions on incompatible versions.

Done. However the TLS support is in the same case and it require at least pymemcache 3.1.0. Do we want to add notes about the TLS feature and minimal pymemcache compatible version too?

4383 · 2021-07-16T16:47:47Z

Since this is passing in the kwargs directly to the pymemcache constructors, there should probably be a docs note about the minimum compatible pymemcache version as this could (probably?) cause exceptions on incompatible versions.

Done. However the TLS support is in the same case and it require at least pymemcache 3.1.0. Do we want to add notes about the TLS feature and minimal pymemcache compatible version too?

Though that could be done with a follow up patch.

jvanasco · 2021-07-16T16:49:30Z

I think that would make sense. I'm just stepping through possible deployment/upgrade scenarios for existing applications, and I see this raising an exception without much helpful information. I don't think it needs to be accurate, but just having something like "the minimum supported/tested pymemcache version is 3.5" in the README or docs would be enough of a hint.

sqla-tester · 2021-08-17T06:12:06Z

Failed to create a gerrit review, git squash against branch 'master' failed

sqla-tester

OK, this is sqla-tester setting up my work on behalf of 4383 to try to get revision 03aee4b of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-08-17T06:15:14Z

Failed to create a gerrit review, git squash against branch 'master' failed

sqla-tester

OK, this is sqla-tester setting up my work on behalf of 4383 to try to get revision 9ad5a92 of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-08-17T06:17:21Z

Patchset 9ad5a92 added to existing Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

4383 · 2021-08-17T06:17:45Z

Changes rebased over the HEAD of master

Socket keepalive capabilities have been introduced [1][2][3] with pymemcache 3.5.0 [4][5]. These changes allow to pass keepalive configuration to the pymemcache client. Retry mechanisms have been implemented [6][7] with pymemcache 3.5.0 [5][8]. These changes allow to instantiate a ``RetryingClient`` by using dogpile.cache. [1] pinterest/pymemcache@b289c87 [2] pinterest/pymemcache@c782de1 [3] pinterest/pymemcache@4d46f5a [4] pinterest/pymemcache@07b5ecc [5] https://pypi.org/project/pymemcache/3.5.0/ [6] pinterest/pymemcache@75fe5c8 [7] https://pymemcache.readthedocs.io/en/latest/getting_started.html#using-the-built-in-retrying-mechanism [8] pinterest/pymemcache@07b5ecc Change-Id: Ia2e76475943bc8ec86b5974217d1c92110be5b43

types-redis and types-decorator seems missing requirements, these changes add them to the mypy tox environment. Change-Id: Iee34ed090895728ae4e46c45eb4ef56ba6a43a4b

4383 · 2021-08-17T07:43:25Z

The latest patch set fix a typo in the commit message

sqla-tester

OK, this is sqla-tester setting up my work on behalf of 4383 to try to get revision f898fe4 of this pull request into gerrit so we can run tests and reviews and stuff

sqla-tester · 2021-08-17T07:43:41Z

Patchset f898fe4 added to existing Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester

mike bayer (zzzeek) wrote:

better, can you go through the other change requests i made on patch set 6, changelog file, imports, versionadded directives, thanks!

View this in Gerrit at https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester · 2021-08-19T16:12:41Z

dogpile/cache/backends/memcached.py

@@ -17,6 +18,8 @@
 from ..api import NO_VALUE
 from ... import util

+log = logging.getLogger(__name__)


mike bayer (zzzeek) wrote:

this should be removed

View this in Gerrit at https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester · 2021-08-19T16:12:41Z

dogpile/cache/backends/memcached.py

@@ -488,6 +533,22 @@ class PyMemcacheBackend(GenericMemcachedBackend):
    :param serde: optional "serde". Defaults to
     ``pymemcache.serde.pickle_serde``
    :param default_noreply: Defaults to False
+    :param socket_keepalive: optional socket keepalive, will be used for


mike bayer (zzzeek) wrote:

these params all need "..versionadded:: 1.1.5"

View this in Gerrit at https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester · 2021-08-19T16:14:10Z

mike bayer (zzzeek) wrote:

also Hervé you have access to gerrit directly here, so it might be easier if you just work with this gerrit rather than with the PR, that way we can both collaborate on pushes.

View this in Gerrit at https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester · 2021-08-31T09:19:14Z

Hervé Beraud (4383) wrote:

PS16 LGTM.
Thanks for your update.

View this in Gerrit at https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965

sqla-tester · 2021-09-02T14:20:42Z

Gerrit review https://gerrit.sqlalchemy.org/c/sqlalchemy/dogpile.cache/+/2965 has been merged. Congratulations! :)

4383 · 2021-09-02T14:33:20Z

Thanks.

@zzzeek Can I request a new release to allow us to use these changes in openstack oslo. Thanks by advance.

* Update oslo.cache from branch 'master' to f4fa6aa6fa2aca23a8f4f9e63c5a57dbcd2d1166 - Add socket keepalive options to oslo.cache This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. This patch is needed to fix a TLS issue on stable branches introduced by pymemcache (since train), where if a cluster node disappear the client will fail without retrying to reconnect or to switch to an other node of the cluster. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive Partial-Bug: #1959562 Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e (cherry picked from commit f4fa6aa)

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. This patch is needed to fix a TLS issue on stable branches introduced by pymemcache (since train), where if a cluster node disappear the client will fail without retrying to reconnect or to switch to an other node of the cluster. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive Partial-Bug: #1959562 Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e (cherry picked from commit f4fa6aa) (cherry picked from commit 2ad2d52)

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. This patch is needed to fix a TLS issue on stable branches introduced by pymemcache (since train), where if a cluster node disappear the client will fail without retrying to reconnect or to switch to an other node of the cluster. Conflicts: - requirements.txt - doc/requirements.txt - oslo_cache/tests/unit/test_cache_basics.py NOTE(hberaud): Conflicts are related to requirements that have been updated in the youngest branches, however, I removed these reqs updated to follow the same logic that have been applied by Moisès in his patches related to TLS, namely, appending the dogpile.cache.pymemcache backend only if the version of dogpile.cache is higher to a specific version [4]. TLS changes are at the origin of the bug fixed here. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive [4] https://opendev.org/openstack/oslo.cache/commit/3e3037886e8021911e10f492c0502951bff5104e Partial-Bug: #1959562 Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e (cherry picked from commit f4fa6aa) (cherry picked from commit 2ad2d52) (cherry picked from commit 7b00cb3)

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. This patch is needed to fix a TLS issue on stable branches introduced by pymemcache (since train), where if a cluster node disappear the client will fail without retrying to reconnect or to switch to an other node of the cluster. Conflicts: - requirements.txt - doc/requirements.txt - oslo_cache/tests/unit/test_cache_basics.py NOTE(hberaud): Conflicts are related to requirements that have been updated in the youngest branches, however, I removed these reqs updated to follow the same logic that have been applied by Moisès in his patches related to TLS, namely, appending the dogpile.cache.pymemcache backend only if the version of dogpile.cache is higher to a specific version [4]. TLS changes are at the origin of the bug fixed here. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive [4] https://opendev.org/openstack/oslo.cache/commit/3e3037886e8021911e10f492c0502951bff5104e This patch also squash parts of the fix submitted with: https://review.opendev.org/c/openstack/oslo.cache/+/834267 Partial-Bug: #1959562 Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e (cherry picked from commit f4fa6aa) (cherry picked from commit 2ad2d52) (cherry picked from commit 7b00cb3) (cherry picked from commit f34ed73)

This patch specifies a set of options required to setup the socket keepalive of the dogpile.cache's pymemcache backend [1][2]. This setup from those options can later on be passed to this backend. This patch also sets up the socket keepalive object based on the configuration options passed via oslo.config and adds it as an argument to be passed to the selected oslo.cache backend. Dogpile.cache will be used as an interface between oslo.cache and pymemcache [3]. This patch is needed to fix a TLS issue on stable branches introduced by pymemcache (since train), where if a cluster node disappear the client will fail without retrying to reconnect or to switch to an other node of the cluster. Conflicts: - requirements.txt - doc/requirements.txt - oslo_cache/tests/unit/test_cache_basics.py NOTE(hberaud): Conflicts are related to requirements that have been updated in the youngest branches, however, I removed these reqs updated to follow the same logic that have been applied by Moisès in his patches related to TLS, namely, appending the dogpile.cache.pymemcache backend only if the version of dogpile.cache is higher to a specific version [4]. TLS changes are at the origin of the bug fixed here. [1] sqlalchemy/dogpile.cache#205 [2] pinterest/pymemcache@b289c87 [3] https://dogpilecache.sqlalchemy.org/en/latest/api.html?highlight=keepalive#dogpile.cache.backends.memcached.PyMemcacheBackend.params.socket_keepalive [4] https://opendev.org/openstack/oslo.cache/commit/3e3037886e8021911e10f492c0502951bff5104e This patch also squash parts of the fix submitted with: https://review.opendev.org/c/openstack/oslo.cache/+/834267 Conflits: CONFLICT (content): Merge conflict in oslo_cache/tests/test_cache.py Partial-Bug: #1959562 Change-Id: I501100e1a48cdd4e094c08046e2150405dcf371e (cherry picked from commit f4fa6aa) (cherry picked from commit 2ad2d52) (cherry picked from commit 7b00cb3) (cherry picked from commit f34ed73) (cherry picked from commit 6194133)

4383 changed the title ~~Allow to pass pymemcache socket keepalive~~ Allow to pass pymemcache socket keepalive and allow to instantiate a RetryingClient Jul 15, 2021

zzzeek requested a review from sqla-tester Jul 15, 2021

sqla-tester reviewed Jul 15, 2021

View changes

zzzeek requested changes Jul 15, 2021

View changes

4383 force-pushed the socket-keepalive branch from 8a7ea5d to 637c562 Compare Jul 16, 2021

zzzeek requested a review from sqla-tester Jul 16, 2021

sqla-tester reviewed Jul 16, 2021

View changes

4383 force-pushed the socket-keepalive branch from 637c562 to ca14bf7 Compare Jul 16, 2021

zzzeek requested a review from sqla-tester Jul 16, 2021

sqla-tester reviewed Jul 16, 2021

View changes

4383 force-pushed the socket-keepalive branch from ca14bf7 to 914f215 Compare Jul 16, 2021

zzzeek requested a review from sqla-tester Jul 16, 2021

sqla-tester reviewed Jul 16, 2021

View changes

4383 force-pushed the socket-keepalive branch from 914f215 to af674ea Compare Jul 16, 2021

4383 force-pushed the socket-keepalive branch from c6827f5 to 03aee4b Compare Aug 17, 2021

4383 requested a review from sqla-tester Aug 17, 2021

sqla-tester reviewed Aug 17, 2021

View changes

4383 force-pushed the socket-keepalive branch from 03aee4b to 9ad5a92 Compare Aug 17, 2021

4383 requested a review from sqla-tester Aug 17, 2021

sqla-tester reviewed Aug 17, 2021

View changes

4383 added 2 commits Aug 17, 2021

Fix mypy's redis requirements issues

f898fe4

types-redis and types-decorator seems missing requirements, these changes add them to the mypy tox environment. Change-Id: Iee34ed090895728ae4e46c45eb4ef56ba6a43a4b

4383 force-pushed the socket-keepalive branch from 9ad5a92 to f898fe4 Compare Aug 17, 2021

4383 requested a review from sqla-tester Aug 17, 2021

sqla-tester reviewed Aug 17, 2021

View changes

sqla-tester reviewed Aug 19, 2021

View changes

sqlalchemy-bot closed this in 3639099 Sep 2, 2021

Allow to pass pymemcache socket keepalive and allow to instantiate a RetryingClient #205

Allow to pass pymemcache socket keepalive and allow to instantiate a RetryingClient #205

Conversation

4383 commented Jul 15, 2021 • edited

4383 commented Jul 15, 2021

sqla-tester left a comment

sqla-tester commented Jul 15, 2021

zzzeek Jul 15, 2021

Choose a reason for hiding this comment

4383 Jul 16, 2021

Choose a reason for hiding this comment

zzzeek Jul 15, 2021

Choose a reason for hiding this comment

4383 Jul 16, 2021

Choose a reason for hiding this comment

4383 commented Jul 16, 2021

zzzeek commented Jul 16, 2021

sqla-tester left a comment

sqla-tester commented Jul 16, 2021

4383 commented Jul 16, 2021

sqla-tester left a comment

sqla-tester commented Jul 16, 2021

zzzeek commented Jul 16, 2021 • edited

4383 commented Jul 16, 2021 • edited

4383 commented Jul 16, 2021

zzzeek commented Jul 16, 2021

sqla-tester left a comment

sqla-tester commented Jul 16, 2021

jvanasco commented Jul 16, 2021

4383 commented Jul 16, 2021

4383 commented Jul 16, 2021

jvanasco commented Jul 16, 2021

sqla-tester commented Aug 17, 2021

sqla-tester left a comment

sqla-tester commented Aug 17, 2021

sqla-tester left a comment

sqla-tester commented Aug 17, 2021

4383 commented Aug 17, 2021 • edited

4383 commented Aug 17, 2021

sqla-tester left a comment

sqla-tester commented Aug 17, 2021

sqla-tester left a comment

sqla-tester Aug 19, 2021

Choose a reason for hiding this comment

sqla-tester Aug 19, 2021

Choose a reason for hiding this comment

sqla-tester commented Aug 19, 2021

sqla-tester commented Aug 31, 2021

sqla-tester commented Sep 2, 2021

4383 commented Sep 2, 2021

4383 commented Jul 15, 2021 •

edited

zzzeek commented Jul 16, 2021 •

edited

4383 commented Jul 16, 2021 •

edited

4383 commented Aug 17, 2021 •

edited