Bring python 3.13 into integration tests #363

dkropachev · 2024-08-09T12:05:13Z

It is resurrection of #336
Brings python 3.13 to the integration tests.
Updates scylla to 5.4 and switches from gabrielfalcao/pyenv-action@v16 to actions/setup-python@v5

.github/workflows/integration-tests.yml

Lorak-mmk

Nit: it is a good practice to have all commits passing tests. IIUC the first commit changes used Scylla version, which breaks some tests, and subsequent commits fix this breakage. If this is true, then could you move first commit to be the last? That would keep all commits passing.

.github/workflows/integration-tests.yml

tests/integration/__init__.py

tests/integration/standard/test_metadata.py

tests/integration/standard/test_cluster.py

mykaul · 2024-08-13T06:06:25Z

I've read somewhere that 3.13 optionally / experimentally removes the GIL. Would be good to test without it, to see where we are at.

so far the code assumed view can be name like: ```python possible_names = [name, f'values({name})'] ``` we have now one more option, that need to be considered: ```python f'keys({name})' ```

some test that was marked with @xfail_scylla, are now passing with newer release, i.e. some issue were fixed. so this new decorator can xfail up to a certion scylla_version note that it support both a oss version, and enterprise version

at some point scylla started validation of the arguments to compaction and newer versions rejects the values passed from this test chnaged it to something within the values expected

test is failing with the following: ``` cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY" ``` seems like scylla doesn't support ordering by clustering keys Ref: scylladb#343

since we want to start test the alpha/beta version of python refactoring a bit the integration test workflow the action for pyenv we are using isn't really getting updates too much, and trying to switch back to the offical use python action that can now have prerelease python versions

dkropachev · 2024-08-16T02:32:24Z

I've read somewhere that 3.13 optionally / experimentally removes the GIL. Would be good to test without it, to see where we are at.

#370

dkropachev · 2024-08-16T02:41:42Z

Nit: it is a good practice to have all commits passing tests. IIUC the first commit changes used Scylla version, which breaks some tests, and subsequent commits fix this breakage. If this is true, then could you move first commit to be the last? That would keep all commits passing.

It is good idea, but, reality is that full test suit takes 30 minutes to complete, say you have 5 commits, it means that I have to spend 150 minutes on testing them.
Now, say I want to change something in the partilar commit, I have to go back to the commit I am changing, spend 30 minutes on testing it and, if code I have changed is not isolated, I have retest every commit that goes after it, does it worth it ? I think no

Lorak-mmk · 2024-08-20T12:43:35Z

Sure, it is not practical to run integration tests on each commit right now, because they take too long. But:

imo it is still worth it to try and have commits without braking tests - it makes it easier to read such commits and if there is a need to bisect something, there will be less fixing to do (even if there still is some fixing)
running unit tests will catch a lot of problems and is much faster.

Lorak-mmk

Note for the future: if you resolved an issue pointed by the comment, please mark the comment as resolved.

Lorak-mmk · 2024-08-20T15:47:06Z

tests/integration/__init__.py

+    if not isinstance(ent_scylla_version, (tuple, list)):
+        ent_scylla_version = (ent_scylla_version,)


What are possible forms of ent_scylla_version? You only pass it in a way that is obvious, e.g. '2023.1', but this code can also hold a tuple of tuple and a list - why?

Lorak-mmk · 2024-08-20T15:49:42Z

tests/integration/__init__.py

+    ent_scylla_version = [Version(v) for v in ent_scylla_version]
+


I'll write my comments as I try to understand this function - hopefully it will be at least a bit helpful with improving comments / changing code.

So here either ent_scylla_version was (tuple, list) - so this will be a [Version(tuple), Version(list)], or it was changed to 1-element tuple, so it will be Version(tuple)- I assume this is the case for passing a string like2023.1`, but I don't get what first variant is for. Why do we have an array of versions???

Actually it looks like Version only accepts string / bytes:

>>> from packaging.version import Version >>> repr(Version((1, 2, 3)))

so I have no idea what the first variant is supposed to do, as it passes tuple and list to Version

Lorak-mmk · 2024-08-20T16:00:45Z

tests/integration/__init__.py

+    # Enterprise releases are tricky, here are the rules:
+    # 1. If something is fixed in `a.b.c` it is also fixed in `a.b.d` where c < d
+    # 2. If something is fixed in `a.1.1` it is also fixed in `c.d.f` where Version(a.b.1) < Version(`c.d.f`)
+    # To make version matching work properly of enterprise we need to have a first major version (`x.1.1`) where it was fixed, in addition to exact version.
+    for ent_v in ent_scylla_version:
+        if ent_v.minor in (1,0) and ent_v.micro in (1,0):
+            first_major = ent_v


I'm not sure I get point 2. The variable b appear without conext and I don't know how to understand it and how to quantify it.
If we quantify it existentially (If something is fixed in a.1.1 then it is also fixed in c.d.f if exists b such that Version(a.b.1) < Version(c.d.f)) it implies that if something is fixed in 2023.1.1 then it is also fixed in 2023.0.2 - because there exists such b (0) that Version(2023.b.1) < Version(2023.0.2).

If we quantify it universally (If something is fixed in a.1.1 then it is also fixed in c.d.f if for all b Version(a.b.1) < Version(c.d.f)) that means that if something is fixed in 2023.1.1 then it is not fixed in 2023.2.1. Why? Point 1 doesn't apply as minor versions don't match, and for b>=2 it is not true that Version(2023.b.1) < Version(2023.2.1)

Neither of those variants sounds probable.

Lorak-mmk

One more thing regarding comment tests/integration: introduced @xfail_scylla_version : commit message mentions xfail_scylla_version instead of xfail_scylla_version_lt

dkropachev marked this pull request as ready for review August 9, 2024 14:15

dkropachev requested review from Lorak-mmk, fruch and sylwiaszunejko and removed request for Lorak-mmk August 9, 2024 14:15

fruch approved these changes Aug 9, 2024

View reviewed changes

.github/workflows/integration-tests.yml Outdated Show resolved Hide resolved

dkropachev self-assigned this Aug 9, 2024

Lorak-mmk mentioned this pull request Aug 12, 2024

CI: add integration tests with python 3.13 #336

Closed

Lorak-mmk requested changes Aug 12, 2024

View reviewed changes

dkropachev force-pushed the int_tests_on_3.13 branch from bae1fbb to 2479b98 Compare August 15, 2024 17:38

fruch added 5 commits August 15, 2024 17:11

cqlengine/management.py: handle more options of views names

b89764b

so far the code assumed view can be name like: ```python possible_names = [name, f'values({name})'] ``` we have now one more option, that need to be considered: ```python f'keys({name})' ```

tests/*cqlengine/management: fix parameters passed to compaction

7ff9347

at some point scylla started validation of the arguments to compaction and newer versions rejects the values passed from this test chnaged it to something within the values expected

dkropachev force-pushed the int_tests_on_3.13 branch from 2479b98 to 4e6329c Compare August 15, 2024 21:25

dkropachev requested a review from Lorak-mmk August 15, 2024 23:04

Lorak-mmk requested changes Aug 20, 2024

View reviewed changes

Lorak-mmk reviewed Aug 20, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bring python 3.13 into integration tests #363

Bring python 3.13 into integration tests #363

dkropachev commented Aug 9, 2024 •

edited

Loading

Lorak-mmk left a comment

mykaul commented Aug 13, 2024

dkropachev commented Aug 16, 2024

dkropachev commented Aug 16, 2024 •

edited

Loading

Lorak-mmk commented Aug 20, 2024

Lorak-mmk left a comment

Lorak-mmk Aug 20, 2024

Lorak-mmk Aug 20, 2024

Lorak-mmk Aug 20, 2024

Lorak-mmk Aug 20, 2024

Lorak-mmk left a comment

		if not isinstance(ent_scylla_version, (tuple, list)):
		ent_scylla_version = (ent_scylla_version,)

		ent_scylla_version = [Version(v) for v in ent_scylla_version]

Bring python 3.13 into integration tests #363

Are you sure you want to change the base?

Bring python 3.13 into integration tests #363

Conversation

dkropachev commented Aug 9, 2024 • edited Loading

Lorak-mmk left a comment

Choose a reason for hiding this comment

mykaul commented Aug 13, 2024

dkropachev commented Aug 16, 2024

dkropachev commented Aug 16, 2024 • edited Loading

Lorak-mmk commented Aug 20, 2024

Lorak-mmk left a comment

Choose a reason for hiding this comment

Lorak-mmk Aug 20, 2024

Choose a reason for hiding this comment

Lorak-mmk Aug 20, 2024

Choose a reason for hiding this comment

Lorak-mmk Aug 20, 2024

Choose a reason for hiding this comment

Lorak-mmk Aug 20, 2024

Choose a reason for hiding this comment

Lorak-mmk left a comment

Choose a reason for hiding this comment

dkropachev commented Aug 9, 2024 •

edited

Loading

dkropachev commented Aug 16, 2024 •

edited

Loading