quincy: test/pybind: replace nose with pytest #55060

cbodley · 2024-01-04T18:54:46Z

backport tracker: https://tracker.ceph.com/issues/63937

backport of #52143
parent tracker: https://tracker.ceph.com/issues/61567

this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/main/src/script/ceph-backport.sh

cbodley · 2024-01-04T18:56:08Z

there were several conflicts around filtering for the crimson-rados suite; hopefully i got everything right

idryomov

there were several conflicts around filtering for the crimson-rados suite; hopefully i got everything right

I spotted some leftovers, mostly around markers. Otherwise LGTM!

qa/suites/crimson-rados/basic/tasks/rados_python.yaml

qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests_old_format.yaml

qa/workunits/rbd/test_librbd_python.sh

src/test/pybind/test_rbd.py

src/test/pybind/test_rados.py

src/test/pybind/pytest.ini

ljflores · 2024-01-04T23:33:56Z

@cbodley let me know when this is ready to get into testing

cbodley · 2024-01-05T14:23:17Z

thanks @idryomov, cleaned up the cherry-picks and documented the conflicts around markers

qa/suites/crimson-rados/basic/tasks/rados_python.yaml

idryomov · 2024-01-05T15:40:19Z

jenkins test make check

idryomov · 2024-01-05T15:40:58Z

jenkins test api

idryomov · 2024-01-05T15:59:27Z

@cbodley Backport looks great, but I noticed that wait marker used in RADOS tests is missing from pytest.ini in main. Is that intentional?

cbodley · 2024-01-05T17:49:36Z

@cbodley Backport looks great, but I noticed that wait marker used in RADOS tests is missing from pytest.ini in main. Is that intentional?

not intentional, no

ljflores · 2024-01-08T22:56:46Z

Waiting on this PR to go into testing / merge to unblock https://trello.com/c/Yjrx9ygD/1911-wip-yuri8-testing-2023-12-15-0911

cbodley · 2024-01-10T18:37:04Z

seeing one failure from test_rbd.py, ex. http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:56:52-rbd-quincy-distro-default-smithi/7510134/teuthology.log

=================================== FAILURES ===================================
_________________________ TestImage.test_diff_iterate __________________________

self = <test_rbd.TestImage object at 0x7fa827636dd8>

    def test_diff_iterate(self):
        check_diff(self.image, 0, IMG_SIZE, None, [])
        self.image.write(b'a' * 256, 0)
        check_diff(self.image, 0, IMG_SIZE, None, [(0, 256, True)])
        self.image.write(b'b' * 256, 256)
        check_diff(self.image, 0, IMG_SIZE, None, [(0, 512, True)])
        self.image.discard(128, 256)
        check_diff(self.image, 0, IMG_SIZE, None, [(0, 512, True)])

        self.image.create_snap('snap1')
        self.image.discard(0, 1 << IMG_ORDER)
        self.image.create_snap('snap2')
        self.image.set_snap('snap2')
>       check_diff(self.image, 0, IMG_SIZE, 'snap1', [(0, 512, False)])

../../../clone.client.0/src/test/pybind/test_rbd.py:1226:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../clone.client.0/src/test/pybind/test_rbd.py:1450: in check_diff
    eq(extents, expected)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = [], b = [(0, 512, False)]

    def assert_equal(a, b):
>       assert a == b
E       AssertionError

../../../clone.client.0/src/test/pybind/assertions.py:2: AssertionError

@idryomov any idea what might be happening there?

idryomov · 2024-01-10T20:45:56Z

@cbodley See https://tracker.ceph.com/issues/63941. I have updated pacific branch in ceph-ci.git, it's possible that other named branched need to be pushed as well.

idryomov · 2024-01-10T20:51:54Z

@cbodley Scratch the above, I see that it's a different test. The root cause is the same binaries vs test mismatch though.

I think you just need to rebase to pick up #54950 that landed in quincy in the interim.

* use fixtures for temporary images and groups * use pytest.skip instead of nose.SkipTest * replace setUp/tearDown with setup/teardown_method * add @pytest.mark.skip_if_crimson * replace nose assertions Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit af04457) Conflicts: qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests.yaml does not exist on quincy qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests_old_format.yaml qa/workunits/rbd/test_librbd_python.sh src/test/pybind/test_rbd.py skip_if_crimson filter not present on quincy

* use a fixture for setup_test() * invoke test_cephfs.py with pytest Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit 0bd813f) Conflicts: src/test/pybind/test_cephfs.py missing ceph#43546 missing ceph#46905

* replace nose.Skiptest with pytest.skip() * replace nose attrs with @pytest.mark * replace nose assertions * replace setUp/tearDown with setup/teardown_method Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit 0d0128f) Conflicts: qa/suites/crimson-rados/basic/tasks/rados_python.yaml src/test/pybind/test_rados.py no rollback or wait markers, added snap marker

* use a fixture for setup_test() * replace nose assertions Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit 136b3ec)

Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit cbdd520) Conflicts: qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests.yaml does not exist on quincy

ljflores · 2024-01-11T20:25:40Z

jenkins test api

cbodley requested review from a team as code owners January 4, 2024 18:54

cbodley added this to the quincy milestone Jan 4, 2024

cbodley added the pybind label Jan 4, 2024

github-actions bot added cephfs Ceph File System core crimson rbd tests labels Jan 4, 2024

cbodley mentioned this pull request Jan 4, 2024

qa/distros: bump container host distros from centos 8->9 #53901

Merged

ljflores approved these changes Jan 4, 2024

View reviewed changes

idryomov reviewed Jan 4, 2024

View reviewed changes

cbodley force-pushed the wip-63937-quincy branch from 75c31c7 to 3552439 Compare January 5, 2024 14:20

cbodley commented Jan 5, 2024

View reviewed changes

qa/suites/crimson-rados/basic/tasks/rados_python.yaml Outdated Show resolved Hide resolved

cbodley force-pushed the wip-63937-quincy branch from 3552439 to affad57 Compare January 5, 2024 14:35

idryomov approved these changes Jan 5, 2024

View reviewed changes

cbodley added the needs-qa label Jan 9, 2024

cbodley added 2 commits January 10, 2024 15:57

test/pybind/cephfs: convert from nose to pytest

79fcb0c

* use a fixture for setup_test() * invoke test_cephfs.py with pytest Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit 0bd813f) Conflicts: src/test/pybind/test_cephfs.py missing ceph#43546 missing ceph#46905

cbodley added 3 commits January 10, 2024 15:57

test/pybind/rgwfs: convert from nose to pytest

32695e8

* use a fixture for setup_test() * replace nose assertions Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit 136b3ec)

qa/suites: install pytest for pybind tasks

49908e1

Signed-off-by: Casey Bodley <cbodley@redhat.com> (cherry picked from commit cbdd520) Conflicts: qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests.yaml does not exist on quincy

cbodley force-pushed the wip-63937-quincy branch from affad57 to 49908e1 Compare January 10, 2024 20:58

idryomov modified the milestones: quincy, v17.2.8 Jan 17, 2024

vshankar approved these changes Jan 18, 2024

View reviewed changes

cbodley merged commit 7fa3524 into ceph:quincy Jan 18, 2024
10 checks passed

cbodley deleted the wip-63937-quincy branch January 18, 2024 15:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

quincy: test/pybind: replace nose with pytest #55060

quincy: test/pybind: replace nose with pytest #55060

cbodley commented Jan 4, 2024

cbodley commented Jan 4, 2024

idryomov left a comment

ljflores commented Jan 4, 2024

cbodley commented Jan 5, 2024

idryomov commented Jan 5, 2024

idryomov commented Jan 5, 2024

idryomov commented Jan 5, 2024

cbodley commented Jan 5, 2024

ljflores commented Jan 8, 2024

cbodley commented Jan 10, 2024

idryomov commented Jan 10, 2024

idryomov commented Jan 10, 2024 •

edited

ljflores commented Jan 11, 2024

quincy: test/pybind: replace nose with pytest #55060

quincy: test/pybind: replace nose with pytest #55060

Conversation

cbodley commented Jan 4, 2024

cbodley commented Jan 4, 2024

idryomov left a comment

Choose a reason for hiding this comment

ljflores commented Jan 4, 2024

cbodley commented Jan 5, 2024

idryomov commented Jan 5, 2024

idryomov commented Jan 5, 2024

idryomov commented Jan 5, 2024

cbodley commented Jan 5, 2024

ljflores commented Jan 8, 2024

cbodley commented Jan 10, 2024

idryomov commented Jan 10, 2024

idryomov commented Jan 10, 2024 • edited

ljflores commented Jan 11, 2024

idryomov commented Jan 10, 2024 •

edited