Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Ceph S3 Version to Newest #7326

Merged
merged 1 commit into from
Jul 3, 2023
Merged

Conversation

nadavMiz
Copy link
Contributor

@nadavMiz nadavMiz commented Jun 6, 2023

Explain the changes

  1. update CEPH_TESTS_VERSION to the latest version
  2. update the testing script to use the new testing framework tox (see https://github.com/ceph/s3-tests#readme for more details)
  3. changed test names to the new format: use s3tests_boto3/functional/test_s3.py::test_basic_key_count instead of s3tests_boto3.functional.test_s3:test_basic_key_count
  4. removed S3_CEPH_TEST_STEMS as changing test names is no longer needed. moved tests that previously didn't run because they didn't match its regex (but showed as if they did) to the blacklist. those are the tests related to iam and sts
  5. added new tests that don't pass to the pending list
  6. changed s3-tests repo permissions, because write permission is required for tox to create files
  7. changed manually changing of test_s3select.py in test_ceph_s3_deploy.sh from line 14 to line 16 because the file has been changed
  8. changed keyword used to identify skipped tests from 'SKIP' to 'skipped' in order to identify skipped tests
  9. moved test_count_json_operation and test_column_sum_min_max tests to the pending list, since they no longer pass
  10. change the way we only run boto3 tests as described in https://github.com/ceph/s3-tests/pull/482/files#diff-7b3ed02bc73dc06b7db906cf97aa91dec2b2eb21f2d92bc5caa761df5bbc168fR44-R48
  11. change the check of aws4 to check by test names ending with aws4 instead of using the list. moved aws4 tests to the pending list because of the problem with tox trying to parse the endpoint as if it is an Amazon URL. s3-tests issue: aws4 tests fails because of "UnboundLocalError: local variable 'region_name' referenced before assignment" ceph/s3-tests#520
  12. update the commit of the operator to the latest in ceph-s3-tests.yaml
  13. remove filtering based on not fails_on_rgw on tests job since it is not relevant any more
  14. update ceph_s3_tests_guide.md with latest changes

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

  1. Run ceph s3 tests job (https://github.com/noobaa/noobaa-core/blob/master/docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md#run-all-ceph-s3-tests)
  2. All tests should pass
  • Doc added/updated
  • Tests added

@nadavMiz nadavMiz requested review from nimrod-becker, shirady, a team and Neon-White and removed request for a team June 6, 2023 12:24
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
src/deploy/NVA_build/Tests.Dockerfile Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 10 times, most recently from d97384b to ff207d6 Compare June 8, 2023 14:28
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
src/test/system_tests/ceph_s3_tests/test_ceph_s3.js Outdated Show resolved Hide resolved
@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 3 times, most recently from 786efde to fc94df4 Compare June 22, 2023 11:25
@nadavMiz
Copy link
Contributor Author

I have added call to kubectle get pods in case ceph tests are failing. its worth mentioning that on the previous run some tests failled but no reset were spotted for ny of the pods. so its not a sure way of knowing when the fail was a temporary issue

@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 2 times, most recently from 7be92a9 to 825de58 Compare June 26, 2023 11:55
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 2 times, most recently from b8e288a to 03ef778 Compare June 27, 2023 09:59
@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 3 times, most recently from d665387 to 7e9fcd1 Compare July 2, 2023 07:12
@nadavMiz nadavMiz requested a review from shirady July 2, 2023 11:50
Copy link
Contributor

@shirady shirady left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
docs/dev_guide/ceph_s3_tests/ceph_s3_tests_guide.md Outdated Show resolved Hide resolved
@nadavMiz nadavMiz force-pushed the new-ceph-tests branch 3 times, most recently from d558efe to 6ececb2 Compare July 3, 2023 10:35
Signed-off-by: nadav mizrahi <nadav.mizrahi16@gmail.com>
@nadavMiz nadavMiz merged commit b3ea3a5 into noobaa:master Jul 3, 2023
6 checks passed
@guymguym
Copy link
Member

guymguym commented Jul 6, 2023

@nadavMiz Notice that it's probably better not to add 100KB images to the repo.
I mean, it's not much, but we don't want it to add loading time to cloning.
Next time lets think how to add this information to the repo instead of an image - maybe just copying the text to a markdown file would have been good too.
Anyway, removing things from history is a complicated thing (at least it was, I haven't checked for new options) so we might need to live with it, and just think about it next time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants