Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split test_seldon_servers in order to be able to run tests on each SeldonDeployment separately #229

Closed
orfeas-k opened this issue Nov 27, 2023 · 1 comment · Fixed by #228
Closed

Split test_seldon_servers in order to be able to run tests on each SeldonDeployment separately #229

orfeas-k opened this issue Nov 27, 2023 · 1 comment · Fixed by #228
Labels
enhancement New feature or request
Projects
MLOps Solution Issues

Comments

@orfeas-k
Copy link
Contributor

Why it needs to get done

We 've seen those tests on servers (SeldonDeployment CRs) being flaky during the past (e.g. #203), for reasons that we believe are due to resources. Being able to split those and run each server in its own environment will hopefully fix this.

What needs to get done

Configure test_seldon_servers in order to be able to run tests only on one server

When is the task considered done

  • Once we can trigger tests using tox only on one server.
  • CI runs each tests for each server on distinct GH runners
@orfeas-k orfeas-k added the enhancement New feature or request label Nov 27, 2023
@syncronize-issues-to-jira Syncronize issues to Jira
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5049.

This message was autogenerated

This was referenced Nov 27, 2023
Closed
Merged
@NohaIhab NohaIhab added this to Labeled in MLOps Solution Issues Nov 30, 2023
MLOps Solution Issues automation moved this from Labeled to Done Dec 4, 2023
orfeas-k added a commit that referenced this issue Dec 4, 2023
@orfeas-k
feat: Configure test_seldon_servers integration tests to be splitable (…
3b8bb88 
…#228)

- tests(integration): Configure test_seldon_servers test in order to be able 
  to run tests only on one of the SeldonDeployments CRs. This can be done 
  using pytest's-k flag and the corresponding keyword. You can find keywords 
  in CONTRIBUTING.md or by looking at the id field of seldon_servers.py objects.
- tests(integration): Remove limits from SeldonDeployments applied.
- Update CONTRIBUTING.md with instructions on how to run tests separately.
- ci: Run test_seldon_servers integration test for each SeldonDeployment CR 
  in a distinct GH runner. 
- ci: Extract automatically tox integration environments using a script.

Fixes #229
orfeas-k added a commit that referenced this issue Jan 23, 2024
@orfeas-k
feat: Configure test_seldon_servers integration tests to be splitable (…
3470205 
…#228)

- tests(integration): Configure test_seldon_servers test in order to be able 
  to run tests only on one of the SeldonDeployments CRs. This can be done 
  using pytest's-k flag and the corresponding keyword. You can find keywords 
  in CONTRIBUTING.md or by looking at the id field of seldon_servers.py objects.
- tests(integration): Remove limits from SeldonDeployments applied.
- Update CONTRIBUTING.md with instructions on how to run tests separately.
- ci: Run test_seldon_servers integration test for each SeldonDeployment CR 
  in a distinct GH runner. 
- ci: Extract automatically tox integration environments using a script.

Fixes #229
@orfeas-k orfeas-k mentioned this issue Jan 23, 2024
Merged
kimwnasptd pushed a commit that referenced this issue Jan 23, 2024
@orfeas-k
feat: Configure test_seldon_servers integration tests to be splitable (…
1f0e269 
…#228) (#234)

- tests(integration): Configure test_seldon_servers test in order to be able 
  to run tests only on one of the SeldonDeployments CRs. This can be done 
  using pytest's-k flag and the corresponding keyword. You can find keywords 
  in CONTRIBUTING.md or by looking at the id field of seldon_servers.py objects.
- tests(integration): Remove limits from SeldonDeployments applied.
- Update CONTRIBUTING.md with instructions on how to run tests separately.
- ci: Run test_seldon_servers integration test for each SeldonDeployment CR 
  in a distinct GH runner. 
- ci: Extract automatically tox integration environments using a script.

Fixes #229
@orfeas-k orfeas-k mentioned this issue Jan 31, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
MLOps Solution Issues
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Configure test_seldon_servers integration tests to be splitable canonical/seldon-core-operator
1 participant
@orfeas-k