Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Fix Integration Tests, Send Push Notifications to Teams, Add Merge Gate #2279

Merged
merged 44 commits into from
Aug 2, 2023

Conversation

awharrison-28
Copy link
Contributor

@awharrison-28 awharrison-28 commented Aug 2, 2023

Motivation and Context

This PR addresses multiple issues with the integration tests. Fixes integration test breaks related to qdrant and chroma package updates. Labels known flaky tests as xfail. Adds merge queue gates. Adds push notifications to teams for test results.

Description

  • integration tests are now 1x daily down from 2x daily
  • adds a run condition for merge_group
  • if run for merge_group, a subset of the tests will gate the merge
  • ends the results of each test matrix to a dedicated teams channel
  • update qdrant client requirement in the pyproject.toml
  • remove persist_directory from chroma, as the default is now persistent
  • default with_embedding to false in qdrant_memory_store
  • modify chroma test setup and teardown to create the database in a known location and clear the database of entries at the end of each test (I would have just deleted the database at the end of each test, but there are test agent permissions issues with this in CI)
  • marked known flaky tests (pinecone and sequential_planner) as allowed to fail

Fixes #2284

@awharrison-28 awharrison-28 requested a review from a team as a code owner August 2, 2023 03:08
@shawncal shawncal added python Pull requests for the Python Semantic Kernel memory connector labels Aug 2, 2023
@shawncal shawncal self-requested a review August 2, 2023 16:08
@awharrison-28 awharrison-28 force-pushed the feature-python-integration-tests branch from 51e90ae to cf1d92d Compare August 2, 2023 18:11
Copy link
Member

@dmytrostruk dmytrostruk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left small comment, but in general LGTM!

awharrison-28 and others added 2 commits August 2, 2023 12:31
…store.py

Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
@awharrison-28 awharrison-28 added this pull request to the merge queue Aug 2, 2023
Merged via the queue into main with commit 419e596 Aug 2, 2023
25 checks passed
mkarle pushed a commit to mkarle/semantic-kernel that referenced this pull request Aug 2, 2023
…Merge Gate (microsoft#2279)

This PR addresses multiple issues with the integration tests. Fixes
integration test breaks related to qdrant and chroma package updates.
Labels known flaky tests as xfail. Adds merge queue gates. Adds push
notifications to teams for test results.

- integration tests are now 1x daily down from 2x daily
- adds a run condition for `merge_group`
- if run for merge_group, a subset of the tests will gate the merge
- ends the results of each test matrix to a dedicated teams channel
- update qdrant client requirement in the pyproject.toml
- remove `persist_directory` from chroma, as the default is now
persistent
- default with_embedding to false in qdrant_memory_store
- modify chroma test setup and teardown to create the database in a
known location and clear the database of entries at the end of each test
(I would have just deleted the database at the end of each test, but
there are test agent permissions issues with this in CI)
- marked known flaky tests (pinecone and sequential_planner) as allowed
to fail

Fixes microsoft#2284

---------

Co-authored-by: Abby Harrison <abharris@microsoft.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
SOE-YoungS pushed a commit to SOE-YoungS/semantic-kernel that referenced this pull request Nov 1, 2023
…Merge Gate (microsoft#2279)

### Motivation and Context
This PR addresses multiple issues with the integration tests. Fixes
integration test breaks related to qdrant and chroma package updates.
Labels known flaky tests as xfail. Adds merge queue gates. Adds push
notifications to teams for test results.

### Description
- integration tests are now 1x daily down from 2x daily
- adds a run condition for `merge_group`
- if run for merge_group, a subset of the tests will gate the merge
- ends the results of each test matrix to a dedicated teams channel
- update qdrant client requirement in the pyproject.toml
- remove `persist_directory` from chroma, as the default is now
persistent
- default with_embedding to false in qdrant_memory_store
- modify chroma test setup and teardown to create the database in a
known location and clear the database of entries at the end of each test
(I would have just deleted the database at the end of each test, but
there are test agent permissions issues with this in CI)
- marked known flaky tests (pinecone and sequential_planner) as allowed
to fail

Fixes microsoft#2284

---------

Co-authored-by: Abby Harrison <abharris@microsoft.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
memory connector python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python: Integration Test Status
3 participants