Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change contrib-tests.yml tests to use --skip-openai #1132

Merged
merged 6 commits into from Jan 10, 2024

Conversation

maxim-saplin
Copy link
Collaborator

Why are these changes needed?

contrib-tests.yml workflow used pip uninstall -y openai trick to skip the tests depending on OpenAI. Changed it to use pytest argument --skip-openai instead

Related issue number

Closes #1112

Notes

As of now the tests in contrib-tests.yml are mostly skipped cause they depend on OpenAI (~70-80% of individual tests are skipped). It makes sense to tidy up the file and remove all those OpenAI dependent tests as they don't do anything. Here're the breakdown for individual files:

RetrieveChatTest

	• test/test_retrieve_utils.py
	• [x][Always Skipped] test/agentchat/contrib/test_retrievechat.py
	• test/agentchat/contrib/test_qdrant_retrievechat.py


[x]CompressionTest
	
	• [x][Always Skipped]test/agentchat/contrib/test_compressible_agent.py


[x]GPTAssistantAgent
	
	• [x][Always Skipped]test/agentchat/contrib/test_gpt_assistant.py


[x]TeachableAgent

	• [x][Always Skipped]test/agentchat/contrib/test_teachable_agent.py


[x]LMMTest
	• test/agentchat/contrib/test_img_utils.py
	• test/agentchat/contrib/test_lmm.py
[x]test/agentchat/contrib/test_llava.py - depends on LLAVA imports

Checks

@codecov-commenter
Copy link

codecov-commenter commented Jan 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (78a2d84) 31.80% compared to head (29d110f) 35.87%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1132      +/-   ##
==========================================
+ Coverage   31.80%   35.87%   +4.06%     
==========================================
  Files          29       31       +2     
  Lines        4112     4298     +186     
  Branches      961     1042      +81     
==========================================
+ Hits         1308     1542     +234     
+ Misses       2710     2650      -60     
- Partials       94      106      +12     
Flag Coverage Δ
unittests 35.85% <ø> (+4.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@afourney afourney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but may conflict with #1072 since several tests needed to be corrected in that PR as well.

Copy link
Collaborator

@sonichi sonichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a small issue.

test/agentchat/contrib/test_qdrant_retrievechat.py Outdated Show resolved Hide resolved
test/agentchat/contrib/test_retrievechat.py Outdated Show resolved Hide resolved
@maxim-saplin
Copy link
Collaborator Author

@sonichi, I've modified both files, please check

@sonichi sonichi added this pull request to the merge queue Jan 10, 2024
Merged via the queue into microsoft:main with commit 083ba32 Jan 10, 2024
76 of 84 checks passed
@maxim-saplin maxim-saplin deleted the fix-contrib-tests branch January 10, 2024 08:03
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* Fixing RetrieveChat part

* All 5 test groups are updated to --skip-openai

* sys.path.append() fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev development experience/productivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Issue]: test in contrib-tests.yml should also use --skip-openai to skip the openai test
6 participants