Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs/Contributing] Clarification on running non-OpenAI tests #1052

Merged
merged 3 commits into from
Dec 28, 2023
Merged

[Docs/Contributing] Clarification on running non-OpenAI tests #1052

merged 3 commits into from
Dec 28, 2023

Conversation

maxim-saplin
Copy link
Collaborator

As a fresh contributor I struggled with running tests locally and it took me some time to understand that:

  1. There're 2 kinds of tests: A (not depending on OpenAI endpoints) and B (requiring a live OpenAI endpoint, being skipped is openai is not installed)
  2. build.yaml (which is a mandatory checkpoint for any PR) only runs A

Made a clarification to Contributing page.

P.S.: As a follow-up it's worth to clarify how to run OpenAI tests locally. Guidance from someone is welcome. It seems there're 2 dependencies: (1) a well formed OAI_CONFIG_LIST file (there're plenty of hard-coded model names spread across tests, haven't identified the, all) and (2) Docker Desktop up-and-running

Checks

@codecov-commenter
Copy link

codecov-commenter commented Dec 24, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (70cc1f4) 30.27% compared to head (29ebaae) 30.27%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1052   +/-   ##
=======================================
  Coverage   30.27%   30.27%           
=======================================
  Files          30       30           
  Lines        3980     3980           
  Branches      897      897           
=======================================
  Hits         1205     1205           
  Misses       2696     2696           
  Partials       79       79           
Flag Coverage Δ
unittests 30.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@sonichi sonichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc looks OK for the current setup. There are better ways to organize the tests, I think.
It'll be good to create an issue in case we forget to update this part when we reorganize the tests.

website/docs/Contribute.md Show resolved Hide resolved
website/docs/Contribute.md Show resolved Hide resolved
website/docs/Contribute.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@sonichi sonichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created issues for the TODOs. Approving now.

@sonichi sonichi added this pull request to the merge queue Dec 28, 2023
@ekzhu ekzhu added the documentation Improvements or additions to documentation label Dec 28, 2023
Merged via the queue into microsoft:main with commit 30daa1e Dec 28, 2023
16 checks passed
@maxim-saplin maxim-saplin deleted the contrib-run-tests-non-oai branch December 29, 2023 08:02
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* add agent notebook and documentation

* fix bug

* set flush to True when printing msg in agent

* add a math problem in agent notebook

* remove

* header

* improve notebook doc

* notebook update

* improve notebook example

* improve doc

* improve notebook doc

* improve print

* doc

* human_input_mode

* human_input_mode str

* indent

* indent

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* renaming and doc format

* typo

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
…oft#1052)

* Update Contribute.md

* Removed mentioning of OAI_CONFIG_LIST

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants