Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enabling automation of experiments running v2.0 #469

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8726ab8
Revising to enable automation of experiments running v1.0
xisen-w Nov 4, 2024
b44bef5
Any new updates
xisen-w Nov 15, 2024
c100876
Revising to enable automation of experiments running v1.0
xisen-w Nov 4, 2024
18370d4
Any new updates
xisen-w Nov 15, 2024
21a99d2
Add template
you-n-g Nov 15, 2024
86ae0b2
Stoping tracking additional env
xisen-w Nov 20, 2024
f94dbff
Merge branch 'automated-evaluation' of https://github.com/microsoft/R…
xisen-w Nov 20, 2024
66ffd6d
Uploading relevant envs
xisen-w Nov 20, 2024
0ef80a5
Adding tests
xisen-w Nov 20, 2024
907d980
Updating
xisen-w Nov 20, 2024
51388d1
Updated collect.py to extract result from trace
xisen-w Nov 23, 2024
af6220e
Update .gitignore to remove the unecessary ones
xisen-w Nov 23, 2024
54c3c6d
"Remove unnecessary files"
xisen-w Nov 23, 2024
78708e4
Merge branch 'automated-evaluation' of https://github.com/microsoft/R…
xisen-w Nov 25, 2024
3f131f3
Merge branch 'main' into automated-evaluation
xisen-w Nov 25, 2024
38bb9e6
Updated to enable automatic collection of experiment result information
xisen-w Nov 25, 2024
10b0053
Updating the env files & Upading test_system file
xisen-w Nov 25, 2024
238f492
Updated relevant env for better testing
xisen-w Nov 25, 2024
68ca63a
Updated README.md
xisen-w Nov 25, 2024
8b18fad
reverting gitignore back
xisen-w Nov 25, 2024
2395dc5
Updates
xisen-w Dec 3, 2024
b7cc98e
README update
xisen-w Dec 3, 2024
0b5a09d
Updates on env README
xisen-w Dec 3, 2024
24cd0c2
Updating collect.py
xisen-w Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updates on env README
  • Loading branch information
xisen-w committed Dec 3, 2024
commit 0b5a09d1eec7ebf462d779e6c3f7188a703d8d9b
5 changes: 3 additions & 2 deletions scripts/exp/ablation/README.md
Original file line number Diff line number Diff line change
@@ -4,9 +4,10 @@ This document outlines the environment configurations for the ablation studies.

| Name | .env | Description | Available? |
|-----------|--------------|-------------------------------------------|------------|
| max | max.env | Enables all features | No |
| minicase | minicase.env | Enables minicase and DS-Agent | Yes |
| basic | basic.env | Standard case of RDAgent | Yes |
| minicase | minicase.env | Enables minicase and DS-Agent | Yes |
| pro | pro.env | Standard case with vector RAG | Yes |
| max | max.env | Enables all features | No |

## Notes

1 change: 0 additions & 1 deletion scripts/exp/ablation/env/max.env
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, it needs to add the path to the knowledge base.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KG_IF_USING_VECTOR_RAG here should be set to false.

Original file line number Diff line number Diff line change
@@ -2,4 +2,3 @@ KG_IF_USING_VECTOR_RAG=False
KG_IF_USING_GRAPH_RAG=True
KG_IF_ACTION_CHOOSING_BASED_ON_UCB=True
#KG_KNOWLEDGE_BASE_PATH= TODO: Specify Your Knowledge Base Path
CHAT_MODEL
1 change: 1 addition & 0 deletions scripts/exp/ablation/env/mini-case.env
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
KG_IF_USING_VECTOR_RAG=True
KG_IF_USING_GRAPH_RAG=False
KG_IF_ACTION_CHOOSING_BASED_ON_UCB=True
# MIGHT BE LEGACY

1 change: 1 addition & 0 deletions scripts/exp/ablation/env/pro.env
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, it also needs to add the path to the knowledge base.

Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
KG_IF_USING_VECTOR_RAG=True
KG_IF_USING_GRAPH_RAG=False
KG_IF_ACTION_CHOOSING_BASED_ON_UCB=True
# MIGHT BE LEGACY
Loading
Oops, something went wrong.