Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Feature selection better #340

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Support COSTEER Multi-Dimension for output & bug-fix
  • Loading branch information
xisen-w committed Sep 25, 2024
commit 632b13cb58fba7536a45286149598725846c7d3b
2 changes: 1 addition & 1 deletion rdagent/scenarios/kaggle/developer/runner.py
Original file line number Diff line number Diff line change
@@ -62,7 +62,7 @@ def init_develop(self, exp: KGFactorExperiment | KGModelExperiment) -> KGFactorE
)

org_data_path = (
Path(FACTOR_IMPLEMENT_SETTINGS.data_folder) / KAGGLE_IMPLEMENT_SETTING.competition / "X_valid.pkl"
Path(KAGGLE_IMPLEMENT_SETTING.local_data_path) / KAGGLE_IMPLEMENT_SETTING.competition / "X_valid.pkl"
)
with open(org_data_path, "rb") as f:
org_data = pickle.load(f)
6 changes: 5 additions & 1 deletion rdagent/scenarios/kaggle/experiment/prompts.yaml
Original file line number Diff line number Diff line change
@@ -314,7 +314,11 @@ kg_feature_simulator: |-

kg_model_output_format: |-
For feature related tasks, the output should be a pandas DataFrame with the new features. The columns should be the new features, and the rows should correspond to the number of samples in the input DataFrame.
For model related tasks, the output should be an np.ndarray with the appropriate number of predictions, each prediction being a single value. The output should be a 2D array with dimensions corresponding to the number of predictions and 1 column (e.g., (8, 1) if there are 8 predictions).
For model related tasks:
1. the output should be an np.ndarray with the appropriate number of predictions & the appropriate values within each prediction
2. the output should be a 2D array with dimensions corresponding to the number of predictions and the number of things to output. Eg, if 4 predictions, each prediction needs to predict 3 probabilities, then (4,3). Or (8, 1) if there are 8 predictions but each prediction is only one value.
3. please reference the competition's submission requirement and align with that.
Submission Requirements here:\n: {{submission_specifications}}

kg_model_simulator: |-
The models will be trained on the competition dataset and evaluated on their ability to predict the target. Metrics like accuracy and AUC-ROC is used to evaluate the model performance.