Updated README.md

scripts/exp/tools/README.md

@@ -1,138 +1,148 @@
-Tools Directory
+### Tools Directory
 
 This directory provides scripts to run experiments with different environment configurations, collect results, and demonstrate usage through an example script.
 
-Directory Structure
+### Directory Structure
 
 scripts/exp/tools/
 ├── run_envs.sh       # Script for running experiments
 ├── collect.py        # Results collection and summary
-├── test_system.sh    # Example usage script
+├── test_system.sh    # Usage script for rdagent kaggle loop
 ├── README.md         # This documentation
 
 Tools Overview
 
 	1.	run_envs.sh: Executes experiments with different environment configurations in parallel.
 	2.	collect.py: Collects and summarizes experiment results into a single file.
-	3.	test_system.sh: Demonstrates how to use the above tools together for experiment execution and result collection.
+	3.	test_system.sh: Demonstrates how to use the above tools together for experiment execution and result collection [for rdagent kaggle loop]
 
-Getting Started
+### Getting Started
 
 Prerequisites
 
-	1.	Ensure the scripts have execution permissions:
+1. Ensure the scripts have execution permissions:
 
+```
 chmod +x scripts/exp/tools/run_envs.sh
 chmod +x scripts/exp/tools/test_system.sh
+```
 
+2. Place your .env files in the desired directory for environment configurations.
 
-	2.	Install required dependencies for Python scripts:
+### Usage
 
-pip install -r requirements.txt
-
-
-	3.	Place your .env files in the desired directory for environment configurations.
-
-Usage
-
-1. Running Experiments with Different Environments
+#### 1. Running Experiments with Different Environments
 
 The run_envs.sh script allows running a command with multiple environment configurations in parallel.
 
-Command Syntax
+**Command Syntax**
 
+```
 ./run_envs.sh -d <dir_to_.envfiles> -j <number_of_parallel_processes> -- <command>
+```
 
-Example Usage
+**Example Usage**
 
 Basic example:
 
+```
 ./run_envs.sh -d env_files -j 1 -- echo "Hello"
+```
 
 Practical example:
 
-dotenv run -- ./run_envs.sh -d /h/home/v-xisenwang/RD-Agent/scripts/exp/ablation/env -j 1 -- python /home/v-xisenwang/RD-Agent/rdagent/app/kaggle/loop.py
+```
+dotenv run -- ./run_envs.sh -d RD-Agent/scripts/exp/ablation/env -j 1 -- python RD-Agent/rdagent/app/kaggle/loop.py
+```
+
+**Explanation:**
+
+| Option | Description |
+| --- | --- |
+| `-d` | Specifies the directory containing .env files. |
+| `-j` | Number of parallel processes to run (e.g., 1 for sequential execution). |
+| `--` | Separates script options from the command to execute. |
+| `<command>` | The command to execute with the environment variables loaded. |
 
-Explanation:
-	•	-d: Specifies the directory containing .env files.
-	•	-j: Number of parallel processes to run (e.g., 1 for sequential execution).
-	•	--: Separates script options from the command to execute.
-	•	<command>: The command to execute with the environment variables loaded.
 
-2. Collecting Results
+####2.Collecting Results
 
 The collect.py script processes logs and generates a summary JSON file.
 
-Command Syntax
+**Command Syntax**
 
+```
 python collect.py --log_path <path_to_logs> --output_name <summary_filename>
+```
 
-Example Usage
+**Example Usage**
 
 Collect results from logs:
 
+```
 python collect.py --log_path logs --output_name summary.json
+```
+**Explanation:**
 
-Explanation:
-	•	--log_path: Required. Specifies the directory containing experiment logs.
-	•	--output_name: Optional. The name of the output summary file (default: summary.json).
+| Option | Description |
+| --- | --- |
+| `--log_path` | Required. Specifies the directory containing experiment logs. |
+| `--output_name` | Optional. The name of the output summary file (default: summary.json). |
 
-3. Example Workflow
+#### 3. Example Workflow [for rdagent kaggle loop]
 
 Use the test_system.sh script to demonstrate a complete workflow.
 
-Steps:
+**Steps:**
 
-	1.	Ensure the scripts are executable:
+1.	Ensure the scripts are executable:
 
+```
 chmod +x scripts/exp/tools/run_envs.sh
 chmod +x scripts/exp/tools/test_system.sh
+```
 
+2.	Run the test system:
 
-	2.	Run the test system:
-
+```
 ./scripts/exp/tools/test_system.sh
-
-
+```
 
 This will:
 	1.	Load environment configurations from .env files.
 	2.	Execute experiments using the configurations.
-	3.	Collect and summarize results.
-
-Troubleshooting
-
-Permission Denied
-
-If you encounter a PermissionError when running scripts:
-	1.	Ensure the script has execution permissions:
-
-chmod +x ./scripts/exp/tools/run_envs.sh
-chmod +x ./scripts/exp/tools/test_system.sh
 
+3. Find your logs in the logs directory.
 
-	2.	Verify file ownership:
+4. Use the collect.py script to summarize results:
 
-ls -l ./scripts/exp/tools/
+```
+python collect.py --log_path logs --output_name summary.json
+```
 
+### Troubleshooting
 
-	3.	Change ownership or update permissions if necessary:
+#### Permission Denied
 
-sudo chown $USER:$USER ./scripts/exp/tools/*
-chmod 755 ./scripts/exp/tools/*
+If you encounter a PermissionError when running scripts:
 
-Capitalized Environment Variables
+1.Ensure the script has execution permissions:
 
-To standardize variable names, .env files should use uppercase variable names with underscores (e.g., MY_VARIABLE). For example:
+```
+chmod +x ./scripts/exp/tools/run_envs.sh
+chmod +x ./scripts/exp/tools/test_system.sh
+```
 
-IF_USING_VECTOR_RAG=true
-IF_USING_GRAPH_RAG=false
-MODEL_FEATURE_SELECTION_CODER=advanced
+2.Verify file ownership:
 
-Notes
+``` 
+ls -l ./scripts/exp/tools/
+```
 
-	•	Use the -j parameter to scale parallel processes as needed.
-	•	Ensure .env files are correctly formatted to avoid errors.
-	•	Customize test_system.sh as per your project’s requirements.
+### Notes
+* Scale parallel processes as needed using the -j parameter.
+* Avoid errors by ensuring .env files are correctly formatted.
+* Modify test_system.sh to meet your project's specific needs.
+* Add other metrics interested in collect.py to summarize automatically.
 
 For further assistance, refer to the comments within the scripts or reach out to the development team.