Skip to content
This repository was archived by the owner on May 29, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f7d8dd9
feat: add support for the latest autonomy framework version
Adamantios Jan 3, 2025
726df76
feat: remove all unused docker networks
Adamantios Jan 3, 2025
74366c5
chore: bump trader version
annasambrook Jan 8, 2025
ed85d14
chore: update tools accuracy hash
annasambrook Jan 8, 2025
6958a53
update: delete mech chain id
annasambrook Jan 8, 2025
8d317ee
Merge pull request #418 from valory-xyz/chore/update-tools-accuracy-hash
Adamantios Jan 8, 2025
2bd7aad
Merge branch 'refs/heads/develop' into chore/bump-trader-version
annasambrook Jan 8, 2025
a58f487
chore: bump trader version
annasambrook Jan 8, 2025
96481f1
update: remove mech chain id
annasambrook Jan 8, 2025
c59648f
Merge pull request #417 from valory-xyz/chore/bump-trader-version
Adamantios Jan 8, 2025
74c9ec0
chore: bump trader version
annasambrook Jan 8, 2025
448e4dd
update: delete mech chain id
annasambrook Jan 8, 2025
37e7944
add: analyse agent logs script
annasambrook Jan 8, 2025
b1c403b
enhance: add all analyse logs options
annasambrook Jan 9, 2025
e568145
update: readme
annasambrook Jan 9, 2025
f3a0872
fix: update the mech subgraph's url
Adamantios Jan 7, 2025
e6a2fba
Update analyse_logs.py
annasambrook Jan 9, 2025
403f318
Update README.md
annasambrook Jan 9, 2025
f1f3d57
Revert "chore: bump trader version"
annasambrook Jan 9, 2025
430f5de
Revert "update: delete mech chain id"
annasambrook Jan 9, 2025
8820802
Merge remote-tracking branch 'origin/feat/analyse-agent-logs-script' …
annasambrook Jan 9, 2025
3b4ac02
feat: remove all unused docker networks
Adamantios Jan 3, 2025
14cc168
Merge pull request #421 from valory-xyz/fix/subgraph
Adamantios Jan 9, 2025
41c026c
Revert "feat: remove all unused docker networks"
annasambrook Jan 9, 2025
351aa4a
Merge pull request #420 from valory-xyz/feat/analyse-agent-logs-script
Adamantios Jan 9, 2025
8be5390
Merge pull request #416 from valory-xyz/feat/support-latest-framework
Adamantios Jan 9, 2025
8293536
update RPC
dagacha Jan 9, 2025
bd2f877
Merge pull request #422 from valory-xyz/docs/update-minor
Adamantios Jan 9, 2025
f6ffaa1
fix: check k8s and docker filepaths for logs
annasambrook Jan 9, 2025
4ab47b2
fix: hardcoded names of the docker containers
Adamantios Jan 9, 2025
5990673
Merge pull request #424 from valory-xyz/fix/docker-containers-names
Adamantios Jan 9, 2025
7646122
fix: fallback in case an old framework version is used
Adamantios Jan 9, 2025
5ecc661
fix: iterate through build directories to find logs
annasambrook Jan 9, 2025
d1bed2f
Merge pull request #423 from valory-xyz/feat/analyse-agent-logs-script
Adamantios Jan 9, 2025
2f01c60
Merge pull request #425 from valory-xyz/fix/fallback
Adamantios Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Ensure your machine satisfies the requirements:
## Resource Requirements

- You need xDAI on Gnosis Chain in one of your wallets.
- You need an RPC for your agent instance. We recommend [Nodies RPC](https://www.nodies.app/).
- You need an RPC for your agent instance. We recommend [Quicknode RPC](https://www.quicknode.com/).
- (From release v0.16.0 onwards) You will need a Subgraph API key that can be obtained at [The Graph](https://thegraph.com/studio/apikeys/).

## Run the Service
Expand Down Expand Up @@ -105,7 +105,7 @@ Services can become staked by invoking the `stake()` contract method, where serv
Once the command has completed, i.e. the service is running, you can see the live logs with:

```bash
docker logs trader_abci_0 --follow
docker logs $(docker ps --filter "name=trader" --format "{{.Names}}" | grep "_abci" | head -n 1) --follow
```

To stop your agent, use:
Expand Down Expand Up @@ -153,16 +153,16 @@ Note: In this case, if the service is staked, then it will not update the on-cha
cd trader; poetry run python ../report.py; cd ..
```

3. Use this command to investigate your agent's logs:
3. Use the `analyse_logs.py` script to investigate your agent's logs:

```bash
cd trader; poetry run autonomy analyse logs --from-dir trader_service/abci_build/persistent_data/logs/ --agent aea_0 --reset-db; cd ..
cd trader; poetry run python ../analyse_logs.py --agent aea_0 --reset-db; cd ..
```

For example, inspect the state transitions using this command:
For example, inspect the state transitions using the following command:

```bash
cd trader; poetry run autonomy analyse logs --from-dir trader_service/abci_build/persistent_data/logs/ --agent aea_0 --fsm --reset-db; cd ..
cd trader; poetry run python ../analyse_logs.py --agent aea_0 --fsm --reset-db; cd ..
```

This will output the different state transitions of your agent per period, for example:
Expand Down Expand Up @@ -411,7 +411,11 @@ Error: Service terminatation failed with following error; ChainInteractionError(

## Build deployments without executing the service

The script builds both a Docker Compose deployment (on `./trader/trader_service/abci_build`) and a Kubernetes deployment (on `./trader/trader_service/abci_build_k8s`). Then, by default, the script will launch the local Docker Compose deployment. If you just want to build the deployment without executing the service (for example, if you are deploying to a custom Kubernetes cluster), then execute the script as
The script builds both a Docker Compose deployment (on `./trader/trader_service/abci_build_????`)
and a Kubernetes deployment (on `./trader/trader_service/abci_build_k8s`).
Then, by default, the script will launch the local Docker Compose deployment.
If you just want to build the deployment without executing the service
(for example, if you are deploying to a custom Kubernetes cluster), then execute the script as:

```bash
./run_service.sh --build-only
Expand Down
153 changes: 153 additions & 0 deletions analyse_logs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
import os
import subprocess
import sys
import argparse


def _parse_args():
"""Parse the script arguments."""
parser = argparse.ArgumentParser(description="Analyse agent logs.")

parser.add_argument(
"--service-dir",
default="trader_service",
help="The service directory containing build directories (default: 'trader_service')."
)
parser.add_argument(
"--from-dir",
help="Path to the logs directory. If not provided, it is auto-detected."
)
parser.add_argument(
"--agent",
default="aea_0",
help="The agent name to analyze (default: 'aea_0')."
)
parser.add_argument(
"--reset-db",
action="store_true",
help="Use this flag to disable resetting the log database."
)
parser.add_argument(
"--start-time",
help="Start time in `YYYY-MM-DD H:M:S,MS` format."
)
parser.add_argument(
"--end-time",
help="End time in `YYYY-MM-DD H:M:S,MS` format."
)
parser.add_argument(
"--log-level",
choices=["INFO", "DEBUG", "WARNING", "ERROR", "CRITICAL"],
help="Logging level."
)
parser.add_argument(
"--period",
type=int,
help="Period ID."
)
parser.add_argument(
"--round",
help="Round name."
)
parser.add_argument(
"--behaviour",
help="Behaviour name filter."
)
parser.add_argument(
"--fsm",
action="store_true",
help="Print only the FSM execution path."
)
parser.add_argument(
"--include-regex",
help="Regex pattern to include in the result."
)
parser.add_argument(
"--exclude-regex",
help="Regex pattern to exclude from the result."
)

return parser.parse_args()


def find_build_directory(service_dir):
"""Find the appropriate build directory within the service directory."""
try:
# create a list of all build directories
build_dirs = [
d for d in os.listdir(service_dir)
if d.startswith("abci_build_") and os.path.isdir(os.path.join(service_dir, d))
]
# iterate through the build directories to find the one that contains logs
for build_dir in build_dirs:
build_dir = os.path.join(service_dir, build_dir)
logs_dir = os.path.join(build_dir, "persistent_data", "logs")
# Check if the logs folder exists and contains files
if os.path.exists(logs_dir) and os.listdir(logs_dir):
return build_dir
return os.path.join(service_dir, "abci_build")
except FileNotFoundError:
print(f"Service directory '{service_dir}' not found")
sys.exit(1)


def run_analysis(logs_dir, **kwargs):
"""Run the log analysis command."""
command = [
"poetry", "run", "autonomy", "analyse", "logs",
"--from-dir", logs_dir,
]
if kwargs.get("agent"):
command.extend(["--agent", kwargs.get("agent")])
if kwargs.get("reset_db"):
command.extend(["--reset-db"])
if kwargs.get("start_time"):
command.extend(["--start-time", kwargs.get("start_time")])
if kwargs.get("end_time"):
command.extend(["--end-time", kwargs.get("end_time")])
if kwargs.get("log_level"):
command.extend(["--log-level", kwargs.get("log_level")])
if kwargs.get("period"):
command.extend(["--period", kwargs.get("period")])
if kwargs.get("round"):
command.extend(["--round", kwargs.get("round")])
if kwargs.get("behaviour"):
command.extend(["--behaviour", kwargs.get("behaviour")])
if kwargs.get("fsm"):
command.extend(["--fsm"])
if kwargs.get("include_regex"):
command.extend(["--include-regex", kwargs.get("include_regex")])
if kwargs.get("exclude_regex"):
command.extend(["--exclude-regex", kwargs.get("exclude_regex")])

try:
subprocess.run(command, check=True)
print("Analysis completed successfully.")
except subprocess.CalledProcessError as e:
print(f"Command failed with exit code {e.returncode}")
sys.exit(e.returncode)
except FileNotFoundError:
print("Poetry or autonomy not found. Ensure they are installed and accessible.")
sys.exit(1)


if __name__ == "__main__":
# Parse user arguments
args = _parse_args()

# Determine the logs directory
if args.from_dir:
logs_dir = args.from_dir
if not os.path.exists(logs_dir):
print(f"Specified logs directory '{logs_dir}' not found.")
sys.exit(1)
else:
# Auto-detect the logs directory
build_dir = find_build_directory(args.service_dir)
logs_dir = os.path.join(build_dir, "persistent_data", "logs")
if not os.path.exists(logs_dir):
print(f"Logs directory '{logs_dir}' not found.")
sys.exit(1)

# Run the analysis
run_analysis(logs_dir, **vars(args))
29 changes: 16 additions & 13 deletions report.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,9 @@
SECONDS_PER_DAY = 60 * 60 * 24

OUTPUT_WIDTH = 80
TRADER_CONTAINER_PREFIX = "trader"
AGENT_CONTAINER_IDENTIFIER = "abci"
NODE_CONTAINER_IDENTIFIER = "tm"


class ColorCode:
Expand Down Expand Up @@ -238,19 +241,19 @@ def _warning_message(current_value: int, threshold: int = 0, message: str = "")

def _get_agent_status() -> str:
client = docker.from_env()
trader_abci_container = (
client.containers.get("trader_abci_0")
if "trader_abci_0" in [c.name for c in client.containers.list()]
else None
)
trader_tm_container = (
client.containers.get("trader_tm_0")
if "trader_tm_0" in [c.name for c in client.containers.list()]
else None
)

is_running = trader_abci_container and trader_tm_container
return _color_bool(is_running, "Running", "Stopped")
agent_running = node_running = service_running = False
for container in client.containers.list():
container_name = container.name
if TRADER_CONTAINER_PREFIX in container_name:
if AGENT_CONTAINER_IDENTIFIER in container_name:
agent_running = True
if NODE_CONTAINER_IDENTIFIER in container_name:
node_running = True
if agent_running and node_running:
service_running = True
break

return _color_bool(service_running, "Running", "Stopped")


def _parse_args() -> Any:
Expand Down
29 changes: 21 additions & 8 deletions run_service.sh
Original file line number Diff line number Diff line change
Expand Up @@ -737,7 +737,7 @@ directory="trader"
service_repo=https://github.com/$org_name/$directory.git
# This is a tested version that works well.
# Feel free to replace this with a different version of the repo, but be careful as there might be breaking changes
service_version="v0.21.4"
service_version="v0.22.0"

# Define constants for on-chain interaction
gnosis_chain_id=100
Expand Down Expand Up @@ -832,10 +832,15 @@ command -v docker >/dev/null 2>&1 ||
exit 1
}

docker rm -f abci0 node0 trader_abci_0 trader_tm_0 &> /dev/null ||
containers=$(docker ps --filter name=trader_* -aq) &> /dev/null ||
{ echo >&2 "Docker is not running!";
exit 1
}
if [[ -n "$containers" ]]; then
docker rm -f $containers
fi

docker network prune --force

try_read_storage

Expand Down Expand Up @@ -1275,8 +1280,7 @@ export DISABLE_TRADING=false
export STOP_TRADING_IF_STAKING_KPI_MET=true
export RESET_PAUSE_DURATION=45
export MECH_WRAPPED_NATIVE_TOKEN_ADDRESS=$WXDAI_ADDRESS
export MECH_CHAIN_ID=ethereum
export TOOLS_ACCURACY_HASH=QmebjcPizAdVFSUAfMBgAGFJhLPVBMvV68LxhSq4LPvv9d
export TOOLS_ACCURACY_HASH=QmZSkE49cnp3KeR9r6bp3hP4M2LPAmG4beHq4isz55ghv5

if [ -n "$SUBGRAPH_API_KEY" ]; then
export CONDITIONAL_TOKENS_SUBGRAPH_URL="https://gateway-arbitrum.network.thegraph.com/api/$SUBGRAPH_API_KEY/subgraphs/id/7s9rGBffUTL8kDZuxvvpuc46v44iuDarbrADBFw5uVp2"
Expand All @@ -1287,9 +1291,9 @@ if [ -n "$SUBGRAPH_API_KEY" ]; then
fi

service_dir="trader_service"
build_dir="abci_build"
directory=$(ls -d "$service_dir"/abci_build_???? 2>/dev/null || echo "$service_dir/abci_build")
build_dir=$(basename "$directory")
build_dir_k8s="abci_build_k8s"
directory="$service_dir/$build_dir"

if [ -d $directory ]
then
Expand Down Expand Up @@ -1329,19 +1333,28 @@ if [[ -d "$build_dir_k8s" ]]; then
echo "Directory removed: $build_dir"
fi
export OPEN_AUTONOMY_PRIVATE_KEY_PASSWORD="$password" && poetry run autonomy deploy build --kubernetes "../../$keys_json_path" --n $n_agents -ltm
build_dir=$(ls -d abci_build_???? 2>/dev/null || echo "abci_build")
mv $build_dir $build_dir_k8s
echo "Kubernetes deployment built on ./trader/$service_dir/$build_dir_k8s"

export OPEN_AUTONOMY_PRIVATE_KEY_PASSWORD="$password" && poetry run autonomy deploy build "../../$keys_json_path" --n $n_agents -ltm
build_dir=$(ls -d abci_build_???? 2>/dev/null || echo "abci_build")
echo "Docker Compose deployment built on ./trader/$service_dir/$build_dir"

cd ..

# warm start is disabled as no global weights are provided to calibrate the tools' weights
# warm_start

add_volume_to_service_docker_compose "$PWD/trader_service/abci_build/docker-compose.yaml" "trader_abci_0" "/data" "$path_to_store"
add_volume_to_service_k8s "$PWD/trader_service/abci_build_k8s/build.yaml"
directory="$service_dir/$build_dir"
if [ "$build_dir" = "abci_build" ]; then
suffix="abci_build"
else
suffix=${build_dir##*_}
fi
abci_0="trader${suffix}_abci_0"
add_volume_to_service_docker_compose "$PWD/$directory/docker-compose.yaml" "$abci_0" "/data" "$path_to_store"
add_volume_to_service_k8s "$PWD/$service_dir/$build_dir_k8s/build.yaml"
sudo chown -R $(whoami) "$path_to_store"

if [[ "$build_only" == true ]]; then
Expand Down
Loading