enhance the uids selection process

README.md

@@ -48,7 +48,7 @@
 - [Registering your wallet](#registering-your-wallet)
 - [Running a Miner](#running-a-miner)
 - [Running a Validator](#running-a-validator)
-- [New Releases](#new-releases)
+- [Releases](#releases)
 - [Troubleshooting](#troubleshooting)
   - [Troubleshooting Subtensor](#troubleshooting-subtensor)
 - [License](#license)
@@ -372,28 +372,9 @@ pm2 start neurons/validator.py \
 
 > NOTE: to access the wandb UI to get statistics about the miners, you can click on this [link](https://wandb.ai/eclipsevortext/subvortex-team) and choose the validator run you want.
 
-## New Releases
+## Releases
 
-When a new version of the subnet is released, each miner/validatior have to be updated.
-
-> Be sure you are in the SubVortex directory
-
-Get the lastest version of the subnet
-
-```
-git pull
-```
-
-Install the dependencies
-
-```
-pip install -r requirements.txt
-pip install -e .
-```
-
-Restart miners/validators if running them in your base environment or restart pm2 by executing `pm2 restart all` if you are using pm2 as process manager.
-
-> NOTE: to access the wandb UI to get statistics about the miners, you can click on this [link](https://wandb.ai/eclipsevortext/subvortex-team) and choose the validator run you want.
+- [Release-2.1.0](./scripts/release/release-2.1.0/RELEASE-2.1.0.md)
 
 ## Troubleshooting
 

scripts/redis/docs/redis-backup.md

@@ -0,0 +1,77 @@
+This guide provides step-by-step instructions for creating and restoring a dump in Redis. Redis is an open-source, in-memory data structure store used as a database, cache, and message broker. Dumps are a way to back up and restore data in Redis.
+
+<br />
+
+Table of Contents
+---
+
+- [Create a dump](#create-a-dump)
+- [Restore a dump](#restore-a-dump)
+
+<br />
+
+## Creating a Redis Dump
+
+To create a dump of your Redis database, follow these steps:
+
+1. **Connect to Redis**: Open a terminal or command prompt and connect to your Redis instance using the `redis-cli` command:
+
+   ```bash
+   redis-cli -a $(sudo grep -Po '^requirepass \K.*' /etc/redis/redis.conf)
+   ```
+
+2. **Create the Dump**: Use the `SAVE` command to create a dump of the current database. This command saves the dataset to a file called `dump.rdb` in the Redis data directory.
+
+   ```bash
+   SAVE
+   ```
+
+3. **Create the Dump**: Exit redis environment
+
+   ```bash
+   exit
+   ```
+
+4. **Make a copy**: Copy the dump file `dump.rdb` in `/var/lib/redis` and make a copy of it
+
+   ```bash
+   sudo cp /var/lib/redis/dump.rdb /var/lib/redis/dump.bak.rdb
+   ```
+
+5. **Verify the Dump**: Check that the copy of the dump file (`dump.bak.rdb`) has been created in `/var/lib/redis`.
+   ```bash
+   ls /var/lib/redis
+   ```
+
+## Restoring a Redis Dump
+
+To restore a dump in Redis, follow these steps:
+
+1. **Stop the Redis Server**: If Redis is running, stop the Redis server:
+
+   ```bash
+   sudo systemctl stop redis-server.service
+   ```
+
+2. **Replace the Dump File**: Replace the existing `dump.rdb` file in the Redis data directory with the dump file you want to restore.
+
+3. **Start the Redis Server**: Start the Redis server again.
+
+   ```bash
+   sudo systemctl start redis-server.service
+   ```
+
+4. **Verify the Restoration**: Connect to Redis using `redis-cli` and verify that the data has been restored correctly:
+
+   ```bash
+   redis-cli
+   KEYS *
+   ```
+
+   This command will display all keys in the database, confirming that the restoration was successful.
+
+## Additional Notes
+
+- It's important to ensure that Redis is stopped before replacing the dump file to avoid data corruption.
+
+For more information about Redis and its commands, refer to the [Redis Documentation](https://redis.io/documentation).

scripts/release/release-2.1.0/RELEASE-2.1.0.md

@@ -0,0 +1,179 @@
+This guide provides step-by-step instructions for the release 2.1.0.
+
+Previous Release: 2.0.0
+
+<br />
+
+---
+
+- [Validator](#validators)
+  - [Rollout Process](#validator-rollout-process)
+  - [Rollback Process](#validator-rollback-process)
+- [Miner](#miner)
+  - [Rollout Process](#miner-rollout-process)
+  - [Rollback Process](#miner-rollback-process)
+- [Additional Resources](#additional-resources)
+
+---
+
+<br />
+
+# Validator
+
+## Rollout Process <a id="validator-rollout-process"></a>
+
+1. **Backup Database**: Before starting the rollout process, backup your database using the [Backup Guide](../../redis/docs/redis-backup.md#create-a-dump).
+
+2. **Upgrade Subnet**: Check if you are on main or on a tag
+
+   ```bash
+   git branch -vvv
+   ```
+
+   You will see something similar
+
+   ```bash
+   # If you are on a tag
+   * (HEAD detached at v0.2.4)  d6e233a Merge pull request #13 from eclipsevortex/release/0.2.4
+
+   # If you are on main
+   * main                       13e555e [origin/main] Merge pull request #19 from eclipsevortex/release/2.0.0
+   ```
+
+   > IMPORTANT <br />
+   > The \* tell you your active branch. It has to be hear on the tag on the main branch.
+
+   If you are on a tag branch, checkout main
+
+   ```bash
+   git checkout main
+   ```
+
+   Otherwise/Then, get the latest version of the subnet
+
+   ```bash
+   git pull
+   ```
+
+   Then, install the dependencies
+
+   ```bash
+   pip install -r requirements.txt
+   pip install -e .
+   ```
+
+3. **Restart validator**: Restart your validator to take the new version into the new version
+
+   ```bash
+   pm2 restart validator-92
+   ```
+
+4. **Check logs**: Check the validator logs to see if you see some `New Block`
+   ```bash
+   pm2 logs validator-92
+   ```
+
+<br />
+
+## Rollback Process <a id="validator-rollback-process"></a>
+
+If any issues arise during or after the rollout, follow these steps to perform a rollback:
+
+1. **Rollback Database**: Rollback the database by running in **SubVortex** directory
+
+   ```bash
+   python3 ./scripts/migrations/release-2.1.0/migration.py --run-type rollback
+   ```
+
+   You should see
+
+   ```bash
+    2024-03-29 22:08:27.867 |       INFO       | Loading database from localhost:6379
+    2024-03-29 22:08:27.901 |       INFO       | Rollback starting
+    2024-03-29 22:08:27.907 |       INFO       | Rollback done
+    2024-03-29 22:08:27.908 |       INFO       | Checking rollback...
+    2024-03-29 22:08:27.910 |       INFO       | Rollback checked successfully
+   ```
+
+   If any issue, restore your backup database using the [Backup Guide](../../migrations/backup.md#restore-a-dump).
+
+2. **Downgrade Subnet**: Get the tags
+
+   ```bash
+   git fetch --tags
+   ```
+
+   Check tag v2.0.0 exist
+
+   ```bash
+   git tag
+   ```
+
+   Checkout the tag
+
+   ```bash
+   git checkout tags/v2.0.0
+   ```
+
+   you will see
+
+   ```
+   Note: switching to 'tags/v0.2.4'.
+
+   You are in 'detached HEAD' state. You can look around, make experimental
+   changes and commit them, and you can discard any commits you make in this
+   state without impacting any branches by switching back to a branch.
+
+   If you want to create a new branch to retain commits you create, you may
+   do so (now or later) by using -c with the switch command. Example:
+
+   git switch -c <new-branch-name>
+
+   Or undo this operation with:
+
+   git switch -
+
+   Turn off this advice by setting config variable advice.detachedHead to false
+
+   HEAD is now at d6e233a Merge pull request #13 from eclipsevortex/release/0.2.4
+   ```
+
+   Then install the dependencies
+
+   ```bash
+   pip install -r requirements.txt
+   pip install -e .
+   ```
+
+3. **Restart validator**: Restart your validator to take the new version into the new version
+
+   ```bash
+   pm2 restart validator-92
+   ```
+
+4. **Check logs**: Check the validator logs to see if you see some `New Block`
+   ```bash
+   pm2 logs validator-92
+   ```
+
+<br />
+
+# Miner
+
+## Rollout Process <a id="miner-rollout-process"></a>
+
+There is no rollout for miners.
+
+## Rollback Process <a id="miner-rollback-process"></a>
+
+There is no rollback for miners.
+
+<br />
+
+# Additional Resources
+
+- [Backup Guide](../../redis/docs/redis-backup.md): Detailed instructions for backing up and restoring your database.
+
+<br />
+
+For any further assistance or inquiries, please contact [**SubVortex Team**](https://discord.com/channels/799672011265015819/1215311984799653918)

scripts/release/release-2.1.0/migration.py

@@ -0,0 +1,93 @@
+import asyncio
+import argparse
+import bittensor as bt
+from redis import asyncio as aioredis
+
+from subnet.shared.utils import get_redis_password
+from subnet.shared.checks import check_environment
+
+
+def check_redis(args):
+    try:
+        asyncio.run(check_environment(args.redis_conf_path))
+    except AssertionError as e:
+        bt.logging.warning(
+            f"Something is missing in your environment: {e}. Please check your configuration, use the README for help, and try again."
+        )
+
+
+def rollout():
+    bt.logging.info("No rollout")
+
+
+async def rollback(args):
+    try:
+        bt.logging.info(
+            f"Loading database from {args.database_host}:{args.database_port}"
+        )
+        redis_password = get_redis_password(args.redis_password)
+        database = aioredis.StrictRedis(
+            host=args.database_host,
+            port=args.database_port,
+            db=args.database_index,
+            password=redis_password,
+        )
+
+        bt.logging.info("Rollback starting")
+        async for key in database.scan_iter("selection:*"):
+            await database.delete(key)
+        bt.logging.info("Rollback done")
+
+        bt.logging.info("Checking rollback...")
+        count = 0
+        async for key in database.scan_iter("selection:*"):
+            count += 1
+        if count == 0:
+            bt.logging.info("Rollback checked successfully")
+        else:
+            bt.logging.error(
+                f"Check rollback failed! You still have {count} keys to remove."
+            )
+
+    except Exception as e:
+        bt.logging.error(f"Error during rollback: {e}")
+
+
+async def main(args):
+    if args.run_type == "rollout":
+        rollout()
+    else:
+        await rollback(args)
+
+
+if __name__ == "__main__":
+    try:
+        parser = argparse.ArgumentParser()
+        parser.add_argument(
+            "--run-type",
+            type=str,
+            default="rollout",
+            help="Type of migration you want too execute. Possible values are rollout or rollback)",
+        )
+        parser.add_argument(
+            "--redis_password",
+            type=str,
+            default=None,
+            help="password for the redis database",
+        )
+        parser.add_argument(
+            "--redis_conf_path",
+            type=str,
+            default="/etc/redis/redis.conf",
+            help="path to the redis configuration file",
+        )
+        parser.add_argument("--database_host", type=str, default="localhost")
+        parser.add_argument("--database_port", type=int, default=6379)
+        parser.add_argument("--database_index", type=int, default=1)
+        args = parser.parse_args()
+
+        asyncio.run(main(args))
+    except KeyboardInterrupt:
+        print("KeyboardInterrupt")
+    except ValueError as e:
+        print(f"ValueError: {e}")