2.3.0 - Post-release analysis

[p-ferreira]

• explore incentive landscape (p1) @p-ferreira @bkb2135
  • create a generic script to facilitate future development
• load all current runs and look for repetitive challenges (p2) @bkb2135
  • create a generic script to facilitate future development
• ensuring that validators fail gracefully (p3) @dbobrenko
  • Reference run: https://wandb.ai/opentensor-dev/alpha-validators/runs/9qwcfbit?nw=nwuserpedromacrocosmos
  • PR: Fix repeated tasks on CUDA OOM #243

[p-ferreira]

@bkb2135 results on load all current runs and look for repetitive challenges (p2):
Challenges are not repeated, except from validator Rizzo, which has a pretty high presence of NaNs. I'm not totally sure why, but other validators are fine
https://macrocosmosworkspace.slack.com/archives/C06ULL8R84Q/p1716489431129969?thread_ts=1716489389.636009&cid=C06ULL8R84Q

[p-ferreira]

Code that facilitates the creation of historic tables on network params (like incentive, emission, consensus, etc) from the metagraph. The code iterates back over the metagraph given a block_start, last_n_steps and block interval step:

from tqdm.notebook import tqdm 
import pandas as pd
import bittensor as bt

def create_uid_hotkey_history_df(netuid, last_n_steps=30, step=100, block_start = None, block_stop = None):
    subtensor = bt.subtensor(network='archive')
    metagraph = subtensor.metagraph(netuid, lite=False)

    if block_start is not None:
        metagraph.sync(block=block_start)

    current_block = metagraph.block

    data = []

    for interval in tqdm(list(range(0, last_n_steps))):
        block = current_block - (interval * step)

        if block_stop is not None:
            if block <= block_stop:
                break

        metagraph.sync(block=int(block), lite=True)
        bt.logging.info(f'Syncing block {block}')


        for uid in metagraph.uids:            
            hotkey = metagraph.hotkeys[uid]
            emission = metagraph.emission[uid]
            coldkey = metagraph.coldkeys[uid]
            stake = metagraph.S[uid]
            consensus = metagraph.C[uid]
            incentive = metagraph.I[uid]

            #series = pd.Series([uid.item(), hotkey, emission.item(), block.item(), coldkey, stake.item(), consensus.item()], index=df.columns)
            data.append([uid.item(), hotkey, emission.item(), block.item(), coldkey, stake.item(), consensus.item(), incentive.item()])

    df = pd.DataFrame(data, columns=['uid', 'hotkey', 'emission', 'block', 'coldkey', 'stake', 'consensus', 'incentive'])
    return df

[p-ferreira]

Regarding explore incentive landscape (p1):

The plot shows that we did had a weird period around the incentives but things seems to be improved once the majority of validators were running the latest version.

Data:
https://huggingface.co/datasets/macrocosm-os/experiments/blob/main/historic/block_3001610_to_3021510.parquet

[p-ferreira]

Continuing to monitor the incentives, we can observe that the network has aligned itself with a slight increase of incentives over the weekend following the release.

[p-ferreira]

New goals for subnet health analysis:

• Identify impact of weight copying on S1 (@p-ferreira)