TODO

• Update with revised code

Basic settings

• Set the number of GPUs to use:
  NUM_GPUS=num_of_gpus
• Set the rank of the current node (in multi-node training):
  NODE_RANK=rank_of_node
• Set the number of nodes to use (in multi-node training):
  NUM_NODES=num_of_nodes
• Set the address of the master node:
  MASTER_ADDR=addr_of_master
• Set the port of master:
  MASTER_PORT=port_of_master
• Set the GPUs to use:
  CUDA_VISIBLE_DEVICES=gpu_devices

Inference

To train the model, run the following command:

CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES torchrun \
    --nproc_per_node=${NUM_GPUS} \
    --nnodes=${NUM_NODES} \
    --node_rank=${NODE_RANK} \
    --master_addr=${MASTER_ADDR} \
    --master_port=${MASTER_PORT} \
    main.py \
    --name={EXPERIMENT_NAME}\
    --data={DATA_NAME}\
    --data_path={DATA_PATH}\
    --split={SPLIT}         #   {SPLIT} = {'train', 'test', 'val', 'testdev'}