add subplot?

[xwjBupt]

Hi!

it would be very helpful if the metric can be add in one tag like the tensorboard?

eg:

writer.add_scalar(f'train_detail_loss/WholeLoss' %
                      , WholeLoss, epoch)

writer.add_scalar(f'train_detail_loss/OriginalLoss' %
                     OriginalLoss, epoch)

besides, if the number of params is too big, the web page is all blank after ' aim up ' command?

thank you!!

[SGevorg]

Hi @xwjBupt, for subplots use context arguments in track functions.

Here is how it works. For example you could write:
aim.track('loss', WholeLoss, epoch, loss_type='whole')
aim.track('loss', OriginalLoss, epoch, loss_type='original')

As a result you would have two separate metrics - one for whole loss and another for original loss. Also you would have then available context.loss_type for search and other comparison features. You have freedom to use any group of parameters here as long as it makes sense for you.

Here is an experiment (with 3 losses: subset=val, subset=test, subset=train and other metrics).
You can use context to search, group, divide into charts etc. Here is an example to play with.

This adds lots more flexibility than the TensorBoard format (train_detail_loss/WholeLoss) as you can add more context for your metrics to apply in search and additional aggregation, comparison purposes.

More about track function and context in docs
Also docs on how to search using context.
Here is also a full code example.

[SGevorg]

@xwjBupt I pinned this Q for other users in case they have same question.
Thanks for raising this issue ⭐

[xwjBupt]

Hi @SGevorg thanks for your reply, but you did not answer my second question,which is if the params is too big,the page will be blank?

[SGevorg]

@xwjBupt sure we are looking into it.
Opening a separate issue with high priority to address the second part. If possible could you share samples of params you have used?
It's possible the issue could come from a format of the params or something of that sort

[xwjBupt]

@SGevorg sure,following is the code i use:

import torch
import time
import argparse
import loguru
from loguru import logger
import aim

from aim.pytorch_lightning import AimLogger

from argparse import ArgumentParser

import torch
import pytorch_lightning as pl
from torch.nn import functional as F
from torch.utils.data import DataLoader, random_split
import torch.nn as nn
from tqdm import tqdm

try:
    from torchvision.datasets.mnist import MNIST
    from torchvision import transforms
except Exception as e:
    from tests.base.datasets import MNIST
from aim import Session


class Net(nn.Module):
    def __init__(self, ):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(28 * 28, 2048)
        self.l2 = torch.nn.Linear(2048, 10)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = torch.relu(self.l1(x))
        x = torch.relu(self.l2(x))
        return x


if __name__ == '__main__':
    parser = argparse.ArgumentParser('setup record')

    # default method log
    parser.add_argument("--method", default='baseline-gt')
    parser.add_argument('--description', default='1e-6_cosann@20_wce-aspp',
                        help='baseline on brats18_subset')
    parser.add_argument("--seed", default=2564)

    parser.add_argument("--debug", default=True)  # for wandb to record
    # for wandb to record
    parser.add_argument("--commit_info", default='for train')

    parser.add_argument("--netname", default='Unet3D')
    parser.add_argument("--add_tail", default=False)

    parser.add_argument("--gtcount", default=True)
    parser.add_argument("--perception", default=False)
    parser.add_argument("--DDP", default=False)

    # dataset
    parser.add_argument("--dataset", default='Brats2018_subset',
                        choices=['Brats2019_Kflod', 'Brats2018_Kfold', 'Brats2018_subset', 'Brats2020_Kfold',
                                 'Brats2018_Val2019', 'Brats2019', 'Brats2018', 'Brats2020'])
    parser.add_argument("--datasetkwargs",
                        default={'shape': 128, 'nonzero': False})
    parser.add_argument("--datasplit", default=None)
    parser.add_argument("--datafolds", default=5)
    parser.add_argument("--dataselect", default=None)
    parser.add_argument("--dataratio", default=0.2)
    parser.add_argument("--dataaug", default=['Flip', 'Rotate', 'Intensity', ],
                        help='Elastic,Rotate,Flip,Gamma,Rician,Gaussian,Scale,Intensity,Brightness,ALL,None')

    # loss
    parser.add_argument("--lossname", default='WCE_Dice',
                        choices=['BCEWithLogitsLoss', 'BCEDiceLoss', 'CrossEntropyLoss', 'WeightedCrossEntropyLoss',
                                 'PixelWiseCrossEntropyLoss', 'GeneralizedDiceLoss', 'DiceLoss', 'TagsAngularLoss',
                                 'MSELoss', 'SmoothL1Loss', 'L1Loss', 'WeightedSmoothL1Loss', 'WCE_Dice'])
    parser.add_argument(
        "--losskwargs", default={'wce_weight': 0.1})
    parser.add_argument("--lossweight", default=None)
    parser.add_argument("--loss_balance_weight", default=[0.00075, 0.75])

    # metric
    parser.add_argument("--best_WholeLoss", default=float('inf'))
    parser.add_argument("--best_OriginalLoss", default=float('inf'))
    parser.add_argument("--best_OriginalLoss_epoch", default=-1)
    parser.add_argument("--best_WholeLoss_epoch", default=-1)

    # optimizer and scheuler
    parser.add_argument("--lr", default=5e-7)
    parser.add_argument("--opt", default='adam',
                        choices=['adam', 'sgd'])
    parser.add_argument("--num_gpus", default=[0])
    parser.add_argument("--bs", default=8)
    parser.add_argument("--works", default=8)

    parser.add_argument("--maingpu_bs", default=2)
    parser.add_argument("--warmup_epoch", default=5)
    parser.add_argument("--warmup_method", default='Exp_up',
                        choices=['Linear_up', 'Sin_up', 'Exp_up'])
    parser.add_argument("--momentum", default=0.95)
    parser.add_argument("--lr_decaystart", default=25)

    parser.add_argument("--lr_changer", default='cosann',
                        choices=['cosine', 'step', 'expotential', 'rop', 'cosann', 'cosstart', 'null'])
    parser.add_argument("--MultiStepLR_milestones", default=10)
    parser.add_argument("--MultiStepLR_gamma", default=0.995)

    parser.add_argument("--StepLR_step_size", default=10)
    parser.add_argument("--StepLR_gamma", default=0.995)

    parser.add_argument("--CosineAnnealingLR_T_max", default=131)
    parser.add_argument("--CosineAnnealingLR_eta_min", default=1e-8)

    parser.add_argument("--ExponentialLRepLR_gamma", default=0.95)

    parser.add_argument("--ReduceLROnPlateauLR_mode", default='max')
    parser.add_argument("--ReduceLROnPlateauLR_factor", default=0.75)
    parser.add_argument("--ReduceLROnPlateauLR_patience", default=15)
    parser.add_argument("--ReduceLROnPlateauLR_min_lr", default=5e-9)

    parser.add_argument("--CosineAnnealingWarmRestarts_T0", default=30)
    parser.add_argument("--CosineAnnealingWarmRestarts_Tmul", default=1)
    parser.add_argument("--CosineAnnealingWarmRestarts_min", default=1e-9)
    # resume and pretrian
    parser.add_argument('--pretrained',
                        default=  # None
                        'best_baseline_on_20.pth'
                        # '/home/xwj/code/MyBrats/Brats2018/Unet-baseline/init16-wbias-128/07-22_10-02/fold_all/model/best_OriginalLoss_epoch_0153_0.1205.pth'
                        )

    # display
    parser.add_argument("--epochs", default=20, type=int)
    parser.add_argument("--epoch_dis", default=50)
    parser.add_argument("--iter_dis", default=60)
    parser.add_argument("--start_epoch", default=0)
    parser.add_argument("--dense_val", default=250)

    parser.add_argument("--resume",
                        default=None
                        # '/tf//songc/XU/MyBrats/Brats2020/Unet-baseline/wgtcount-wcorner-w10perception7-@ploss0.0892-smallLr-fix/09-25_02-11/fold_all/model/last_check_fold_all.pth'
                        )

    args = parser.parse_args()
    dic = vars(args)

    timestamp = time.strftime("%m-%d_%H-%M", time.localtime())
    a = {'bs': 12, 'method': 'tryaim', 'time': timestamp}
    # aim.set_params(a, name='hparams')  # Passing name argument is optional
    hyperparam_dict = {
        'learning_rate': 0.01,
        'batch_size': 12}
    myaimlogger = Session(experiment='line')
    myaimlogger.set_params(dic, name='params')

    dataset = MNIST('', train=True, download=True, transform=transforms.ToTensor())
    mnist_test = MNIST('', train=False, download=True, transform=transforms.ToTensor())
    mnist_train, mnist_val = random_split(dataset, [55000, 5000])

    train_loader = DataLoader(mnist_train, batch_size=args.bs)
    val_loader = DataLoader(mnist_val, batch_size=args.bs)
    test_loader = DataLoader(mnist_test, batch_size=args.bs)
    net = Net().cuda()
    opt = torch.optim.Adam(net.parameters(), lr=0.00001)

    for i in range(args.epochs):
        for index, data in tqdm(enumerate(train_loader)):
            x, y = data
            x = x.cuda()
            y = y.cuda()
            out = net(x)
            loss = F.cross_entropy(out, y)

            myaimlogger.track(loss.item(), name='loss', epoch=index, subset='train', dataset='train',loss_type='original')
            myaimlogger.track(loss.item() * 5, name='loss', epoch=index, subset='train', dataset='train',loss_type='original_5')

            opt.zero_grad()
            loss.backward()
            opt.step()

        logger.info('train epoch %d loss %.5f' % (i, loss.item()))
        for index, data in tqdm(enumerate(val_loader)):
            x, y = data
            x = x.cuda()
            y = y.cuda()
            out = net(x)
            loss = F.cross_entropy(out, y)
            myaimlogger.track(loss.item(), name='val_loss', epoch=index, subset='val', dataset='val')
            myaimlogger.track(loss.item() * 5, name='val_loss_5', epoch=index, subset='val', dataset='val')

        logger.info('val epoch %d loss %.5f' % (i, loss.item()))

[SGevorg]

Thanks @xwjBupt a lot for this much info! Looking into it.

[SGevorg]

@xwjBupt the fix is in: #257 (comment)
Could you confirm if the solution works?