Textual Inversion error (TestTubeLogger)

[wasyleque]

Linux. GTX1080Ti 11GB, everything else working, but after command

python3 ./main.py -t --base ./configs/stable-diffusion/v2-finetune.yaml --actual_resume ./models/ldm/stable-diffusion-v1/model.ckpt -n ewa_tmp --gpus 0 --data_root ./ewa_tmp/ --init_word 'ewunia'

i got this error:

Restored from ./models/ldm/stable-diffusion-v1/model.ckpt with 0 missing and 2 unexpected keys
Unexpected Keys: ['model_ema.decay', 'model_ema.num_updates']
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/piotr/Documents/InvokeAI/./main.py:784 in │
│ │
│ 781 │ │ else: │
│ 782 │ │ │ logger_cfg = OmegaConf.create() │
│ 783 │ │ logger_cfg = OmegaConf.merge(default_logger_cfg, logger_cfg) │
│ ❱ 784 │ │ trainer_kwargs['logger'] = instantiate_from_config(logger_cfg) │
│ 785 │ │ │
│ 786 │ │ # modelcheckpoint - use TrainResult/EvalResult(checkpoint_on=metric) to │
│ 787 │ │ # specify which metric is used to determine best models │
│ │
│ /home/piotr/Documents/InvokeAI/ldm/util.py:90 in instantiate_from_config │
│ │
│ 87 │ │ elif config == 'is_unconditional': │
│ 88 │ │ │ return None │
│ 89 │ │ raise KeyError('Expected key target to instantiate.') │
│ ❱ 90 │ return get_obj_from_str(config['target'])( │
│ 91 │ │ **config.get('params', dict()), **kwargs │
│ 92 │ ) │
│ 93 │
│ │
│ /home/piotr/Documents/InvokeAI/ldm/util.py:100 in get_obj_from_str │
│ │
│ 97 │ if reload: │
│ 98 │ │ module_imp = importlib.import_module(module) │
│ 99 │ │ importlib.reload(module_imp) │
│ ❱ 100 │ return getattr(importlib.import_module(module, package=None), cls) │
│ 101 │
│ 102 │
│ 103 def _do_parallel_data_prefetch(func, Q, data, idx, idx_to_fn=False): │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: module 'pytorch_lightning.loggers' has no attribute 'TestTubeLogger'

[VincentLu91]

I came cross this error before and heard that TestTubeLogger is no longer supported in pytorch_lightning versions 1.7 and up. So you might want to install an earlier version of pytorch_lightning i.e., 1.6 and try training again

[Any-Winter-4079]

On M1 we switched to CSVLogger (to use pytorch lightning 1.7.X)
I'd suggest both cuda and m1 use CSVLogger.
For example, we have:

        # default logger configs
        if torch.cuda.is_available():
            def_logger = 'testtube'
            def_logger_target = 'TestTubeLogger'
        else:
            def_logger = 'csv'
            def_logger_target = 'CSVLogger'

But we could unify it so everyone uses CSVLogger.

[wasyleque]

I came cross this error before and heard that TestTubeLogger is no longer supported in pytorch_lightning versions 1.7 and up. So you might want to install an earlier version of pytorch_lightning i.e., 1.6 and try training again
Thanks everything working, but in manual there is an error

https://invoke-ai.github.io/InvokeAI/features/TEXTUAL_INVERSION/
python3 ./main.py -t
--base ./configs/stable-diffusion/v1-finetune.yaml
--actual_resume ./models/ldm/stable-diffusion-v1/model.ckpt
-n my_cat
--gpus 0
--data_root D:/textual-inversion/my_cat
--init_word 'cat'

There should be:
--gpus 0, \

[lstein]

There's no reason not to change to use CSVLogger on al platforms. @Any-Winter-4079 could you make this fix and fix the docs too? I have never been able to run Textual Inversion on my system without crashing (this is an old problem, and goes back to pre-1.0) and can't test even minor changes.

[lstein]

Just made PR #1190 that should fix this issue.

[Any-Winter-4079]

#1190 has been merged, so I'm closing the issue.