parsing of track_grad_norm when passed as argument

[tshrjn]

When using Trainer.from_argparse_args & passing track_grad_norm as cli arg.
Following error is thrown:

track_grad_norm can be an int, a float or 'inf' (infinity norm).

To Reproduce

Run python demo.py --gpus "1," --track_grad_norm 2

Code sample

import argparse
from pytorch_lightning import Trainer
from pl_bolts.models.gans import BasicGAN

trainer = Trainer()

parser = argparse.ArgumentParser(description='demo')
parser = trainer.add_argparse_args(parser)
args = parser.parse_args()

model = BasicGAN(args)
trainer = Trainer.from_argparse_args(args)
trainer.fit(model)

Environment

* CUDA:
        - GPU:
                - GeForce RTX 2080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.1
        - pyTorch_debug:     False
        - pyTorch_version:   1.6.0
        - pytorch-lightning: 0.9.0rc16
        - tensorboard:       2.2.0
        - tqdm:              4.45.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                -
        - processor:         x86_64
        - python:            3.6.10
        - version:           #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019

[tshrjn]

Also, I think bolts can easily be used for continuous testing with a few basic models & iter.combination of all Trainer args for say 10 steps each. This would greatly improve the catching of small issues.

[tshrjn]

Another similar issue is with limit_train_batches which according to the docs can be a float, but is expected by the argparse to be a float. Thus, again motivating a better testing mechanism.

[awaelchli]

argparse does not support multiple types. The only solution I see is to define it as string and parse it back to the real type when we load the args.