torchscript model tests have blind spots

[fmassa]

🐛 Bug

#3027 uncovered that some models in torchvision are not being tested on CI for torchscript compatibility.

I believe the main reason was that scripting a model in the past could take a significant amount of time, which could make the CI timeout, so only models added in

vision/test/test_models.py

Lines 47 to 79 in 8c28175

	script_test_models = {
	'deeplabv3_resnet50': {},
	'deeplabv3_resnet101': {},
	'mobilenet_v2': {},
	'resnext50_32x4d': {},
	'fcn_resnet50': {},
	'fcn_resnet101': {},
	'googlenet': {
	'unwrapper': lambda x: x.logits
	},
	'densenet121': {},
	'resnet18': {},
	'alexnet': {},
	'shufflenet_v2_x1_0': {},
	'squeezenet1_0': {},
	'vgg11': {},
	'inception_v3': {
	'unwrapper': lambda x: x.logits
	},
	'r3d_18': {},
	"fasterrcnn_resnet50_fpn": {
	'unwrapper': lambda x: x[1]
	},
	"maskrcnn_resnet50_fpn": {
	'unwrapper': lambda x: x[1]
	},
	"keypointrcnn_resnet50_fpn": {
	'unwrapper': lambda x: x[1]
	},
	"retinanet_resnet50_fpn": {
	'unwrapper': lambda x: x[1]
	}
	}

seem to be tested for scriptability.

Today, operator fusion happens during the first forward pass of the model.
We should at least check that all models support being converted to torch.jit.script. At best, we should execute the scripted model and compare the outputs with the non-scripted model, but this might introduce a lot of overhead if compilation is slow.