Torch 1.10

[LysandreJik]

Authorize torch 1.10

[JohnnyC08]

As a side note. I'm excited that label smoothing is now integrated into cross entropy! https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html

Excited to replace my current implementation with this.

[LysandreJik]

@anton-l @patrickvonplaten @Narsil there's the following error when passing to torch 1.10:

udioClassificationPipelineTests.test_small_model_pt _____________
[gw0] linux -- Python 3.7.12 /usr/local/bin/python

self = <tests.test_pipelines_audio_classification.AudioClassificationPipelineTests testMethod=test_small_model_pt>

    @require_torch
    def test_small_model_pt(self):
        model = "anton-l/wav2vec2-random-tiny-classifier"
    
        audio_classifier = pipeline("audio-classification", model=model)
    
        audio = np.ones((8000,))
        output = audio_classifier(audio, top_k=4)
        self.assertEqual(
            nested_simplify(output, decimals=4),
            [
                {"score": 0.0843, "label": "on"},
                {"score": 0.0840, "label": "left"},
                {"score": 0.0837, "label": "off"},
>               {"score": 0.0835, "label": "yes"},
            ],
        )
E       AssertionError: Lists differ: [{'score': 0.0842, 'label': 'no'}, {'score': 0.0838, 'lab[77 chars]ht'}] != [{'score': 0.0843, 'label': 'on'}, {'score': 0.084, 'labe[77 chars]es'}]
E       
E       First differing element 0:
E       {'score': 0.0842, 'label': 'no'}
E       {'score': 0.0843, 'label': 'on'}
E       
E       - [{'label': 'no', 'score': 0.0842},
E       ?              -                 ^
E       
E       + [{'label': 'on', 'score': 0.0843},
E       ?             +                  ^
E       
E       -  {'label': 'up', 'score': 0.0838},
E       ?             ^^                ^^
E       
E       +  {'label': 'left', 'score': 0.084},
E       ?             ^^^^                ^
E       
E       -  {'label': 'go', 'score': 0.0837},
E       ?             -
E       
E       +  {'label': 'off', 'score': 0.0837},
E       ?              ++
E       
E       -  {'label': 'right', 'score': 0.0834}]
E       ?             ^^^^^                 ^
E       
E       +  {'label': 'yes', 'score': 0.0835}]

Do you know where it might come from?

[anton-l]

@LysandreJik I can't track down which commit made the outputs different yet. The checkpoint certainly didn't change. Is it only torch 1.10 that gets affected?

[LysandreJik]

It seems so!

[sgugger]

Thanks for all of your work on this upgrade!