Error running librispeech_asr_imperceptible_defended.json

[lcadalzo]

File "/opt/conda/lib/python3.8/site-packages/art/defences/preprocessor/mp3_compression.py", line 119, in __call__
raise ValueError("Mp3 compression can only be applied to temporal data across at least one channel.")
ValueError: Mp3 compression can only be applied to temporal data across at least one channel.

To reproduce:

python -m armory run scenario_configs/librispeech_asr_imperceptible_defended.json --check

[davidslater]

This looks like an ART error, at this line:
https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/art/estimators/speech_recognition/pytorch_deep_speech.py#L683

File "/workspace/art/attacks/evasion/imperceptible_asr/imperceptible_asr_pytorch.py", line 474, in _forward_1st_stage                                                                loss, decoded_output = self.estimator.compute_loss_and_decoded_output(                                                                                                                                      │    └ <property object at 0x7f089100c810>                                                                                                                                           └ <art.attacks.evasion.imperceptible_asr.imperceptible_asr_pytorch.ImperceptibleASRPyTorch object at 0x7f08215a92b0>                                          File "/workspace/art/estimators/speech_recognition/pytorch_deep_speech.py", line 619, in compute_loss_and_decoded_output                                                             inputs, targets, input_rates, target_sizes, batch_idx = self._preprocess_transform_model_input(                                                                                                                                              │    └ <function PyTorchDeepSpeech._preprocess_transform_model_input at 0x7f087ea20b80>                                                                                              └ PyTorchDeepSpeech(channels_first=None, model=DeepSpeech(
                                                            (conv): MaskConv(                                                                                                        
                                                              (seq_module): Sequential(                                                                                              
                                                                (0): Conv2d(...                                                                                                      
File "/workspace/art/estimators/speech_recognition/pytorch_deep_speech.py", line 683, in _preprocess_transform_model_input
preprocessed_x_i, _ = self._apply_preprocessing(x=x[i], y=None, no_grad=False)                                                                                                       
                      │    │                      │ └ 0                                                                                                                              
                      │    │                      └ tensor([[ 0.0000e+00, -3.0518e-05,  3.0518e-05,  ..., -1.2207e-04,
                      │    │                                 -1.2207e-04, -9.1553e-05]], device='cuda:0', dtyp...
                      │    └ <function PyTorchEstimator._apply_preprocessing at 0x7f0890fbeaf0>
                      └ PyTorchDeepSpeech(channels_first=None, model=DeepSpeech(
                          (conv): MaskConv(
                            (seq_module): Sequential(
                              (0): Conv2d(...
File "/workspace/art/estimators/pytorch.py", line 209, in _apply_preprocessing
x, y = preprocess(x, y)
│      │          │  └ None
│      │          └ tensor([ 0.0000e+00, -3.0518e-05,  3.0518e-05,  ..., -1.2207e-04,
│      │                    -1.2207e-04, -9.1553e-05], device='cuda:0', dtype=t...
│      └ <art.defences.preprocessor.mp3_compression.Mp3Compression object at 0x7f087c13cc40>
└ tensor([ 0.0000e+00, -3.0518e-05,  3.0518e-05,  ..., -1.2207e-04,
          -1.2207e-04, -9.1553e-05], device='cuda:0', dtype=t...
File "/workspace/art/defences/preprocesso
r/mp3_compression.py", line 119, in __call__
raise ValueError("Mp3 compression can only be applied to temporal data across at least one channel.")
ValueError: Mp3 compression can only be applied to temporal data across at least one channel.

It breaks up the inputs into non-batches (removes the first dimension), but the mp3 defense cannot handle non-batches.

[davidslater]

imperceptible_asr_pytorch.py is incompatible with the numpy-based Mp3Compression defense.

Seeing if this can be mapped to the non-pytorch version.

[davidslater]

The Non-PyTorch version of ImperceptibleASR has been refactored in such a way to make it unable to be generated directly from config.

I will remove this config from armory.

[davidslater]

Actually, there is a Mp3CompressionPyTorch defense. Will attempt to make that work.

[davidslater]

Made a PR into ART for an (seemingly related but actually) unrelated Mp3Compression bug I found:
Trusted-AI/adversarial-robustness-toolbox#1680

[davidslater]

Made a second bug issue for ART that if resolved (and combined with the Mp3Compression fix) should work:
Trusted-AI/adversarial-robustness-toolbox#1683