the difference between err1_spatical and err1

[jiaxue-ai]

when I adjusted your code to the method that use colour images only with 3D filter (get ride of fusion layer), an interesting thing I find is that the result of err1 and err1_spatical are always different, observe from the code, err1_spatial is extracted from dagnn.Loss layers

function stats = extractStats(net)
% -------------------------------------------------------------------------
sel = find(cellfun(@(x) isa(x,'dagnn.Loss'), {net.layers.block})) ;
stats = struct() ;
for i = 1:numel(sel)
stats.(net.layers(sel(i)).name) = net.layers(sel(i)).block.average ;
end

and err1 is computed by comparing difference between label and prediction

function [err1, err5] = error_multiclass(opts, labels, predictions)
% -------------------------------------------------------------------------
[~,predictions] = sort(predictions, 3, 'descend') ;
error = ~bsxfun(@eq, predictions, reshape(labels, 1, 1, 1, [])) ;
err1 = sum(sum(sum(error(:,:,1,:)))) ;
err5 = sum(sum(sum(min(error(:,:,1:5,:),[],3)))) ;

so what's the relation between them?

[jiaxue-ai]

and the predictions for err1 is that

[frame_predictions{:,fr}] = deal(net.vars(pred_layers).value) ;

[feichtenhofer]

err1 is the final test error which uses both streams and multiple frames for predicitions.