ValueError: could not broadcast input array when extract features from CAD dataset

[qdinfish]

Hi
When I try to run extract features from CAD dataset, got following error. Could you help check it?

data_confs Namespace(batch_size={'test': 120, 'trainval': 120}, data_type='img', dataset_folder='/media/dev1/0C78B85C78B845EC/database/CAD/imgs_ranked', label_type='activity')
AlexNet_LSTM(
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
(1): ReLU(inplace)
(2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(4): ReLU(inplace)
(5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(7): ReLU(inplace)
(8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(9): ReLU(inplace)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(11): ReLU(inplace)
(12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(fc): Sequential(
(0): Dropout(p=0.5)
(1): Linear(in_features=9216, out_features=4096, bias=True)
(2): ReLU(inplace)
)
(LSTM): LSTM(4096, 3000, batch_first=True)
(classifier): Linear(in_features=3000, out_features=5, bias=True)
)
trainval 18290
The features files are created at /media/dev1/0C78B85C78B845EC/database/CAD/feas/activity/trainval.npy

0 / 18290
Traceback (most recent call last):
File "GAR.py", line 41, in
Action.extractFeas()
File "/home/dev1/Group-Activity-Recognition-1/Runtime/Action_Level.py", line 58, in extractFeas
feas[i*batch_size:(i+1)*batch_size,:-1] = fea.data.cpu().numpy()
ValueError: could not broadcast input array from shape (10,85152) into shape (10,35480)

[ruiyan1995]

For old version code，I just run test on VD，so you need to modify the batchsize=5 in Configs for CAD！I have updated my code to adapt both VD and CAD. You can check it on github.------------------ Original ------------------ From: "qdinfish"<notifications@github.com> Date: Mon, Jul 22, 2019 10:31 PM To: "ruiyan1995/Group-Activity-Recognition"<Group-Activity-Recognition@noreply.github.com>; Cc: "Subscribed"<subscribed@noreply.github.com>; Subject: [ruiyan1995/Group-Activity-Recognition] ValueError: could notbroadcast input array when extract features from CAD dataset (#6) Hi When I try to run extract features from CAD dataset, got following error. Could you help check it? data_confs Namespace(batch_size={'test': 120, 'trainval': 120}, data_type='img', dataset_folder='/media/dev1/0C78B85C78B845EC/database/CAD/imgs_ranked', label_type='activity') AlexNet_LSTM( (features): Sequential( (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2)) (1): ReLU(inplace) (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False) (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2)) (4): ReLU(inplace) (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False) (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (7): ReLU(inplace) (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (9): ReLU(inplace) (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (11): ReLU(inplace) (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False) ) (fc): Sequential( (0): Dropout(p=0.5) (1): Linear(in_features=9216, out_features=4096, bias=True) (2): ReLU(inplace) ) (LSTM): LSTM(4096, 3000, batch_first=True) (classifier): Linear(in_features=3000, out_features=5, bias=True) ) trainval 18290 The features files are created at /media/dev1/0C78B85C78B845EC/database/CAD/feas/activity/trainval.npy 0 / 18290 Traceback (most recent call last): File "GAR.py", line 41, in Action.extractFeas() File "/home/dev1/Group-Activity-Recognition-1/Runtime/Action_Level.py", line 58, in extractFeas feas[i*batch_size:(i+1)*batch_size,:-1] = fea.data.cpu().numpy() ValueError: could not broadcast input array from shape (10,85152) into shape (10,35480) &mdash; You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[qdinfish]

I got the folloiwng erro with the new code :(

Please wait for tracking and ranking! about 240min + 180min
Traceback (most recent call last):
File "GAR.py", line 22, in
Pre.Processing(opt.dataset_root, opt.dataset_name, operation='track', ranked=True)
File "/home/dev1/newcode/Group-Activity-Recognition/Pre/Processing.py", line 20, in init
model_confs = Configs.Model_Configs(dataset_name, 'action').configuring()
TypeError: 'module' object is not callable

[qdinfish]

I updated the old version code and it works fine for the CAD , but the Confusion matrix is as below, looks queue and waiting can't be differentiated. Your thougts ?

probs = F.softmax(outputs.data)
Classification report for classifier
precision recall f1-score support

       0       0.63      0.99      0.77       209
       1       1.00      0.14      0.25       141
       2       0.97      1.00      0.98        90
       3       1.00      0.99      1.00       181

micro avg 0.80 0.80 0.80 621
macro avg 0.90 0.78 0.75 621
weighted avg 0.87 0.80 0.75 621

Confusion matrix:
[[ 98.56 0. 1.44 0. ]
[ 85.82 14.18 0. 0. ]
[ 0. 0. 100. 0. ]
[ 0.55 0. 0. 99.45]]
0.7987117552334944
[[206 0 3 0]
[121 20 0 0]
[ 0 0 90 0]
[ 1 0 0 180]]
infer one sequence, takes 0.00754817403649 s

[ruiyan1995]

It's too easy to overfit on CAD, so you should change the lr. I think it should achieve at least 90% performance.

[qdinfish]

I changed lr from 0.0001 to 0.00001 or 0.001, 0.8099838969404187 is the best result :(.
Do you have used other trick ? Thansk a lot!

Confusion matrix:
[[ 98.09 0.48 1.44 0. ]
[ 77.3 22.7 0. 0. ]
[ 0. 0. 100. 0. ]
[ 2.76 0. 0. 97.24]]
0.8099838969404187
[[205 1 3 0]
[109 32 0 0]
[ 0 0 90 0]
[ 5 0 0 176]]

[ruiyan1995]

I changed lr from 0.0001 to 0.00001 or 0.001, 0.8099838969404187 is the best result :(.
Do you have used other trick ? Thansk a lot!

Confusion matrix:
[[ 98.09 0.48 1.44 0. ]
[ 77.3 22.7 0. 0. ]
[ 0. 0. 100. 0. ]
[ 2.76 0. 0. 97.24]]
0.8099838969404187
[[205 1 3 0]
[109 32 0 0]
[ 0 0 90 0]
[ 5 0 0 176]]

Step One: make sure that your cnn+lstm feature is right;
Step Two: a small batch_size=500 and init_lr=0.0001
I have a good result at the first epoch! It's very easy to overfit.

[qdinfish]

I used the same lr and batch ( batch_size=500 and init_lr=0.0001), the Acc can reach 1 and test acc is not stable , it can reach from 0.69 to 0.90.
I'm not sure whether the exact feature from cnn+lstm are correct, could you attach your extract files ? Then I can try it in my server.

[ruiyan1995]

I used the same lr and batch ( batch_size=500 and init_lr=0.0001), the Acc can reach 1 and test acc is not stable , it can reach from 0.69 to 0.90.
I'm not sure whether the exact feature from cnn+lstm are correct, could you attach your extract files ? Then I can try it in my server.

The test acc is indeed really unstable. Nobody explains for it in papers or releases the scripts for CAD, and I hold that the small size of CAD is the main reason for it. If you got the acc of 90% for it, the code has been worked now. If you have any other idea for it, please feel free to contact me.