Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to do the implementation with my Dataset #5

Open
vaibhavsah opened this issue Aug 7, 2021 · 12 comments
Open

How to do the implementation with my Dataset #5

vaibhavsah opened this issue Aug 7, 2021 · 12 comments

Comments

@vaibhavsah
Copy link

Hi

I am a newbie to PyTorch. I want to use this model for my thesis. Can you please help me in explaining how to run this code on my dataset. Can you please update the Readme file with simple steps to run and evaluate the model on custom dataset.
@vaibhavsah
Copy link
Author

@rishikksh20 @aarti9
In my dataset I have 6 classes for identification. So on using the code in develop branch, I am getting an error for no support for multi-class classification by the loss function. Please help me out there.
Screenshot 2021-08-09 at 12 02 17 PM

RuntimeError: multi-target not supported at /pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15

Please help me out.

@aarti9
Copy link
Collaborator

aarti9 commented Aug 9, 2021 via email

@vaibhavsah
Copy link
Author

Thanks @aarti9. You may connect with me on sah.vaibhav@gmail.com
I have also mailed you about the issue.

@vaibhavsah
Copy link
Author

As the code was not working, after a bit research, I tried updating the loss function calculation as
loss_func = nn.CrossEntropyLoss(weight=class_weights.to(device))

Where class_weights are calculated as:
tensor([0.0045, 0.0042, 0.0048, 0.0038, 0.0070, 0.0065])

But this also didn't work. It seems there is some issue in the dimensions of target variable in train_epoch function. Please help me fix it.

@aarti9
Copy link
Collaborator

aarti9 commented Aug 9, 2021 via email

@vaibhavsah
Copy link
Author

Earlier I used that only, which was already there in your code. The thing is that I have made some changes in the Dataset Preprocessing to read the videos from the dataframe. But not sure if that is causing the errors.

class DatasetProcessing(data.Dataset):
    def __init__(self, df, root_dir):
        super(DatasetProcessing, self).__init__()
        # List of all videos path
        video_list = df["Video"].apply(lambda x: root_dir + '/' + x)
        # for root, dirs, files in os.walk(videos_path):
        #     for file in files:
        #         fullpath = os.path.join(root, file)
        #         if ('.mp4' in fullpath):
        #             video_list.append(fullpath)
        self.video_list = np.asarray(video_list)

        self.df = df
        # self.framespath = framespath

    def __getitem__(self, index):
        # Ensure that the raw videos are in respective folders and folder name matches the output class label
        video_label = self.video_list[index].split('/')[-2]
        video_name = self.video_list[index].split('/')[-1]
        # pklFileName = os.path.splitext(video_name)[0]
        # with open(self.framespath + '/' + pklFileName + '.pkl', 'rb') as f:
        #     w_list = pickle.load(f)
        #
        # return w_list[0], w_list[1]

        video_frames, len_ = get_frames(self.video_list[index], n_frames = 15)
        print(len(video_frames))
        video_frames = np.asarray(video_frames)
        video_frames = video_frames/255
        class_list = ['Run', 'Walk', 'Wave', 'Sit', 'Turn', 'Stand']
        # print(class_list)
        class_id_loc = np.where(class_list == video_label)
        label = class_id_loc
        d = torch.as_tensor(np.array(video_frames).astype('float'))
        l = torch.as_tensor(np.array(label).astype('float'))
        return (d, l)

    def __len__(self):
        return self.video_list.shape[0]

@vaibhavsah
Copy link
Author

vaibhavsah commented Aug 9, 2021 •
edited

This is the file I have created on Colab. Colab Sheet

@vaibhavsah
Copy link
Author

@aarti9 I changed my approach. Divided my test and train data in folders and used your develop branch from fresh. But a new issue arrived. It's saying that ViViT has no attribute evaluate. What's so strange here ? Please acknowledge.

AttributeError: 'ViViT' object has no attribute 'evaluate'

Screenshot 2021-08-09 at 2 32 10 PM

@vaibhavsah
Copy link
Author

@aarti9 Instead of model.evaluate(), I ran evaluate. That did the trick to run the code.
But the thing is that for any number of epochs, or any values of parameters, the accuracy output is fixed to 21.09%. It seems strange.
There might be some fix needed with how the accuracy is being calculated. Can you please check at your end, what can be the issue.

@2000222
Copy link

2000222 commented Oct 27, 2021

@vaibhavsah Hello, have u fixed the problem of the 21.09% accuracy? Also, I faced some issues about training vivit on custom dataset, could u please share the Colab sheet again, the link has broken. Thanks.

@vaibhavsah
Copy link
Author

vaibhavsah commented Oct 27, 2021 via email

@vaibhavsah
Copy link
Author

@2000222 Sorry, even I couldn't find the solution. Had to switch to basic Transformer model

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@aarti9 @vaibhavsah @2000222