Skip to content

Conversation

@jiata
Copy link
Contributor

@jiata jiata commented Jun 24, 2020

Description

set num workers to 0 for action rec

Related Issues

Checklist:

  • I have followed the contribution guidelines and code style for this project.
  • I have added tests covering my contributions.
  • I have updated the documentation accordingly.
  • This PR is being made to staging and not master
  • I will squash merge this PR into staging

@jiata jiata requested a review from PatrickBue as a code owner June 24, 2020 19:47
@PatrickBue
Copy link
Contributor

PatrickBue commented Jun 24, 2020

@jiata Out of interest, why is this change needed?

@jiata
Copy link
Contributor Author

jiata commented Jun 24, 2020

@jiata Out of interest, why is this change needed?

bc of that test that is failing in staging. for that particular machine the DataLoader requires more than what that vm has available. On my machine, i dont see this error.

pytorch/pytorch#8976

shuffle=False,
num_workers=db_num_workers(),
num_workers=0,
# num_workers=db_num_workers(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about setting num_workers = db_num_workers(8) and if that doesn't work then 4, 2, or eventually just 0 as done here? Having 0 workers significantly slows down image reading, which for action recognition likely makes a big difference (it does for image classification)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying with significantly minified data since this is a memory issue

@jiata jiata requested a review from PatrickBue June 25, 2020 20:53
import pytest
import scrapbook as sb

from utils_cv.common.data import unzip_url
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] "unzip_url" not used

@jiata jiata merged commit d570167 into staging Jun 26, 2020
@jiata jiata deleted the jiata-staging-ci branch June 26, 2020 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants