Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[BUGFIX] gc before fork in loader #1525

Merged
merged 1 commit into from
Feb 22, 2021
Merged

[BUGFIX] gc before fork in loader #1525

merged 1 commit into from
Feb 22, 2021

Conversation

szha
Copy link
Member

@szha szha commented Feb 20, 2021

Description

Same workaround from apache/mxnet#19925 for dataset loader in gluonnlp.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • avoid dangling data before fork by waiting and gc

cc @dmlc/gluon-nlp-team

Signed-off-by: Sheng Zha <zhasheng@amazon.com>
@szha szha requested a review from a team as a code owner February 20, 2021 23:03
@github-actions
Copy link

@codecov
Copy link

codecov bot commented Feb 20, 2021

Codecov Report

Merging #1525 (175272c) into master (0211c65) will increase coverage by 0.26%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1525      +/-   ##
==========================================
+ Coverage   86.27%   86.54%   +0.26%     
==========================================
  Files          55       55              
  Lines        7513     7522       +9     
==========================================
+ Hits         6482     6510      +28     
+ Misses       1031     1012      -19     
Impacted Files Coverage Δ
src/gluonnlp/data/loading.py 81.75% <100.00%> (+5.90%) ⬆️
src/gluonnlp/data/filtering.py 82.60% <0.00%> (+4.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0211c65...175272c. Read the comment docs.

@szha szha requested a review from sxjscience February 21, 2021 18:01
@leezu leezu merged commit 223f1f6 into dmlc:master Feb 22, 2021
@szha szha deleted the multiworker branch February 22, 2021 02:47
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants