Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trinity.C: MT_SHADOW_ENUMERATOR needs to be polished #15

Closed
yatli opened this issue Feb 24, 2017 · 0 comments
Closed

Trinity.C: MT_SHADOW_ENUMERATOR needs to be polished #15

yatli opened this issue Feb 24, 2017 · 0 comments
Assignees
Milestone

Comments

@yatli
Copy link

yatli commented Feb 24, 2017

https://github.com/Microsoft/GraphEngine/blob/multi_cell_lock/src/Trinity.C/src/Storage/MTHash/MT_SHADOW_ENUMERATOR.cpp#L43

The enumerator reports the status code from local memory storage use cell operation back to LMS enumeration routine, but if someone deleted the cell, the status code should be E_CELL_NOT_FOUND, and will cause the enumeration routine to stop the enumeration on the current trunk and move to the next one before all other entries are enumerated.

Also, https://github.com/Microsoft/GraphEngine/blob/multi_cell_lock/src/Trinity.C/src/Storage/MTHash/MT_SHADOW_ENUMERATOR.cpp#L20

Currently the memory cost of an enumerator grows linear to the data size. This is bad if we have a lot of threads doing enumerations together.

@yatli yatli self-assigned this Feb 24, 2017
@yatli yatli added this to the Planned milestone Mar 1, 2017
@yatli yatli mentioned this issue Dec 10, 2017
5 tasks
yatli pushed a commit that referenced this issue Dec 10, 2017
@yatli yatli closed this as completed Dec 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant