Use GiGL DistServer for DistLoader#487
Conversation
kmontemayor2-sc
requested review from
mkolodner-sc,
nshah-sc,
svij-sc,
xgao4-sc,
yliu2-sc and
zfan3-sc
as code owners
|
/unit_test_py |
|
/integration_test |
|
/e2e_test |
GiGL Automation@ 23:57:49UTC : 🔄 @ 01:19:24UTC : ✅ Workflow completed successfully. |
GiGL Automation@ 23:57:51UTC : 🔄 @ 01:04:08UTC : ❌ Workflow failed. |
GiGL Automation@ 23:57:58UTC : 🔄 @ 01:19:26UTC : ✅ Workflow completed successfully. |
kmontemayor2-sc
changed the title
|
/integration_test |
GiGL Automation@ 01:29:17UTC : 🔄 @ 02:44:42UTC : ✅ Workflow completed successfully. |
|
/unit_test_py |
|
/integration_test |
|
/e2e_test |
GiGL Automation@ 22:28:35UTC : 🔄 @ 23:39:28UTC : ✅ Workflow completed successfully. |
GiGL Automation@ 22:28:37UTC : 🔄 @ 23:43:17UTC : ❌ Workflow failed. |
GiGL Automation@ 22:28:39UTC : 🔄 @ 23:51:26UTC : ❌ Workflow failed. |
4 participants
Scope of work done
turns out, we need to make some modifications to the DistServer in order for it to work at scale for graph store mode (and also how we setup the cluster).
The DistServer has lots of global locks which slow every thing down, when they are occasionally not needed, or can be per-produer. Will sene out follow up PRs to address this.
So copying over more code into out code base to support this. We should more strongly consider something like #483 but in the mean time this should be ~sufficient.
Where is the documentation for this feature?: N/A
Did you add automated tests or write a test plan?
Updated Changelog.md? NO
Ready for code review?: NO