-
Notifications
You must be signed in to change notification settings - Fork 198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add multiple GPU support #760 #924
Add multiple GPU support #760 #924
Conversation
I found an issue of the inconsistency of
Possible scenario:
So, from DTD 1.12,
Also I will rename |
bae92f4
to
df2fc0c
Compare
Lines 637 to 639 in 58c81fb
|
27b4a61
to
e6ff394
Compare
1f793fa
to
76d547b
Compare
@larsbijl @bcipriano I think we can start review technical design and code since, at least all test passed. |
in the services menu of cuegui |
76d547b
to
a958c3a
Compare
will update (migration V10 -> V11, VERSION 0.9 -> 0.10) when #936 merged |
a958c3a
to
d2a82dc
Compare
Co-authored-by: Lars van der Bijl <285658+larsbijl@users.noreply.github.com>
Co-authored-by: Lars van der Bijl <285658+larsbijl@users.noreply.github.com>
Co-authored-by: Lars van der Bijl <285658+larsbijl@users.noreply.github.com>
Co-authored-by: Lars van der Bijl <285658+larsbijl@users.noreply.github.com>
e942206
to
dad1300
Compare
@bcipriano @larsbijl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did one last review, changes LGTM.
After the changes on the gpu PR AcademySoftwareFoundation#924 the performance of the booking query degraded up to 4 times the previous throughput. Creating some indexes for columns that changed names seems to have fixed the problem. Signed-off-by: Diego Tavares <dtavares@imageworks.com>
* Add new indexes to improve booking performance After the changes on the gpu PR #924 the performance of the booking query degraded up to 4 times the previous throughput. Creating some indexes for columns that changed names seems to have fixed the problem. Signed-off-by: Diego Tavares <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> --------- Signed-off-by: Diego Tavares <dtavares@imageworks.com> Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com>
…tion#1304) * Add new indexes to improve booking performance After the changes on the gpu PR AcademySoftwareFoundation#924 the performance of the booking query degraded up to 4 times the previous throughput. Creating some indexes for columns that changed names seems to have fixed the problem. Signed-off-by: Diego Tavares <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> --------- Signed-off-by: Diego Tavares <dtavares@imageworks.com> Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com>
…tion#1304) * Add new indexes to improve booking performance After the changes on the gpu PR AcademySoftwareFoundation#924 the performance of the booking query degraded up to 4 times the previous throughput. Creating some indexes for columns that changed names seems to have fixed the problem. Signed-off-by: Diego Tavares <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> * Update cuebot/src/main/resources/conf/ddl/postgres/migrations/V18_Add_New_Indexes Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com> --------- Signed-off-by: Diego Tavares <dtavares@imageworks.com> Signed-off-by: Diego Tavares da Silva <dtavares@imageworks.com>
…columns - Fix the column indexing on the "addColumn" of class CueJobMonitorTree. - This bug was introduced after the merge from the pull request "Add multiple GPU support AcademySoftwareFoundation#760 (AcademySoftwareFoundation#924)" on 4/18/22 at 11:45 AM where the following new columns were introduced on the CueJobMonitorTree: "Gpus", "Min Gpus", "Max Gpus", "MaxGpuMem" and the indexing of the columns were wrongly defined.
Fix "Monitor Cue" with incorrect column indexing for "Min" and "Max" columns - Fix the column indexing on the "addColumn" of class CueJobMonitorTree. - This bug was introduced after the merge from the pull request "Add multiple GPU support #760 (#924)" on 4/18/22 at 11:45 AM where the following new columns were introduced on the CueJobMonitorTree: "Gpus", "Min Gpus", "Max Gpus", "MaxGpuMem" and the indexing of the columns were wrongly defined.
…dation#1431) Fix "Monitor Cue" with incorrect column indexing for "Min" and "Max" columns - Fix the column indexing on the "addColumn" of class CueJobMonitorTree. - This bug was introduced after the merge from the pull request "Add multiple GPU support AcademySoftwareFoundation#760 (AcademySoftwareFoundation#924)" on 4/18/22 at 11:45 AM where the following new columns were introduced on the CueJobMonitorTree: "Gpus", "Min Gpus", "Max Gpus", "MaxGpuMem" and the indexing of the columns were wrongly defined.
Plan
Keeping original code and order as much as possible. This pull request concentrates on adding multiple GPU support. No intention of doing cosmetic changes.
This is the first iteration/attempt of shipping multiple GPU support. It may not cover all the use-cases.
Commits
Add GetDefault and SetDefault to AllcationInterface #939 and Fix PyOutline test #942
Add DB migration for supporting multiple GPU
Sync with V10 migration
Bump minor version number
Add Job Spec DTD 1.12
Update proto files for multiple GPU support
Sync with proto changes
Replace gpu with gpus and gpu_memory
[RQD] Support multiple GPUs with nvidia-smi
[PyOutline] Support gpus and gpu_memory
gpu
gpu_memory
= The amount of GPU memorygpus
= The number of GPU units[PyCue] Support gpus and gpu_memory
[cuegui] Sync with proto changes
python setup.py test