Determine --maxSharedMemory automatically #206
Comments
4 tasks
vasdommes
changed the title
vasdommes
added a commit
that referenced
this issue
Mar 26, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Mar 26, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Mar 27, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Mar 27, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Mar 27, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Mar 27, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Mar 28, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Mar 28, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Mar 28, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Mar 28, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 2, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Apr 2, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 4, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Apr 4, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 8, 2024
Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Apr 8, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 13, 2024
Our memory estimates turned out to be very imprecise (by a factor of ~2), so we do not want to override --maxSharedMemory explicitly set by user. Before calling initialize_bigint_syrk_context(), we look at /proc/meminfo MemAvailable and calculate how much memory will be allocated for different BigFloat matrices (excluding what was already allocated, e.g. SDP or X,Y). We set --maxSharedMemory to 90% of remaining RAM. What will be allocated by SDP_Solver: - dX, dY (from step()) - R, Z (from compute_search_direction()) - A_X_inv, A_Y - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q by approx_objective: - schur_complement, schur_complement_cholesky - schur_off_diagonal - Q - new SDP (created in quadratic_approximate_objectives): bilinear_bases, bases_blocks, free_var_matrix TODO: check for other temporary matrices, see if we can ignore some of them TODO: account for buffers for reduce-scatter Q. We can do it in BigInt_Shared_Memory_Syrk_Context, when we know the split factor.
vasdommes
added a commit
that referenced
this issue
Apr 13, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 17, 2024
…fers in --maxSharedMemory limit (update for #206 Determine --maxSharedMemory automatically) Strictly speaking, these buffers are not shared-memory. But their size depends on output_window_split_factor, so it is natural to account them together with shared memory windows.
vasdommes
added a commit
that referenced
this issue
Apr 17, 2024
Fix #206 Determine --maxSharedMemory automatically, if not set by user.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
See details in parent issue #207
--maxSharedMemorycan be set by user, see #203When creating shared memory windows, we should also check available memory, estimate how much memory we'll need for another parts of computation, and see how much shared memory we can actually use.
The text was updated successfully, but these errors were encountered: