Topic/oshmem mem prefetch v2.x #2872
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The hook is called from memheap when memory range is going to be allocated by smalloc(), realloc() and others. ucx spml uses this hook to call ucp_mem_advise in order to speedup non blocking memory mapping. Signed-off-by: Alex Mikheev <alexm@mellanox.com> (cherry picked from commit 986ca00)
Signed-off-by: Alex Mikheev <alexm@mellanox.com> (cherry picked from commit 9da9e62)
yosefe
reviewed
Jan 29, 2017
oshmem/mca/spml/ucx/spml_ucx.c
Outdated
| &mca_spml_ucx.ucp_peers[i].ucp_conn); | ||
| if (UCS_OK != err) { | ||
| SPML_ERROR("ucp_ep_create failed!!!\n"); | ||
| SPML_ERROR("ucp_ep_create failed: %s\n", ucs_status_string(err)); |
oshmem/mca/spml/ucx/spml_ucx.c
Outdated
| err = ucp_worker_flush(mca_spml_ucx.ucp_worker); | ||
| if (UCS_OK != err) { | ||
| SPML_ERROR("fence failed"); | ||
| SPML_ERROR("fence failed: %s", ucs_status_string(err)); |
oshmem/mca/spml/ucx/spml_ucx.c
Outdated
| err = ucp_worker_flush(mca_spml_ucx.ucp_worker); | ||
| if (UCS_OK != err) { | ||
| SPML_ERROR("fence failed"); | ||
| SPML_ERROR("fence failed: %s", ucs_status_string(err)); |
mike-dubman
reviewed
Jan 29, 2017
|
|
||
| *p_buff = (void*) addr; | ||
| /* no barrier because it is not required by spec! */ | ||
| MCA_SPML_CALL(memuse_hook(addr, 1<<order)); |
Member
There was a problem hiding this comment.
1LL<<order
2nd param is size_t or cast order to (size_t)
mike-dubman
reviewed
Jan 29, 2017
oshmem/mca/spml/ucx/spml_ucx.c
Outdated
|
|
||
| void mca_spml_ucx_memuse_hook(void *addr, size_t length) | ||
| { | ||
| int my_pe = oshmem_my_proc_id(); |
Member
There was a problem hiding this comment.
plz split into var and assignment
Contributor
Author
|
@yosefe @miked-mellanox can you rollback the merge of original comits to master ? I would rather fix it there and than cherry pick again |
mike-dubman
reviewed
Jan 29, 2017
oshmem/mca/spml/ucx/spml_ucx.c
Outdated
|
|
||
| status = ucp_mem_advise(mca_spml_ucx.ucp_context, ucx_mkey->mem_h, ¶ms); | ||
| if (UCS_OK != status) { | ||
| SPML_ERROR("ucp_mem_advise failed addr %p len %llu", |
Member
There was a problem hiding this comment.
maybe warn? as it is not destructive?
Contributor
Author
There was a problem hiding this comment.
There is no option for warning. It is either verbose or error.
Contributor
|
@alex-mikheev i think it would be easier to submit a fix commit both to master and to this PR |
Signed-off-by: Alex Mikheev <alexm@mellanox.com>
mike-dubman
approved these changes
Jan 29, 2017
yosefe
approved these changes
Jan 29, 2017
Member
|
@alex-mikheev - please PR it into v2.x and v2.0 |
Member
|
bot:lanl:retest |
Member
|
@hppritcha Please merge this ASAP. The LANL failure looks unrelated to this commit. |
Member
|
bot:lanl:retest |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@yosefe @miked-mellanox @jladd-mlnx
please take a look