Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UCP: Copy memh flags from parent to child #9891

Merged
merged 1 commit into from
May 28, 2024

Conversation

iyastreb
Copy link
Contributor

What

UCP_MEMH_FLAG_IMPORTED was added in #8584, as the only memory handle flag. But this flag is only set in the memh parent, and never propagated to user memh. Despite that, flag is used on user memh:

ucp_am_params_check_memh(const ucp_request_param_t *param, uint32_t *flags_p)
{
 ...
    if (ucs_unlikely(param->memh->flags & UCP_MEMH_FLAG_IMPORTED)) {
        if (ENABLE_PARAMS_CHECK &&
            ucs_unlikely(*flags_p & UCP_AM_SEND_FLAG_EAGER)) {
            return UCS_ERR_INVALID_PARAM;
        }

        *flags_p |= UCP_AM_SEND_FLAG_RNDV;
    }

I think this flag must copied from parent to child in ucp_memh_init_from_parent, so that we can distinguish whether user memh is attached to imported memory or not

brminich
brminich previously approved these changes May 21, 2024
@yosefe
Copy link
Contributor

yosefe commented May 22, 2024

@iyastreb can we add a unit test for it?

@iyastreb
Copy link
Contributor Author

@iyastreb can we add a unit test for it?

Sure, good idea indeed!

@iyastreb iyastreb force-pushed the ucp/copy-memh-flags-parent branch 2 times, most recently from 41f7582 to b7b605d Compare May 23, 2024 06:50
@@ -1744,6 +1745,7 @@ ucp_memh_import_slow(ucp_context_h context, ucs_rcache_t *existing_rcache,
user_memh->parent = memh;
} else {
memh = user_memh;
memh->flags |= UCP_MEMH_FLAG_IMPORTED;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can set it once for both cases outside of if..else on line 1752 (and remove on line 1744)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

}

void test_ucp_mmap::test_rereg_imported_mem(ucp_mem_h memh,
uint64_t memh_pack_flags,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems memh_pack_flags is not really needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

uint64_t memh_pack_flags,
size_t size)
ucp_mem_h test_ucp_mmap::import_memh(ucp_mem_h exported_memh,
uint64_t memh_pack_flags)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems this method is always packign exported key, so memh_pack_flags is not really nedeed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, true, I'll remove this param from this function and 2 others

~mem_chunk();
ucp_rkey_h unpack(ucp_ep_h, ucp_md_map_t md_map = 0);
};
struct mem_chunk {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe move mem_chunk to test_ucp_mmap, so it will not be in top-level namespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, why not

EXPECT_FALSE(mem.memh->flags & UCP_MEMH_FLAG_IMPORTED);

ucp_mem_h imported_memh = import_memh(mem.memh);
EXPECT_TRUE(!!(imported_memh->flags & UCP_MEMH_FLAG_IMPORTED));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!! probably not needed here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@iyastreb iyastreb force-pushed the ucp/copy-memh-flags-parent branch from 4415e56 to 2e6e8f7 Compare May 27, 2024 07:36
@brminich brminich merged commit 1b0f7f1 into openucx:master May 28, 2024
142 checks passed
@iyastreb iyastreb deleted the ucp/copy-memh-flags-parent branch May 28, 2024 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants