Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Obtain source rank from psm2_mq_status ? #10

Closed
danghvu opened this issue Mar 24, 2017 · 3 comments
Closed

Obtain source rank from psm2_mq_status ? #10

danghvu opened this issue Mar 24, 2017 · 3 comments

Comments

@danghvu
Copy link

danghvu commented Mar 24, 2017

Section 4.3.2.1 specifies a msg_source in the field of psm2_mq_status.

However I do not find this field in the master branch code: https://github.com/01org/opa-psm2/blob/master/psm2_mq.h#L380

What is the proper way to obtain the source rank?

@rwmcguir
Copy link
Contributor

It appears the PDF file is incorrect, digging through the history (far older that what git hub contains) I don't see any reference to this member variable. I will have the PDF corrected.

Let me get back with you on obtaining the source rank.

@rwmcguir
Copy link
Contributor

Assuming you are using PSM2 natively and not through MPI. In this context rank is purely a MPI defined notion. PSM2 does not need rank information to operate or distinguish messages. If you are using MPI in conjunction with PSM2 you will need to exchange rank information yourself and build a lookup table using epaddrs. You can also try to extract it from the tag. PSM2 doesn't track rank information internally other than for attempting load balancing for HW resources during job startup.

The preferred method is to switch to the API using the psm2_mq_status2 structure. This has the remote epaddr as part of the status field.

/** Remote peer's epaddr */
psm2_epaddr_t msg_peer;

It is also recommend if you look into the psm2_mq.h file for examples. A good number of refernces on how MPI uses tags and rank id is present. The new tags are larger in psm2 vs psm1 so more information can be packed in, so any place you see data structures or APIs that are duplicated except with a 2 in them, that is expansion for PSM2 using the 96 bit tags (vs 48 bits in TrueScale PSM1).

I hope this assists you.

@danghvu
Copy link
Author

danghvu commented Mar 24, 2017

Thanks, turns out it's easier to use the tag interface to transfer the source id. Update to the pdf will be great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants