Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

06 of 10 LNX series - Add FI_PEER as a capability #10030

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

06 of 10 LNX series - Add FI_PEER as a capability #10030

wants to merge 5 commits into from
+59 −22

Commits on May 16, 2024

  1. util: Consider provider exclusion 

    When checking fabric attributes with ofi_check_fabric_attr() make sure to
consider provider exclusion.

When checking to see if a provider name is given, only consider ones which
are not excluded using the '^' character.

Signed-off-by: Amir Shehata <shehataa@ornl.gov>
    @amirshehataornl
    amirshehataornl committed May 16, 2024
    Configuration menu
    Copy the full SHA
    71eb274 View commit details
    Browse the repository at this point in the history

  2. include/rdma: Add address matching call back to peer_srx 

    It is not efficient to do a reverse lookup on the AV table when a message
is received. Some providers do not store the fi_addr_t associated with the
peer in the header passed on the wire. And it is not practical to require
providers to add that to wire header, as it would break backwards
compatibility.

In order to handle this case, an address matching callback is added to the
peer_srx.peer_ops structure. This allows the provider receiving the
message to register an address matching callback. This callback is called
by the owner provider to match an fi_addr_t with provider specific address
in the message received.

The callback allows the receiving provider to do an O(1) index into the AV
table to lookup the address of the peer, and then compare that with the
source address in the received message.

As part of this change provider specific address information needs to be
passed to the owner provider, which the owner will need to give back to the
receiving provider, when it attempts to do address matching.

Update the SHM and LINKx providers to conform with the API changes

Signed-off-by: Amir Shehata <shehataa@ornl.gov>
    @amirshehataornl
    amirshehataornl committed May 16, 2024
    Configuration menu
    Copy the full SHA
    bcc5181 View commit details
    Browse the repository at this point in the history

  3. include/rdma: new fi_peer_match structure 

    Add a new structure fi_peer_match to collect the parameters which need
to be passed to the get_msg and get_tag functions.

Update the util_get_tag() and util_get_msg() function callbacks.
Compilation gives a warning but not failing. This causes memory
corruption when the callbacks are called.

Signed-off-by: Amir Shehata <shehataa@ornl.gov>
    @amirshehataornl
    amirshehataornl committed May 16, 2024
    Configuration menu
    Copy the full SHA
    3ddf85f View commit details
    Browse the repository at this point in the history

  4. srx: Add memory registration cb to fi_ops_srx_peer 

    Add a memory registration callback to the fi_ops_srx_peer. This allows
core providers to expose a memory registration callback which the parent
or peer provider can use to register memory on the receive path.

For example the CXI provider registers memory with the NIC on the receive
path. When using the peer infrastructure this can not happen because we
do not know which provider will perform the receive operation. But if
the source NID is specified then we can know and therefore we can
perform the receive buffer registration at the top of the receive path.

Signed-off-by: Amir Shehata <shehataa@ornl.gov>
    @amirshehataornl
    amirshehataornl committed May 16, 2024
    Configuration menu
    Copy the full SHA
    293275d View commit details
    Browse the repository at this point in the history

  5. fabric: Add FI_PEER capability 

    Add FI_PEER capability bit

Signed-off-by: Amir Shehata <shehataa@ornl.gov>
    @amirshehataornl
    amirshehataornl committed May 16, 2024
    Configuration menu
    Copy the full SHA
    5f915c9 View commit details
    Browse the repository at this point in the history