Squelch libibverbs complaints about lack of usnic userspace plugin #565
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Defer initializing the CPCs until we know that we have devices/ports to use. This both prevents some useless work at startup when there are no devices/ports to use, and also prevents librdmacm complaining that there are no verbs-capable RDMA devices available (e.g., if a Cisco usNIC device is present, but does not present a verbs RDMA interface).
libibverbs will complain to stderr if it sees device entries in /sys/class/infiniband for which it has no userspace plugins. The Cisco usNIC device no longer exports a verbs interface, thereby causing libibverbs to emit this annoying stderr warning. To avoid this, use the public ibv API to register a "fake" libibverbs driver at run-time (right after we call ibv_fork_init(), but -- critically -- *before* we call ibv_get_device_list()). The purpose of this driver is solely to convince libibverbs that there *is* a driver for /sys/class/infininband/usnic_verbs devices. ...although this driver will never return a valid ibv context (and therefore will never be used).
When a libibverbs driver returns NULL for its context, it's the Open MPI libibverbs fake driver. Hence, this device is simply not supported -- ignore it.
|
Sent from
|
Refer to this link for build results (access rights to CI server needed): |
jsquyres
added a commit
that referenced
this pull request
May 5, 2015
Squelch libibverbs complaints about lack of usnic userspace plugin
markalle
pushed a commit
to markalle/ompi
that referenced
this pull request
Sep 12, 2020
Add MPI_ROOT to the orted environment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After #555 is merged, install a fake usnic ibv driver so that libibverbs does not complain about a lack of a userspace plugin for usnic_verbs kernel devices.
See comments / commit messages for further explanation.
(note: this PR contains a first commit that is part of #555 -- when #555 is merged, that commit will be removed from this PR)