Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

efa: Elastic Fabric Adapter (EFA) userspace RDMA provider #475

Merged
merged 5 commits into from May 27, 2019
Merged

efa: Elastic Fabric Adapter (EFA) userspace RDMA provider #475

merged 5 commits into from May 27, 2019

Conversation

gal-pressman
Copy link
Contributor

Introduce a provider that exposes EFA devices to user applications via
the verbs interface.

@rleon
Copy link
Member

rleon commented Feb 25, 2019

Please post it to ML too, it is new driver.

Thanks

@gal-pressman
Copy link
Contributor Author

I'll send it to the list as well.

jgunthorpe
jgunthorpe requested changes Feb 25, 2019
View reviewed changes
Copy link
Member

@jgunthorpe jgunthorpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you outline how much of the standard example 'tests' work with this driver? Does ud_pingpong work?

README.md Outdated Show resolved Hide resolved
debian/control Outdated Show resolved Hide resolved
debian/control Outdated Show resolved Hide resolved
debian/ibverbs-providers.install Outdated Show resolved Hide resolved
debian/libibverbs-dev.install Outdated Show resolved Hide resolved
providers/efa/verbs.c Outdated Show resolved Hide resolved
redhat/rdma-core.spec Outdated Show resolved Hide resolved
redhat/rdma-core.spec Outdated Show resolved Hide resolved
redhat/rdma-core.spec Outdated Show resolved Hide resolved
suse/rdma-core.spec Outdated Show resolved Hide resolved
@gal-pressman
Copy link
Contributor Author

Thanks for the review Jason.
Regarding your question, ibv_ud_pingpong is working.
perftest's ib_send_lat and ib_send_bw over UD are working as well.

@gal-pressman
Copy link
Contributor Author

Patches sent to the mailing list:
https://patchwork.kernel.org/cover/10829741/

jgunthorpe
jgunthorpe reviewed Mar 5, 2019
View reviewed changes
ccan/CMakeLists.txt Outdated Show resolved Hide resolved
jgunthorpe
jgunthorpe reviewed Mar 5, 2019
View reviewed changes
Copy link
Member

@jgunthorpe jgunthorpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See

https://github.com/rustyrussell/ccan/blob/master/ccan/ilog/ilog.h

Maybe use that stuff instead

ccan/align.h Outdated Show resolved Hide resolved
ccan/align.h Outdated Show resolved Hide resolved
@gal-pressman
Copy link
Contributor Author

See

https://github.com/rustyrussell/ccan/blob/master/ccan/ilog/ilog.h

Maybe use that stuff instead

We can use the examples in the comments for fls and roundup.
I can copy ilog.h to ccan and use it in util/align.h, what do you think?

@jgunthorpe
Copy link
Member

My general guidance is to use ccan stuff when it makes sense..

In this case the badly named 'align queue size' functions seems to be some kind of 'round up power of 2' which is basically
1 << (ilog2(val -1) + 1)

Right? So it seems like a reasonable choice to use it and provide a better named wrapper for this common case

@gal-pressman
Copy link
Contributor Author

My general guidance is to use ccan stuff when it makes sense..

In this case the badly named 'align queue size' functions seems to be some kind of 'round up power of 2' which is basically
1 << (ilog2(val -1) + 1)

Right? So it seems like a reasonable choice to use it and provide a better named wrapper for this common case

Will do

bdrung
bdrung requested changes Mar 7, 2019
View reviewed changes
debian/ibverbs-providers.lintian-overrides Outdated Show resolved Hide resolved
debian/control Outdated Show resolved Hide resolved
@gal-pressman
Copy link
Contributor Author

Let me know if you want me to rebase or wait until the review is over.

@rleon
Copy link
Member

rleon commented Mar 12, 2019

We are waiting for a refreshed version for both kernel and user space.

Thanks

@gal-pressman
Copy link
Contributor Author

Userspace is updated, kernel will be pushed later.

jgunthorpe
jgunthorpe requested changes May 10, 2019
View reviewed changes
Copy link
Member

@jgunthorpe jgunthorpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also make sure to update the kernel-headers part with the final commit id

ccan/compiler.h Outdated Show resolved Hide resolved
ccan/ilog.h Show resolved Hide resolved
libibverbs/verbs.h Show resolved Hide resolved
providers/efa/efa.c Show resolved Hide resolved
providers/efa/man/efadv_create_driver_qp.3.md Outdated Show resolved Hide resolved
util/util.h Outdated Show resolved Hide resolved
@gal-pressman
Copy link
Contributor Author

Updated the pull request, thanks.

@gal-pressman
Copy link
Contributor Author

Added the check

@gal-pressman
Add align and roundup_pow_of_two helpers
7a4e013 
Add align and roundup_pow_of_two helper functions to util common code.
Adapt mlx4 and mlx5 providers to use these functions, EFA will follow in
downstream patch.

ilog.[ch] and compiler.h were added to ccan in order to implement the
functions.

Signed-off-by: Gal Pressman <galpress@amazon.com>
@gal-pressman
Update kernel headers
267e528 
To commit b79656ed44c6 ("RDMA/ipoib: Allow user space differentiate
between valid dev_port").

Signed-off-by: Gal Pressman <galpress@amazon.com>
@gal-pressman
efa: Elastic Fabric Adapter (EFA) userspace RDMA provider
f7f275f 
Introduce a provider that exposes EFA devices to user applications via
the verbs interface.

Signed-off-by: Gal Pressman <galpress@amazon.com>
@gal-pressman
efa: Elastic Fabric Adapter (EFA) direct verbs
b14693d 
In addition to verbs implementation, efa provider implements direct verb
for QP creation which allows users to create Scalable Reliable Datagram
(SRD) QPs.

Signed-off-by: Gal Pressman <galpress@amazon.com>
@gal-pressman
efa: Elastic Fabric Adapter (EFA) direct verbs man pages
46c66b5 
Document EFA direct verbs feature.

Signed-off-by: Gal Pressman <galpress@amazon.com>
@jgunthorpe jgunthorpe merged commit 0da42a6 into linux-rdma:master May 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jgunthorpe jgunthorpe jgunthorpe requested changes

@bdrung bdrung bdrung requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@gal-pressman @rleon @jgunthorpe @bdrung