Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

msg/async: Revert RDMA-CM #15262

Merged
merged 19 commits into from Jun 1, 2017

Conversation

Projects
None yet
4 participants
@Adirl
Copy link

Adirl commented May 24, 2017

No description provided.

amirv and others added some commits May 23, 2017

Revert "msg/async/rdma: Use RDMA resources only after fork"
This reverts commit 972c741.

Change-Id: I4a5627ce02ef8e055c2088db41ac4897dd9a3425
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async: Postpone bind if network stack is not ready"
This reverts commit a5b87e2.

Change-Id: Ia1bc0535636de27d599cfb2dc7be2a555e0a2cd7
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: get_device() by ibv_context"
This reverts commit f47025f.

Change-Id: Icc3667da1828868bc5001d1d8e8b2ec14e5f6c8e
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Pass specific ConnMgr info in constructor"
This reverts commit ce46776.

Change-Id: I5098c109e7be17402ca512c25ba0c250ad610cea
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Make poll_blocking() poll for async events in…
… addition to tx/rx"

This reverts commit 2386cea.

Change-Id: I93a23f216cd4ce55051b125e9dd7edb3e297aaad
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Make port number an attribute of the Connecti…
…on not of the Device"

This reverts commit af2f1ec.

Change-Id: I40f1da9da613dfb90aa8f09073c70ebb30b59aa1
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Fix small memory leaks detected by valgrind"
This reverts commit 23fd3ef.

Change-Id: Id04d88f4d08174632271c6eb224212155e0a4a38
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Device::last_poll_dev must be positive"
This reverts commit fb7787d.

Change-Id: I42d47cd683bd69ac2ebac1a6379a1ee0f2c1d543
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Debug prints for ibv_*"
This reverts commit 4207804.

Change-Id: If3a7b236636af0492bbe3576d25042b7a4dbf762
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "Revert "msg/async/rdma: Debug prints for ibv_*""
This reverts commit 7e0f074.

Change-Id: Iaccdd1739647a82aeafeff3ab46ed9f4d3fe0782
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Debug prints for ibv_*"
This reverts commit 7d1475f.

Change-Id: Ieae3ea07f4e40d4558332e15fe6fbfcd22e0553e
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Introduce RDMAConnMgr"
This reverts commit 91bb13d.

Change-Id: I91dbe3c02651402f0e97e89151ec6a5154afdca3
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Extract sockets stuff from RDMAStack.h"
This reverts commit 1890e92.

Change-Id: I66a33e55206b3f8f8c85d1f1ee1a2e85e3c2a3ad
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Initialize device on first connect"
This reverts commit 94eddb1.

Change-Id: Ia16384eaed1295af6a4352e03308bc13bfb2f6d2
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Move async event handling to Device"
This reverts commit 4b10ba7.

Change-Id: I9a294890c1cf01461356d87491bfa5e954b42444
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Move resource handling to Device"
This reverts commit 9cba3b3.

Change-Id: I5953a9380ecd5a07e8ea107cf1fc35d0772e8f61
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Revert "msg/async/rdma: Introduce Device.{cc,h}"
This reverts commit d930a3d.

Change-Id: Ib55ff961af125be5800ea90b2cce6588e3122f4d
Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
msg/async: Postpone bind if network stack is not ready
RDMAStack shouldn't access hardware from the parent process.
The only reason to do so, is because bind is called before the fork.
After this patch the bind is postponed until the NetworkStack reports
that it is ready to bind.
For NetworkStack types will always return true, except the RDMAStack
which will return true only after the fork (after
AsyncMessenger::ready() is called).

This patch is based on a patch by Haomai Wang <haomai@xsky.com>

Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
(cherry picked from commit a5b87e2)

Change-Id: I5f8be4f93a5fe0f2a7ef3b29d755be52b0790c06
msg/async/rdma: Use RDMA resources only after fork
Thanks to previous patch [1], no need to access RDMA resources before
the fork. Initialize Infiniband class only before a connection is
established or a listener is created. [1] is making sure that the call
to RDMAWorker::listen() is postponed till after the fork.

[1] - 7393db45644d ("msg/async: Postpone bind if network stack is not ready")

While backporting from master branch into the stable branch had to pull
some missing stuff from master:
- Make Infiniband ctor an empty shell. Actual initialization is done
  through a call to init()
- RDMADispatcher polling thread is started only after Infiniband::init()
  is called

(cherry picked from commit 972c741)

Issue: 995322
Signed-off-by: Amir Vadai <amir@vadai.me>
Change-Id: Iab1c450937713e6c4b83daf03c903e2fe5562ba2
@Adirl

This comment has been minimized.

Copy link
Author

Adirl commented May 24, 2017

@Adirl

This comment has been minimized.

Copy link
Author

Adirl commented May 28, 2017

@yuyuyu101

This comment has been minimized.

Copy link
Member

yuyuyu101 commented May 28, 2017

locally tested passed

@yuyuyu101

This comment has been minimized.

Copy link
Member

yuyuyu101 commented May 28, 2017

@liewegas the reason why we removing rdma-cm support is rdma-cm feature breaks rdmastack. we roll back to the original to keep L release stable.

@yuyuyu101

This comment has been minimized.

Copy link
Member

yuyuyu101 commented May 28, 2017

just keep the stable rdma usage and give more efforts on rdma-cm later

@liewegas liewegas changed the title Revert RDMA-CM msg/async: Revert RDMA-CM May 28, 2017

@liewegas liewegas merged commit 703055b into ceph:master Jun 1, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.