Merge pull request #1413 from shefty/master

verbs/man/mr: Describe the visible behavior of relaxed ordering

libibverbs/man/ibv_reg_mr.3

@@ -34,27 +34,48 @@ The argument
 describes the desired memory protection attributes; it is either 0 or the bitwise OR of one or more of the following flags:
 .PP
 .TP
-.B IBV_ACCESS_LOCAL_WRITE \fR  Enable Local Write Access
+.B IBV_ACCESS_LOCAL_WRITE \fR
+Enable Local Write Access
 .TP
-.B IBV_ACCESS_REMOTE_WRITE \fR Enable Remote Write Access
+.B IBV_ACCESS_REMOTE_WRITE \fR
+Enable Remote Write Access
 .TP
-.B IBV_ACCESS_REMOTE_READ\fR   Enable Remote Read Access
+.B IBV_ACCESS_REMOTE_READ\fR
+Enable Remote Read Access
 .TP
-.B IBV_ACCESS_REMOTE_ATOMIC\fR Enable Remote Atomic Operation Access (if supported)
+.B IBV_ACCESS_REMOTE_ATOMIC\fR
+Enable Remote Atomic Operation Access (if supported)
 .TP
-.B IBV_ACCESS_FLUSH_GLOBAL\fR  Enable Remote Flush Operation with global visibility placement type (if supported)
+.B IBV_ACCESS_FLUSH_GLOBAL\fR
+Enable Remote Flush Operation with global visibility placement type (if supported)
 .TP
-.B IBV_ACCESS_FLUSH_PERSISTENT\fR  Enable Remote Flush Operation with persistence placement type (if supported)
+.B IBV_ACCESS_FLUSH_PERSISTENT\fR
+Enable Remote Flush Operation with persistence placement type (if supported)
 .TP
-.B IBV_ACCESS_MW_BIND\fR       Enable Memory Window Binding
+.B IBV_ACCESS_MW_BIND\fR
+Enable Memory Window Binding
 .TP
-.B IBV_ACCESS_ZERO_BASED\fR    Use byte offset from beginning of MR to access this MR, instead of a pointer address
+.B IBV_ACCESS_ZERO_BASED\fR
+Use byte offset from beginning of MR to access this MR, instead of a pointer address
 .TP
-.B IBV_ACCESS_ON_DEMAND\fR    Create an on-demand paging MR
+.B IBV_ACCESS_ON_DEMAND\fR
+Create an on-demand paging MR
 .TP
-.B IBV_ACCESS_HUGETLB\fR      Huge pages are guaranteed to be used for this MR, applicable with IBV_ACCESS_ON_DEMAND in explicit mode only
+.B IBV_ACCESS_HUGETLB\fR
+Huge pages are guaranteed to be used for this MR, applicable with IBV_ACCESS_ON_DEMAND in explicit mode only
 .TP
-.B IBV_ACCESS_RELAXED_ORDERING\fR Allow system to reorder accesses to the MR to improve performance
+.B IBV_ACCESS_RELAXED_ORDERING\fR
+This setting allows the NIC to relax the order that data is transfered between the
+network and the target memory region.  Relaxed ordering allows network initiated
+writes (such as incoming message send or RDMA write operations) to reach memory
+in an arbitrary order.  This can improve the performance of some applications.
+However, relaxed ordering has the following impact: RDMA write-after-write message
+order is no longer guaranteed.  (Send messages will still match posted receive buffers
+in order.)  Back-to-back network writes that target the same memory region
+leave the region in an unknown state.  Relaxed ordering does not change completion
+semantics, such as data visibility.  That is, a completion still ensures that all
+data is visible, including data from prior transfers.  Relaxed ordered operations
+will also not bypass atomic operations.
 .PP
 If
 .B IBV_ACCESS_REMOTE_WRITE