Xrdssi v2 #542

abh3 · 2017-07-13T02:08:59Z

No description provided.

o Allow the SSI framework to coexist with a filesystem o Fully objectify access to the the SSI framework (no extern C) o Vector all errors via the callback to avoid split error checks o Fully implement the QueryResource() interface o Introduce a provider class to rationalize classes o Some bug fixes along the way

…ework. Set default debugging level to off. Turn it on if envar XRDSSIDEBUG is set.

Fix pedantic compiler complaints. Add pthread library to the XrdSsiShMap.so build.

Conflicts: docs/PreReleaseNotes.txt

Do not use the message response thread for callbacks.

Cleanup some tracing information to get a better idea of the flow.

the event that the session is unprovisioned before they complete.

This allows the callback to call Finished() and delete the request.

Conflicts: docs/PreReleaseNotes.txt

Clang reports a problem with the variable being used to calculate the size of the string buffer in XrdSsiFileReq::Alloc - XrdSsiFileReq.cc:142:33: error: invalid use of member 'rID' in static member The fix is to use sizeof() on the same variable that is receiving the string.

Use correct snprintf sizeof variable

Add rUser member to the resource object (xfer query ID). Add rInfo member to the resource object (xfer CGO information). Add uConn argument to Provision(). This specifies how TCP connections should be handled relative to rUser. Add respwt directive to specify default response wait time.

Conflicts: docs/PreReleaseNotes.txt

Increase maximum number of parallel requests in a session to 1024. Add check if a responder is trying to respond to the wrong request. Above check usually caches responding to a cancelled request. Allow short data responses to be immediately sent. Various fixes and improvements.

Fix sizeof() issue when clearing a buffer. Conflicts: docs/PreReleaseNotes.txt

Conflicts: docs/PreReleaseNotes.txt

Remove XrdSsiSession.hh from public headers.

* Do an automatic unbind when the responder object is deleted.

…ater.

Conflicts: src/XrdPlugins.cmake

simonmichal

Are we sure that SSI should be part of xrootd-server-libs? We could also create a separate package for ssi, say xrootd-ssi-libs and xrootd-ssi-devel.
libXrdSsi and libXrdSsiLog are not versioned and the unversioned libs are not packaged.
IMHO it's better to keep all SSI specific staff in XrdSsi.cmake and don't split it between XrdSsi.cmake and XrdPlugins.cmake.

simonmichal · 2017-07-14T09:54:24Z

src/CMakeLists.txt

@@ -29,6 +29,7 @@ include( XrdXml )
 include( XrdPosix )
 include( XrdFfs )
 include( XrdPlugins )
+include( XrdSsi )


Do we want to include XrdSsi in 4.7.0? If not this needs to be optional (similarly as for http and cepth).

Yes, we want SSI and http to be part of the core. People depend on it being packaged that way. Ceph is problematic because of the library dependencies. However, let's keep Ceph he way it is for now.

However, I will consolidate the SSI stuff as you suggest.

simonmichal · 2017-07-14T09:57:21Z

src/XrdPlugins.cmake

@@ -9,6 +9,8 @@ set( LIB_XRD_N2NO2P     XrdN2No2p-${PLUGIN_VERSION} )
 set( LIB_XRD_PSS        XrdPss-${PLUGIN_VERSION} )
 set( LIB_XRD_GPFS       XrdOssSIgpfsT-${PLUGIN_VERSION} )
 set( LIB_XRD_ZCRC32     XrdCksCalczcrc32-${PLUGIN_VERSION} )
+set( LIB_XRD_SSI        XrdSsi-${PLUGIN_VERSION} )


IMHO it would be better to keep all XrdSsi cmake staff in XrdSsi.cmake. Adding it partially to XrdPlugins.cmake is misleading and clutters the file.

simonmichal · 2017-07-14T09:58:42Z

src/XrdPlugins.cmake

+  XrdUtils
+  XrdServer )
+
+set_target_properties(


The library is not versioned!

Also, the library is installed but is not packaged.
Note this creates libXrdSsi.so which is installed but not packaged. libXrdSsi.so.* are, in turn not created but they are packaged.

Sorry about that! Will fix.

simonmichal · 2017-07-14T09:58:59Z

src/XrdPlugins.cmake

+  XrdUtils
+  XrdServer )
+
+set_target_properties(


The library is not versioned!

Also, the library is installed but is not packaged.
Note this creates libXrdSsiLog.so which is installed but not packaged. libXrdSsiLog.so.* are, in turn not created but they are packaged.

simonmichal · 2017-07-14T12:02:55Z

packaging/rhel/xrootd.spec.in

@@ -733,6 +733,10 @@ fi
 %{_libdir}/libXrdN2No2p-4.so
 %{_libdir}/libXrdOssSIgpfsT-4.so
 %{_libdir}/libXrdServer.so.*
+%{_libdir}/libXrdSsi-4.so.*


Are we sure this should go to server-libs, and not to a new package, .e.g. ssi-libs ?
Actually, since libXrdSsi is not versioned this fails as there are no libXrdSsi-4.si.*, there is only the libXrdSsi-4.so, which in turn is installed but not packaged.

Yes, I will give this another go. There is no reason for it not being part of server libs like the other libs.

simonmichal · 2017-07-14T12:03:39Z

packaging/rhel/xrootd.spec.in

@@ -733,6 +733,10 @@ fi
 %{_libdir}/libXrdN2No2p-4.so
 %{_libdir}/libXrdOssSIgpfsT-4.so
 %{_libdir}/libXrdServer.so.*
+%{_libdir}/libXrdSsi-4.so.*
+%{_libdir}/libXrdSsiLib.so.*
+%{_libdir}/libXrdSsiLog-4.so.*


Same here, since libXrdSsiLog is not versioned this fails as there are no libXrdSsiLog-4.si.*, there is only the libXrdSsiLog-4.so, which in turn is installed but not packaged.

simonmichal · 2017-07-14T12:05:35Z

src/XrdHeaders.cmake

@@ -125,6 +125,18 @@ set( XROOTD_PRIVATE_HEADERS
  XrdOfs/XrdOfsHandle.hh
  XrdOfs/XrdOfsTrace.hh
  XrdOfs/XrdOfsTPCInfo.hh
+  XrdSsi/XrdSsiAtomics.hh


Are we sure we don't want this in a separate package, say ssi-devel (together with *.so files) ?

The only reason these are in the private section is because the Castor people wanted some additional enhancements that may or may not break ABI. So, this is temporary until we get that straightened out. After that the headers will go into the public section.

abh3 · 2017-07-14T18:58:21Z

OK, I will work on it today and reissue the pull.

ljanyst · 2017-07-14T19:36:28Z

FYI, there is no need to close and reissue. Any new commits you push to the branch will automatically appear as a revision of this pull request.

abh3 · 2017-07-14T23:14:13Z

OK, I have fixed hopefully all of he packaging issues. If not, please let me know.

simonmichal

It's almost perfect ;-)

The only problem now is that following *.so are installed but not packaged:
/usr/lib64/libXrdSsiLib.so
/usr/lib64/libXrdSsiShMap.so

I think they should go into server-devel, just add the following to the respective section in the spec file:
%{_libdir}/libXrdSsiLib.so
%{_libdir}/libXrdSsiShMap.so

Michal

abh3 and others added 30 commits March 18, 2015 14:31

Implement Version 1 of the Scalable Service Interface

79ea801

Add cmake file for XrdSsi to make it build.

8dfb8af

Merge branch 'master' into xrdssi

afed79c

Keep debugging from making helgrind complain for client-side ssi fram…

b4355ae

…ework. Set default debugging level to off. Turn it on if envar XRDSSIDEBUG is set.

Add a shared meory package, XrdSsiShMap, to the ssi package.

f43a621

Add unit test program, xrdshmap, to teh test suite.

db3603d

Fix pedantic compiler complaints. Add pthread library to the XrdSsiShMap.so build.

Merge branch 'master' into xrdssi

3c18696

Conflicts: docs/PreReleaseNotes.txt

Use traditional locking for request object handling.

08a37ff

Do not use the message response thread for callbacks.

Make sure messages are properly sequenced.

5db1766

Cleanup some tracing information to get a better idea of the flow.

Make sure to remove session reference in all defered tasks in

04b79d1

the event that the session is unprovisioned before they complete.

Do not hold the request lock when calling a request callback method.

561c7ae

This allows the callback to call Finished() and delete the request.

Merge branch 'master' into xrdssi

2738c01

Conflicts: docs/PreReleaseNotes.txt

Merge branch 'master' into xrdssi

9796f82

Merge branch 'master' into xrdssi

f7ed00e

Conflicts: docs/PreReleaseNotes.txt

Merge branch 'master' into xrdssi

638828f

Conflicts: docs/PreReleaseNotes.txt

Merge pull request #281 from timj/xrdssi

25e7dbd

Use correct snprintf sizeof variable

Check for nil pointer to avoid SEGV.

5ae5805

Merge branch 'master' into xrdssi

f875af9

Conflicts: docs/PreReleaseNotes.txt

Merge branch 'master' into xrdssi

b4adcd7

Merge branch 'master' into xrdssi

513d683

Make it easier to use the ssi interface by removing some rules.

71dbc11

Fix invalid sizeof() for clearing a memory buffer.

4acbd57

Merge branch 'master' into xrdssi

3ce6d12

Fix sizeof() issue when clearing a buffer. Conflicts: docs/PreReleaseNotes.txt

Merge branch 'master' into xrdssi

e17a260

Merge remote-tracking branch 'origin/master' into xrdssi

42c84df

Conflicts: docs/PreReleaseNotes.txt

Don't complain about missing net I/F if running in the cmsd.

3ec2b25

abh3 added 15 commits February 10, 2017 18:10

[XrdSsi] Additional cleanup and enhancements.

38c643a

si] Additional bug fixes and improvements.

1b3ab4e

[XrdSs] Add missing file to repo.

f371071

[XrdSsi] Implement reusable resources.

ee94694

Return a string for endpoint name to avoid scope confusion.

0033bd7

Remove unused member.

ac62749

Correct spelling in public headers.

3346eb4

Remove XrdSsiSession.hh from public headers.

Merge branch 'master' into xrdssiV2

c3ea084

* Increase number of outsanding requests to 1 server to 16 million.

575ef67

* Do an automatic unbind when the responder object is deleted.

Merge branch 'master' into xrdssiV2

373be3f

Accomodate offsetof MACRO use in new compilers.

f7372e2

Protect against SEGV when nvalid calling sequence is used.

972ffff

Move SSI headers from public to private for now to avoid ABI issues l…

bcff981

…ater.

Add SSI libraries to the rpm spec file.

c42aab7

Merge branch 'master' into xrdssiV2

1c45a9a

Conflicts: src/XrdPlugins.cmake

abh3 requested a review from simonmichal July 13, 2017 02:09

abh3 self-assigned this Jul 13, 2017

abh3 added the enhancement label Jul 13, 2017

simonmichal suggested changes Jul 14, 2017

View reviewed changes

abh3 closed this Jul 14, 2017

Fix packaging issues (hopefully all of the)

e2d1b84

abh3 reopened this Jul 14, 2017

Fix possible use of uninitialized variable.

06ffc7f

simonmichal suggested changes Jul 19, 2017

View reviewed changes

Correct v1 to v2 ommision that disabled FS+SSI mode.

93cf45d

simonmichal merged commit c4f43a2 into master Aug 1, 2017

amadio deleted the xrdssiV2 branch August 25, 2023 12:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Xrdssi v2 #542

Xrdssi v2 #542

abh3 commented Jul 13, 2017

simonmichal left a comment

simonmichal Jul 14, 2017

abh3 Jul 14, 2017

abh3 Jul 14, 2017

simonmichal Jul 14, 2017

abh3 Jul 14, 2017

simonmichal Jul 14, 2017

simonmichal Jul 14, 2017

abh3 Jul 14, 2017

simonmichal Jul 14, 2017

simonmichal Jul 14, 2017

simonmichal Jul 14, 2017

abh3 Jul 14, 2017

simonmichal Jul 14, 2017

simonmichal Jul 14, 2017

abh3 Jul 14, 2017

abh3 commented Jul 14, 2017

ljanyst commented Jul 14, 2017

abh3 commented Jul 14, 2017

simonmichal left a comment

Xrdssi v2 #542

Xrdssi v2 #542

Conversation

abh3 commented Jul 13, 2017

simonmichal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abh3 commented Jul 14, 2017

ljanyst commented Jul 14, 2017

abh3 commented Jul 14, 2017

simonmichal left a comment

Choose a reason for hiding this comment