Add Linux support for non-system tirpc library

[jlmuir]

Add support on Linux for using a tirpc library that is not a system library.

[AppVeyorBot]

❌ Build asyn 1.0.290 failed (commit 8cacd75dde by @jlmuir)

[jwlodek]

I think in most cases in AD for example, ***_EXTERNAL specifies whether to use a version of the library built as a part of the EPICS build system or not, rather than a system version vs. source install. Maybe we could automatically use the system version if TIRPC_INCLUDE and TIRPC_LIB are not set, and we can comment them out by default? Then we wouldn't need the TIRPC_EXTERNAL.

Something like: https://github.com/areaDetector/ADCore/blob/0ebde8129e652876b7fd71b459275725f70b2da6/ADApp/commonDriverMakefile#L230

[jlmuir]

I was basically trying to be consistent with areaDetector's configure/EXAMPLE_CONFIG_SITE.local, which contains the following comment:

# Configure which 3rd party libraries to use and where to find them.
# For each library XXX the following definitions are used:
# WITH_XXX      Build the plugins and drivers that require this library.
#               Build the source code for this library in ADSupport if XXX_EXTERNAL=NO.
# XXX_EXTERNAL  If NO then build the source code for this library in ADSupport.
# XXX_INCLUDE   If XXX_EXTERNAL=YES then this is the path to the include files for XXX.
#               However, if XXX is a system library whose include files are in a 
#               standard include search path then do not define XXX_INCLUDE.
# XXX_LIB       If XXX_EXTERNAL=YES then this is the path to the library files for XXX.
#               However, if XXX is a system library whose library files in a 
#               standard library search path then do not define XXX_LIB.

But I'd be happy with the approach you proposed too. Thanks!

[jlmuir]

I also see an approach like what you're describing in measComp/measCompApp/src/Makefile. So, do you want to make the change, or do you want me to change this PR?

[jlmuir]

Didn't hear from you, so went ahead and made the change that I think you're wishing for and force-pushed. Thanks!

[AppVeyorBot]

❌ Build asyn 1.0.293 failed (commit a804e432b1 by @jlmuir)

[MarkRivers]

I think this is a slippery slope. These are the SYS_LIBS referenced in asyn/asyn/Makefile:

  asyn_SYS_LIBS_Linux += tirpc
  asyn_SYS_LIBS += usb-1.0
  asyn_SYS_LIBS += ftdi1
  asyn_SYS_LIBS += ftdi
  asyn_SYS_LIBS += gpib

If we add comments for non-system locations of tirpc, then don't we also need to add them for usb-1.0, ftdi1, ftdi, and gpib?

Can you explain why you are not installing tirpc as a system library?

[jlmuir]

Re a slippery slope, I would say that supporting all library dependencies being non-system libraries would be good. For example, I think ADSupport mostly (fully?) does this. Thinking of libraries and programs that I've built from source that are not an EPICS module, it seems that most, if not all, of them support specifying the location of all dependent libraries, includes, and programs, and do not assume system locations. That's part of writing portable software.

The reason I'm not installing tirpc as a system library is that I'm planning to package all EPICS modules that I use as pkgsrc packages. I want to use pkgsrc because it's a cross-platform package management system. This means I can have consistent builds and installs of packages on macOS and Linux and possibly others. To make the binary packages that I build of these pkgsrc packages as portable as possible, I want to build them on a minimal system platform and with dependencies satisfied by other pkgsrc packages, not system packages. This way, dependencies can be cleanly specified and satisfied for each pkgsrc package, and I don't have library dependencies that can change out from under me (e.g., the user can't remove or upgrade a system package that a pkgsrc package depends on that then breaks a pkgsrc package). And anyone who uses these packages can install a package and know that it will work. If I didn't do it this way, then there'd be no guarantee that installing a binary pkgsrc package would even work because it could depend on a set of system packages that have to be installed separately on the system in order for it to work. And then I'd have to maintain a separate list of system packages per platform (and per package if I wanted to allow the user to only install what was really needed) that the user would need to install separately in order for a given pkgsrc package to actually work. Bad business.

[jlmuir]

Another reason that's not mine but could be someone else's is, if a user wants to build and install asyn and dependencies in their home directory, for example, or some other location that does not require administrator privileges to install to. By supporting a non-system tirpc, the user could install tirpc in their home directory and then be able to build asyn against it. This situation could arise on a system where the user doesn't have administrator privileges, and they either can't ask for other system packages to be installed, or the system administrator does not want to install other system packages.

[AndreFavotto]

I was roaming around looking for something else and stumbled up on this issue.

@jlmuir, I don't know how invested you still are on this after 1.5 years, but what you described as a goal on #216 (comment) with pkgsrc is exactly what we have at ESS with e3, using conda.
I would recommend that you take a look at https://e3.pages.ess.eu/ and try it out.

Having dependencies isolated from the system is exactly what conda does, and is heavily battle-tested with multiple IOCs with different dependencies (including epics-base itself!) running on the same host.

[AndreFavotto]

Regardless, I would say that changing the shared upstream code for fulfilling the needs of a specific build system could easily become a messy-never-ending task.
For that reason, any changes needed for your specific builds should be applied as patches on your application level. For our case, that's the conda recipes.

[jlmuir]

Regardless, I would say that changing the shared upstream code for fulfilling the needs of a specific build system could easily become a messy-never-ending task. For that reason, any changes needed for your specific builds should be applied as patches on your application level. For our case, that's the conda recipes.

Hmm, I disagree. It's not for fulfilling the needs of a specific build system; it's for fulfilling the needs of any build system. I don't know of a single build system that doesn't need to control where external programs and libraries are located. And building and installing by hand generally has the same need. And as I noted, it wouldn't even be possible if I didn't have elevated privileges to install (or the ability to request that a system administrator install) additional programs and libraries to a system location. So, this PR was basically making it so that the location of the tirpc library could be specified instead of it being hard-coded as being at a system location.

[AndreFavotto]

Fair enough. I agree that supporting non-system installations of external dependencies is useful, and I have used the same pattern myself for Paho dep on mqtt support and others. My concern is primarily about where this responsibility belongs at current state of things.

If we add configurable paths whenever a dependency cannot be installed system-wide, we will potentially need to do the same for libusb, libftdi, GPIB, etc, as Mark pointed out, and that's only talking about asyn, which is what I meant with the "never-ending" thing. IMO that suggests this is a broader dependency-management problem on EPICS (?) rather than something specific to TIRPC/asyn.

A non-system TIRPC installation is a valid use case, and we actually also use it on e3 I mentioned, but I'm questioning whether adding TIRPC-specific configuration to asyn is the right architectural level for solving it, especially if the goal is to support fully unprivileged EPICS installations in general

[jlmuir]

IMO, making the location of library and program dependencies configurable is the right way, regardless of how many there are. And that's how some EPICS modules work already.

I do acknowledge, though, that asyn, or any EPICS module, could target a platform as supported and thereby have the right, so to speak, to expect that the system libraries that come with that platform exist in the standard system locations.

For example, asyn could officially support macOS Tahoe 26, and then it could reasonably depend on system libraries that come with macOS Tahoe 26. This would make asyn work on macOS Tahoe 26. (macOS Tahoe 26 doesn't actually provide tirpc as a system library, but it provides rpcsvc, which I think would work, but I haven't tried it to confirm.) It would not work on macOS Tahoe 26, though, for any build that wants to provide the tirpc library (or another library like rpcsvc providing the same API) in a non-system location.

Expanding support to POSIX-compliant platforms would be a possibility, but (I don't actually know) I doubt that POSIX says that the tirpc library must be provided as a system library, and so if that's the case, asyn couldn't support POSIX-compliant platforms without providing a way to configure where the tirpc library is located.

Linux as a platform target is harder than macOS because Linux is a kernel, not a distribution, so really it would have to be expanding support to a particular Linux distribution as a platform. And then you're toast because there are many Linux distributions, and then even within a Linux distribution, there are different install sets, which is the case I ran into that prompted me to create this PR: I had a minimal RHEL 9 install that did not include the tirpc library. And I wouldn't be surprised if there are other Linux distributions that either don't include it at all, or don't include it in a minimal install set (if they have the notion of more than one install set).

IMO that suggests this is a broader dependency-management problem on EPICS (?) rather than something specific to TIRPC/asyn.

I don't know what you mean. Are you saying that you think EPICS should include the tirpc library (i.e., what some people refer to as "vendoring," where it actually bundles the tirpc library source code with its own source code)? You could say that, but I don't think that's a good approach. Are you saying that you think the EPICS build system should provide a way to specify the location of any external library or program that an EPICS module might need? You could say that, and that could probably work and make it easier for an EPICS module to allow the location of its external dependencies to be configured in a standard way. I'd be fine with that, but I unfortunately don't have the time to design, propose, and implement it ATM. Or are you saying something else?