Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MacOS (arm64) build support #36

Closed
robcarruthers opened this issue Apr 24, 2023 · 20 comments · Fixed by #37
Closed

MacOS (arm64) build support #36

robcarruthers opened this issue Apr 24, 2023 · 20 comments · Fixed by #37

Comments

@robcarruthers
Copy link
Contributor

I'd be keen to follow this through and help in anyway I can. But i'll need some help...

My prime objective is to get a working version of liblxi on my M1 Macbook Pro and build a ruby wrapper for my own lab tests, but happy to contribute and detail the results. I've tried installing from source but I'm getting errors. I'm not familiar with meson and after ignoring the warning, downloaded the latest source release and tried to build it. I'm getting an error when I run meson setup build:

❯ meson setup build
The Meson build system
Version: 1.1.0
Source dir: /Users/rob/Downloads/liblxi-1.18
Build dir: /Users/rob/Downloads/liblxi-1.18/build
Build type: native build
Project name: liblxi
Project version: 1.18
C compiler for the host machine: cc (clang 14.0.3 "Apple clang version 14.0.3 (clang-1403.0.22.14.1)")
C linker for the host machine: cc ld64 857.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
Found pkg-config: /opt/homebrew/bin/pkg-config (0.29.2)
Found CMake: /opt/homebrew/bin/cmake (3.24.2)
Run-time dependency avahi-client found: NO (tried pkgconfig, framework and cmake)
Run-time dependency libxml-2.0 found: YES 2.9.4
Run-time dependency threads found: YES
Has header "avahi-client/client.h" : NO
Run-time dependency libtirpc found: NO (tried pkgconfig, framework and cmake)

src/meson.build:22:12: ERROR: Dependency "libtirpc" not found, tried pkgconfig, framework and cmake

I then tried:

❯ brew install libtirpc
libtirpc: Linux is required for this software.
Error: libtirpc: An unsatisfied requirement failed this build.

So tried installing it from source but make check gave a bunch of errors about
'call to undeclared function mutex_lock'

It looks like has others may have come close to success a few years ago #14, but I'm not sure if they ever got it working.

Any help or direction would be greatly appreciated.
@lundmar
Copy link
Contributor

lundmar commented Apr 24, 2023

Hi @robcarruthers

Any help getting lxi-tools working on MacOS is appreciated 👍🏻

However, my ability to help in this matter is limited as I don't use nor have access to macOS. I'm hoping someone some day will help drive porting efforts to macOS properly.

I do not use brew but it looks like you may have to fix brews libtirpc formulae first. It is missing a pkg-config configuration file so that meson can detect it's build settings.

For example, libtirpc-dev on Ubuntu includes the following file:
/usr/lib/x86_64-linux-gnu/pkgconfig/libtirpc.pc:

prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include

Name: libtirpc
Description: Transport Independent RPC Library
Requires:
Version: 1.3.2
Libs: -L${libdir} -ltirpc
Libs.private: -lpthread
Cflags: -I${includedir}/tirpc

I bet that is what is missing in the brew formulae.

@lundmar
Copy link
Contributor

lundmar commented Apr 24, 2023

Oh, I missed the important part. It is possible that libtirpc only works on linux systems but I suspect there must be some other packages that includes the RPC implementation for macOS.

@lundmar
Copy link
Contributor

lundmar commented Apr 24, 2023 •
edited

You could try simply remove libtirpcs check for linux. It may be that libtirpc supports mac too but if it does not you will have to figure out an alternative way to install "Sun's Transport-Independent RPC library".

https://github.com/Homebrew/homebrew-core/blob/1eea9fb106953f283124a1b97a4fd8014323d4e0/Formula/libtirpc.rb#L13

@robcarruthers
Copy link
Contributor Author

Thanks for the information and pointers. It makes sense to confirm libtirpc supports Mac and unfortunately it would appear from this comment its linux only.

However there appears to a Arch linux arm version and others have built it from source. I'm out of my depth on the first option and I would guess from the date of the second it was an intel Mac (I'll test this once I get home and pull out my old intel laptop).

I'll keep looking into alternatives...

@lundmar
Copy link
Contributor

lundmar commented Apr 25, 2023 •
edited

FYI - the specific headers that we are looking for are:

vxi11core.h:#include <rpc/rpc.h>
vxi11core_svc.c:#include <rpc/pmap_clnt.h>

On Linux these RPC headers are provides by the libtirpc package. Before that I believe they were provided via glibc. So we need to figure out which software component provides these for mac. It is possible they are already part of the mac toolchain. If that is the case, then it is a matter of removing the libtirpc check from meson.

@robcarruthers
Copy link
Contributor Author

Thanks for that. I found the files you mentioned above in the SDKs for this Mac, so I guess means they are part of the system. They look very similar to files in libtirpc.

/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/include/rpc/rpc.h
/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/include/rpc/pmap_clnt.h

How do I go about removing the libtirpc check from meson?
Is it as simple as removing the checks in the src/ meson.build file ie. delete lines 22-24

@robcarruthers
Copy link
Contributor Author

OK, so I deleted the lines 22-24 and 37 from src/meson.build and tried to build:

❯ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /opt/homebrew/bin/ninja -C /Users/rob/Downloads/liblxi-1.18/build
ninja: Entering directory `/Users/rob/Downloads/liblxi-1.18/build'
[6/7] Compiling C object src/liblxi.1.dylib.p/vxi11.c.o
../src/vxi11.c:137:42: warning: passing 'const char *' to parameter of type 'char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    vxi11_data->rpc_client = clnt_create(address, DEVICE_CORE, DEVICE_CORE_VERSION, "tcp");
                                         ^~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/rpc/clnt.h:318:39: note: passing argument to parameter here
extern CLIENT *clnt_create      __P((char *, unsigned int, unsigned int, char *));
                                           ^
1 warning generated.
[7/7] Linking target src/liblxi.1.dylib
FAILED: src/liblxi.1.dylib 
cc  -o src/liblxi.1.dylib src/liblxi.1.dylib.p/lxi.c.o src/liblxi.1.dylib.p/mdns.c.o src/liblxi.1.dylib.p/tcp.c.o src/liblxi.1.dylib.p/vxi11.c.o src/liblxi.1.dylib.p/vxi11core_clnt.c.o src/liblxi.1.dylib.p/vxi11core_xdr.c.o -Wl,-dead_strip_dylibs -Wl,-headerpad_max_install_names -Wl,-undefined,dynamic_lookup -shared -install_name @rpath/liblxi.1.dylib -compatibility_version 1 -current_version 1 -fvisibility=hidden -Wl,-init,init -lxml2
ld: -init function (init) found in linked dylib, must be in dylib being linked for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

I'm not really sure what's going on here, but the rpc headers from the OS seem to be giving warnings which is promising.
Any ideas what's causing the error?

@lundmar
Copy link
Contributor

lundmar commented Apr 25, 2023

Try remove the , '-Wl,-init,init' part from src/meson.build

@robcarruthers
Copy link
Contributor Author

That seems to compile and build OK, now when I try to run a test file I get:

❯ clang test/search.c -o search
test/search.c:2:10: fatal error: 'lxi.h' file not found
#include <lxi.h>
         ^~~~~~~
1 error generated.

@lundmar
Copy link
Contributor

lundmar commented Apr 25, 2023

Make sure that liblxi gets installed correctly so that the systems toolchain can find the header file in its default include search paths.

On your typical linux system, lxi.h is installed here:

/usr/include/lxi.h

@robcarruthers
Copy link
Contributor Author

The default include search path is:

❯ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

However the meson build command seems to pickup the default brew locations:

❯ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /opt/homebrew/bin/ninja -C /Users/rob/Downloads/liblxi-1.18/build
ninja: Entering directory `/Users/rob/Downloads/liblxi-1.18/build'
[0/1] Regenerating build files.
The Meson build system
Version: 1.1.0
Source dir: /Users/rob/Downloads/liblxi-1.18
Build dir: /Users/rob/Downloads/liblxi-1.18/build
Build type: native build
Project name: liblxi
Project version: 1.18
C compiler for the host machine: cc (clang 14.0.3 "Apple clang version 14.0.3 (clang-1403.0.22.14.1)")
C linker for the host machine: cc ld64 857.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
Found pkg-config: /opt/homebrew/bin/pkg-config (0.29.2)
Found CMake: /opt/homebrew/bin/cmake (3.24.2)
Run-time dependency avahi-client found: NO (tried pkgconfig, framework and cmake)
Dependency libxml-2.0 found: YES 2.9.4 (cached)
Dependency threads found: YES unknown (cached)
Has header "avahi-client/client.h" : NO (cached)
Configuring lxi_connect.3 using configuration
Configuring lxi_disconnect.3 using configuration
Configuring lxi_discover.3 using configuration
Configuring lxi_discover_if.3 using configuration
Configuring lxi_init.3 using configuration
Configuring lxi_receive.3 using configuration
Configuring lxi_send.3 using configuration
Build targets in project: 1

Found ninja-1.11.1 at /opt/homebrew/bin/ninja
Cleaning... 0 files.
[1/1] Linking target src/liblxi.1.dylib

❯ meson install -C build
ninja: Entering directory `/Users/rob/Downloads/liblxi-1.18/build'
ninja: no work to do.
Installing src/liblxi.1.dylib to /opt/homebrew/lib
Installing /Users/rob/Downloads/liblxi-1.18/src/lxi.h to /opt/homebrew/include/
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_connect.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_disconnect.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_init.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_discover.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_discover_if.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_receive.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/man/lxi_send.3 to /opt/homebrew/share/man/man3
Installing /Users/rob/Downloads/liblxi-1.18/build/meson-private/liblxi.pc to /opt/homebrew/lib/pkgconfig
Installing symlink pointing to liblxi.1.dylib to /opt/homebrew/lib/liblxi.dylib

Pushing forward, I added an absolute path to the include in search.c, but this gave more linking errors:

❯ clang test/search.c
Undefined symbols for architecture arm64:
  "_lxi_discover", referenced from:
      _main in search-314c34.o
  "_lxi_init", referenced from:
      _main in search-314c34.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I guess I need to fix the install paths before moving forward. I'm not sure how meson is getting the homebrew locations. What would be the best way to specify these or rather pickup the default include search paths?

@lundmar
Copy link
Contributor

lundmar commented Apr 26, 2023

The default include search path is:

❯ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

This looks to be just the location of your toolchain, not an include path.

You really should try make e.g. a homebrew recipe for liblxi. This will solve your issues with include paths as brew will manage these for you.

Actually, someone has made a homebrew formulae for one of my other open source projects tio (https://tio.github.io/) which provides a good example of a brew formulae that uses meson. With a few modification you can convert this formulae to build liblxi.

@lundmar
Copy link
Contributor

lundmar commented Apr 26, 2023

Here is the homebrew formulae for tio: https://github.com/Homebrew/homebrew-core/blob/master/Formula/tio.rb

@robcarruthers
Copy link
Contributor Author

It would appear things are working...

I forked the liblxi repo and made the changes to the src/meson.build. Then created a local brew formula with brew create, updated the default meta data to use the forked repo and used the install method from the tio formula. Then installed:

❯ HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source --verbose --debug liblxi

This worked fine but simply installed the library in the same place as building with clang, which gave the same error.

After a number of rabbit holes I gave the problem to chatGPT, which promptly recommended I updated the build command for test/search.c to tell the compiler where to find the header file -I/opt/homebrew/include and tell the linker where the dylibs are -L/opt/homebrew/lib -llxi:

❯ clang -I/opt/homebrew/include test/search.c -o search -L/opt/homebrew/lib -llxi

❯ ./search 
Searching for LXI devices - please wait...

Broadcasting on interface lo0
Broadcasting on interface en0

I need to go back over all this and confirm my testing and It'll be a few days before I can get home and test this properly with some instruments and then I plan to put in a PR. But in the interim is there anything you would like included or tested to ensure it all works correctly?

@lundmar
Copy link
Contributor

lundmar commented Apr 28, 2023 •
edited

Yes, you need to specify include and library paths if liblxi is manually installed outside of normal toolchain system search paths. ChartGPT knows he he.

Just seeing it detect instruments would be a good test. That, and of course sending SCPI commands.

Just be aware that the mdns discovery search feature will not work on mac because we are still missing a liblxi mdns backend implementation for macos. Someone tried to embed a self hosted mdns implementation in liblxi to support mac but that turned out to be the wrong way of doing it because that conflicts with the existing system mdns implementation (Avahi on linux, Bonjour on mac). What is needed is a liblxi mdns backend that uses macOS Bonjour mdns service. Anyway, this is separate discussion but an important todo for full macOS support :)

@robcarruthers robcarruthers mentioned this issue Apr 30, 2023
Merged
@lundmar
Copy link
Contributor

lundmar commented Apr 30, 2023

Thanks for working this issue out. Oh, and if you ever feel inclined to want to use liblxi to search for the most modern LXI instruments via mDNS, feel free to help and try implement and test a liblxi Bonjour mDNS backend for mac ;)

According to chatgpt it works as simple as this using Apples Bonjour API to search for e.g. the two service type _lxi._tcp and _vxi-11._tcp:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <arpa/inet.h>
#include <dns_sd.h>

void browse_callback(DNSServiceRef sdRef, DNSServiceFlags flags, uint32_t interfaceIndex, DNSServiceErrorType errorCode, const char *serviceName, const char *regtype, const char *replyDomain, void *context)
{
    if (errorCode == kDNSServiceErr_NoError) {
        printf("Service discovered: %s.%s.%s\n", serviceName, regtype, replyDomain);
        DNSServiceRef *resolve_sdRef = (DNSServiceRef *)context;
        DNSServiceErrorType resolve_error = DNSServiceResolve(resolve_sdRef, 0, interfaceIndex, serviceName, regtype, replyDomain, (DNSServiceResolveReply)resolve_callback, NULL);
        if (resolve_error != kDNSServiceErr_NoError) {
            printf("Resolve failed with error code %d\n", resolve_error);
        }
    }
}

void resolve_callback(DNSServiceRef sdRef, DNSServiceFlags flags, uint32_t interfaceIndex, DNSServiceErrorType errorCode, const char *fullname, const char *hosttarget, uint16_t port, uint16_t txtLen, const unsigned char *txtRecord, void *context)
{
    if (errorCode == kDNSServiceErr_NoError) {
        printf("Service resolved: %s at %s:%d\n", fullname, inet_ntoa(*(struct in_addr *)&hosttarget), ntohs(port));
        // Use the resolved IP address and port number to connect to the service and start using it
    }
}

int main()
{
    DNSServiceRef browse_sdRef, resolve_sdRef;
    DNSServiceErrorType browse_error = DNSServiceBrowse(&browse_sdRef, kDNSServiceFlagsShareConnection, kDNSServiceInterfaceIndexAny, "_lxi._tcp,_vxi-11._tcp", NULL, (DNSServiceBrowseReply)browse_callback, &resolve_sdRef);
    if (browse_error != kDNSServiceErr_NoError) {
        printf("Browse failed with error code %d\n", browse_error);
        exit(1);
    }

    DNSServiceErrorType resolve_error = kDNSServiceErr_NoError;
    while (resolve_error == kDNSServiceErr_NoError) {
        int nfds = DNSServiceRefSockFD(browse_sdRef);
        fd_set readfds;
        FD_ZERO(&readfds);
        FD_SET(nfds, &readfds);

        struct timeval tv;
        tv.tv_sec = 1;
        tv.tv_usec = 0;

        int result = select(nfds + 1, &readfds, NULL, NULL, &tv);
        if (result > 0) {
            DNSServiceErrorType browse_error = DNSServiceProcessResult(browse_sdRef);
            if (browse_error != kDNSServiceErr_NoError) {
                printf("Browse failed with error code %d\n", browse_error);
                exit(1);
            }
            DNSServiceErrorType resolve_error = DNSServiceProcessResult(resolve_sdRef);
            if (resolve_error != kDNSServiceErr_NoError) {
                printf("Resolve failed with error code %d\n", resolve_error);
            }
        } else if (result == -1) {
            printf("Error in select()\n");
            exit(1);
        }
    }

    return 0;
}

If this example works on mac, it should be fairly simple to hook it up in a similar way as the current Avahi mDNS backend is for Linux.

liblxi could use some more mac OS love. Just saying ;)

@robcarruthers
Copy link
Contributor Author

robcarruthers commented Apr 30, 2023 via email

@lundmar
Copy link
Contributor

lundmar commented Apr 30, 2023

Great, a brew formulae would be nice.

Regarding mac and mDNS Bonjour, it would be really great if you could take a crack at it sometime. And of course, I'm here to help out. Then perhaps we can finally have liblxi support mac 100% :)

FYI - I'm working on implementing the HiSLIP (High Speed LAN Instrument) protocol. This is the new protocol used by the most modern instruments - it is much faster than your traditional way of communicating with instruments which uses the old VXI11 protocol which is slow exactly because it uses the RPC implementation that we have been touching in this issue. Luckily, since I'm implementing the HiSLIP protocol from scratch, it should just work on all platforms, including mac. With mDNS and HiSLIP support, liblxi will get in a very nice shape to support modern instruments adhering to the latest LXI standards.

@robcarruthers
Copy link
Contributor Author

robcarruthers commented Apr 30, 2023 via email

@lundmar
Copy link
Contributor

lundmar commented Apr 30, 2023

Actually, you don't really need mDNS enabled test instruments to test the Bonjour example that I posted. You can simply change or add to the protocol type string, eg. change "_lxi._tcp" to "_http._tcp" or "_printer._tcp", etc. This way you will very likely find any web server enabled product or printer on your network. If that works, any search protocol string will work.

If you can get that mDNS example compiling and working on mac that would be a great start. After that it will take little modification to fit it into liblxi.

Regarding getting lxi cli working on mac. That should be very easy when you have a working brew liblxi formulae available. Besides liblxi, lxi cli only requires readline and lua 5.x, all of which are already available in brew.

This was referenced May 2, 2023
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

macOS: Build support robcarruthers/liblxi
2 participants
@lundmar @robcarruthers