cross-compiling for other platform

[HaoboGu]

Hello,

I want to know can I build executable which loads pytorch model and does inference for other platforms? How to do it?

I've tried CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags '-s -w --extldflags "-static -fpic"' but failed. The executable cannot load libtorch on linux.

Thanks.

[sugarme]

Hi @HaoboGu ,

What platform are you using to compile and intend to run on which platform?

To compile for running on linux Amd64, I do

CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build

To compile for running on linux Arm64:

CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc CXX=aarch64-linux-gnu-g++ go build

Of course, the compiled file does not include Libtorch! You have to install libtorch in your server as many C/C++ libraries.

On other note, if you train your model in Amd64 and compile to run inference on Arm64 platform, the result may be different as Pytorch libtorch for 2 platforms are different (somebody mentioned it and I had one model that infers differently with the 2 platform). I haven't tried on Nvidia Jetson for GPU yet but if you can, I would love to know about.

Hope that helps.

[HaoboGu]

Hello @sugarme , thanks for the reply! I'm trying to build a distributed linux executable from MacOS, which means I have to include libtorch in the compiled executable. Is it possible?

[sugarme]

@HaoboGu ,

Yes. What I meant is if you are going to deploy to a linux server, the server should have Libtorch installed.

[HaoboGu]

I am going to distribute the executable to Linux/macos/Windows. It's actually a server runs on end users' local machine. The end users' machine is not guaranteed to have libtorch installed.

[sugarme]

@HaoboGu ,
You can always pack libtorch with the compiled file and make sure libtorch is linked at the end-user system.
Another way would be docker?

[HaoboGu]

@sugarme Thanks, I'll have a try to pack them