Build static binary and deploy to AWS Lambda (using serverless)

[kahlil29]

Description

I'm trying to compile a static binary and deploy it to an AWS Lambda (using serverless).
Earlier I faced a cross-compile issue because I was trying to build on Mac and run it on Linux (I think the default is Amazon Linux for Lambdas).
So due to that I'm now building it on an EC2 instance (same OS that will be used to run it).

When I run make to build/compile, it gives the following warning (which does not seem to be related or relevant to the isssue) :

# gocv.io/x/gocv
dnn.cpp: In function 'cv::Mat* Net_BlobFromImage(Mat, double, Size, Scalar, bool, bool)':
dnn.cpp:119:102: warning: passing NULL to non-pointer argument 1 of 'cv::Scalar_<_Tp>::Scalar_(_Tp) [with _Tp = double]' [-Wconversion-null]
         return new cv::Mat(cv::dnn::blobFromImage(*image, scalefactor, sz, NULL, swapRB, crop, ddepth));
                                                                                                      ^

The build command is :

env GOOS=linux go build -ldflags="-s -w" -o bin/hello hello/main.go

I then run sls deploy or run it locally using sls invoke local -f hello (hello is currently the name of the function/program)

It errors out (in runtime) with the following error :

$ sls invoke local -f hello
Serverless: Packaging service...
Serverless: Excluding development dependencies...
/var/task/bin/hello: error while loading shared libraries: libopencv_highgui.so.4.2: cannot open shared object file: No such file or directory

I encounter the same error in the logs of the deployed Lambda.
Is there anything I need to do in order to properly package the gocv dependency? Or is this something to do with shared libs? I took a look at #601 and tried re-building opencv from source by using cmake with the following options:

cmake -DBUILD_SHARED_LIBS=OFF -DOPENCV_GENERATE_PKGCONFIG=ON -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules ../opencv/

but it does not affect the error(error is still the same).

Your Environment

• Operating System and version: Ubuntu 16.04.6 LTS
• OpenCV version used: 4.2.0
• How did you install OpenCV? From source (But it seems like Gocv does a temp install and build of opencv in the /tmp directory ? )
• GoCV version used: 0.22.0
• Go version: go version go1.14 linux/amd64
• Did you run the env.sh or env.cmd script before trying to go run or go build? No

[EeFey]

I got the same error as #615 when I tried to build on Windows 10.

Then I tried to build on Ubuntu, and I got the same error..
/var/task/main: error while loading shared libraries: libopencv_highgui.so.4.2: cannot open shared object file: No such file or directory

@kahlil29 Do you know if there is any way to get gocv to run on AWS Lambda?

[kahlil29]

After trying for a bit, I kind of gave up. Am not allowed (it's not feasible) to spend more time on this for now (officially, at atleast). But I'm going to try if and when I have some spare time. if I can get it to work, I will let you know. Please let me know if you get it to work or find the solution!

…

On Thu, 19 Mar, 2020, 12:12 am EeFey, ***@***.***> wrote: I got the same error as #615 <#615> when I tried to build on Windows 10. Then I tried to build on Ubuntu, and I got the same error.. /var/task/main: error while loading shared libraries: libopencv_highgui.so.4.2: cannot open shared object file: No such file or directory @kahlil29 <https://github.com/kahlil29> Do you know if there is any way to get gocv to run on AWS Lambda? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#636 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6CJ3KQ64ODX3RVOSLBSCLRIDXFRANCNFSM4LG46WMQ> .

[kahlil29]

@EeFey Any luck with this? I'm probably going to have to revisit this problem sometime soon and try to crack it.

[EeFey]

@kahlil29 sadly no, and I have moved on to alternatives...

[hbmask]

//        return new cv::Mat(cv::dnn::blobFromImage(*image, scalefactor, sz, NULL, swapRB, crop, ddepth));
        return new cv::Mat(cv::dnn::blobFromImage(*image, scalefactor, sz, cv::Scalar(), swapRB, crop, ddepth));

[kahlil29]

@deadprogram Do you have any insights/thoughts on this?

[JoshChristie]

Not entirely sure if this will fix your specific problem but you could try:

1. Try build with -extldflags "-static":
   go build -extldflags "-static" -o output -v main.go

2. Use YES when you compile opencv:
   -D OPENCV_GENERATE_PKGCONFIG=YES

[dcu]

did anybody try to copy the dynamic libraries (listed by ldd) to the lambda package in the lib/ directory ?

[deadprogram]

This is a new PR to address the static build issue on Linux: #802

Windows will be worked on in a subsequent PR.