Unable to run scenedetect on bluemix cf app

[yashgoel95]

I am running a python flask app on bluemix. The code runs perfectly well on local machine( macbook pro) but for some reason I am unable to run it on bluemix. Error is shown below:

TypeError: 'int' object is not iterable

The reason I am getting this error is because opencv-python which is being installed (via pip) does not support ffmpeg and thus it is unable to run this line of code.

cap = cv2.VideoCapture(video_file_path)
cap.isOpened()

returns false

Any solution for this?

[ghost]

Try installing python-opency using os package manager like APT or yum

[yashgoel95]

For some reason sudo apt-get does not work on bluemix (Access issues) and simple apt-get is being skipped.

[ghost]

You should be able to sudo. https://developer.ibm.com/answers/questions/187174/virtual-images-cannot-sudo-within-a-ubunutu-image.html

[kartikarora]

I'll elaborate what @yashgoyalMIT was trying to say.

We are using cloud foundry to deploy the flask app on bluemix. Cloud foundry skips the apt-get command, hence python-opencv does not get installed.

[ghost]

You should find a the correct "buildpack" which contains OpenCV, without it it wont run.
There is also a possibility that your platform (if its similar to Google App Engine) limits any compiled C/C++ python modules, in which case you cannot use OpenCV.

[Breakthrough]

Unfortunately, by default OpenCV relies on ffmpeg for video I/O. It's usually best to compile OpenCV from source where you can't obtain a Linux package, rather than use the pip builds. Most Linux package managers will download all other required system libraries, whereas pip only downloads the OpenCV binaries.

Are you able to build OpenCV from source locally? If you have a machine that matches the server architecture, you may have better luck compiling the OpenCV Python module locally, and installing that version on your server via pip (albeit specifying the path to the compiled module). You can also attempt to cross-compile OpenCV for the target server architecture.

In these cases, you need to ensure that OpenCV is compiled with ffmpeg support, and that an ffmpeg package/library is available on the target system. You can also look into compiling OpenCV with libav support instead, which is a supported alternative to ffmpeg, if that is supported on your target system.

[yashgoel95]

I am able to build OpenCV from source locally but the problem comes up when I try the same on Bluemix Cloud Foundry App. The build fails on sudo make install due to access issues.

[kartikarora]

@Breakthrough Thanks for the reply. We were able to compile the OpenCV with ffmpeg and the python bindings for it using your docs here. Worked like a charm on our local machines (Ubuntu 16.04 64bit, the server is ubuntu 14.04 64bit)

We then tried to do the same on the linux server and the cloud foundry container on it. Since it's a cloud infrastructure, we don't have sudo rights, hence sudo make install fails. Moreover, the container itself doesn't have sudo rights.

I have been trying to get help on their slack team as well, they are suggesting building ffmpeg first, then opencv, then the python bindings from source itself and not use sudo. This raises an issue of building and installing ffmpeg and opencv, without sudo!

[Breakthrough]

@kartikarora / @yashgoyalMIT if make is successful, you can try to just copy the compiled Python module cv2.so from the output folder build/lib into your local Python packages folder (e.g. /usr/local/lib/python2.7/dist-packages/), and see if you have some success with that route. If there are still runtime issues related to library dependencies, as you mentioned, you might want to look into installing the binaries only locally.

The actual folder to place the compiled files when calling make install can be set when calling cmake before building (via the -D CMAKE_INSTALL_PREFIX=/usr/local option). There are other build flags you can configure to specify the paths to the various required libraries, if you can't install them system-wide either (see this page for details).

Once the install prefix is configured to a folder you have write permissions to, you should be able to call make install without the sudo prefix. From there, you could install the compiled Python module (which should be copied to the install directory after calling make install) by manually copying the binary Python module cv2.so to /usr/local/lib/python2.7/dist-packages/ (or wherever your Python installation is located).

Please feel free to keep us posted with any progress; this should definitely be possible, so I'm interested in knowing what solution ends up working.

[Breakthrough]

@kartikarora / @yashgoyalMIT any luck using the cv2.so module you compiled?

[kartikarora]

@Breakthrough I've been trying to put all the cv2.so to virtual environment's site-packages and the shared libraries into CloudFoundry's LD_LIBRARY_PATH. So far I've been successful with putting opencv and ffmpeg's .so files. But it seems there are more files which I need to put there.

@yashgoyalMIT tried building a docker image of our local system and upload it on a container. He has been successful in building and running it. Now we looking into scaling it and running our flask app behind an apache/nginx server to bind the app to multiple ports and for load balancing.

[yashgoel95]

@Breakthrough The only way I could get this to work was by building a docker image with all the dependencies for the flask app and then deploying it to IBM Bluemix container service which runs the docker containers in a hosted cloud environment on IBM Bluemix. This is what my Dockerfile looks like:

FROM ubuntu:latest

COPY requirements.txt /flask-app/requirements.txt

RUN apt-get update -y
RUN apt-get install -y python-pip libffi-dev libssl-dev python-dev build-essential python-setuptools

COPY . /flask-app
WORKDIR /flask-app

RUN apt-get install python-opencv -y
RUN pip install -r requirements.txt
RUN apt-get install ffmpeg -y

EXPOSE 5000

ENTRYPOINT ["python"]
CMD ["restapi.py"]

Advantages of this approach is that I have more control over my application but there is one downside that the whole delivery pipeline is slower as compared to cloud foundry app delivery pipeline even when all the build steps are cached in the next cycle. The delay is mostly on Bluemix's end.

Either ways the issue is resolved. Thanks a lot for the support.

[Breakthrough]

@yashgoyalMIT glad you found an approach to resolve the issue, using a Docker container seems like a novel approach (although as you mentioned, deployment will be a lot slower in that regard). Thanks again for your submission & feedback :)