Cannot Run Sarracenia Due to Missing Magic, Part 2

[gc-nrcan-michael]

...continuation of Issue #721

Given this Dockerfile:

FROM ubuntu:latest


RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    libssl-dev \
    libffi-dev \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY ./app .

RUN pip3 install -r requirements.txt

RUN pip install metpx-sr3
RUN pip3 install metpx-sr3[amqp,mqtt,vip]
RUN pip3 install metpx-sr3


EXPOSE 5671
EXPOSE 5672
EXPOSE 15671
EXPOSE 15672


CMD [ "python3", "./__main__.py" ]

with install:

metpx-sr3==3.00.43

the following error is produced (again, run on Docker in Windows):

Traceback (most recent call last):
  File "/app/./__main__.py", line 7, in <module>
    import sarracenia.moth
  File "/usr/local/lib/python3.10/dist-packages/sarracenia/__init__.py", line 65, in <module>
    import magic
  File "/usr/local/lib/python3.10/dist-packages/magic/__init__.py", line 209, in <module>
    libmagic = loader.load_lib()
  File "/usr/local/lib/python3.10/dist-packages/magic/loader.py", line 49, in load_lib
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation

[petersilva]

sigh... this is unfortunate. in the RUN line that installs python3- packages using apt, please add a line like:

           python3-magic \

say, between python3-pip and python3-setuptools ?

The problem is that the magic library requires a C dependency that cannot be installed using pip. That should solve your immediate problem.

[petersilva]

while the above should fix it, it should not have failed in the first place.... oh... the python module was there... it tested for the python module, so that succeeded, it just failed when it tried to use it because of the missing c library... hmm... I have to think about that.

one option is to remove python3-magic from the hard deps (setup.py and requirements.txt)

[gc-nrcan-michael]

I tested the container using your above fix and it worked.

This is an improvement from the previous version which required two extra installs, now what's left is just loading Sarracenia without python3-magic.

[petersilva]

The other improvement is that, I think windows is ok now. This only occurs one minimalist linux installs (normal linux systems would include the requisite library as part of linux's own "batteries included") it would only be omitted in cloud minimalist, stripped down deployments. regardless, have to check better.

[petersilva]

so instead of just seeing if the the module is in the import path, the test for module presence is to actually try to import it, wrapped in a try/except. that way, if something fails during the import, it should be marked as missing. I think it should degrade properly now.

[petersilva]

There is a PR for this now.

[gc-nrcan-michael]

Unless there is a way to access that PR version through a PIP install, it's probably best to wait until then to test on our repos.

[petersilva]

That's fine, fwiw, you can clone any branch, and use pip to install it locally:

     git clone --branch v03_issue753 https://github.com/MetPX/sarracenia sr3
     cd sr3
     pip install .

to include features, pip install .[amqp,mqtt,vip] (add the features after the dot.)
Does that count as a pip install? if you mean straight from pypi, that will need to wait until the next release.

[gc-nrcan-michael]

Alright, so I tried your suggestion, using the following Dockerfile:

FROM ubuntu:latest


RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    libssl-dev \
    libffi-dev \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    && rm -rf /var/lib/apt/lists/*


RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*
RUN git clone --branch v03_issue753 https://github.com/MetPX/sarracenia sr3
WORKDIR /sr3
RUN pip3 install .
RUN pip install .[amqp,mqtt,vip]


WORKDIR /app

COPY ./app .

RUN pip3 install -r requirements.txt


EXPOSE 5671
EXPOSE 5672
EXPOSE 15671
EXPOSE 15672


CMD [ "python3", "./__main__.py" ]

No magic, and it still worked!

This fix solves the issue.

[petersilva]

Great! so that confirms the fix works for you, and should be merged for inclusion in the next release.