Dockerfile not working with m1 macs

[new5558]

Description

Cannot build docker image with m1 macos

Expected results

Docker build succesfully

Current results

[+] Building 76.5s (9/10)                                                                                                                                       
 => [internal] load build definition from Dockerfile                                                                                                       0.1s
 => => transferring dockerfile: 405B                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/python:3.8-slim-buster                                                                                  3.1s
 => [internal] load build context                                                                                                                          5.3s
 => => transferring context: 86.98MB                                                                                                                       5.3s
 => CACHED [1/6] FROM docker.io/library/python:3.8-slim-buster@sha256:03c12f7bbd977120133b73e4b3ef5c5707ca09be338156dc02306d41633db4c0                     0.0s
 => [2/6] COPY . .                                                                                                                                         0.4s
 => [3/6] RUN apt-get update && apt-get install -y --no-install-recommends build-essential libicu-dev libicu63 pkg-config && rm -rf /var/lib/apt/lists/*  42.4s
 => [4/6] RUN pip3 install --upgrade pip setuptools                                                                                                        5.6s
 => ERROR [5/6] RUN if [ -f docker_requirements.txt ]; then pip3 install -r docker_requirements.txt; fi                                                   19.5s 
------                                                                                                                                                          
 > [5/6] RUN if [ -f docker_requirements.txt ]; then pip3 install -r docker_requirements.txt; fi:                                                               
#9 0.859 Collecting PyYAML==5.4
#9 1.018   Downloading PyYAML-5.4.tar.gz (174 kB)
#9 1.117      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.8/174.8 kB 1.8 MB/s eta 0:00:00
#9 1.250   Installing build dependencies: started
#9 4.203   Installing build dependencies: finished with status 'done'
#9 4.204   Getting requirements to build wheel: started
#9 5.003   Getting requirements to build wheel: finished with status 'done'
#9 5.006   Preparing metadata (pyproject.toml): started
#9 5.265   Preparing metadata (pyproject.toml): finished with status 'done'
#9 5.582 Collecting numpy==1.22.*
#9 5.622   Downloading numpy-1.22.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.4 MB)
#9 7.750      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.4/13.4 MB 6.5 MB/s eta 0:00:00
#9 7.837 Collecting python-crfsuite==0.9.7
#9 7.889   Downloading python-crfsuite-0.9.7.tar.gz (432 kB)
#9 7.939      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 432.2/432.2 kB 8.9 MB/s eta 0:00:00
#9 7.991   Preparing metadata (setup.py): started
#9 8.186   Preparing metadata (setup.py): finished with status 'done'
#9 8.257 Collecting requests==2.25.*
#9 8.295   Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
#9 8.306      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 8.4 MB/s eta 0:00:00
#9 8.407 Collecting gensim==4.0.*
#9 8.744   Downloading gensim-4.0.1.tar.gz (23.1 MB)
#9 11.81      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.1/23.1 MB 7.3 MB/s eta 0:00:00
#9 12.49   Preparing metadata (setup.py): started
#9 16.66   Preparing metadata (setup.py): finished with status 'done'
#9 16.73 Collecting nltk==3.6.6
#9 16.78   Downloading nltk-3.6.6-py3-none-any.whl (1.5 MB)
#9 16.98      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 7.6 MB/s eta 0:00:00
#9 17.04 Collecting emoji==0.5.2
#9 17.09   Downloading emoji-0.5.2-py3-none-any.whl (41 kB)
#9 17.10      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.1/41.1 kB 11.5 MB/s eta 0:00:00
#9 17.39 Collecting attacut==1.0.6
#9 17.44   Downloading attacut-1.0.6-py3-none-any.whl (1.3 MB)
#9 17.62      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 7.3 MB/s eta 0:00:00
#9 17.94 Collecting epitran==1.9
#9 17.99   Downloading epitran-1.9-py2.py3-none-any.whl (137 kB)
#9 18.02      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.7/137.7 kB 5.8 MB/s eta 0:00:00
#9 18.33 Collecting sacremoses==0.0.41
#9 18.40   Downloading sacremoses-0.0.41.tar.gz (883 kB)
#9 18.52      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 883.5/883.5 kB 7.2 MB/s eta 0:00:00
#9 18.57   Preparing metadata (setup.py): started
#9 18.77   Preparing metadata (setup.py): finished with status 'done'
#9 18.88 Collecting sentencepiece==0.1.91
#9 18.92   Downloading sentencepiece-0.1.91.tar.gz (500 kB)
#9 18.99      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 500.5/500.5 kB 8.0 MB/s eta 0:00:00
#9 19.02   Preparing metadata (setup.py): started
#9 19.19   Preparing metadata (setup.py): finished with status 'error'
#9 19.20   error: subprocess-exited-with-error
#9 19.20   
#9 19.20   × python setup.py egg_info did not run successfully.
#9 19.20   │ exit code: 1
#9 19.20   ╰─> [5 lines of output]
#9 19.20       Package sentencepiece was not found in the pkg-config search path.
#9 19.20       Perhaps you should add the directory containing `sentencepiece.pc'
#9 19.20       to the PKG_CONFIG_PATH environment variable
#9 19.20       No package 'sentencepiece' found
#9 19.20       Failed to find sentencepiece pkgconfig
#9 19.20       [end of output]
#9 19.20   
#9 19.20   note: This error originates from a subprocess, and is likely not a problem with pip.
#9 19.20 error: metadata-generation-failed
#9 19.20 
#9 19.20 × Encountered error while generating package metadata.
#9 19.20 ╰─> See above for output.
#9 19.20 
#9 19.20 note: This is an issue with the package mentioned above, not pip.
#9 19.20 hint: See above for details.
------
executor failed running [/bin/sh -c if [ -f docker_requirements.txt ]; then pip3 install -r docker_requirements.txt; fi]: exit code: 1

Steps to reproduce

1. git clone https://github.com/PyThaiNLP/pythainlp && cd pythainlp
2. docker build .

Context

Your environment

• PyThaiNLP version: latest commits of dev branch
• Python version: python:3.8-slim-buster (dockerimage)
• Operating system and version (distro, 32/64-bit): MacOS
• More info (Docker, VM, etc.): Mac M1 Docker

Possible solution

I am trying to bump up the dependencies version of the affected package. Will open PR if able to solve the problem

Files

[new5558]

After some investigation, the only reason that Dockerfile is not working in m1 macs is the dependencies in docker_requirements.txt:

1. nlpo3 do not support m1 macs, waiting for a fix: Can't install on apple silicon (arm64) platform nlpo3#58
2. Sentencepiece 0.1.91 not support m1 macs. This is fixed at 0.1.97: Difficulty installing on M1 mac (solved) google/sentencepiece#763
3. Fairseq installation error can be fixed by upgrading from 0.10.2 to 0.12.2. This newer version although not claiming to support M1, can be installed without errors.
4. current h5py version 3.1.0 not support m1 macs, latest version (3.7.0) support it: Add support for Apple Silicon wheels h5py/h5py#2065

Since nlpo3 is still blocking the fix, I don't think I can open a fix PR for now.

In the meantime, For others who want to contribute to the project using Dockerfile on m1 macs, I found a hacky fix by upgrading/removing the above-listed affected dependencies in docker_requirements.txt and setup.py. Please note that this method causes errors when using functionality that requires nlpo3.

[wannaphong]

I still waiting MacStadium to use free-credit for build nlpO3 to Mac M1. It's in the MacStadium's processing.

[wannaphong]

Now, nlpO3-python 1.2.6 support apple silicon. You can install by pip install nlpo3==1.2.6 for mac.