zsh: segmentation fault poetry run python -m selfie

[seyedasfar]

Uploading alice-whatsapp.txt triggers a segmentation fault error: zsh: segmentation fault poetry run python -m selfie.

Try setting ulimit to ulimit -s -H, but issue persists.

      System Version: macOS 14.4 (23E214)
      Chip: Apple M1 Pro
      Memory: 16 GB
      
      Python 3.12.2

Full Log

user@192 selfie % ./start.sh
selfie/.env already exists. Skipping copy.
Installing Python dependencies with Poetry...
All set!
Building UI with Yarn...
Building Selfie UI...
yarn install v1.22.22
[1/4] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 0.11s.
yarn run v1.22.22
$ next build
   ▲ Next.js 14.1.0
   - Environments: .env.production

   Creating an optimized production build ...
 ✓ Compiled successfully
 ✓ Linting and checking validity of types    
 ✓ Collecting page data    
 ✓ Generating static pages (5/5) 
 ✓ Collecting build traces    
 ✓ Finalizing page optimization    

Route (app)                              Size     First Load JS
┌ ○ /                                    212 kB          296 kB
└ ○ /_not-found                          889 B          85.5 kB
+ First Load JS shared by all            84.6 kB
  ├ chunks/69-154bfabf0e4affbe.js        29.2 kB
  ├ chunks/fd9d1056-ae8ff97023ed9a8d.js  53.4 kB
  └ other shared chunks (total)          2.08 kB


○  (Static)  prerendered as static content

✨  Done in 8.59s.
Moving built files to Selfie API app...
Selfie UI built and integrated with API.
Running llama-cpp-python-cublas.sh to enable hardware acceleration...
Installing accelerated llama-cpp-python...
Looking in indexes: https://pypi.org/simple, https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/basic/cpu
Collecting llama-cpp-python
  Using cached llama_cpp_python-0.2.56-cp312-cp312-macosx_14_0_arm64.whl
Collecting typing-extensions>=4.5.0 (from llama-cpp-python)
  Using cached typing_extensions-4.10.0-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from llama-cpp-python)
  Using cached numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl.metadata (61 kB)
Collecting diskcache>=5.6.1 (from llama-cpp-python)
  Using cached diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from llama-cpp-python)
  Using cached Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python)
  Using cached MarkupSafe-2.1.5-cp312-cp312-macosx_10_9_universal2.whl.metadata (3.0 kB)
Using cached diskcache-5.6.3-py3-none-any.whl (45 kB)
Using cached Jinja2-3.1.3-py3-none-any.whl (133 kB)
Using cached numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl (13.7 MB)
Using cached typing_extensions-4.10.0-py3-none-any.whl (33 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-macosx_10_9_universal2.whl (18 kB)
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.10.0
    Uninstalling typing_extensions-4.10.0:
      Successfully uninstalled typing_extensions-4.10.0
  Attempting uninstall: numpy
    Found existing installation: numpy 1.26.4
    Uninstalling numpy-1.26.4:
      Successfully uninstalled numpy-1.26.4
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 2.1.5
    Uninstalling MarkupSafe-2.1.5:
      Successfully uninstalled MarkupSafe-2.1.5
  Attempting uninstall: diskcache
    Found existing installation: diskcache 5.6.3
    Uninstalling diskcache-5.6.3:
      Successfully uninstalled diskcache-5.6.3
  Attempting uninstall: jinja2
    Found existing installation: Jinja2 3.1.3
    Uninstalling Jinja2-3.1.3:
      Successfully uninstalled Jinja2-3.1.3
  Attempting uninstall: llama-cpp-python
    Found existing installation: llama_cpp_python 0.2.56
    Uninstalling llama_cpp_python-0.2.56:
      Successfully uninstalled llama_cpp_python-0.2.56
Successfully installed MarkupSafe-2.1.5 diskcache-5.6.3 jinja2-3.1.3 llama-cpp-python-0.2.56 numpy-1.26.4 typing-extensions-4.10.0
Installation complete. Please check for any errors above.
Running selfie...
INFO:     Started server process [41101]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8181 (Press CTRL+C to quit)
INFO:     127.0.0.1:50473 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:50473 - "GET /_next/static/css/9baa70274bc79aac.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:50474 - "GET /_next/static/chunks/webpack-cef5cbeda6b80f37.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50475 - "GET /_next/static/chunks/fd9d1056-ae8ff97023ed9a8d.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50476 - "GET /_next/static/chunks/69-154bfabf0e4affbe.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50477 - "GET /_next/static/chunks/d1150664.830ca1ada94502b9.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50478 - "GET /_next/static/chunks/main-app-6f3abb298864e7ec.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50473 - "GET /_next/static/chunks/8e1d74a4-79cbde03b075351f.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50474 - "GET /_next/static/chunks/154.b2d5381c019f03ab.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50478 - "GET /_next/static/chunks/631f9b06-9e3e2f1af9cc18ce.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50476 - "GET /_next/static/chunks/c916193b-34e846c9cb02c705.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50475 - "GET /_next/static/chunks/app/page-c09d9203fed7f65b.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50474 - "GET /_next/static/chunks/app/layout-d40d9d019e064b61.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50473 - "GET /_next/static/chunks/564-c06e481c036b3af0.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:50473 - "GET /v1/connectors HTTP/1.1" 200 OK
INFO:     127.0.0.1:50474 - "GET /v1/documents HTTP/1.1" 200 OK
INFO:     127.0.0.1:50473 - "GET /v1/connectors/whatsapp HTTP/1.1" 200 OK
INFO:selfie.parsers.chat:Selected parser: WhatsAppParser
INFO:selfie.embeddings:Initializing DataIndex
INFO:selfie.embeddings:Embeddings file not found, starting with a new embeddings index.
INFO:selfie.embeddings:Indexing 1 documents started at 1710659723.635367
INFO:selfie.embeddings:Starting upsert operation
./start.sh: line 38: 41101 Segmentation fault: 11  poetry run python -m selfie $GPU_FLAG

[tnunamak]

Hi @usama-asfar, thank you for reporting this. I believe the cause of this issue may be the same as neuml/txtai#377.

Can you please try running Selfie with:

OMP_NUM_THREADS=1 ./start.sh
or
OMP_NUM_THREADS=1 poetry run python -m selfie --gpu?

---

From the txtai FAQ

[seyedasfar]

Thank you @tnunamak, the issue was resolved running Selfie with OMP_NUM_THREADS=1 ./start.sh.