Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CRITICAL] WORKER TIMEOUT #799

Closed
ippocratis opened this issue Mar 24, 2023 · 6 comments
Closed

[CRITICAL] WORKER TIMEOUT #799

ippocratis opened this issue Mar 24, 2023 · 6 comments
Labels
ARM backend bug Something isn't working

Comments

@ippocratis
Copy link

ippocratis commented Mar 24, 2023 •
edited
Loading

uname -a

Linux sway 5.15.78-1-MANJARO-ARM-RPI #1 SMP PREEMPT Sat Nov 12 17:13:38 UTC 2022 aarch64 GNU/Linux

guicron_django.log

[2023-03-23 17:58:55 +0000] [226] [INFO] Booting worker with pid: 226
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 17:59:53 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:226)
[2023-03-23 17:59:54 +0000] [146] [WARNING] Worker with pid 226 was terminated due to signal 9
[2023-03-23 17:59:54 +0000] [252] [INFO] Booting worker with pid: 252
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 18:00:24 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:252)
[2023-03-23 18:00:24 +0000] [252] [INFO] Worker exiting (pid: 252)
[2023-03-23 18:00:25 +0000] [146] [WARNING] Worker with pid 252 was terminated due to signal 9
[2023-03-23 18:00:25 +0000] [259] [INFO] Booting worker with pid: 259
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
Unauthorized: /api/searchtermexamples/
Unauthorized: /api/albums/place/list/
Unauthorized: /api/persons/
Unauthorized: /api/albums/thing/list/
Unauthorized: /api/albums/user/list/
[2023-03-23 18:03:40 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:259)
[2023-03-23 18:03:41 +0000] [146] [WARNING] Worker with pid 259 was terminated due to signal 9
[2023-03-23 18:03:41 +0000] [285] [INFO] Booting worker with pid: 285
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[2023-03-23 18:04:40 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:285)
[2023-03-23 18:04:41 +0000] [146] [WARNING] Worker with pid 285 was terminated due to signal 9
[2023-03-23 18:04:41 +0000] [311] [INFO] Booting worker with pid: 311
use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
Unauthorized: /api/sitesettings
Unauthorized: /api/rqavailable/
Unauthorized: /api/user/1/
Unauthorized: /api/user/
Unauthorized: /api/searchtermexamples/
Unauthorized: /api/albums/thing/list/
Unauthorized: /api/albums/user/list/
Unauthorized: /api/persons/
Unauthorized: /api/dirtree/
Unauthorized: /api/albums/place/list/
Unauthorized: /api/albums/date/list/
[2023-03-24 07:59:07 +0000] [146] [CRITICAL] WORKER TIMEOUT (pid:311)
[2023-03-24 07:59:08 +0000] [146] [WARNING] Worker with pid 311 was terminated due to signal 9

From the admin page of the webUI scan is stale
Screenshot_20230324-133546_Iceraven.png

rqworker.log

use SECRET_KEY from env
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
17:44:48 Worker rq:worker:ea2e93bb090e4fab9716304800f2d562: started, version 1.10.1
17:44:48 Subscribing to channel rq:pubsub:ea2e93bb090e4fab9716304800f2d562
17:44:48 *** Listening on default...
17:44:48 Cleaning registries for queue: default
17:44:48 default: api.apps.build_index() (37bd72f5-7c2b-44e1-b8a4-1acb7714869f)
18:11:45 default: Job OK (37bd72f5-7c2b-44e1-b8a4-1acb7714869f)
18:11:45 Result is kept for 60 seconds
18:11:45 Cleaning registries for queue: default
18:11:45 default: api.apps.build_index() (1645d968-8ae7-47a2-9ef7-4335a2da84cf)
18:32:38 default: Job OK (1645d968-8ae7-47a2-9ef7-4335a2da84cf)
18:32:38 Result is kept for 60 seconds
18:32:38 Cleaning registries for queue: default
18:32:38 default: api.apps.build_index() (8e557a7f-f65a-4a91-9d21-a1980dda8f20)
18:47:08 default: Job OK (8e557a7f-f65a-4a91-9d21-a1980dda8f20)
18:47:08 Result is kept for 60 seconds
18:47:09 default: api.apps.build_index() (616a193d-3076-430c-a9b3-f89264278a37)
19:07:22 default: Job OK (616a193d-3076-430c-a9b3-f89264278a37)
19:07:22 Result is kept for 60 seconds
19:07:22 Cleaning registries for queue: default
19:07:22 default: api.apps.build_index() (c9b44047-4919-4d5c-b284-7789d9cd5820)
19:23:53 default: Job OK (c9b44047-4919-4d5c-b284-7789d9cd5820)
19:23:53 Result is kept for 60 seconds
19:23:53 Cleaning registries for queue: default
19:23:53 default: api.apps.build_index() (9d296957-6099-4fdd-977d-742448a02699)
19:41:32 default: Job OK (9d296957-6099-4fdd-977d-742448a02699)
19:41:32 Result is kept for 60 seconds
19:41:32 Cleaning registries for queue: default
19:41:32 default: api.apps.build_index() (1c939256-332d-4826-90ef-fd1d35704484)
20:01:57 default: Job OK (1c939256-332d-4826-90ef-fd1d35704484)
20:01:57 Result is kept for 60 seconds
20:01:58 Cleaning registries for queue: default
20:01:58 default: api.apps.build_index() (cc9c0ff6-f1c7-43ec-a79e-004d9855c07d)
20:18:03 default: Job OK (cc9c0ff6-f1c7-43ec-a79e-004d9855c07d)
20:18:03 Result is kept for 60 seconds
20:18:03 Cleaning registries for queue: default
20:45:03 Cleaning registries for queue: default
21:12:03 Cleaning registries for queue: default
21:39:03 Cleaning registries for queue: default
22:06:04 Cleaning registries for queue: default
22:33:04 Cleaning registries for queue: default
23:00:04 Cleaning registries for queue: default
23:27:04 Cleaning registries for queue: default
23:54:05 Cleaning registries for queue: default
23:59:25 default: api.apps.build_index() (d1e31451-b473-463e-b738-db377e4945c0)
00:01:55 Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rq/worker.py", line 1061, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 821, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 844, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/code/api/apps.py", line 19, in build_index
    build_image_similarity_index(user)
  File "/code/api/image_similarity.py", line 78, in build_image_similarity_index
    for photo in paginator.page(page).object_list:
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 287, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rq/worker.py", line 1061, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 821, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.10/dist-packages/rq/job.py", line 844, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/code/api/apps.py", line 19, in build_index
    build_image_similarity_index(user)
  File "/code/api/image_similarity.py", line 78, in build_image_similarity_index
    for photo in paginator.page(page).object_list:
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 287, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

00:22:10 Cleaning registries for queue: default
00:49:10 Cleaning registries for queue: default
01:16:10 Cleaning registries for queue: default
01:43:11 Cleaning registries for queue: default
02:10:11 Cleaning registries for queue: default
02:37:11 Cleaning registries for queue: default
03:04:11 Cleaning registries for queue: default
03:31:12 Cleaning registries for queue: default
03:58:12 Cleaning registries for queue: default
04:25:12 Cleaning registries for queue: default
04:52:12 Cleaning registries for queue: default
05:19:13 Cleaning registries for queue: default
05:46:13 Cleaning registries for queue: default
06:13:13 Cleaning registries for queue: default
06:40:13 Cleaning registries for queue: default
07:07:13 Cleaning registries for queue: default
07:34:13 Cleaning registries for queue: default
07:59:33 default: api.apps.build_index() (b165fe1a-a1f8-495f-af7d-64980af66cd2)
08:25:15 default: Job OK (b165fe1a-a1f8-495f-af7d-64980af66cd2)
08:25:15 Result is kept for 60 seconds
08:25:15 Cleaning registries for queue: default
08:25:15 default: api.apps.build_index() (4415b9c6-e52b-4688-9092-063c15c9987d)

My docker-compose.yml

version: "3.8"
services:
  proxy:
    image: reallibrephotos/librephotos-proxy:${tag}
    container_name: proxy
    restart: unless-stopped
    volumes:
      - ${scanDirectory}:/data
      - ${data}/protected_media:/protected_media
    ports:
      - ${httpPort}:80
    depends_on:
      - backend
      - frontend

  db:
    image: postgres:13
    container_name: db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=${dbUser}
      - POSTGRES_PASSWORD=${dbPass}
      - POSTGRES_DB=${dbName}
    volumes:
      - ${data}/protected_media/db:/var/lib/postgresql/data
    command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
    #Checking health of Postgres db
    healthcheck:
      test: psql -U ${dbUser} -d ${dbName} -c "SELECT 1;"
      interval: 5s
      timeout: 5s
      retries: 5

  frontend:
    image: reallibrephotos/librephotos-frontend:${tag}
    container_name: frontend
    restart: unless-stopped
    depends_on:
      - backend

  backend:
    image: reallibrephotos/librephotos:${tag}
    container_name: backend
    restart: unless-stopped
    volumes:
      - ${scanDirectory}:/data
      - /run/media/ippo/TOSHIBA/protected_media:/protected_media
      - /run/media/ippo/TOSHIBA/uhuru-logs:/logs
      - ${data}/cache:/root/.cache
    environment:
      - SECRET_KEY=${shhhhKey}
      - BACKEND_HOST=backend
      - ADMIN_EMAIL=${adminEmail}
      - ADMIN_USERNAME=${userName}
      - ADMIN_PASSWORD=${userPass}
      - DB_BACKEND=postgresql
      - DB_NAME=${dbName}
      - DB_USER=${dbUser}
      - DB_PASS=${dbPass}
      - DB_HOST=${dbHost}
      - DB_PORT=5432
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - MAPBOX_API_KEY=${mapApiKey}
      - WEB_CONCURRENCY=${gunniWorkers}
      - SKIP_PATTERNS=${skipPatterns}
      - ALLOW_UPLOAD=${allowUpload}
      - DEBUG=0
      - HEAVYWEIGHT_PROCESS=${HEAVYWEIGHT_PROCESS}

    # Wait for Postgres
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy

  redis:
    image: redis:6
    container_name: redis
    restart: unless-stopped
    healthcheck:
      test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
      interval: 5s
      timeout: 5s
      retries: 12
@derneuere derneuere added bug Something isn't working backend ARM labels Mar 24, 2023
@derneuere derneuere mentioned this issue Mar 26, 2023
Closed
3 tasks
@derneuere
Copy link
Member

I bumped a bunch of dependencies. It should be faster now, can you test it again?

@ippocratis
Copy link
Author

I bumped a bunch of dependencies. It should be faster now, can you test it again?

Most recent Dev build on dockerhub 193e6e4f0dbc from 20h ago?

@derneuere
Copy link
Member

Yes, exactly :)

@ippocratis
Copy link
Author

ippocratis commented Apr 3, 2023 •
edited
Loading

switched to dev

docker logs backend --follow

workers exiting in a loop with high cpu demand

Extension modules: greenlet._greenlet, zope.interface._zope_interface_coptimizations, gevent.libev.corecext, gevent._gevent_c_greenlet_primitives, gevent._gevent_c_hub_local, gevent._gevent_c_waiter, gevent._gevent_c_hub_primitives, gevent._gevent_c_ident, gevent._gevent_cgreenlet, gevent._gevent_c_abstract_linkable, gevent._gevent_cevent, gevent._gevent_cqueue, gevent._gevent_c_semaphore, gevent._gevent_c_imap, gevent._gevent_clocal, charset_normalizer.md, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, markupsafe._speedups (total: 30)
[2023-04-03 09:13:39 +0000] [57] [WARNING] Worker with pid 124 was terminated due to signal 4
[2023-04-03 09:13:39 +0000] [130] [INFO] Booting worker with pid: 130
use SECRET_KEY from env
Fatal Python error: Illegal instruction

Current thread 0x0000ffffa6e414c0 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1233 in create_module
  File "<frozen importlib._bootstrap>", line 573 in module_from_spec
  File "<frozen importlib._bootstrap>", line 676 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/usr/local/lib/python3.11/dist-packages/torch/__init__.py", line 229 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/code/api/im2txt/sample.py", line 5 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/code/api/models/photo.py", line 19 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/code/api/models/person.py", line 7 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/code/api/models/album_auto.py", line 6 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/code/api/models/__init__.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1206 in _gcd_import
  File "/usr/lib/python3.11/importlib/__init__.py", line 126 in import_module
  File "/usr/local/lib/python3.11/dist-packages/django/apps/config.py", line 269 in import_models
  File "/usr/local/lib/python3.11/dist-packages/django/apps/registry.py", line 116 in populate
  File "/usr/local/lib/python3.11/dist-packages/django/__init__.py", line 24 in setup
  File "/usr/local/lib/python3.11/dist-packages/django/core/wsgi.py", line 12 in get_wsgi_application
  File "/code/ownphotos/wsgi.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1206 in _gcd_import
  File "/usr/lib/python3.11/importlib/__init__.py", line 126 in import_module
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/util.py", line 359 in import_app
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/wsgiapp.py", line 48 in load_wsgiapp
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/wsgiapp.py", line 58 in load
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/base.py", line 67 in wsgi
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/base.py", line 146 in load_wsgi
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/base.py", line 134 in init_process
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/ggevent.py", line 146 in init_process
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/arbiter.py", line 589 in spawn_worker
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/arbiter.py", line 622 in spawn_workers
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/arbiter.py", line 551 in manage_workers
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/arbiter.py", line 211 in run
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/base.py", line 72 in run
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/base.py", line 231 in run
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/app/wsgiapp.py", line 67 in run
  File "/usr/local/bin/gunicorn", line 8 in <module>

Extension modules: greenlet._greenlet, zope.interface._zope_interface_coptimizations, gevent.libev.corecext, gevent._gevent_c_greenlet_primitives, gevent._gevent_c_hub_local, gevent._gevent_c_waiter, gevent._gevent_c_hub_primitives, gevent._gevent_c_ident, gevent._gevent_cgreenlet, gevent._gevent_c_abstract_linkable, gevent._gevent_cevent, gevent._gevent_cqueue, gevent._gevent_c_semaphore, gevent._gevent_c_imap, gevent._gevent_clocal, charset_normalizer.md, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, markupsafe._speedups (total: 30)
[2023-04-03 09:14:05 +0000] [57] [WARNING] Worker with pid 130 was terminated due to signal 4
[2023-04-03 09:14:05 +0000] [136] [INFO] Booting worker with pid: 136
use SECRET_KEY from env

from random internet search i see Illegal instruction is probaply coused from wrong cpu architecture? (i run librephotos on ARM)

@derneuere
Copy link
Member

Dependent on pytorch/pytorch#97272 (comment)

@derneuere
Copy link
Member

Updated to pytorch 2.0.1. Should fix the error in dev and will be available in the next latest release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ARM backend bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@derneuere @ippocratis