Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Illegal instruction trying to start hass #94228

Closed
lionslair opened this issue Jun 8, 2023 · 24 comments
Closed

Illegal instruction trying to start hass #94228

lionslair opened this issue Jun 8, 2023 · 24 comments

Comments

@lionslair
Copy link

lionslair commented Jun 8, 2023

The problem

I am trying to upgrade to 2023.6.0 using the core installation. I have been running all previous installs via core with no issues.

I was already running 2023.5.4 on python3.11

What version of Home Assistant Core has the issue?

2023.6.0

What was the last working version of Home Assistant Core?

2023.5.4

What type of installation are you running?

Home Assistant Core

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

This is the contents of home-assistant.log.fault

Fatal Python error: Illegal instruction

Current thread 0x00007f664bfff700 (most recent call first):
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6650bb2700 (most recent call first):
File "/usr/lib/python3.11/logging/handlers.py", line 1528 in dequeue
File "/usr/lib/python3.11/logging/handlers.py", line 1579 in _monitor
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6654011740 (most recent call first):
File "/usr/lib/python3.11/selectors.py", line 468 in select
File "/usr/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
File "/usr/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
File "/usr/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/main.py", line 214 in main
File "/srv/homeassistant/.virtualenv/bin/hass", line 8 in

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Fatal Python error: Illegal instruction

Current thread 0x00007f664bfff700 (most recent call first):
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
  File "/usr/lib/python3.11/threading.py", line 975 in run
  File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6650bb2700 (most recent call first):
  File "/usr/lib/python3.11/logging/handlers.py", line 1528 in dequeue
  File "/usr/lib/python3.11/logging/handlers.py", line 1579 in _monitor
  File "/usr/lib/python3.11/threading.py", line 975 in run
  File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6654011740 (most recent call first):
  File "/usr/lib/python3.11/selectors.py", line 468 in select
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
  File "/usr/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
  File "/usr/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/__main__.py", line 214 in main
  File "/srv/homeassistant/.virtualenv/bin/hass", line 8 in <module>

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)

Additional information

I can rollback to 2023.5.4 no issues

@lionslair lionslair changed the title Illega; instruction trying to start hass Illegal instruction trying to start hass Jun 8, 2023
@geijt
Copy link

geijt commented Jun 8, 2023

I have the same error when upgrading Core from 2023.5.4 to 2023.6.0.

Fatal Python error: Illegal instruction

Thread 0x00007f2c87eff6c0 (most recent call first):
  File "/srv/homeassistant/lib64/python3.11/site-packages/pyudev/_util.py", line 164 in eintr_retry_call
  File "/srv/homeassistant/lib64/python3.11/site-packages/pyudev/_os/poll.py", line 94 in poll
  File "/srv/homeassistant/lib64/python3.11/site-packages/pyudev/_util.py", line 164 in eintr_retry_call
  File "/srv/homeassistant/lib64/python3.11/site-packages/pyudev/monitor.py", line 537 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2c94adb6c0 (most recent call first):
  File "/usr/lib64/python3.11/threading.py", line 320 in wait
  File "/usr/lib64/python3.11/concurrent/futures/_base.py", line 451 in result
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/recorder/core.py", line 684 in _wait_startup_or_shutdown
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/recorder/core.py", line 739 in _run
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/recorder/core.py", line 695 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2c96dfd6c0 (most recent call first):
  File "/usr/lib64/python3.11/concurrent/futures/thread.py", line 81 in _worker
  File "/usr/lib64/python3.11/threading.py", line 975 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2c975fe6c0 (most recent call first):
  File "/usr/lib64/python3.11/concurrent/futures/thread.py", line 81 in _worker
  File "/usr/lib64/python3.11/threading.py", line 975 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2caacfd6c0 (most recent call first):
  File "/usr/lib64/python3.11/concurrent/futures/thread.py", line 81 in _worker
  File "/usr/lib64/python3.11/threading.py", line 975 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2cab4fe6c0 (most recent call first):
  File "/usr/lib64/python3.11/selectors.py", line 415 in select
  File "/usr/lib64/python3.11/subprocess.py", line 2075 in _communicate
  File "/usr/lib64/python3.11/subprocess.py", line 1207 in communicate
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/util/package.py", line 105 in install_package
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/requirements.py", line 101 in _install_with_retry
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/requirements.py", line 113 in _install_requirements_if_missing
  File "/usr/lib64/python3.11/concurrent/futures/thread.py", line 58 in run
  File "/usr/lib64/python3.11/concurrent/futures/thread.py", line 83 in _worker
  File "/usr/lib64/python3.11/threading.py", line 975 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f2cabcff6c0 (most recent call first):
  File "/usr/lib64/python3.11/logging/handlers.py", line 1528 in dequeue
  File "/usr/lib64/python3.11/logging/handlers.py", line 1579 in _monitor
  File "/usr/lib64/python3.11/threading.py", line 975 in run
  File "/usr/lib64/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib64/python3.11/threading.py", line 995 in _bootstrap

Current thread 0x00007f2caf977740 (most recent call first):
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/auth/jwt_wrapper.py", line 47 in _decode_payload
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/auth/jwt_wrapper.py", line 65 in decode_payload
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/auth/__init__.py", line 558 in async_validate_access_token
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/websocket_api/auth.py", line 84 in async_handle
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/websocket_api/http.py", line 290 in async_handle
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/websocket_api/http.py", line 53 in get
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/view.py", line 148 in handle
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/auth.py", line 236 in auth_middleware
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/ban.py", line 80 in ban_middleware
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/request_context.py", line 28 in request_context_middleware
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/forwarded.py", line 100 in forwarded_middleware
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/components/http/security_filter.py", line 85 in security_filter_middleware
  File "/srv/homeassistant/lib64/python3.11/site-packages/aiohttp/web_middlewares.py", line 117 in impl
  File "/srv/homeassistant/lib64/python3.11/site-packages/aiohttp/web_app.py", line 504 in _handle
  File "/srv/homeassistant/lib64/python3.11/site-packages/aiohttp/web_protocol.py", line 433 in _handle_request
  File "/usr/lib64/python3.11/asyncio/events.py", line 80 in _run
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 1922 in _run_once
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 607 in run_forever
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 640 in run_until_complete
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/runner.py", line 179 in run
  File "/srv/homeassistant/lib64/python3.11/site-packages/homeassistant/__main__.py", line 214 in main
  File "/srv/homeassistant/bin/hass", line 8 in <module>

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, PIL._imaging, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, greenlet._greenlet, MySQLdb._mysql (total: 34)

@lionslair
Copy link
Author

I have decided to migrate to the docker version. Have ported over lost nothing yet.

@jrtallen
Copy link

jrtallen commented Jun 8, 2023

Same issue here:

Fatal Python error: Illegal instruction

Thread 0x00007f501dcd66c0 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/pyudev/_util.py", line 164 in eintr_retry_call
File "/srv/homeassistant/lib/python3.11/site-packages/pyudev/_os/poll.py", line 94 in poll
File "/srv/homeassistant/lib/python3.11/site-packages/pyudev/_util.py", line 164 in eintr_retry_call
File "/srv/homeassistant/lib/python3.11/site-packages/pyudev/monitor.py", line 537 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f501e7d76c0 (most recent call first):
File "/usr/lib/python3.11/threading.py", line 320 in wait
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 451 in result
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/recorder/core.py", line 684 in _wait_startup_or_shutdown
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/recorder/core.py", line 739 in _run
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/recorder/core.py", line 695 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f5038ff96c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f50397fa6c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f5039ffb6c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f503a7fc6c0 (most recent call first):
File "/usr/lib/python3.11/selectors.py", line 415 in select
File "/usr/lib/python3.11/subprocess.py", line 2059 in _communicate
File "/usr/lib/python3.11/subprocess.py", line 1207 in communicate
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/util/package.py", line 105 in install_package
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/requirements.py", line 101 in _install_with_retry
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/requirements.py", line 113 in _install_requirements_if_missing
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f503affd6c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f503b7fe6c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f5040b136c0 (most recent call first):
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f50413146c0 (most recent call first):
File "/usr/lib/python3.11/logging/handlers.py", line 1528 in dequeue
File "/usr/lib/python3.11/logging/handlers.py", line 1579 in _monitor
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Current thread 0x00007f5044e39040 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/auth/jwt_wrapper.py", line 47 in _decode_payload
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/auth/jwt_wrapper.py", line 65 in decode_payload
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/auth/init.py", line 558 in async_validate_access_token
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/websocket_api/auth.py", line 84 in async_handle
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/websocket_api/http.py", line 290 in async_handle
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/websocket_api/http.py", line 53 in get
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/view.py", line 148 in handle
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/auth.py", line 236 in auth_middleware
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/ban.py", line 80 in ban_middleware File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/request_context.py", line 28 in request_context_middleware
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/forwarded.py", line 100 in forwarded_middleware
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/http/security_filter.py", line 85 in security_filter_middleware
File "/srv/homeassistant/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 117 in impl
File "/srv/homeassistant/lib/python3.11/site-packages/aiohttp/web_app.py", line 504 in _handle
File "/srv/homeassistant/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 433 in _handle_request
File "/usr/lib/python3.11/asyncio/events.py", line 80 in _run
File "/usr/lib/python3.11/asyncio/base_events.py", line 1922 in _run_once
File "/usr/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
File "/usr/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/main.py", line 214 in main
File "/srv/homeassistant/bin/hass", line 8 in

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml, PIL._imaging, simplejson._speedups, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, greenlet._greenlet (total: 21)

@lionslair
Copy link
Author

Looking like it was not just a me thing

@I-Kuznetsov
Copy link

The same.

Fatal Python error: Illegal instruction

Thread 0x00007f6789ffb700 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f678a7fc700 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Current thread 0x00007f678affd700 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f678b7fe700 (most recent call first):
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6790b40700 (most recent call first):
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 139 in _write_to_self
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 812 in call_soon_threadsafe
File "/usr/local/lib/python3.11/asyncio/futures.py", line 403 in _call_set_state
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 340 in _invoke_callbacks
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 550 in set_result
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 64 in run
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6791341700 (most recent call first):
File "/usr/local/lib/python3.11/logging/handlers.py", line 1528 in dequeue
File "/usr/local/lib/python3.11/logging/handlers.py", line 1579 in _monitor
File "/usr/local/lib/python3.11/threading.py", line 975 in run
File "/usr/local/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6794987280 (most recent call first):
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 119 in _read_from_self
File "/usr/local/lib/python3.11/asyncio/events.py", line 80 in _run
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1922 in _run_once
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/main.py", line 214 in main
File "/srv/homeassistant/bin/hass", line 8 in

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)

@geijt
Copy link

geijt commented Jun 9, 2023

I might have an answer on why we get this error.

In HA 2023.6.0 the orjson dependency is updated from 3.8.10 to 3.8.12 and it seems that orjson changed the wheel in 3.8.11 to require x86-64-v2. My (old) CPU only supports x86-64-v1 and other people also had that issue with orjson so they reverted that requirement in 3.8.14. See here for more info.

@lionslair
Copy link
Author

ah. I thought it might be linked to wheel. I remember when it went from 3.9 to 3.10 I had some funny issues to. This time just could not work it out so switched to docker.

@jrtallen
Copy link

jrtallen commented Jun 9, 2023

I can confirm that if I upgrade Home Assistant Core to 2023.6.1 and I manually upgrade orjson to the latest version available, hass no longer crashes. However, it has numerous other problems. I should have manually updated orjson to 3.8.14 rather than latest to test this. I'll try that at some point today...

@golds69
Copy link

golds69 commented Jun 10, 2023

I can confirm that if I upgrade Home Assistant Core to 2023.6.1 and I manually upgrade orjson to the latest version available, hass no longer crashes. However, it has numerous other problems. I should have manually updated orjson to 3.8.14 rather than latest to test this. I'll try that at some point today...

This helps, but after restarting hass, orjson is updated to version 3.8.12 again and hass starts again with an error.

@geijt
Copy link

geijt commented Jun 10, 2023

Not really surprising, but disappointing, that it will auto update after restart. Let's hope that they can update the home assistant dependency of orjson to at least 3.8.14 or higher soon.

@osenft
Copy link

osenft commented Jun 11, 2023

I ran into the same issue on a AMD Athlon II X4 640 CPU. I can confirm that orjson 3.8.14 does fix the issue. Restarting Home Assistant did NOT downgrade the lib from 3.8.14 to 3.8.12 for me.

@doudz
Copy link
Contributor

doudz commented Jun 12, 2023

Same problem for me and upgrading manually to orjson 3.8.14 fix it !
python3 -m pip install orjson==3.8.14

@gimisa
Copy link

gimisa commented Jun 13, 2023

It did it for me too thanks a lot doudz

python3 -m pip install orjson==3.8.14
have to leave the line in , in start script as follow

pulseaudio --start ( need for audio running sox)
source bin/activate
python3 -m pip install orjson==3.8.14 ( needed otherwise HomeAssistant kept on replacing with orjson=3.8.12 )
hass

@andynash
Copy link

Upgrading orjson to 3.8.14 allowed me to start Home Assistant, however there were some errors with integrations like Cloud, SmartThings etc.

I'd have tried a restart to see if that would fix those, however orjson was auto-downgraded to 3.8.12 so Home Assistant wouldn't start more than once.

For now I've downgraded to 2023.5.4 and will wait for an official fix - there's nothing in 2023.6 I need badly enough to go further into the rabbit hole.

@andrey-krasnov
Copy link

Same problem for me and upgrading manually to orjson 3.8.14 fix it ! python3 -m pip install orjson==3.8.14

Newer versions also fit for my instance: python3 -m pip install orjson --upgrade

@skylogic004
Copy link

I was getting the "Illegal instruction" error too. I spent the last few days debugging and finally got it working. I'm guessing other people getting this error are finding this thread, so I'll summarize what I learned and what worked.

Some context: I was updating the homeassistant python package from 2022.6.7 to 2023.1.7 (I wasn't trying the latest HA version at this point, but later in this post I switch to 2023.6.3). I'm using an old RPi 1 B+. I was running Python 3.9.

The issue: running the following code

source /srv/homeassistant/bin/activate
hass

Produces the error Illegal instruction

I tried a bunch of things (which didn't work) so I started digging into the source code to figure out where this error message is coming from. Eventually I determined the error is produced upon running this line (of python code):

import jwt

which is from the pyjwt python package.

Key takeaway so far: if you're getting the "Illegal instruction" error, try running python -c "import jwt". If there's no output, then jwt is fine, but if it says "Illegal instruction" then you know that pyjwt is the source of the problem.

I had version 2.5 of pyjwt. I tried installing a bunch of different (earlier and later) versions, but they all produced the same error. Then I tried a different python virtual environment and found that pyjwt worked there, so I decided to make a whole new virtual environment. Since I was running Python 3.9 before, I decided it was a good time to upgrade to 3.10. I used https://www.python.org/ftp/python/3.10.12/Python-3.10.12.tgz (I'll skip installation instructions; you can look them up elsewhere). Then, to test: in the new virtual environment I ran pip install wheel pyjwt==2.5.0 and python -c "import jwt" and it worked.

Then I tried pip install homeassistant (version 2023.6.3) and got an error about orjson. It seems people in this thread also have issues with orjson, so I'll report what worked for me. The error message was:

Collecting orjson==3.8.12
Downloading orjson-3.8.12.tar.gz (669 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 669.2/669.2 kB 551.8 kB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [13 lines of output]
error: failed to parse manifest at /tmp/pip-install-b00ndvyy/orjson_7591cd510ded4c3aa8758373a4b30229/Cargo.toml

Caused by:
failed to parse the edition key

Caused by:
supported edition values are 2015 or 2018, but 2021 is unknown
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with cargo build?
Caused by: cargo metadata exited with an error:
Checking for Rust toolchain....
Running maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-yjauqncr --interpreter /srv/testenv/bin/python3.10

Since maturin is a tool for building and publishing Rust-based Python packages, I wanted to try upgrading rust but I couldn't figure out how to upgrade the existing one on my rpi. So instead I installed a new one. I don't know much about rust, but apparently "rust" and "rustup" both install the rust compiler but rustup is more developer-oriented. Following standard installation procedure, I used curl https://sh.rustup.rs -sSf | sh (using my "homeassistant" OS user). It said "error: cannot install while Rust is installed" but offered to continue anyway so I said "yes" (I suppose I have 2 versions installed now, so I'll have to be wary of that going forward). It installed itself to /home/homeassistant/.cargo/bin.

Finally, running pip install orjson==3.8.12 was successful now. Likewise, I was able to do pip install homeassistant and everything worked.

Final takeaways:

  • if you get the Illegal instruction error, try running python -c "import jwt". If there's no output, then jwt is fine, but if it says "Illegal instruction" then you know that pyjwt is the source of the problem. My solution was a new virtual environment.
  • if you get the maturin failed error message (when installing orjson), my solution was to install rustup (i.e. get the latest version the rust compiler).

Hope this helps someone!

@gimisa
Copy link

gimisa commented Jun 28, 2023 via email

@andynash
Copy link

@skylogic004 I get neither the jwt nor maturin errors, and both orjson 3.1.12 (HA requirement) and 3.8.14 (needed to fix the illegal instruction error and allow HA 2023.6 to start) install just fine, so I think these may be errors related to your specific environment.

@gimisa Yes, having orjson upgrade at each start-up to 3.8.14 should work, however this is the rabbit hole I was referring to.

Firstly I'd need to come up with some sort of custom start-up, like your HAstart.sh. I don't like custom fixes unless they are absolutely necessary and I currently use HA Core as provided by the HA team, this would introduce non-standard complexity that I'd like to avoid.

Secondly, as I mentioned before, even when HA starts using 3.8.14 I was seeing errors in other in various integrations, likely caused by the fact that I was using a newer version of orjson than HA requires.

I could go down both these routes to get HA 2023.6 working, and I may or may not be successful, but I'd rather see the underlying problem solved properly.

The second issue in particular could take an unknown amount of work/time to fix (and a fix would not be guaranteed) so that's a path I won't go down.

I've just tried upgrading to 2023.6.3 (latest) and the problem is still there. I'll try again when 2023.7 comes out next week.

Hopefully one of the HA Core developers will take a look at this - anyone know how best to bring it to their attention?

@bdraco
Copy link
Member

bdraco commented Jun 29, 2023

Please try 2023.7.0b0 it has orjson 3.9.1

#94514

@jrtallen
Copy link

Installed 2023.7.0b0 and it seems to be about the same as when I forced orjson 3.8.14 to install. There are numerous components that fail, namely cloud, tts.google.translate, mobile_app, and default_config.

@gimisa
Copy link

gimisa commented Jun 29, 2023 via email

@jrtallen
Copy link

jrtallen commented Jul 5, 2023

I installed 2023.7.0b5 today and all issues appear to be resolved in this version.

@bdraco
Copy link
Member

bdraco commented Jul 5, 2023

Thanks for confirming

@bdraco bdraco closed this as completed Jul 5, 2023
@andynash
Copy link

I just upgraded to 2023.7.1 to see if this was fixed. On restarting HA hung. Restarting using sysctl from the command line worked, but I had errors just like these:

Installed 2023.7.0b0 and it seems to be about the same as when I forced orjson 3.8.14 to install. There are numerous components that fail, namely cloud, tts.google.translate, mobile_app, and default_config.

A further soft restart, and (so far) things seem back to normal 🤞

Realise this is closed, and this doesn't change that, but posting in case someone gets to the first stage and thinks of reverting to 2023.5 - it might be worth a further soft restart before you do.

@github-actions github-actions bot locked and limited conversation to collaborators Aug 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests