Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ownphotos-backend invalid instruction #54

Closed
grafixoner opened this issue Jan 11, 2019 · 9 comments
Closed

ownphotos-backend invalid instruction #54

grafixoner opened this issue Jan 11, 2019 · 9 comments

Comments

@grafixoner
Copy link

grafixoner commented Jan 11, 2019 •
edited

Backend services are not starting (getting a 502 bad gateway error).

I notice in my backend logs I have this.

...done.
./entrypoint.sh: line 18: 40 Illegal instruction (core dumped) python manage.py shell <<EOF
from api.models import User
User.objects.filter(email='$ADMIN_EMAIL').delete()
User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF
Running backend server...
[2019-01-10 21:24:31 +0000] [43] [INFO] Starting gunicorn 19.8.1
[2019-01-10 21:24:31 +0000] [43] [INFO] Listening at: http://0.0.0.0:8001 (43)
[2019-01-10 21:24:31 +0000] [43] [INFO] Using worker: sync
[2019-01-10 21:24:31 +0000] [47] [INFO] Booting worker with pid: 47
[2019-01-10 21:24:32 +0000] [49] [INFO] Booting worker with pid: 49
[2019-01-10 21:24:33 +0000] [51] [INFO] Booting worker with pid: 51
[2019-01-10 21:24:34 +0000] [53] [INFO] Booting worker with pid: 53

Obviously the big problem is the core dump in the entrypoint.sh from the following lines.

python manage.py shell <<EOF
from api.models import User
User.objects.filter(email='$ADMIN_EMAIL').delete()
User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF

Also seems odd the worker keeps restarting like it can't connect either, probably related.

I'm using the latest images and have tried resetting everything and pruning the volumes many times.. Also the other odd thing is on my Macbook Pro the same docker-compose.yml works just fine but not on my Nextcloud server.

As noted that you were thinking it was odd architecture and it's not it's a Core2 Duo with Intel CPU.

  • Architecture: x86_64
  • CPU op-mode(s): 32-bit, 64-bit
  • Byte Order: Little Endian
  • CPU(s): 2
  • On-line CPU(s) list: 0,1
  • Thread(s) per core: 1
  • Core(s) per socket: 2
  • Socket(s): 1
  • NUMA node(s): 1
  • Vendor ID: GenuineIntel
  • CPU family: 6
  • Model: 23
  • Model name: Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz
  • Stepping: 6
  • CPU MHz: 800.000
  • CPU max MHz: 2500.0000
  • CPU min MHz: 800.0000
  • BogoMIPS: 4999.55
  • Virtualization: VT-x
  • L1d cache: 32K
  • L1i cache: 32K
  • L2 cache: 6144K
  • NUMA node0 CPU(s): 0,1
  • Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm kaiser tpr_shadow vnmi flexpriority dtherm
@guysoft
Copy link
Collaborator

guysoft commented Jan 12, 2019

What is the output of uname -a?

@grafixoner grafixoner mentioned this issue Jan 12, 2019
Open
@guysoft
Copy link
Collaborator

guysoft commented Jan 20, 2019

Closing if no reply

@guysoft guysoft closed this as completed Feb 5, 2019
@vkadam
Copy link

vkadam commented Feb 13, 2019

Came to this issue from #52, should I open new issue?
Here is output of the my uname -a
Linux 5a5cb9ca9e3b 4.4.59+ #23824 SMP PREEMPT Tue Dec 25 18:27:56 CST 2018 x86_64 x86_64 x86_64 GNU/Linux

@guysoft
Copy link
Collaborator

guysoft commented Feb 14, 2019 •
edited

@vkadam if you are getting invalid instruction AND your linux kernel is complied with x86_64 x86_64 Then something is strange, because it means your device indeed has an intel 64bit instruction set, but is also complaning on an invalid instruction.

I need more logs output to understand what is going on.
For example what does this say?:

 sudo docker-compose logs --tail 100 --follow

@guysoft
Copy link
Collaborator

guysoft commented Feb 19, 2019

@vkadam I would say "closing if no reply" but the issue is already closed. Any progress?

@vkadam
Copy link

vkadam commented Feb 19, 2019

Apologies for delay, here is log

root@vishal-dsm:/volume2/docker/ownphotos# docker logs --tail 100 --follow ownphotos-backend
photo not added to album place
15:41:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
  8%|▊         | 3886/46623 [13:39:22<187:03:46, 15.76s/it]1
393cba6b8bfae540adae73b2804306c41
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:26:54) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3887/46623 [13:39:36<179:39:51, 15.13s/it]1
2334aed57088f719bd8d0932e824de8d1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:27:47) while time zone support is active.
  RuntimeWarning)
photo not added to album place
15:42:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
  8%|▊         | 3888/46623 [13:39:52<182:48:30, 15.40s/it]1
9dfade3363b322e10f807d97db7a572d1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:28:01) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3889/46623 [13:40:06<180:00:29, 15.16s/it]1
179126ecd1848e7ef3b85f0e43acbe311
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:28:04) while time zone support is active.
  RuntimeWarning)
photo not added to album place
15:42:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
  8%|▊         | 3890/46623 [13:40:20<176:22:22, 14.86s/it]1
7e405e529d0a5072ad28b1c28ce055721
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:28:12) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3891/46623 [13:40:32<165:54:45, 13.98s/it]1
a8ceeb19c6034335c83ceb214174b75e1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:28:37) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3892/46623 [13:40:44<156:24:49, 13.18s/it]1
d7bdc841881bb2468a83c149b461dcf41
15:43:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:29:41) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3893/46623 [13:40:55<150:19:05, 12.66s/it]1
0bef53e5ed2a451f3d4ce5fe0e07acb91
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:29:54) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3894/46623 [13:41:06<144:45:46, 12.20s/it]1
b5e5b2995514e3aee2623c0d38dc36271
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:30:16) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3895/46623 [13:41:19<146:02:44, 12.30s/it]1
447a23832005ca8d2d34d1f1304d609b1
15:43:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:30:22) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3896/46623 [13:41:31<145:32:57, 12.26s/it]1
e284913c412722847fefe3954c7259521
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:11) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
  8%|▊         | 3897/46623 [13:41:44<147:22:38, 12.42s/it]1
934af31457ac38d9133d06857e3432921
15:44:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:17) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3898/46623 [13:41:56<146:48:37, 12.37s/it]1
fb7d48fbcd31cef816650a028d13d76b1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:25) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3899/46623 [13:42:07<140:35:00, 11.85s/it]1
3c95768df454168133a457a8bf3f3a8a1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:27) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3900/46623 [13:42:18<138:42:58, 11.69s/it]1
0c64449ea359196e337ee34078b5fef01
15:44:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:33) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
face saved
  8%|▊         | 3901/46623 [13:42:33<150:31:48, 12.68s/it]1
7cc703c8004ac6cfba7e7b68c543dc151
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:36) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
face saved
15:45:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
  8%|▊         | 3902/46623 [13:42:49<164:11:08, 13.84s/it]1
6bdfefdbe59e9d5a7df3a01c126c08c41
Not Found: /api/jobs/
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:39) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
face saved
  8%|▊         | 3903/46623 [13:43:06<174:39:21, 14.72s/it]1
9a619d82c6d1a38ef6dd37dedb64f60d1
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:57) while time zone support is active.
  RuntimeWarning)
photo not added to album place
15:45:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
face saved
face saved
  8%|▊         | 3904/46623 [13:43:25<190:42:01, 16.07s/it]1
07010d9e78bd620350d108d09535e11d1
[2019-02-19 15:45:37 +0000] [82] [CRITICAL] WORKER TIMEOUT (pid:562)
[2019-02-19 15:45:39 +0000] [589] [INFO] Booting worker with pid: 589
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:32:59) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
face saved
  8%|▊         | 3905/46623 [13:43:42<193:31:49, 16.31s/it]1
059853349fab926cca55e3ef2a7da4b71
/miniconda/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
15:46:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:33:02) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3906/46623 [13:43:56<184:52:06, 15.58s/it]1
c753b70d440615752fdfd6d6c9e1f45c1
[2019-02-19 15:46:09 +0000] [82] [CRITICAL] WORKER TIMEOUT (pid:589)
[2019-02-19 15:46:09 +0000] [589] [INFO] Worker exiting (pid: 589)
[2019-02-19 15:46:10 +0000] [596] [INFO] Booting worker with pid: 596
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:33:06) while time zone support is active.
  RuntimeWarning)
photo not added to album place
/miniconda/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
  8%|▊         | 3907/46623 [13:44:10<180:08:08, 15.18s/it]1
1077b26ce8a7284fae967e807395b7741
15:46:30 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:33:07) while time zone support is active.
  RuntimeWarning)
photo not added to album place
  8%|▊         | 3908/46623 [13:44:25<177:50:19, 14.99s/it]1
d4139db443cd3dd95617056ac165fad51
[2019-02-19 15:46:40 +0000] [82] [CRITICAL] WORKER TIMEOUT (pid:596)
[2019-02-19 15:46:40 +0000] [596] [INFO] Worker exiting (pid: 596)
[2019-02-19 15:46:41 +0000] [604] [INFO] Booting worker with pid: 604
/miniconda/lib/python3.7/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_timestamp received a naive datetime (2015-01-04 18:33:09) while time zone support is active.
  RuntimeWarning)
photo not added to album place
face saved
  8%|▊         | 3909/46623 [13:44:41<180:59:08, 15.25s/it]1
5675e43569341911b87ac290f1c808421
/miniconda/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
15:47:00 Sent heartbeat to prevent worker timeout. Next one should arrive within 35 seconds.

@guysoft
Copy link
Collaborator

guysoft commented Feb 21, 2019 •
edited

@vkadam
Backend seems to be working fine for you. So it might be something else, not the machine code instructions.
Where are you getting the invalid instruction?

OP says its at
./entrypoint.sh: line 18: 40 Illegal instruction (core dumped) python manage.py shell <<EOF

But I don't see it at your log output.

@vkadam
Copy link

vkadam commented Feb 21, 2019

@guysoft agree, not seeing the same Illegal instruction error. But somehow that worker is timing out and restarting again. I ok to continue discussion under #52.

@grafixoner
Copy link
Author

grafixoner commented Mar 20, 2019 •
edited

FYI I tried a few days later after posting this and had zero problems. This was after beating my head against a wall for like 3 days.

Everything is working great on my end now and have even started implementing some features and also deep object learning sets, that I need for a photo set, and once rounded out will deliver back to the community. (time permitting)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@guysoft @grafixoner @vkadam