Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unable to start dashy due to OOM on initial build #380

Closed
4 tasks done
jacobsandersen opened this issue Dec 23, 2021 · 9 comments
Closed
4 tasks done

[BUG] Unable to start dashy due to OOM on initial build #380

jacobsandersen opened this issue Dec 23, 2021 · 9 comments
Labels
🛠️ Build Changes [PR] Includes modifications to build, CI or actions 🕸️ Inactive ❕ Medium Priority [ISSUE] Needs to be dealt with at some point

Comments

@jacobsandersen
Copy link

I've gone ahead and starred the repo and I do see similar issues but none with this exact failure. I'm not new to GitHub. This is indeed the first time I'm interacting.

I hope it's a valid issue. If not, I apologize.

This is attempting to reopen #379...


Environment

Self-Hosted (Docker)

Version

latest

Describe the problem

I am unable to start dashy when running in docker via docker-compose due to a OOM during build time.

The error:

Attaching to dashy
dashy            | 
dashy            | <--- Last few GCs --->
dashy            | 
dashy            | 
dashy            | <--- JS stacktrace --->
dashy            | 
dashy            | 
dashy            | #
dashy            | # Fatal process OOM in insufficient memory to create an Isolate
dashy            | #
dashy            | 

I am using this docker-compose setup:

  dashy:
    container_name: "dashy"
    image: "lissy93/dashy:arm64v8"
    env_file:
      - "./dashy/.env"
    restart: unless-stopped
    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s
    labels:
      - "traefik.http.routers.dashy.rule=Host(`omitted`)"
      - "traefik.http.routers.dashy.entrypoints=https"
      - "traefik.http.routers.dashy.tls=true"

I have applied these environment variables, the NODE_OPTIONS in an attempt to resolve the OOM:

NODE_ENV=production
NODE_OPTIONS=--max-old-space-size=4096

This is my system information (uname -a): Linux pine 5.4.17-2102.202.5.el8uek.aarch64 #2 SMP Sat May 22 16:34:31 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux

lscpu:

Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per cluster: 4
Socket(s):           -
Cluster(s):          1
NUMA node(s):        1
Vendor ID:           ARM
Model:               1
Model name:          Neoverse-N1
Stepping:            r3p1
BogoMIPS:            50.00
NUMA node0 CPU(s):   0-3
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs

free -m:

              total        used        free      shared  buff/cache   available
Mem:          23114        5940        2940         642       14234       13063
Swap:          8191         192        7999

As far as I know I should have more than enough memory to run dashy, but as I've seen in other issues the initial build process does kick up the memory usage significantly. This does seem expected. But it doesn't explain why I can't seem to resolve the issue with the NODE_OPTIONS environment variable.

Any assistance would be much appreciated because this project looks amazing.

Additional info

I think I included everything necessary but if not please let me know.

Please tick the boxes

@jacobsandersen
Copy link
Author

jacobsandersen commented Dec 23, 2021

Seems to be resolved by changing the alpine version to 3.14.

I copied the arm64v8 dockerfile to my system and set docker-compose to build from that.

  • I changed line 8 to use alpine 3.14.
    FROM arm64v8/alpine:3.14

  • I added this at line 20:

# Clone repo
RUN git clone https://github.com/Lissy93/dashy.git ${DIRECTORY}
  • I commented out the COPY instructions at lines 25 and 31.

Result: https://gist.github.com/simpleauthority/14550a8d162511c91f235ac28476b649

And I'm happy to say I am now using dashy :)

Sorry for the initial issue that got closed and the reopen :L

GitHub Gist: instantly share code, notes, and snippets.

@Lissy93
Copy link
Owner

Lissy93 commented Dec 23, 2021

Hiya Jacob,
Sorry that your first issue was closed, your issue is valid, and a good question. I'm glad you've got it working now :)

I guess one thing to keep in mind, is running the app with your own Docker file (instead of pulling from DockerHub), will be harder to keep it up-to-date with WatchTower. Was there a feature in the hosted Dockerfile that missing?

Feel free to reach out if there's anything else you need :)

@jacobsandersen
Copy link
Author

Hiya Jacob, Sorry that your first issue was closed, your issue is valid, and a good question. I'm glad you've got it working now :)

I guess one thing to keep in mind, is running the app with your own Docker file (instead of pulling from DockerHub), will be harder to keep it up-to-date with WatchTower. Was there a feature in the hosted Dockerfile that missing?

Feel free to reach out if there's anything else you need :)

Yeah I know it will be harder, so it's not my favored solution. The only thing about the hosted dockerfile is that it wouldn't build. I believe it was because of the alpine version, because changing from alpine 3.12 to 3.14 in my own dockerfile made it work with no extra env vars.

@Lissy93
Copy link
Owner

Lissy93 commented Dec 24, 2021

Thanks for the explanation, so this definitely sounds like an issue with Dashy. I'm going to reopen this ticket and get the Dockerfile fixed :)

When I tried just running docker run -p 8080:80 lissy93/dashy:arm64v8 on my Pi 3, it seemed to work, so wondering what device you're using, and if there were any other changes you made prior to the run command?

@Lissy93 Lissy93 reopened this Dec 24, 2021
@Lissy93 Lissy93 added ❕ Medium Priority [ISSUE] Needs to be dealt with at some point 🛠️ Build Changes [PR] Includes modifications to build, CI or actions labels Dec 25, 2021
@jacobsandersen
Copy link
Author

jacobsandersen commented Dec 27, 2021

@Lissy93 No changes were made, this is the exact entry in docker-compose.yml that I was using:

  dashy:
    container_name: "dashy"
    image: "lissy93/dashy:arm64v8"
    env_file:
      - "./dashy/.env"
    restart: unless-stopped
    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s
    labels:
      - "traefik.http.routers.dashy.rule=Host(`omitted`)"
      - "traefik.http.routers.dashy.entrypoints=https"
      - "traefik.http.routers.dashy.tls=true"

So besides putting env information in a separate file and adding labels for traefik, it should pretty much be what you suggest.

The system it's on is a VM from Oracle Public Cloud. What I know is that the CPU is their Ampere A1 Flex offering, which is a Neoverse-N1 ARM model (https://www.arm.com/products/silicon-ip-cpu/neoverse/neoverse-n1).

I have really no idea why it didn't want to build and gave an OOM error. I initially tried to fix that by adding an environment variable to increase node's memory but that did nothing.

I also have no idea why changing from Alpine 3.12 to 3.14 worked. Even more strange, removing the memory environment variable and running at Alpine 3.14 worked. So, the only thing that ended up being changed that fixed the issue is the Alpine version.

Really very strange. If I can attempt to get more info for you let me know.

Accelerating the transformation to a Scalable Cloud to Edge Infrastructure

@Lissy93
Copy link
Owner

Lissy93 commented Dec 27, 2021

That's super helpful, thank you :) I'll hopefully get a fix pushed out this week

@liss-bot
Copy link
Collaborator

This issue has gone 6 weeks without an update. To keep the ticket open, please indicate that it is still relevant in a comment below. Otherwise it will be closed in 5 working days.

@liss-bot liss-bot added the ⚰️ Stale [ISSUE] [PR] No activity for over 1 month label Jan 27, 2022
@liss-bot
Copy link
Collaborator

liss-bot commented Feb 1, 2022

This issue was automatically closed because it has been stalled for over 6 weeks with no activity.

@liss-bot liss-bot closed this as completed Feb 1, 2022
@liss-bot liss-bot removed the ⚰️ Stale [ISSUE] [PR] No activity for over 1 month label Feb 1, 2022
@Lissy93
Copy link
Owner

Lissy93 commented Feb 5, 2022

This should now be fixed in 2.0.2 :)
Thank you for your patience, and feel free to reach out if there's anything else that you need.

asterling8516 pushed a commit to asterling8516/dashy that referenced this issue Nov 23, 2023
Closes Lissy93#380

Signed-off-by: Bjorn Lammers <walkxnl@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🛠️ Build Changes [PR] Includes modifications to build, CI or actions 🕸️ Inactive ❕ Medium Priority [ISSUE] Needs to be dealt with at some point
Projects
None yet
Development

No branches or pull requests

3 participants