Memory & Storage historical plot

[dawonn-haval]

What's your browser and operating system?

Ubuntu, Chrome

What is the problem?

When a task fails, It would be nice to be able to see a plot of the memory and storage use during the task to see if the system simply ran out of memory / storage.

What should be the expected behavior?

The diagnostics tab shows a pie graph, which is great for seeing the current state of the system. It would be super helpful if this were a plot / graph so that we could look at how much RAM / Storage was in use during the processing. If it maxes out and then drops, it can be easily determined that more RAM is required.

How can we reproduce this?

Process a large dataset with little ram, and let it run over night. Wake up disappointed and try again with more RAM... :)

[dawonn-haval]

For anyone else in a similar situation; I installed Prometheus with the node_exporter 'plugin' to record system metrics. That should solve the immediate need, but would still be nice to have some integrated solution. :)

[pierotofy]

Indeed, I think this has been asked in the past too. It would be a matter of adding a prometheus docker-compose file that pulls the prometheus docker image and links it with the rest of the stack. Perhaps you might be interested in contributing to this?

[dawonn-haval]

Would really love to, but my todo list is already long and full of terrors... :(

If you're gonna bundle it, Grafana would probably be a better user experience. The default prometheus installation is rather bare-bones and terse. (Intentionally so, I think)

It might also be wise to investigate InfluxDB. I have heard good things, but ultimately I don't know much about the TSDB's, I just googled things until it did what I wanted... :)

[pierotofy]

I'm going to close this; it's definitely something that can be monitored with external tools and an integration in docker-compose would be nice, a PR is still welcomed but we're not going to keep this feature request open. 🙏