Dashing Maintenance

[pushmatrix]

Here is an outline of the next tasks that @tylermauthe and I will be looking at for Dashing. We'll be trying to tackle one of these a week, and we'll be posting our findings here.

• Investigate memory leaks and performance issues
  Memory leaks are causing low-memory devices like the PI to crap out after a while. Let's take a look at where these are happening, and nip them in the bud. There are also issues with the list/number widget where raspberry PIs are not able to render them within ugly flashing on the screen. Let's see if we can optimize that.
• Switch to Puma
  Thin is causing many issues, and we' like to investigate using Puma. One added benefit is now Dashing can run on jruby!
• "Inbox-zero" the issues/pr list
  Let's go through the PR list and figure out which ones should be included. Also, let's close up old issues and ones that don't deal with issues in dashing itself. Non-related issues should be sent to Stack Overflow.
• Convert app.rb to use a classful sinatra approach instead of the current classless one.
  Makes it easier to monkeypatch and extend.

[agassner]

+1 for the memory leak issue.
Running dashing in a docker container with limited memory has been quite a challenge.
After a quick investigation it seems that rufus scheduler could possibly be the reason - keeping objects referenced and not allowing them to be GC'd

[pushmatrix]

@agassner How much memory?

[agassner]

@pushmatrix 512MB

[pushmatrix]

Is your box crashing or is the browser tab crashing?

[agassner]

Box(server) is crashing.
Browsers are in multiple screens(separate machines) showing different dashboards and all working fine.

[robacarp]

hey @pushmatrix, you said you're running dashing on puma back at Shopify. I spent some time reading through the puma-related issue chatter and don't see a certain path forward (some people report success, while others don't). What method are you using to launch with puma instead of thin?

[terraboops]

@agassner - We're investigating memory leaks in the client side.

@pushmatrix - Maybe we should update Rufus to the latest? (#237)

[terraboops]

• Add therubyracer gem in a sensible fashion to enable JS engine portability on Linux

[mal]

+1 for server leaks

I've had to limit the server to 1 gig of memory, it now crashes every hour or so, but previously it was taking out the entire machine it's on! ;(

[MartynKeigher]

Hey @pushmatrix and @tylermauthe!!

Was wanting to know when the switch to PUMA is on the roadmap! This month, next month, Xmas? I'm loving dashing more and more, and as i add more and more... that need for puma is, well... ya know! ;p

That (swith to puma) along with FontAwesome 4.4 being put into the master branch are my 2 items for a HUGE +1!

FWIW: Here is a step by step I did to switch to the current master (at the time of this post) to Puma... http://pastebin.com/6LvgBFmb (cc: @robacarp)

Keep up the awesome work guys!!

://mk

[robacarp]

@MartynKeigher thanks, i'll have a look

[pushmatrix]

@robacarp I unfortunately was mistaken. I thought we were running Puma at Shopify, but it turns out we aren't :(. It was a discussion a few ops peeps had and I had thought they switched it, but didn't. Sad...

I will have to look into it further then. I'd like to have Puma in shortly. Can't give an exact date @MartynKeigher, but will look into your pastebin. Thanks for putting that together.

[davidhrbac]

+1 for the memory leak issue.

[simzen85]

This is what dashing does to my MBP https://monosnap.com/file/xHVCgRgST7Qp7U8NjcaHS5NPVs8lbx

[terraboops]

Doing this helped me out, I was hitting memory caps on a Heroku free dyno: https://github.com/Shopify/dashing/wiki/How-To:-Prevent-a-job-from-overlapping-with-itself

Based on some anecdotal testing of my own, I've become convinced that the memory leak I was experiencing was related to my jobs. If you're doing something like creating an array outside your job and filling it indefinitely during your job, you're going to have problems eventually. However, I couldn't find anything obvious like that -- though it may be that a library I was using does something like that.

[terraboops]

Also: Sorry about the delays on Puma everyone, life happens. I took the work done in #427 and updated to the latest release here: https://github.com/tylermauthe/dashing/tree/puma_server

I will endeavour to clean up the outstanding isues (outlined in #427) by the end of Feb.