Add disk and network metrics to Prometheus and fix dashboard

[kathryn-zhou]

Why are these changes needed?

Replacement PR for #14061 because of #14133

Related issue number

Closes #14133

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[rkooo567]

Hey can we add a regression test here to avoid a similar issue?

[rkooo567]

What was the root cause of the previous failure?

[simon-mo]

The root cause was a rename in variable net -> network_speed in Python the missed renaming of net -> networkSpeed in javascript. There's no way to test it using our current stack (because javascript fail to render). If we get someone to work on this, using the javascript testing stack (jest, puppeteer) should help.

[rkooo567]

Why don't we have a simple sanity check with https://selenium-python.readthedocs.io/getting-started.html? (maybe one test that uses seleium to click a button or sth)

[simon-mo]

Yes we could. Maybe create an issue for it? 

On February 17, 2021, GitHub ***@***.***> wrote: Why don't we have a simple sanity check with <https://selenium- python.readthedocs.io/getting-started.html>? (maybe one test that uses seleium to click a button or sth)

— You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <https://github.com/ray- project/ray/pull/14144#issuecomment-780250639>, or unsubscribe <https://github.com/notifications/unsubscribe- auth/AFBD7A4YHQF2TGWN7CYRFLLS7MR27ANCNFSM4XXKWNYA>.

[rkooo567]

Can we just do this in this pr? Dashboard error we saw is a significant regression for ppl like me who have engagement with users (with the master wheel). If you guys have no bandwidth, I can try making a pr

[simon-mo]

let's do another PR so this PR is unblocked. One issue is that these kind of browser based testing can be very flaky because you don't know how long javascript takes to render. And installing browser tool like selenium itself is hard to do it right. I would say it takes around 4-8 hours to finish end to end. @kathryn-zhou can you start taking a look at selenium, or even better i heard cypress is more reliable testing tool: https://docs.cypress.io/guides/overview/key-differences.html#Architecture

[rkooo567]

I see. I wasn't aware that this could be complicated. I agree not to include it in this PR in that case. @kathryn-zhou can you create a separate issue?

[simon-mo]

lint passed.