Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource Usage UI #1592

Merged
merged 42 commits into from Jul 24, 2017
Merged

Resource Usage UI #1592

merged 42 commits into from Jul 24, 2017

Conversation

@kwm4385
Copy link
Contributor

@kwm4385 kwm4385 commented Jul 14, 2017

New Features

  • Cluster-level utilization data added to /utilization page (formerly /slave-usage). Max over/under CPU and memory numbers are linked to the corresponding request. Colored sections of the breakdown link to the appropriate filter on the Requests page.
    local hubteamqa com-3334-singularity-ui-utilization
  • Resource Usage section on Request Detail page added underneath Task History. Will be shown open by default if CPU has been over-allocated in the past 24 hours.
    image
  • Over/under CPU/memory filters on the Requests page allow searching of requests based on resource utilization.

Modifications

  • /slave-usage page has been renamed to /utilization. The tiles within the Slave Health section have had their details replaced by popovers to eliminate them being covered by the tooltip component. Overall layout was improved and code refactored to better align with project standards.

TODO:

  • Add filters for utilization levels on Requests page
  • Display 24-hr total CPU/mem stats on utilization page
kwm4385 added 16 commits Jul 13, 2017
@kwm4385 kwm4385 added the UI label Jul 14, 2017
@kwm4385 kwm4385 added the hs_staging label Jul 14, 2017
@ssalinas ssalinas modified the milestone: 0.17.0 Jul 14, 2017
kwm4385 added 6 commits Jul 14, 2017
@kwm4385 kwm4385 requested a review from ssalinas Jul 14, 2017
@kwm4385
Copy link
Contributor Author

@kwm4385 kwm4385 commented Jul 14, 2017

kwm4385 added 3 commits Jul 17, 2017
@kwm4385
Copy link
Contributor Author

@kwm4385 kwm4385 commented Jul 17, 2017

Top section now includes data from

// across all slaves
long totalMemBytesUsed;
long totalMemBytesAvailable;
double totalCpuUsed;
double totalCpuAvailable;

image

@kwm4385 kwm4385 changed the title (WIP) Resource Usage UI Resource Usage UI Jul 17, 2017
<div className="col-md-10">
<div className="row">
<div className="col-md-3">
<div className="aggregate">

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

This seems like a good thing to break out into a separate component.

import Utils from '../../../utils';
import { STAT_NAMES, HUNDREDTHS_PLACE } from '../Constants';

const getPctSlaveUsage = (slaves, slaveUsages, usageCallback, resourceCallback) => {

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

Mapper might be a more descriptive suffix than Callback for the parameter names. You could also avoid passing around the slaves and slaveUsages parameters around with

const getPctSlaveUsage = (usageMapper, resourceMapper) => (slaves, slaveUsages) => { //...}

const getCpuUtilizationPct = getPctSlaveUsage(
  usage => usage.cpusUsed,
  slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.cpusUsedStat)
);

const getMemUtilizationPct = getPctSlaveUsage(
  usage => usage.memoryBytesUsed,
  slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.memoryBytesUsedStat)
);
const totalResource = slaves.map(resourceCallback)
.reduce((acc, val) => acc + parseFloat(val), 0);

return Utils.roundTo((totalUsage / totalResource) * 100, HUNDREDTHS_PLACE);

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

This line might be good to abstract out into a helper function since it does one very specific thing.

<h4>Current</h4>
<div className="row">
<div className="col-xs-12">
<div className="aggregate graph col-xs-2">

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

This is very similar to ClusterAggregates.jsx. I'd strongly suggest creating a new component.

@@ -56,6 +56,7 @@ const maybeLink = (name, value) => {
};

const SlaveResourceHealthMenuItems = ({stats}) => {
stats = _.filter(stats, (stat) => _.values(STAT_NAMES).includes(stat.name));

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

If you're going to use includes (ES2016), I'd suggest that you don't use _.contains elsewhere in your code. You can also directly use Array.prototype.filter

used={utilization.cpuUsed / utilization.numTasks}
style={utilization.cpuUsed >= utilization.cpuReserved ? 'danger' : null}
>
<p>{Utils.roundTo(utilization.cpuUsed / utilization.numTasks, 2)} of {utilization.cpuReserved / utilization.numTasks} CPU reserved</p>

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

Move these percentage calculations into a separate utility function instead of using the magic number 2 all over the place.


return utilization ? (
<CollapsableSection id="request-utilization" title="Resource usage" subtitle="(past 24 hours)" defaultExpanded={isCpuOverAllocated}>
{isFetching ? <div className="page-loader fixed" /> : attributes}

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

This loading component would be good to abstract out too.

@@ -80,7 +92,7 @@ export default class RequestFilters extends React.Component {
return (
<NavItem
key={index}
className={classNames({'separator-pill': _.contains([3, 5], index)})}
className={classNames({'separator-pill': _.contains([3, 5, 7], index)})}

This comment has been minimized.

margin-left : 10px
width : 160px

small

This comment has been minimized.

@andyhuang91

andyhuang91 Jul 17, 2017

This would override the color of <small> tags everywhere. If you actually want to do that, move this rule out of the slaveUsage.styl, otherwise scope this override to just this page.

kwm4385 added 10 commits Jul 18, 2017
css
@kwm4385 kwm4385 added the hs_qa label Jul 19, 2017
@kwm4385 kwm4385 added the hs_stable label Jul 21, 2017
@ssalinas
Copy link
Member

@ssalinas ssalinas commented Jul 24, 2017

This one is looking good. Already found a few requests that were really over-allocated using it. Going to merge so @baconmania can work off of some of this to eventually add disk utilization details once we gather them as well

@ssalinas ssalinas merged commit 8730005 into master Jul 24, 2017
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/push The Travis CI build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@ssalinas ssalinas deleted the resource-usage-ui branch Jul 24, 2017
@baconmania baconmania modified the milestones: 0.18.0, 0.17.0 Sep 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.