New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pantsd] Add RunTracker stats. #5374

Merged
merged 5 commits into from Jan 23, 2018

Conversation

Projects
None yet
3 participants
@kwlzn
Copy link
Member

kwlzn commented Jan 23, 2018

[omerta pants (kwlzn/pantsd_stats)]$ ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": -1,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 0,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 15,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list :: --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 15,
  "resulting_graph_size": 14634
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list :: --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 6198,
  "resulting_graph_size": 14634
}

Fixes #4644

@kwlzn kwlzn force-pushed the twitter:kwlzn/pantsd_stats branch from 2995776 to b8ada86 Jan 23, 2018

@stuhood stuhood added this to the 1.4.x milestone Jan 23, 2018

@kwlzn kwlzn requested review from ity and stuhood Jan 23, 2018

@illicitonion
Copy link
Contributor

illicitonion left a comment

Looks good! :)

@@ -80,8 +80,6 @@ def __init__(self, options, run_tracker, target_roots,
self._workspace = workspace or (ScmWorkspace(self._scm) if self._scm else None)
self._replace_targets(target_roots)
self._invalidation_report = invalidation_report
# TODO(#4769): This should not be exposed to anyone.
# Note that the Context created in unit tests by BaseTest uses a different codepath.

This comment has been minimized.

@illicitonion

illicitonion Jan 23, 2018

Contributor

This note is probably still useful...

(I think @stuhood had plans to unify them? :))

This comment has been minimized.

@kwlzn

kwlzn Jan 23, 2018

Member

that work should already be tracked in #4401 fwict

@@ -144,6 +142,11 @@ def scm(self):
"""
return self._scm

@property
def scheduler(self):

This comment has been minimized.

@illicitonion

illicitonion Jan 23, 2018

Contributor

@ity FYI, this may be useful :)

This comment has been minimized.

@stuhood

stuhood Jan 23, 2018

Member

If possible, it would be good not to expose this as a public-seeming API, and continue to access the private field (or to expose a method that closes over the entire "get the graph len and record it in the runtracker" operation). The goal of #4769 is to give Tasks the ability to request that things run in the scheduler in a declarative way. Having imperative access is supposed to just be a stopgap.

This comment has been minimized.

@kwlzn

kwlzn Jan 23, 2018

Member

sgtm - fixed.

self.resident_graph_size = None
self.resulting_graph_size = None

def set_resident_graph_size(self, size):

This comment has been minimized.

@illicitonion

illicitonion Jan 23, 2018

Contributor

s/resident/already_resident/?

This comment has been minimized.

@kwlzn

kwlzn Jan 23, 2018

Member

renamed to preceding_graph_size

@stuhood
Copy link
Member

stuhood left a comment

Thanks Kris.

@@ -144,6 +142,11 @@ def scm(self):
"""
return self._scm

@property
def scheduler(self):

This comment has been minimized.

@stuhood

stuhood Jan 23, 2018

Member

If possible, it would be good not to expose this as a public-seeming API, and continue to access the private field (or to expose a method that closes over the entire "get the graph len and record it in the runtracker" operation). The goal of #4769 is to give Tasks the ability to request that things run in the scheduler in a declarative way. Having imperative access is supposed to just be a stopgap.

@kwlzn kwlzn merged commit 84c4de7 into pantsbuild:master Jan 23, 2018

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

stuhood added a commit that referenced this pull request Jan 24, 2018

[pantsd] Add RunTracker stats. (#5374)
[omerta pants (kwlzn/pantsd_stats)]$ ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": -1,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 0,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list : --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 15,
  "resulting_graph_size": 81
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list :: --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 15,
  "resulting_graph_size": 14634
}
[omerta pants (kwlzn/pantsd_stats)]$ PANTS_ENABLE_PANTSD=True ./pants -q list :: --run-tracker-stats-local-json-file=test.json >/dev/null; cat test.json | jq '.pantsd_stats'

{
  "resident_graph_size": 6198,
  "resulting_graph_size": 14634
}

Fixes #4644
@stuhood

This comment has been minimized.

Copy link
Member

stuhood commented Jan 24, 2018

Leaving a note here for posterity: in order to cherry-pick this one, I needed to cherry-pick #5239 as well.

stuhood added a commit that referenced this pull request Jan 24, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment