New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[engine] Initial Trace implementation for rust engine #4076

Merged
merged 11 commits into from Nov 19, 2016

Conversation

Projects
None yet
3 participants
@baroquebobcat
Contributor

baroquebobcat commented Nov 18, 2016

Problem

Re-implementing trace was deferred during the initial work on the Rust scheduler. See #4007.

Solution

This patch partially reintroduces it. The rust implementation will now produce traces that include the subjects and some of the information about failures. Some types of failures will not have good messages yet. This is because we haven't landed a patch to fix #4025. Also, the trace function will produce traces for each of the roots.

Result

Example output:

$ echo "WUT" >> BUILD.tools 
$ ./pants --enable-v2-engine list //:
Exception caught: (<class 'pants.build_graph.address_lookup_error.AddressLookupError'>)
  File "/.../pants/src/python/pants/bin/pants_exe.py", line 50, in <module>
    main()
  File "/.../pants/src/python/pants/bin/pants_exe.py", line 44, in main
    PantsRunner(exiter).run()
  File "/.../pants/src/python/pants/bin/pants_runner.py", line 57, in run
    options_bootstrapper=options_bootstrapper)
  File "/.../pants/src/python/pants/bin/pants_runner.py", line 46, in _run
    return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
  File "/.../pants/src/python/pants/bin/local_pants_runner.py", line 37, in run
    self._run()
  File "/.../pants/src/python/pants/bin/local_pants_runner.py", line 77, in _run
    self._exiter).setup()
  File "/.../pants/src/python/pants/bin/goal_runner.py", line 184, in setup
    goals, context = self._setup_context()
  File "/.../pants/src/python/pants/bin/goal_runner.py", line 157, in _setup_context
    self._daemon_graph_helper
  File "/.../pants/src/python/pants/bin/goal_runner.py", line 101, in _init_graph
    graph, address_mapper = graph_helper.create_build_graph(target_roots, self._root_dir)
  File "/.../pants/src/python/pants/bin/engine_initializer.py", line 88, in create_build_graph
    for _ in graph.inject_specs_closure(target_roots.as_specs()):
  File "/.../pants/src/python/pants/engine/legacy/graph.py", line 205, in inject_specs_closure
    for address in self._inject(specs):
  File "/.../pants/src/python/pants/engine/legacy/graph.py", line 225, in _inject
    self._index(target_entries)
  File "/.../pants/src/python/pants/engine/legacy/graph.py", line 83, in _index
    'Build graph construction failed for {}:\n{}'.format(node, trace))

Exception message: Build graph construction failed for (SiblingAddresses(directory=u''), Select(Collection.of(HydratedTarget))):
Computing =Collection.of(HydratedTarget) for SiblingAddresses(directory=u'')
  Computing =Collection.of(HydratedTarget) for SiblingAddresses(directory=u'')
    Computing =HydratedTarget for SiblingAddresses(directory=u'')
      Computing =Collection.of(Address) for SiblingAddresses(directory=u'')
        Computing =Collection.of(Address) for SiblingAddresses(directory=u'')
          Computing =AddressFamily for SiblingAddresses(directory=u'')
            Computing =AddressFamily for Dir(path=u'')
              Computing =AddressFamily for Dir(path=u'')
                Throw("EntryId(17) failed!")

Computing =Collection.of(Address) for SiblingAddresses(directory=u'')
  Computing =Collection.of(Address) for SiblingAddresses(directory=u'')
    Computing =AddressFamily for SiblingAddresses(directory=u'')
      Computing =AddressFamily for Dir(path=u'')
        Computing =AddressFamily for Dir(path=u'')
          Throw("EntryId(17) failed!")

Testing

Local manual testing with ./pants list and a corrupted BUILD file. Running tests that had been skipped due to this feature being missing. CI away on #4076

baroquebobcat added some commits Nov 18, 2016

Trace for the rust engine
Unfortunately, this still has some proof of concept bits in it. But, it does provide traces.

My caveat is that the traces do not include node information or the message from the failure. But you do see the subjects and the overall structure.

Additionally they print to stdout.
append to the files, since we're doing multiple traces, don't want to…
… truncate

Also, add 4025 to skips where relevant
Merge branch 'master' into nhoward/trace_for_engine
Had a couple isort errors in ...tasks2. Fixed those.

@baroquebobcat baroquebobcat changed the title from Trace for the rust engine to [engine] Initial Trace implementation for rust engine Nov 19, 2016

@baroquebobcat

This comment has been minimized.

Show comment
Hide comment
@baroquebobcat

baroquebobcat Nov 19, 2016

Contributor

This patch was initially reviewed on https://rbcommons.com/s/twitter/r/4392/
@stuhood and @kwlzn gave ship its on it there. I pinged @benjyw, @JieGhost as reviewers as well.

Contributor

baroquebobcat commented Nov 19, 2016

This patch was initially reviewed on https://rbcommons.com/s/twitter/r/4392/
@stuhood and @kwlzn gave ship its on it there. I pinged @benjyw, @JieGhost as reviewers as well.

@benjyw

This comment has been minimized.

Show comment
Hide comment
@benjyw

benjyw Nov 19, 2016

Contributor

I've been skipping over these because I know ~nothing about Rust.

On Fri, Nov 18, 2016 at 6:19 PM, Nick Howard notifications@github.com
wrote:

This patch was initially reviewed on https://rbcommons.com/s/
twitter/r/4392/
@stuhood https://github.com/stuhood and @kwlzn
https://github.com/kwlzn gave ship its on it there. I pinged @benjyw
https://github.com/benjyw, @JieGhost https://github.com/JieGhost as
reviewers as well.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#4076 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAfS_Ir3xg1EIaiVl8zu74bPSO3mXWQMks5q_lyngaJpZM4K2pZC
.

Contributor

benjyw commented Nov 19, 2016

I've been skipping over these because I know ~nothing about Rust.

On Fri, Nov 18, 2016 at 6:19 PM, Nick Howard notifications@github.com
wrote:

This patch was initially reviewed on https://rbcommons.com/s/
twitter/r/4392/
@stuhood https://github.com/stuhood and @kwlzn
https://github.com/kwlzn gave ship its on it there. I pinged @benjyw
https://github.com/benjyw, @JieGhost https://github.com/JieGhost as
reviewers as well.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#4076 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAfS_Ir3xg1EIaiVl8zu74bPSO3mXWQMks5q_lyngaJpZM4K2pZC
.

@@ -45,6 +45,16 @@ impl Scheduler {
self.graph.visualize(&self.roots, path, &self.tasks.externs)
}
pub fn trace(&self, path: &Path) -> io::Result<()> {
for root in &self.roots {
let result = self.graph.trace(&root, path, &self.tasks.externs);

This comment has been minimized.

@stuhood

stuhood Nov 19, 2016

Member

It should be equivalent to wrap this line in try!()... equivalent the new ? operator: https://blog.rust-lang.org/2016/11/10/Rust-1.13.html

@stuhood

stuhood Nov 19, 2016

Member

It should be equivalent to wrap this line in try!()... equivalent the new ? operator: https://blog.rust-lang.org/2016/11/10/Rust-1.13.html

@stuhood stuhood merged commit 37122df into pantsbuild:master Nov 19, 2016

1 of 2 checks passed

coverage/coveralls Coverage pending from Coveralls.io
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment