Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

manifest_parser: remove multi-output depslog restriction #1534

Merged
merged 3 commits into from Jan 4, 2020

Conversation

mathstuf
Copy link
Contributor

@mathstuf mathstuf commented Mar 1, 2019

It seems to work as intended.


Cc: @RedBeard0531

@jhasse
Copy link
Collaborator

jhasse commented Mar 6, 2019

A test that it works as intended would be good (not only that it parses without error).

@mathstuf
Copy link
Contributor Author

mathstuf commented Mar 6, 2019

Here's a test file (courtesy of @RedBeard0531)

rule TOUCH
    command = touch $out

rule COPY
    command = echo 'using $in' && for file in $out; do cp $in $$file; done
    deps = msvc
    msvc_deps_prefix = using 

rule CHECK
    command = for file in $in; do if ! test -e $$file; then return 99; fi; done

build source: TOUCH
build dest1 dest2: COPY source
build test1: CHECK dest1
build test2: CHECK dest2
build test3: CHECK dest1 dest2
build test: phony test1 test2 test3
default test

ninja -t deps dest1 and ninja -t deps dest2 both output the expected depslog information. What's the best way to add this to the test suite?

@mathstuf
Copy link
Contributor Author

mathstuf commented Mar 8, 2019

Tests should also cover gcc parsing for the following two cases:

a b: in

and

a: in
b: in

@RedBeard0531
Copy link
Contributor

It should also work with either of:

a: in
b: in

Especially when one of the outputs is an implicit output. This is important for cases like -gsplit-dwarf -MMD because the compiler seems to only list the "primary" output in its deps files.

That said, if it is harder to do this for make-style deps, getting it just for msvc-style would be an improvement for a few of my use cases, so I'd be happy to see msvc-style support land independently if that would get it in an earlier release.

@mathstuf
Copy link
Contributor Author

mathstuf commented Mar 8, 2019

Also found out the place to implement the tests for this (build_test.cc). I'll work on that this weekend.

@mathstuf
Copy link
Contributor Author

mathstuf commented Mar 8, 2019

@RedBeard0531 Noted, I'll add a test for that as well. Though we should get GCC to mention the .dwo file in the deps log if it doesn't already.

@mathstuf
Copy link
Contributor Author

Tests added. @RedBeard0531 he b: in (or even b a: in) is currently treated the same as if you had a depfile not agreeing on the "primary" output name and is basically ignored. It's up to the generator to get the "primary" output correct.

@mathstuf
Copy link
Contributor Author

Odd. Those failures aren't happening locally. Debugging.

@mathstuf
Copy link
Contributor Author

Ugh. I wish GTest gave more information :/ . Can anyone else reproduce the test failures?

@mathstuf
Copy link
Contributor Author

Passing on my local macOS build too. Will try with Windows later.

@jonesmz

This comment was marked as abuse.

@jhasse
Copy link
Collaborator

jhasse commented Mar 11, 2019

I encourage you to consider writing a pull request that makes the configure.py script's "--with-gtest" actually do something (currently it does nothing at all, as the letters "gtest" are found exactly once in configure.py).

I think it would be better to remove that switch and always use googletest.

But probably don't start working on that until you've asked on the Ninja mailing list

Or opened an issue where you suggest the change.

as there's a decent chance (in my opinion) that such a PR would be either ignored or rejected.

PRs should no longer be ignored, I'm trying to change that. If you think there's an open one which should be looked at, just ping me :)

@jhasse
Copy link
Collaborator

jhasse commented Apr 18, 2019

PR for GoogleTest: #1562

You can cherry-pick the commit to get more information about the failed test.

@mathstuf
Copy link
Contributor Author

Thanks, will look at this in the next week or so.

@mathstuf mathstuf force-pushed the remove-depslog-restriction branch from d6830a6 to 347b449 Compare May 3, 2019 13:18
@mathstuf
Copy link
Contributor Author

mathstuf commented May 3, 2019

Ah ha. Installing re2c gets me to agree with the CI now. Tracking it down.

@mathstuf mathstuf force-pushed the remove-depslog-restriction branch from b5bc6a6 to 20f6065 Compare May 3, 2019 17:52
@mathstuf
Copy link
Contributor Author

mathstuf commented May 3, 2019

@jhasse Now that the cc.in file has right code too, CI is passing.

src/depfile_parser_test.cc Outdated Show resolved Hide resolved
@RedBeard0531
Copy link
Contributor

This is a rather severe limitation that prevents listing correct dependencies for a c++20 module scanner that outputs both a dyndeps file and a compiler flags file, when the compile task depends on both but order only for dyndeps and implicit for flags file. Additionally, the overall restriction has been lifted with the dyndeps patch because dyndeps can add implicit outputs to a deps=gcc build.

What is remaining to getting this merged?

@mathstuf
Copy link
Contributor Author

There's the conflict to resolve, but more importantly, there is the test case I commented on above. What was labeled a "buggy" depfile is no accepted. Should it continue to be rejected? If so, further work is necessary.

@mathstuf
Copy link
Contributor Author

Rebased and now rejects the case again.

@petrhosek
Copy link
Contributor

We've the limitations around multiple outputs with deps when building Rust targets with Ninja. I've tested this change locally and it seems to be working as expected. Can we get this merged?

@mathstuf
Copy link
Contributor Author

I've rebased on top of master.

@mathstuf
Copy link
Contributor Author

Any chance of this for 1.10 too? :) @jhasse

@jhasse jhasse added this to the 1.10.0 milestone Nov 18, 2019
@jhasse
Copy link
Collaborator

jhasse commented Nov 18, 2019

Let's see. It probably needs someone more experienced than me to review.

@mathstuf
Copy link
Contributor Author

Hmm. @bradking? @nico? Thoughts on this PR?

Copy link
Contributor

@bradking bradking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change makes sense over all. I've reviewed the logic a few times and it looks okay. The tests look thorough.

src/build.cc Show resolved Hide resolved
src/build.cc Outdated Show resolved Hide resolved
src/build_test.cc Outdated Show resolved Hide resolved
src/depfile_parser.cc Outdated Show resolved Hide resolved
src/graph.cc Show resolved Hide resolved
@mathstuf
Copy link
Contributor Author

Updated to use std:: on all lines added/changed in this PR. I left std::strerror alone since <cstring> is not included anyways.

@jhasse jhasse merged commit 1c5857c into ninja-build:master Jan 4, 2020
@mathstuf mathstuf deleted the remove-depslog-restriction branch January 6, 2020 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants