Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create discovered dependencies file for emit-module jobs #1056

Merged

Conversation

BenchR267
Copy link
Contributor

Emit-module jobs might have different discovered dependencies and run in a different context where they need to get invalidated separately from the compile jobs.
With the new key "emit-module-dependencies" in the output file map the build system can specify that it expects a makefile-style dependency file at the given path.

rdar://91574616

Emit-module jobs might have different discovered dependencies and run in a different context where they need to get invalidated separately from the compile jobs.
With the new key "emit-module-dependencies" in the output file map the build system can specify that it expects a makefile-style dependency file at the given path.

rdar://91574616
@BenchR267
Copy link
Contributor Author

@swift-ci please test

Copy link
Contributor

@artemcm artemcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks!

@BenchR267 BenchR267 merged commit b2eb29f into swiftlang:main Apr 12, 2022
@BenchR267 BenchR267 deleted the ben/91574616-emit-module-dependencies branch April 12, 2022 15:59
@finagolfin
Copy link
Contributor

finagolfin commented Apr 20, 2022

I think something is wrong with this pull, as I've been seeing a bunch of emit-module.d files in my Swift package directory with the latest 5.7 snapshot builds. The first Apr. 12 snapshot doesn't have this problem, so building with swift build -v then comparing to the Apr. 18 snapshot shows that both run this identical command to emit a module:

/home/butta/swift-nio/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-18-a-ubuntu20.04/usr/bin/swiftc -module-name _NIODataStructures -incremental -emit-dependencies -emit-module -emit-module-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/_NIODataStructures.swiftmodule -output-file-map /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/_NIODataStructures.build/output-file-map.json -parse-as-library -c /home/butta/swift-nio/Sources/_NIODataStructures/Heap.swift /home/butta/swift-nio/Sources/_NIODataStructures/PriorityQueue.swift -I /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug -target x86_64-unknown-linux-gnu -swift-version 5 -enable-batch-mode -index-store-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/index/store -Onone -enable-testing -g -j1 -DSWIFT_PACKAGE -DDEBUG -module-cache-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/ModuleCache -parseable-output -parse-as-library -v

but when translated to the swift-frontend only the later snapshot adds an unprefixed -emit-dependencies-path:

Swift version 5.7-dev (LLVM 8bbf73b53c59eeb, Swift a159921d1bf3336)Target: x86_64-unknown-linux-gnu/home/butta/swift-nio/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-18-a-ubuntu20.04/usr/bin/swift-frontend -frontend -emit-module -experimental-skip-non-inlinable-function-bodies-without-types /home/butta/swift-nio/Sources/_NIODataStructures/Heap.swift /home/butta/swift-nio/Sources/_NIODataStructures/PriorityQueue.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -I /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug -enable-testing -g -module-cache-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/ModuleCache -swift-version 5 -Onone -D SWIFT_PACKAGE -D DEBUG -new-driver-path /home/butta/swift-nio/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-18-a-ubuntu20.04/usr/bin/swift-driver -empty-abi-descriptor -resource-dir /home/butta/swift-nio/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-18-a-ubuntu20.04/usr/lib/swift -enable-anonymous-context-mangled-names -module-name _NIODataStructures -emit-module-doc-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/_NIODataStructures.swiftdoc -emit-module-source-info-path /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/_NIODataStructures.swiftsourceinfo -emit-dependencies-path _NIODataStructures.emit-module.d -parse-as-library -o /home/butta/swift-nio/.build/x86_64-unknown-linux-gnu/debug/_NIODataStructures.swiftmodule

which causes all those files to be dumped in the top level of the package, rather than in .build/ where they belong. The trunk snapshots haven't been generated in weeks so I didn't test trunk, but if you have trunk built locally, should be reproducible there too. Adding -Xswiftc -disallow-use-new-driver works around the problem and led me to this driver.

Update: I can reproduce with the just-released Apr. 20 trunk snapshot for Ubuntu too, with the same workaround making it go away, implying the bug is definitely in this repo.

@compnerd
Copy link
Member

@buttaface thanks for identifying the change. I’ve been seeing this on Windows as well.

compnerd added a commit to compnerd/swift-driver that referenced this pull request May 1, 2022
Provide a fallback default for the `emit-module.d` file.  It has been
observed on Windows and Android that builds subsequent to swiftlang#1056 would
emit files into the root of the source tree (specifically, `pwd`). If
there is no explicit location specified for the new output, and given
that there is no primary output associated with the command, the path
that defaults is simply a singular file name component, emitting that
file into the location that the driver was executed from (which comes
out to the root of the package for commandline invocations), dirtying
the source tree.

Rather than sinking the knowledge for the default into
`existingOutputForSingleInput`, where we have no access to the module
name, emit the logic inline and derive a name based on the Swift module
and place it as a peer.  This should help ensure that the serialized
diagnostics do not end up committed accidentally.
compnerd added a commit to compnerd/swift-driver that referenced this pull request May 4, 2022
Provide a fallback default for the `emit-module.d` file.  It has been
observed on Windows and Android that builds subsequent to swiftlang#1056 would
emit files into the root of the source tree (specifically, `pwd`). If
there is no explicit location specified for the new output, and given
that there is no primary output associated with the command, the path
that defaults is simply a singular file name component, emitting that
file into the location that the driver was executed from (which comes
out to the root of the package for commandline invocations), dirtying
the source tree.

Rather than sinking the knowledge for the default into
`existingOutputForSingleInput`, where we have no access to the module
name, emit the logic inline and derive a name based on the Swift module
and place it as a peer.  This should help ensure that the serialized
diagnostics do not end up committed accidentally.
artemcm pushed a commit to artemcm/swift-driver that referenced this pull request May 23, 2022
Provide a fallback default for the `emit-module.d` file.  It has been
observed on Windows and Android that builds subsequent to swiftlang#1056 would
emit files into the root of the source tree (specifically, `pwd`). If
there is no explicit location specified for the new output, and given
that there is no primary output associated with the command, the path
that defaults is simply a singular file name component, emitting that
file into the location that the driver was executed from (which comes
out to the root of the package for commandline invocations), dirtying
the source tree.

Rather than sinking the knowledge for the default into
`existingOutputForSingleInput`, where we have no access to the module
name, emit the logic inline and derive a name based on the Swift module
and place it as a peer.  This should help ensure that the serialized
diagnostics do not end up committed accidentally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants