New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libswift: Start implementing SIL optimizations in Swift #37058
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat! Sorry for the unsolicited review, I just added a couple comments regarding the build setup. Feel free to ignore if they're irrelevant.
I was recently looking into porting some of the DiagnosticConsumer
s to Swift for better Unicode-awareness and I think the infra here could eventually be a good foundation for that too.
CMakeLists.txt
Outdated
|
||
# Workaround to make lldb happy: we have to explicitly add all libswift modules | ||
# to the linker command line. | ||
set(LIBSWIFT_MODULES "SIL" "Optimizer") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be worth adding a comment to the package's targets list pointing to here for anyone adding a new module
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍done
let package = Package( | ||
name: "libswift", | ||
platforms: [ | ||
.macOS("10.9"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless this needs an earlier deployment target, would it make sense to align it with the driver/SwiftPM at 10.15?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, I changed it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to take this back, because it must match the deployment target in the cmake (which is 10.9)
libswift/README.md
Outdated
|
||
Though, there are some small deviations from the C++ SIL design as well. Either due to the nature of the Swift language (e.g. the SIL `Value` is a protocol, not a class), or improvements, which could be done in C++ as well. | ||
|
||
In this early stage, the design of _libswift_ is of course not fixed and will probably get some find tuning. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this early stage, the design of _libswift_ is of course not fixed and will probably get some find tuning. | |
In this early stage, the design of _libswift_ is of course not fixed and will probably get some fine tuning. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, but I pushed a new version anyway where I rewrote this section (and added more documentation in general).
Will libswift eventually depend on https://github.com/apple/swift-tools-support-core? |
@eeckstein some initial review on architecture. This is not the end of my review. I am against invasively placing support for a swiftpm subproject into swift's cmake like this. The build system is already too complex and adding into the custom cmake a sub-buildsystem will make it have even more issues. libswift is not special and should be able to be integrated without putting in conditional control flow into the basic build commands of the cmake (see add_swift_host_library). We need to minimize all non-standard cmake in the tree as much as possible. Some solutions:
I have more comments, but lets talk about this first. |
@tschuett I don't think there is a need for that (right now) |
@gottesmm I don't see the problem here. I agree that our build system is very complex. But, given that we have >15K lines of cmake code in the project, this little additional logic doesn't make a big difference. ad 1. swiftpm is the standard build system for swift code. It is better than cmake in every aspect. It would be a step back use cmake instead of swiftpm. We really shouldn't do this. ad 3. It is important that the tools can be built with the build system (without using build-script). i.e. invoking ad 2. This sounds interesting. If this works (e.g. tracking correct build dependencies), then we could use it. I'll let this up to someone who knows this better than me. |
@eeckstein Its an easy problem unless one is maintaining/evolving it. The problem here is that once we let in one thing another thing comes in. We are specifically trying to not grow the custom cmake part of the build system as much as possible. Integrating another build system into it without Let me respond in a subsequent comment line by line. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need to verbatim repeat all of the broken types and APIs on the Swift side that we've been wanting to fix for years on the C++ side. We know how to simplify all of these things, but no one was sufficiently motivated to do the big search/replace on the C++ code base.
Here I'm drawing particular attention to
- Location
- Builder
- Argument
|
||
import SILBridging | ||
|
||
public struct Location { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The debug scope is missing from this encapsulation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right, this should be actually mapped to SILDebugLocation
.
@@ -49,3 +49,8 @@ struct FunctionPass { | |||
} | |||
} | |||
|
|||
extension Builder { | |||
init(at insertionPoint: Instruction, _ context: FunctionPassContext) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A Builder should always be created with the (insertionPoint, (DebugScope, DebugLocation))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the debug location argument even needed? Isn't this always the location of the insertion point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think the debug info is usually inferred from the insertion point. Initially, you could just infer it and add overrides later when we need it.
The debug location always needs to be consistent with the insertion point, but I think there are a few places where we override it. We might choose location before or after insertion. IIRC inlining changes the scope. More commonly, we need to decide whether to mark a location as "compiler-generated" vs. inheriting from the surrounding code. We might just want to add some kind of enum to the initializer to make that selection.
Either way, I always wanted the SILLocation + DebugScope to be a single "DebugLocation" type with both information.
|
||
private var bridgedInsPoint: BridgedInstruction { insertionPoint.bridged } | ||
|
||
private func notifyInstructionsChanged() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why isn't this interface on a PassContext API instead of calling directly from the Builder into C++?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To minimize the usage of the bridging API. notifyInstructionsChanged()
will be called from a lot of places in the builder and here it's a central place where PassContext_notifyChanges
is called. It makes it easier in case we change the bridging layer at some time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, the C API needs to be called in a central place. That's why I expect that the Swift code to need a PassContext struct to encapsulate the C API. There are more ways to change instructions than just creating new ones
I didn't try to review the build changes. I suspect the build script just needs to communicate the location of bridging headers and installed Swift libraries, which should be something we can pass into the libswift project. It seems wrong that adding a Swift project to the build requires custom logic in the build script just to do this. Whatever happens here, I want to be able to rebuild libswift cleanly without running the build script. As a rule, I don't use the build script to build because it totally destroys my workflow. I often need to rebuild one project in isolation regardless of other changes in the tree. |
@@ -0,0 +1,117 @@ | |||
# Swift implemented in Swift | |||
|
|||
_Libswift_ is the part of the Swift compiler, which is implemented in Swift. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should "is the part" be "is a part" or "is the SIL optimizations part"?
|
||
import SIL | ||
|
||
let mergeCondFailsPass = FunctionPass(name: "merge-condfails", runMergeCondFails) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's more swifty to un-abbreviate "cond" as "condition" in identifiers defined on the Swift side. E.g. mergeCondFailsPass
→ mergeConditionFailsPass
, runMergeCondFails
→ runMergeConditionFails
, condFailToMerge
→ conditionFailToMerge
etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general, I agree. But in this case "CondFail" refers to the "cond_fail" instruction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ 2 tiny non-functional comments.
} | ||
} | ||
|
||
/// Try to merge the cond_fail instructions. Returns true if any could |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function doesn't have a return value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I'll fix it.
struct FunctionPass { | ||
|
||
let name: String | ||
let runFunction: (Function, FunctionPassContext) -> () |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that this, and InstructionPass
, just have a name + closure (which I assume will be unique for each pass), would it make sense for it to be a protocol?
This would mean that instead of writing:
let mergeCondFailsPass = FunctionPass(name: "merge-condfails", runMergeCondFails)
You could write:
struct MergeCondFailsPass: FunctionPass {
var name: String { "merge-condfails" }
/* mutating? In case a pass has state? */
func callAsFunction(_ function: Function, _ context: FunctionPassContext) {
// ...
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A pass should not have any state (that means: stored properties), because the pass object lives forever and it's dangerous to carry over state (like analysis caches) from one run to another. We have this in the C++ optimizer, and it ends up that the run functions of passes always instantiate another "pass" object which does the actual work - a lot of boiler plate.
Providing the run function as a function, makes this clear.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could also make the run function a static requirement, to enforce that there is no instance and hence no state.
The real benefit of using a protocol is that it gives you the full power of subtyping. So you could create a protocol SpecialFunctionPass: FunctionPass
with additional requirements, and which provides additional utility functions, etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, in the existing optimizer we didn't have a use for different kind of function passes. I'm not a big fan of adding abstractions/features if there is not a good use for them. I think simplicity should come first - and I think that not needing to define a separate nominal type for a pass makes it a bit simpler.
I pushed a new version with some changes in the build system. Most importantly: I added bootstrapping support in the build. |
@swift-ci test |
@swift-ci smoke test |
@swift-ci smoke test windows |
1 similar comment
@swift-ci smoke test windows |
@swift-ci smoke test windows |
@swift-ci smoke test |
@swift-ci test |
Build failed |
@swift-ci test |
Build failed |
@swift-ci test linux |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few small things. But overall the cmake isn't that invasive and we should get the fixes so we can use normal cmake relatively soon.
cmake/modules/AddSwift.cmake
Outdated
|
||
if(${SWIFT_HOST_VARIANT_SDK} IN_LIST SWIFT_APPLE_PLATFORMS) | ||
# For some reason, invoking CMAKE_Swift_COMPILER does not work. | ||
set(swift_command "xcrun" "swiftc") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this really not work still? That is surprising to me.
cmake/modules/AddSwift.cmake
Outdated
set_property(TARGET ${executable} APPEND_STRING PROPERTY | ||
LINK_FLAGS ${libswift_ast_path_flags}) | ||
|
||
# Workaround for a linker crash: rdar://77839981 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you describe the crash symptons/etc in the comment here (if possible)?
@gottesmm I pushed a new version |
@swift-ci test |
Build failed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am fine with the cmake. I am a little worried about __has_feature part of the change and I also think that the immediate part is already fixed on main since I changed our tests to always use the just built stdlib. The real solution is to run the JIT out of process but we aren't there yet.
Can you fix my comments and lets find a resolution to my two questions? I am not sure if this should require a whole test cycle though. If you are going to do more changes, feel free to fix them in this PR. Otherwise can you fix in a follow on?
cmake/modules/AddSwift.cmake
Outdated
@@ -477,7 +477,9 @@ function(add_swift_host_library name) | |||
endif() | |||
|
|||
add_library(${name} ${libkind} ${ASHL_SOURCES}) | |||
add_dependencies(${name} ${LLVM_COMMON_DEPENDS}) | |||
if (LLVM_COMMON_DEPENDS) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment here describing what LLVM_COMMON_DEPENDS is and maybe provide a reference to the large comment in ./lib/CMakeLists.txt where we describe it? I just don't want this to be mysterious.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you already did this. Thanks
@@ -597,6 +599,112 @@ function(add_swift_host_library name) | |||
endif() | |||
endfunction() | |||
|
|||
# Add a module of libswift |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add to all of the comments here that this is a temporary addition to AddSwift.cmake until fixes in the swift-driver are available on macOS host toolchains? That way it is 100% clear in the source that this is going away once what I mentioned happens.
|
||
#include <stdint.h> | ||
|
||
#if !__has_feature(nullability) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am fine fixing this in a follow up, but isn't __has_feature an extension? @compnerd is my memory right? I think we may need to do something more here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is handled in type_traits.h, where __has_feature is defined to 0 if it's not available. Also, it's tested with the windows PR testing - so it works for windows.
|
||
/// The header of a Swift object. | ||
/// | ||
/// This must be in sync with HeapObject, which is defined in the runtime lib. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you be more specific with this comment? My feeling is that I want it to be clear that we are matching the ABI of HeapObject. In Sync is good, but I think making it clear we are dealing with layout compatibility for correctness is important to make explicit.
include/swift/SIL/SILModule.h
Outdated
class FixedSizeSlab : public llvm::ilist_node<FixedSizeSlab>, | ||
public SILAllocated<FixedSizeSlab> { | ||
/// The payload for the FixedSizeSlab. | ||
/// This is a super-class rather than a member of FixedSizeSlab to make briding |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
briding
-> binding
.
@@ -88,6 +88,16 @@ class FixedSizeSlab : public llvm::ilist_node<FixedSizeSlab>, | |||
template<typename T> const T *dataFor() const { return (const T *)(&data[0]); } | |||
}; | |||
|
|||
/// A fixed size slab of memory, which can be allocated and freed by the | |||
/// SILModule at (basically) zero cost. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment here explaining in a paragraph below why it is zero cost? Just that way people do not have to look around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add a "See SILModule::allocSlab()", which has more info
include/swift/SIL/SILNode.h
Outdated
@@ -114,7 +115,7 @@ class SILNodePointer { | |||
/// subobject. If the SILNode is actually the base subobject of a | |||
/// ValueBase subobject, the cast will yield a corrupted value. | |||
/// Always use the LLVM casts (cast<>, dyn_cast<>, etc.) instead. | |||
class alignas(8) SILNode { | |||
class alignas(8) SILNode : public SwiftObjectHeader { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment here explaining that SILNode inherits from SwiftObjectHeader to allow for libswift bridging/ to look for information in SwiftObjectHeader.h about how this all works?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll mention libswift/README.md, where this is explained
|
||
/// The C++ version of SwiftObject. | ||
/// | ||
/// In C++ code, never use BridgedSwiftObject directly. SwiftObjectHeader has |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really would like somewhere a large description of how the bridging works that is centralized in the code base. One of those long in source comments. It IMO should describe how we rely on layout compatibility for correctness and also all of the little important bits like this. That way the whole model/its interesting parts is made explicit in the source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's explained in libswift/README.md. I'll mention it here.
#ifndef SWIFT_SILOPTIMIZER_OPTIMIZERBRIDGING_H | ||
#define SWIFT_SILOPTIMIZER_OPTIMIZERBRIDGING_H | ||
|
||
#include "../SIL/SILBridging.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We generally do not do relative paths like this in the include dir. Can you do the full "swift/SIL/SILBridging.h"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case it's needed for the libswift build.
lib/SIL/IR/SILBasicBlock.cpp
Outdated
@@ -26,12 +26,24 @@ | |||
#include "swift/SIL/SILModule.h" | |||
#include "swift/SIL/SILUndef.h" | |||
#include "swift/Strings.h" | |||
extern "C" { | |||
#include "swift/SIL/SILBridging.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just put the extern "C" in swift/SIL/SILBridging.h and #ifdef __cplusplus
? The includers of a header in c++ shouldn't need to know to do this. They should just be able to include SILBridging.h.
Also I think the cmake changes look good! I just want more comments there. |
* add the (still empty) libswift package * add build support for libswift in CMake * add libswift to swift-frontend and sil-opt The build can be controlled with the LIBSWIFT_BUILD_MODE cmake variable: by default it’s “DISABLE”, which means that libswift is not built. If it’s “HOSTTOOLS”, libswift is built with a pre-installed toolchain on the host system.
This is the initial version of a buildable SIL definition in libswift. It defines an initial set of SIL classes, like Function, BasicBlock, Instruction, Argument, and a few instruction classes. The interface between C++ and SIL is a bridging layer, implemented in C. It contains all the required bridging data structures used to access various SIL data structures.
This lets tests run conditionally on libswift: // REQUIRES: libswift
…LOptimizer's PassManager With the macro SWIFT_FUNCTION_PASS a new libswift function pass can be defined in Passes.def. The SWIFT_FUNCTION_PASS_WITH_LEGACY is similar, but it allows to keep an original C++ “legacy” implementation of the pass, which is used if the compiler is not built with libswift.
And two example instruction building functions.
StackList is a very efficient data structure for worklist type things. This is a port of the C++ utility with the same name. Compared to Array, it does not require any memory allocations.
SILPrinter prints the SIL of a function. It's and example pass which demonstrates the basic SIL API of libswift.
But keeping the old pass as legacy pass
Instruction passes are basically visit functions in SILCombine for a specific instruction type. With the macro SWIFT_INSTRUCTION_PASS such a pass can be declared in Passes.def. SILCombine then calls the run function of the pass in libswift.
…tion pass in libswift. But keeping the old visit function as legacy
@swift-ci smoke test |
@swift-ci smoke test macOS |
1 similar comment
@swift-ci smoke test macOS |
With libswift it is possible to add SIL optimization passes written in Swift. It allows to gradually migrate the SIL optimizer from C++ to Swift.
libswift
is a Swift package, parallel to thelib
directory in the repository.This PR is a very first initial version - a proof of concept.
It contains the basic (and still imcomplete) SIL and optimizer API and some example passes.
To see how how a Swift optimization pass looks like, take a look at SILPrinter, MergeCondFails or SimplifyGlobalValue
To see how all this works, take a look at libswift/README.md. It contains a lot more information about design and implementation.