Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to package an iOS dynamic framework into a format Xcode can consume #928

Merged
merged 33 commits into from
Dec 14, 2020

Conversation

mccorkill1
Copy link
Contributor

@mccorkill1 mccorkill1 commented Sep 29, 2020

Background:
This scope of work was to develop a rule that can be used with Bazel that will output a dynamic framework which can be imported and used in Xcode. The existing ios_framework rule did not meet this requirement because it doesn’t include the headers, modulemap, or swiftmodule, and swiftdoc files bundled into the packaged .framework file. Therefore, we wrote the ios_dynamic_framework and watchos_dynamic_framework rules that output a .framework file with the compiled dynamic framework, Info.plist, Headers folder with the generated headers file, and Modules folder with the swiftmodule and modulemap files.

Approach:

  • Copy the ios_framework rule - it already did the packaging of the .framework file and the compiling of the framework

  • Copy the swift_static_framework_aspect - This aspect generates the swiftdoc, headers, and modulemap files, but it has limitations because static frameworks can’t have transitive dependencies. We removed the if statements that enforced that there be no transitive dependencies, used the SwiftInfo provider to get the relevant files for the transitive dependencies, including the swiftmodule file, and returned the references to the files in a new provider called SwiftDynamicFrameworkInfo.

  • Copy swift_static_framework partial - this partial uses the files returned by the SwiftDynamicFrameworkInfo provider and bundles them into the packaged .framework file. It also returns an incomplete CcInfo provider which was needed for the transitive dependencies to work.

  • Generate an objc_provider - In order for the ios_dynamic_framework rule to be used as a dependency of swift_library, it needed to pass an objc_provider that has a reference to the dynamic_framework_file. The dynamic_framework_file is is processed in the framework_provider_partial and returned in the AppleDynamicFramework provider. So after the partials are processed, the objc_provider is made and added to the list of returned providers. The _ios_dynamic_framework_impl felt like the wrong place to do this logic, but it's the only place we had access to all the data

Outstanding questions and callouts:

  • We added the swift_dynamic_framework_aspect to all product_type == framework. This shouldn’t impact anything since the aspect only returns additional information, but wanted to confirm this is the approach was acceptable.

  • We built this rule for ios and watchos. Are tvos and macos a priority for this use case? If so, can they go in a separate PR?
    Edit: We added support for tvos and macos

  • We added framework as a supported type for watchos in rule_factory.bzl and rule_support.bzl

  • Currently the rule only supports swift_library dependencies, and those libraries can only have swift_library dependencies. There is an issue with swift_libraries that have objc_library dependencies that we are aware is in the backlog to add support for.

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@mccorkill1 mccorkill1 marked this pull request as ready for review September 29, 2020 18:12
@mccorkill1 mccorkill1 marked this pull request as draft September 30, 2020 00:53
@mccorkill1 mccorkill1 marked this pull request as ready for review October 9, 2020 18:45
@brentleyjones
Copy link
Collaborator

@mccorkill1 Hi! We can't really look at this until a CLA is signed. that said, I would like a rule that produces .frameworks for Xcode consumption (we have a SwiftUI integration that requires it).

We built this rule for ios and watchos. Are tvos and macos a priority for this use case? If so, can they go in a separate PR?

Personally it would feel odd to me that it only supported some platforms.

@mccorkill1
Copy link
Contributor Author

@brentleyjones I am working on getting added as an authorized user on my company's corporate account. I will work on adding support for the other two platforms in the mean time. Thank you for the feedback!

…now be used as a dependency of a swift_library and have the transitives packaged properly. With that in mind, there is no more need to use ios_dynamic_framework as a dependency of a swift_library. It can solely be used to bundle a compiled swift_library into a framework usable by Xcode.
@mccorkill1
Copy link
Contributor Author

@googlebot I signed it!

@google-cla
Copy link

google-cla bot commented Nov 12, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added cla: yes and removed cla: no labels Nov 12, 2020
@brentleyjones
Copy link
Collaborator

Thanks @mccorkill1! I'll take a look at this soon.

apple/internal/ios_rules.bzl Show resolved Hide resolved
apple/internal/ios_rules.bzl Outdated Show resolved Hide resolved
Copy link
Collaborator

@brentleyjones brentleyjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still making my way though this, but I wanted to leave some comments as I saw them. So far it looks great though!

apple/internal/macos_rules.bzl Outdated Show resolved Hide resolved
apple/internal/macos_rules.bzl Outdated Show resolved Hide resolved
apple/internal/macos_rules.bzl Outdated Show resolved Hide resolved
apple/internal/tvos_rules.bzl Show resolved Hide resolved
apple/internal/tvos_rules.bzl Outdated Show resolved Hide resolved
apple/watchos.bzl Show resolved Hide resolved
apple/macos.bzl Outdated Show resolved Hide resolved
apple/ios.bzl Show resolved Hide resolved
apple/internal/rule_support.bzl Outdated Show resolved Hide resolved
apple/internal/rule_support.bzl Outdated Show resolved Hide resolved
@google-cla
Copy link

google-cla bot commented Nov 13, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added cla: no and removed cla: yes labels Nov 13, 2020
@google-cla
Copy link

google-cla bot commented Nov 13, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@brentleyjones
Copy link
Collaborator

@googlebot I fixed it.

@google-cla
Copy link

google-cla bot commented Nov 13, 2020

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@mccorkill1
Copy link
Contributor Author

@brentleyjones So I've been working on getting the macos_dynamic_framework rule into shape and here's where I'm at: I've got most of it packaging properly, but I ran into a few snags.

  • swiftsourceinnfo files don't seem to be generated/accessible via bazel yet. It's in the works, but this PR hasn't merged yet: Adds support for swiftsourceinfo rules_swift#523

  • I had a hard time getting things to symlink properly once they were packaged into the framework. They looked right outside of the package, but once they were in the .framework file, they looked like copies of the file instead of symlinks. I was also using the declare_symlink function and the documentation says not to depend on it because it's still experimental: https://docs.bazel.build/versions/master/skylark/lib/actions.html#declare_symlink

  • I don't fully understand the CoreResources file or how it's used by Xcode. I know it's generated as a part of the code signing process, but I didn't see a way I could reference it from within the macos_dynamic_framework rule so I could package it in.

I've pushed my work in progress code to this branch on my fork if you want to take a look: https://github.com/mccorkill1/rules_apple/tree/macos_dynamic_framework

I think the best way forward is to remove the support for the macos_dynamic_framework rule from this PR and include it in a separate PR once these issues can be resolved. What do you think?

@keith
Copy link
Member

keith commented Dec 3, 2020

swiftsourceinnfo files don't seem to be generated/accessible via bazel yet. It's in the works, but this PR hasn't merged yet: bazelbuild/rules_swift#523

Did you hit an issue where this is required for something? I doubt it should be, especially for modules compiled with library evolution (which I guess these may or may not be)

@brentleyjones
Copy link
Collaborator

I think the best way forward is to remove the support for the macos_dynamic_framework rule from this PR and include it in a separate PR once these issues can be resolved. What do you think?

I agree. Let's pull out macOS for now and make a new PR with your progress.

@brentleyjones
Copy link
Collaborator

Did you hit an issue where this is required for something? I doubt it should be, especially for modules compiled with library evolution (which I guess these may or may not be)

My example from Xcode had it. I'll second @keith here, and say it's not needed, but once it is generated via bazelbuild/rules_swift#523 we should add it into these rules.

…moved off SwiftInfo.transitive_swift* in the swift_dynamic_framework_aspect
@google-cla
Copy link

google-cla bot commented Dec 3, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

test/starlark_tests/resources/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/tvos/BUILD Outdated Show resolved Hide resolved
@google-cla
Copy link

google-cla bot commented Dec 3, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

Copy link
Collaborator

@brentleyjones brentleyjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple more tags need to be migrated.

test/starlark_tests/targets_under_test/ios/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/ios/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/ios/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/ios/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/ios/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/tvos/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/tvos/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/watchos/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/watchos/BUILD Outdated Show resolved Hide resolved
test/starlark_tests/targets_under_test/watchos/BUILD Outdated Show resolved Hide resolved
@google-cla
Copy link

google-cla bot commented Dec 7, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

Copy link
Collaborator

@brentleyjones brentleyjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mccorkill1 for putting in all the work on this!

@keith @segiddins: Can I get a second pair of eyes on this before we merge it? Thanks!

@mccorkill1
Copy link
Contributor Author

Thank you @brentleyjones for being so responsive and reviewing the PR! I'm really happy with how the code turned out and I'm excited to get it merged in!

apple/internal/ios_rules.bzl Outdated Show resolved Hide resolved

return [file for file in framework_imports if file.short_path.endswith(".h")]

def _swift_dynamic_framework_partial_impl(ctx, swift_dynamic_framework_info):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we're trying to move away from partials taking ctx as a param?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I matched the format of the other partials and am no longer passing in ctx as a param to the rule when I call it. I added the same TODO that I saw in other partials to Remove ctx from the args when ctx is removed from all partials

When I removed it as a parameter I got Error: _swift_dynamic_framework_partial_impl() got unexpected keyword argument: ctx even though I am no longer manually passing in ctx.

…k partial and updated how the binary_target is selected so it not dependent on the order of the deps_list
@google-cla
Copy link

google-cla bot commented Dec 7, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@google-cla
Copy link

google-cla bot commented Dec 7, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

… when it is removed from all partials and fixed a comment
@google-cla
Copy link

google-cla bot commented Dec 8, 2020

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@mccorkill1
Copy link
Contributor Author

@keith @segiddins Can you take a look this? I think it's about ready to be merged in, but I need at least one more approval.

@keith
Copy link
Member

keith commented Dec 14, 2020

need to make sure to squash and merge this one

def _get_header_imports(framework_imports):
"""Get the header files from the list of framework imports"""

return [file for file in framework_imports if file.short_path.endswith(".h")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the extensions here should match what objc_library or cc_library allow, but this can be addressed in a follow-up

@brentleyjones brentleyjones merged commit 572aee2 into bazelbuild:master Dec 14, 2020
@brentleyjones
Copy link
Collaborator

Thanks again @mccorkill1!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants