[TIMOB-25927] Add Swift support for native modules, move SDK-core into "TitaniumKit.framework"

[hansemannn]

JIRA: https://jira.appcelerator.org/browse/TIMOB-25927

---

✅This change will not cause a breaking change for native iOS modules, so no recompiling required for existing Obj-C based modules. But that's so 2017!

🗓 Scheduled for Q3/Q4 and SDK 8.0.0

---

Overview

Replaces the Titanium iOS Core with framework based alternative that is required if we want to deal with native modules that use Swift libraries, as we cannot bake Swift frameworks into static libraries and Swift frameworks do not allow non modular imports. Especially not with our (currently) hacky solution that attempts to reference symlinks from the application data directory. The following tasks represent the base of this effort.

Docs

This PR is introducing Jazzy to generate apidocs for TitaniumKit. We are currently at 23 % docs-coverage, which can be improved (😙). See the docs here.

Playground

You can downloaded a recent packaged version of this PR here and try around with Swift modules:

ti create -t module --name ti.test --id ti.test --code-base swift

You can also test this module "ti.test" that is created via the CLI and can be packaged and ran by Titanium. I also took an existing module (Ti.PSPDFKit) and was able to reduce the module code by > 60 %.

Reviewers

• sgtcoolguy - Overall review and integration in the Titanium ecosystem
• cb1kenobi - All CLI-related changes, especially to the module template generation and the app-, module-build.
• janvennemann - Native framework structure and docs generation / meta-tooling
• vijaysingh-axway - iOS code changes, e.g. moved classes, TiSharedConfig replacement for injected constants and class-extensions

SDK Tasks

• Create universal TitaniumKit.framework
• Move SDK-core to TitaniumKit.framework
• Remove Kroll Thread (use Main Thread) merged via own PR ([TIMOB-26226] iOS: Remove Kroll-Thread #10196)
• Remove TiCore (use native JSCore) merged via own PR ([TIMOB-25748] iOS: Remove TiCore in favor of built-in JavaScriptCore #9798)
• Reference TitaniumKit as a target dependency of the core
• Integrate TitaniumKit into the build pipeline
• Create .podspec for usage with CococaPods
• Be able to recompile core-framework if compiler flags change (e.g. IS_XCODE_* and USE_TI_UIIOSBLURVIEW). Can we use OTHER_C_FLAG=$(inherited) -DISXCODE_*? Update: I got it working by using the concept of class extensions to solve this very clean. During review, we should make sure to not miss any remaining preprocessor statements. The only ones still there seem to be internally now, like image-debugging and DEBUG statements.

Module / CLI Tasks

• Create framework based module template (embedded Swift framework)
• Reference TitaniumKit locally
• Test common Module API's (String, Number, Array, Dictionary, Date, id/Any)
• Update the Scons-CLI to copy TitaniumKit files into the packaged SDK
• Update the Build-CLI to copy TitaniumKit files into generated project build directory
• Update module build
  • Compile (universal) framework instead of static library
  • Package framework instead of static library
  • Link framework based modules in SDK core (update _build.js validateModules step)
  • Keep backwards compatibility for Obj-C based modules (which still need to be recompiled)
  • Update templates to include a Swift module template. Prompt to choose between Swift and ObjC-module
  • Be able to link TitaniumKit into module projects without actually linking the framework?
  • Symlink TitaniumKit core-source to generated include/ directory in packaged SDK to actually allow retain backwards compatibility
  • Fix Swift-based modules from not being copied to the actual app (the .framework is not available)
  • Link Swift framework-dependencies to the "Embedded Frameworks" section automatically (needs an update to the framework-manager from @janvennemann to 1) check modules/iphone/<module>/<version>/<name>.framework to be embedded and move it to the correct section and 2) not require our infamous ti.swiftsupport hook)
• Incremental build seems to be broken, we need to see where it falls through (EDIT: Turned to be a side-effect of [TIMOB-25226] iOS: Use the new Xcode build-system by default on Xcode 10, make configurable on older #10181, fixed in 10223)
• Precompile TitaniumKit.framework and link it to the Xcode-project. Retain TitaniumKit.xcodeproj for easy local development
• Add Always Embed Swift Standard Libraries when Swift-based modules are added, making the ti.swift hook obsolete. EDIT: Added as part of the module template for now, but we should really do this in the _build.js instead. Someone has a proper place for this?
• Figure out, where the (duplicate) <generated-sdk>/iphone/dist directory with the compiled framework comes from. We do not need it twice. It must be either a glitch in the <sdk-repo>/support/iphone/build_titaniumkit.sh (where the framework is compiled) or in the <sdk-repo>/build/ios.js (where the script is invoked and the framework is copied)

Misc / Future Tasks

• Clean up linting once everything is relatively stable
• Investigate support for dropping TitaniumKit into native apps to use Ti-JavaScript in these. Update: Investigated and actually working, but developers cannot use Ti.UI.* API's, since that namespace requires the API's from the non-core project, that cannot be put into a framework so far, because of the usage-based preprocessor statements (e.g. USE_TI_UILISTVIEW). Maybe something for later majors if requested, but recent frameworks failed with this concept, so we should rather focus on hardening the cross-platform idea
• Update Obj-C module template to use TitaniumKit as well (not necessarily required, could be moved to the next feature version)

[sgtcoolguy]

A few thoughts:

• It looks like this moves some of the core to a new framework named TitaniumKit. We distribute that framework as source? Can it be pre-built in our scons build to speed up app builds? Would it not make a difference?
• This model is actually moving much more towards what we did on Windows where we have a TitaniumKit base library that the Windows impl extends. Effectively the entire API is declared in TitaniumKit and then windows provides actual implementations. I think Matt Langstons goal was to eventually move this way and have TitaniumKit be the common base for each platform. Interesting that this does move us closer that way (though I assume it'd be quite a huge undertaking and change to continue towards that, and not sure if that'd mean we'd need to use C++ everywhere versus Obj-C, etc.).
• Are there (eventual) plans to break out each API namespace to their own frameworks as well? i.e. Media, Network, Filesystem, etc.
• How did you decide what actually went in to TitaniumKit itself? It appears to be a mix of the Kroll stuff, Ti.API, common UI code, Ti.App, Ti.Blob, TiFile. In other words it is not at all clear what needed to go in there or how it's divided. Trial and error?
  • The naming has me a little confused when comparing to Windows, as there TitaniumKit is used to effectively outline the full API with no implementation and Titanium is the "core" impl.
  • But in that case, basically the circular dependencies between Blob and File forced us to move both to the very core Other than those two we had the core runtime class that registered all the APIs/native modules./etc. Sort of the equivalent of KrollBridge on iOS?
  • There is no KrollObject/KrollMethod there, that sort of binding layer is effectively what HAL does and TitaniumKit hooks up the JS/C++ API.

[hansemannn]

It looks like this moves some of the core to a new framework named TitaniumKit. We distribute that framework as source? Can it be pre-built in our scons build to speed up app builds? Would it not make a difference?

Initially, I was worrying that having a prepackaged framework could lead to issues because of the preprocessor macros. But as I moved them all out (only leaving the developer-related ones), that is not an issue anymore. Having faster clean builds is a good argument to do it, also that we would need to copy the framework-project into the build folder like we do right now. On incremental builds, the framework is already picked up and performs great. No benchmarking done, yet, but likely soon.

This model is actually moving much more towards what we did on Windows where we have a TitaniumKit base library that the Windows impl extends. Effectively the entire API is declared in TitaniumKit and then windows provides actual implementations. I think Matt Langstons goal was to eventually move this way and have TitaniumKit be the common base for each platform. Interesting that this does move us closer that way (though I assume it'd be quite a huge undertaking and change to continue towards that, and not sure if that'd mean we'd need to use C++ everywhere versus Obj-C, etc.).

It did not have too much in common with the initial C++ based TitaniumKit. Personally, I was never a fan of having a C++ layer, since it's simply to hard to maintain, especially in regards to community contributions. So it's really just the name (see details below) and the idea of a centralized place to precompile common API's.

Are there (eventual) plans to break out each API namespace to their own frameworks as well? i.e. Media, Network, Filesystem, etc.

Yes! Definitely a goal, but especially for UI and Media very critical because of the preprocessor macros that cannot be used if they are prepackaged. Without them, the app size grows and apps may be rejected (like when compiling camera-api's in Ti.Media although not using them).

How did you decide what actually went in to TitaniumKit itself? It appears to be a mix of the Kroll stuff, Ti.API, common UI code, Ti.App, Ti.Blob, TiFile. In other words it is not at all clear what needed to go in there or how it's divided. Trial and error?

It really was a trial and error. I only moved the Kroll* classes over and noticed the dependencies that would need to go as well. After all, I think it was a good idea, because it allows us to use Titanium in native apps as well (simply by using TiApp as the main entry point and inherit from there).

The naming has me a little confused when comparing to Windows, as there TitaniumKit is used to effectively outline the full API with no implementation and Titanium is the "core" impl.

The naming really just was because we are Titanium and frameworks in iOS are usually suffixed Kit, like UIKit, ARKit, IGListKit, ... . I thought naming it TitaniumCoreKit or TitaniumCore, but that made me think of TiCore again, which I am trying to forget asap 😄.

But in that case, basically the circular dependencies between Blob and File forced us to move both to the very core Other than those two we had the core runtime class that registered all the APIs/native modules./etc. Sort of the equivalent of KrollBridge on iOS?

I am not sure I can follow on this one. The circular dependencies did not seem to be an issue as far as I'm concerned.

[sgtcoolguy]

I quickly took a look at the the CI build and while not huge, it looks like the app build for iOS dropped by about 11 seconds here with the pre-built framework. (from 1m 19s to 1m 8s)

It sure would be good to take a look at the build process and time out everything to see where our biggest gains could come. I have to assume that the parse/transpile/file I/O of all the JS files is the largest chunk, but who knows. Makes me wonder if we can't do multiple files in parallel and gain some time back.

[hansemannn]

Some more improvements in iOS build time could come from:

• Only generate missing app-icons once
  • Currently, it is always done, even on incremental builds. Would save 1-2s
• Do not optimize images anymore
  • It is also done for every (even incremental) build right now and should be the developer's decision to take care of. Also uses some kind of jar to optimize them, which is a dependency we do not need
• Do not log files / steps that are being skipped. Although not critical, these are unnecessary

[hansemannn]

One important note before attempting to merge this: In the days before merging, we need to verify all occurrences of classes that have previously been in iphone/Classes/* and now are in iphone/TitaniumKit/Source/**/*. The reason is that those (like KrollContext.[h|m]) might have changed since and Git would not always discover that properly. We need to sync those files and block other iOS PR's from being merged in this timeframe to prevent file overlaps. Thx!

[vijayvikramsingh]

Macro is removed here. I think it should be there.

[sgtcoolguy]

OK, I think it's about time to get this monster PR in...

[hansemannn]

Found one regression, sorry for that! https://jira.appcelerator.org/browse/TIMOB-26439