Source Code for the frameworks for S4TF? Where are they?

[shyamalschandra]

Where are the implementations for the following frameworks?

Example:

1. Tensorflow
2. Datasets
3. ModelSupport
4. ImageClassificationModels
5. Foundation
   etc.

Do you put these on Github? How can I drill-down without understanding the capabilities in a transparent way of these custom Swift frameworks for Tensorflow first? All the classes with their data members and functions along with the respective namespaces?

Thanks in advance!

[shyamalschandra]

I couldn't find anything here: https://github.com/apple/swift/tree/tensorflow

[BradLarson]

Due to the nature of the different repositories, this can be a little confusing. I wrote a bit of a guide to them in the mailing list, so I'll reproduce that below:

At a high level, there are several separate GitHub projects that handle different aspects of Swift for TensorFlow:

https://github.com/tensorflow/swift - Documentation about the project, along with a SIL parsing library and an initial implementation of Swift quotes (an experimental capability).

https://github.com/apple/swift/tree/tensorflow - The tensorflow branch of the Swift compiler, containing the compiler-side additions required to make automatic differentiation work. Many of the additions here will hopefully be upstreamed into the mainline Swift compiler.

https://github.com/tensorflow/swift-apis - The higher-level data structures and algorithms that allow for machine learning models (with an emphasis on deep learning) to be built using Swift for TensorFlow. To date, these have been layered on top of TensorFlow's eager execution environment, and that resulted in a single TensorFlow module that you'd import into your project. This is currently being reworked to separate the lower-level Tensor module containing basic elements like Tensor from the higher-level TensorFlow module containing layers and other structures.

https://github.com/google/swift-jupyter - The Jupyter kernel for Swift, allowing Swift for TensorFlow to be used in Jupyter and Colab.

https://github.com/tensorflow/swift-models - A repository of example machine learning models created with Swift for TensorFlow. This is intended to provide examples of how to use Swift for TensorFlow, to allow for end-to-end tests of automatic differentiation and the learning APIs, and to host benchmarks of model performance.

These are the core repositories directly maintained by the Swift for TensorFlow team, although there are several important third-party repositories that build on top of one or more of the above, such as SwiftAI: https://github.com/fastai/swiftai

If you think we could do a better job of highlighting the various repositories on the main Readme here, maybe we could make the structure of the project more prominent there.

[shyamalschandra]

@BradLarson : Where can I just get the .framework source code for each of the Swift for Tensorflow frameworks along with the data members, functions, and classes inside each?

[8bitmp3]

Hey @shyamalschandra, just to add to what @BradLarson said, most of the packages you inquired about are implemented in the /tensorflow/swift-models/ repo:

• Datasets > https://github.com/tensorflow/swift-models/tree/master/Datasets
• ModelSupport > https://github.com/tensorflow/swift-models/tree/master/Support (I think)
• ImageClassificationModels > https://github.com/tensorflow/swift-models/tree/master/Models/ImageClassification

Meanwhile, Foundation is one of Swift lang's main frameworks: https://developer.apple.com/documentation/foundation

So, for example, you come across the DCGAN example and it starts with importing a bunch of stuff and you want to dive into each package:

import Datasets
import Foundation
import ModelSupport
import TensorFlow
...

One solution is to check the paths in /tensorflow/swift-models/Package.swift:

import PackageDescription

let package = Package(
    name: "TensorFlowModels",
...
    targets: [
            .target(name: "ImageClassificationModels", path: "Models/ImageClassification"),
            .target(name: "Datasets", dependencies: ["ModelSupport"], path: "Datasets"),
            .target(name: "ModelSupport", path: "Support"),
...

Again, this was discovered by searching for each term in the search bar.

Perhaps, the path to ModelSupport could be renamed to ModelSupport for improved usability, but I'm sure it can be discussed at one of the design meetings (cc @saeta).

---

To identify how each S4TF thing was implemented, Github's repo search works really well. For instance, if you're interested in the Conv2D<...> struct's internals, check under /TensorFlow/Layers/: https://github.com/tensorflow/swift-apis/blob/master/Sources/TensorFlow/Layers/Convolutional.swift

Another example: the main Layer protocol's code is directly under .../TensorFlow/ here: https://github.com/tensorflow/swift-apis/blob/master/Sources/TensorFlow/Layer.swift.

And, best of all, everything is written in Swift, since Swift is not a wrapper lang. The API docs are a good guide for all kinds of stuff: https://www.tensorflow.org/swift/api_docs.

Hope this helps and feel free to correct me if I got anything not right.

[shyamalschandra]

@8bitmp3 : Thanks for your response!

[maxvol]

@BradLarson are there any expectations at the moment about when those "Many of the additions here will hopefully be upstreamed into the mainline Swift compiler" ? Is it going to be 5.x, 6.x, or some distant version? Are those additions expected to be merged at once or one by one, spread across different (minor) Swift versions?

[shyamalschandra]

@BradLarson: Also, will the official toolchain be merged with the Swift for Tensorflow toolchain?

[BradLarson]

@maxvol @shyamalschandra - The upstreaming process has been going on for a while now, placing these features behind a compiler flag in the mainline Swift toolchain. Here's a post discussing the trajectory for evaluating adding automatic differentiation into Swift and the plan that was proposed for upstreaming automatic differentiation.

Ideally, at the end of this, not much would be left that would be tremendously different in the tensorflow branch. There's no concrete timeline on this, but the sooner this can be realized, the easier it will be for us as engineers who have had to synchronize a separate and complex branch with upstream changes in the Swift toolchain. We're strongly motivated to make this work.

The feature itself is still subject to the Swift evolution process, but we're hopeful that the design and value of the feature will make it acceptable to the broader community.

[shyamalschandra]

I agree with you, @BradLarson!

[rickwierenga]

@BradLarson : Where can I just get the .framework source code for each of the Swift for Tensorflow frameworks along with the data members, functions, and classes inside each?

I just wanted to add that Swift for TensorFlow is not just a framework, such as Alamofire, but rather a complete toolchain. This means we have access to not only the Swift for TensorFlow library but also Python interop, TensorFlow, etc.

[BradLarson]

Thanks for bringing this up and starting a discussion. We're looking into better ways of surfacing this information.