Data Driven Workflows

[Richard-Gist]

This issue is here to help track some ideas and things related to the milestone. It's also here so the Milestone will stop bouncing between not-complete and complete.

So far we have spike branches: data-spike1 through data-spike3 so far.

The main PR that will bring in Data-Driven Workflows will be the branch data-driven

[Richard-Gist]

Added another spike data-spike4. It shows a registry that can generate a workflow from strings (at this time).

[Richard-Gist]

Working through how to use the Echo library to see if we can use it to automatically find all the FlowRepresentable types in a running app. Having moderate success, but still not there. I opened this discussion: Azoy/Echo#66 on their project to see if they could help me out.

I can get all type descriptors, find my protocol descriptors, and confirm if I have metadata, I can check conformance or check conformance with a metadata type and specific protocol descriptors.

I cannot get from a type descriptor to a metadata type. I cannot get from a metadata type to a FR.Type.

[morganzellers]

EOD:

Created a new branch to spike out using SourceKitten to check for FlowRepresentable conformance.

Spike branch contains a file, findFlowRepresentables.swift which (currently) uses STDIN to parse JSON output from sourcekitten structure and returns the name of the structure that inherits from FlowRepresentable.

Note: Usage directions are at the top of the script file. Though you have to build it, so who's scripting anyways?

Next up is figuring out how to scale this to a project. I think sourcekitten structure can only be passed a single file, so that likely means iterating through all *.swift files in a project and appending the JSON output for each to a file, then passing the file to the script, which decodes the JSON into a list of objects, ultimately outputting the list of FlowRepresentable conforming types to a file that we can then reference.

[Richard-Gist]

EOD: data-spike4 is to a point where it can dynamically find all the FlowRepresentable views that are NOT generic. It can then take these views and register them in the registry from the start of the spike. This can then be used to generate a workflow from an array of strings.

Big caveats: no generic views, the dependency Echo may prevent release, and this spike still has a global mutable state with a singleton registry.

[morganzellers]

EOD: Updated script to find all swift filenames in a directory and its subdirectories. It then generates an array of sourcekitten structure JSON output. I'm currently in the middle of changing the parsing of that JSON over to use dictionaries instead of Swift objects. This parsing determines if a given file conforms to a protocol (FlowRepresentable). Conforming types are added to a list

[morganzellers]

EOD:

Script is now creating a list of the types as well as a list of their associated AST JSON from sourcekitten. Tomorrow I'm taking a crack at using them to generate the Workflow from the SwiftUI Example:

WorkflowLauncher(isLaunched: .constant(true)) {
    thenProceed(with: SwiftCurrentOnboarding.self) {
        thenProceed(with: ContentView.self)
    }
}

[morganzellers]

EOD:

Began the process of switching from SourceKitten to swift-syntax for parsing and checking files. Looks like the experience is going to be much better since things will already be Swift types with swift-syntax. I've added the dependency to the SwiftUIExampleApp and added a target under the example app that is a command line app. At the moment, the script is importing the library and things are compiling.

[Richard-Gist]

EOD:

I made ANOTHER spike, data-spike5, which showcases possible ways of leveraging PlantUML for describing workflows in .puml files.

I think the State diagram ultimately landed on the easiest to write/read/parse. With aliasing being used in FR11, I think we can have a concise and nice-looking diagram that doesn't abuse the language and is functional in our descriptions. It, unfortunately, would not be auto-generated quite the same as other PlantUML diagrams, and it would require some effort to make for a safe experience writing it.

[morganzellers]

EOD:

Script updated to use swift-syntax, but still working on attaining the same functionality as with sourcekitten. Script can be checked out, but you'll need to change the copy script in the build phases of the DataDriven target.

[Richard-Gist]

EOD: I've been playing out some more data ideas. #147 (comment) discusses the JSON schema I just pushed to data-spike5

[morganzellers]

Still working on updating the script to use swift-syntax. Things are going well but the documentation for swift-syntax is pretty bare bones.

[Richard-Gist]

TL;DR - data-spike6 in SwiftUIExampleApp.swift has code we may want to convert into a PR after the holidays.

data-spike6 Is all coded in SwiftUIExampleApp.swift but I've started to separate out the different parts and have been looking into extensibility for this part of the library so that it can be easily enhanced to fit each person's use cases.

From this investigation, I'm liking the FlowRepresentableAggrigator (sp) protocol that allows easy usage of different methods for aggregating (Echo, Registry, Manual) and passing that as a tool to extract the workflow from the provided type names.

I'm not fully sold on my mapping of FR names to FR types, as I think there will be some conflicts with how it is implemented now. But I think it's good enough to keep moving forward for now.

I have codable objects to convert the JSON. They are currently Structs but I think we'll want them to be Classes so they will be easily extended.

I think extracting generation of the FRMetadata out to the FlowRepresentable itself with TylerMetadata has really fixed a lot of the inappropriate overlapping and has generally helped clean the code up quite a bit.

I think after the holidays we may be able to push forward with making an effort to convert this spike into a PR for the data-driven branch.

[morganzellers]

EOD: Basically cleaned up and removed some of the cruft left over from the sourcekitten part of the spike. Next step is to do some testing on the parsing - I know the way I'm currently checking for "FlowRepresentable" conformance has some blind spots.

[morganzellers]

EOD:

Updated script to more flexibly check for FlowRepresentable conformance. It will now check every token between a struct name and a {. Next, need to make sure it can do the same for classes.

[morganzellers]

EOD from yesterday:

Not a ton of progress was made between meetings and end-of-the-year things but did get some more knowledge about handling classes that conform to FlowRepresentable.

[morganzellers]

EOD:

Added support for finding extension and protocol conformance. Added rough support for finding protocol conformance via subclasssing.

Ex:

protocol FRToo: FlowRepresentable {

}

struct LoginView: View, FRToo {}

Am able to find that LoginView conforms to FlowRepresentable

[morganzellers]

EOD:

Have been taking a look at the FileVisitor class in the Sitrep library. I think this sort of direction would be good to take our script in. Having access to a swift class version of the AST will make things so much easier to code to as well as much more flexible. Tomorrow planning to take a crack at implementing a function or two in the morning before playbook time starts.

[morganzellers]

EOD:

Took a swing today at making the protocol and extension conformance checking better. I think all it did was make me want to take a second look at the FileVisitor class in Sitrep. I think the extra functionality they've added via that class will help the script a lot.

[morganzellers]

EOD:

Spiked out some changes (on the data-spike-sitrep branch) to the script using the FileVisitor class from SitRep. I like the direction it brought things in and I plan to implement that style going forward. I think holding the AST's in memory and applying logic to them that way rather than traversing and parsing the source code many times will be much better.

[nickkaczmarek]

Yesterday we implemented the ability to provide platform specific FlowRepresentable names, LaunchStyles, and FlowPersistences as well as the ability to use a * property to default the above types and allow the user to provide overrides as needed. See: this PR

Should enable something like this:

{
    "schemaVersion": "v0.0.1",
    "sequence": [
        {
            "flowRepresentableName": "FR1"
        },
        {
            "flowRepresentableName": "FR2",
            "launchStyle": "modal",
            "flowPersistence": "removedAfterProceeding"
        },
        {
            "flowRepresentableName": {
                "watchOS": "FR3",
                "macOS": "FR3",
                "iOS": "FR3",
                "iPadOS": "FR3",
                "tvOS": "FR3",
                "android": "FRA3"
            },
            "launchStyle": {
                "watchOS": "modal",
                "macOS": "modal",
                "iOS": "modal",
                "iPadOS": "popover",
                "tvOS": "modal",
                "android": "widget"
            },
            "flowPersistence": {
                "watchOS": "removedAfterProceeding",
                "macOS": "removedAfterProceeding",
                "iOS": "removedAfterProceeding",
                "iPadOS": "removedAfterProceeding",
                "tvOS": "removedAfterProceeding",
                "android": "somethingElse"
            }
        },
        {
            "flowRepresentableName": {
                "*": "FR3",
                "android": "FRA3"
            },
            "launchStyle": {
                "*": "modal",
                "iPadOS": "popover",
                "android": "widget"
            },
            "flowPersistence": {
                "watchOS": "removedAfterProceeding",
                "macOS": "removedAfterProceeding",
                "iOS": "removedAfterProceeding",
                "iPadOS": "removedAfterProceeding",
                "tvOS": "removedAfterProceeding",
                "android": "somethingElse"
            }
        }
    ]
}

[nickkaczmarek]

Rebased the data-driven branch on main this morning and updated the tests to be async await as needed. Currently working on introducing a sample app that displays the functionality data driven workflows offers. Saw that Richard had originally put that stuff into the SwiftUIExample app, but I felt like a new project was needed so that's what I'm working on.

[Tyler-Keith-Thompson]

We actually support data-driven workflows now with #193