Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idiomatic F# API #3644

Open
mikhailshilkov opened this issue Dec 11, 2019 · 12 comments
Open

Idiomatic F# API #3644

mikhailshilkov opened this issue Dec 11, 2019 · 12 comments

Comments

@mikhailshilkov
Copy link
Collaborator

@mikhailshilkov mikhailshilkov commented Dec 11, 2019

Pulumi .NET SDK supports F# as one of the languages to author infrastructure programs. It works quite fine, but there are several complications which make F# programs harder to write.

A typical Pulumi program consists of a number of value assignments, each calling a constructor of a resource type with proper arguments. The resource is defined by virtue of calling its constructor: the value doesn't need to be returned or passed to anywhere.

Let's look at two existing examples:

A. Azure resources (full code):

let resourceGroup = ResourceGroup "appservice-rg"

let storageAccount =
    Account("sa",
        AccountArgs
           (ResourceGroupName = io resourceGroup.Name,
            AccountReplicationType = input "LRS",
            AccountTier = input "Standard"))

let sku = PlanSkuArgs(Tier = input "Basic", Size = input "B1")
let appServicePlan = 
    Plan("asp", 
        PlanArgs
           (ResourceGroupName = io resourceGroup.Name,
            Kind = input "App",
            Sku = input sku))

B. Kubernetes deployment (full code):

let deployment = 
    Pulumi.Kubernetes.Apps.V1.Deployment("nginx",
      DeploymentArgs
        (Spec = input (DeploymentSpecArgs
          (Selector = input (LabelSelectorArgs(MatchLabels = appLabels)),
           Replicas = input 1,
           Template = input (
             PodTemplateSpecArgs
              (Metadata = input (ObjectMetaArgs(Labels = appLabels)),
               Spec = input (
                  PodSpecArgs
                    (Containers = 
                      inputList [
                        input (
                          ContainerArgs
                            (Name = input "nginx",
                             Image = input "nginx",
                             Ports = 
                              inputList [
                                input (
                                 ContainerPortArgs
                                   (ContainerPortValue = input 80))]))]))))))))

Here are some challenges:

1. Inputs and outputs.

Resources may contain output values: e.g. resourceGroup.Name in the first snippet. Its value might not be known at the time when this property is used, so a special type Output<T> is used to define the outputs. It's somewhat similar to Task<T>.

Each resource's constructor accepts a property bag where each property typically has type Input<T> or a type derived from it. An input can be a plain value (1, `"nginx") or point to an output value of another resource. This allows chaining the resources and creating dependencies between them.

While C# has implicit conversions between plain values, outputs, and inputs, F# type system is strict and requires conversions. Helper functions input, io, inputList, etc. were created to simplify the conversions, but they still clutter the code, and might be tricky to get right at times.

2. Indentation.

F# is white space significant, so the indentations must be right. The kubernetes example above shows that some resource args may use subtypes and sub-sub-types etc, so indentation gets quite heavy. It's also very strict, and compiler messages are not very helpful in getting it right.

Going one level deeper requires a ( or [, so the final statement ends with something like ))]))])))))))).

3. Side effects.

Resources are defined by calling constructors. Effectively, Pulumi SDK is based on side effects of calling those constructors as opposed to pure functions returning results. This goes against F# philosophy and puts an obstacle against many useful functional techniques.


For inspiration, a comparable kubernetes example in TypeScript looks much more concise:

const deployment = new k8s.apps.v1.Deployment("nginx", {
    spec: {
        selector: { matchLabels: appLabels },
        replicas: 1,
        template: {
            metadata: { labels: appLabels },
            spec: { containers: [{ name: "nginx", image: "nginx" }] }
        }
    }
});

The goal is to bring readability and write-ability of F# programs close to this level.


Good places to give a try to a Pulumi program in F#:

@lukehoban lukehoban mentioned this issue Dec 11, 2019
13 of 38 tasks complete
@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Dec 11, 2019

One possible avenue could be relying on computation expressions with custom operations. The kubernetes deployment example above could look like:

  let deployment = deployment "nginx" {
    spec (
      deploymentSpec {
        replicas 1
        selector (LabelSelectorArgs(MatchLabels = appLabels))
        template (
          podTemplateSpec {
            metadata (ObjectMetaArgs(Labels = appLabels))
            spec (
              podSpec {
                containers [
                  container {
                    name "nginx"
                    image "nginx"
                    ports [ContainerPortArgs(ContainerPortValue = input 80)]}]})})})
  }

While it's more lightweight, it still has many levels and a )]}]})})}) at the end. Besides, this solution requires generation of custom F#-specific assemblies for each Pulumi provider, which is possible but not very compelling.

@vshapenko

This comment has been minimized.

Copy link

@vshapenko vshapenko commented Dec 17, 2019

Take a look on similar problem in Fabulous. Here is a little sample:

         View.Grid(
           rowdefs = [Dimension.Star; Dimension.Auto],
           coldefs = [Dimension.Star],
           rowSpacing = 0.,
           columnSpacing = 0.,
           children = [
             yield view |> row 0
             match navBar with
             | Some bar -> yield bar |> row 1
             | None -> ()
           ]
         )

And in fabulous there is heavy use of optional parameters,yes. We do not need set it all

@Lanayx

This comment has been minimized.

Copy link

@Lanayx Lanayx commented Dec 17, 2019

CE makes it much more attractive, the braces can be easily converted to js-way. Still one thing that doesn't look ok is repeating spec ( elements, without it things should look better. Is it possible to shorten it to the template below?

  let deployment = deployment "nginx" {
      deploymentSpec {
        replicas 1
        selector (LabelSelectorArgs(MatchLabels = appLabels))
        template (
              metadata (ObjectMetaArgs(Labels = appLabels))
              podSpec {
                containers [
                  container {
                    name "nginx"
                    image "nginx"
                    ports [ContainerPortArgs(ContainerPortValue = 80)]
                  }
                ]
              }
        )
      }
  }
@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Dec 17, 2019

@Lanayx Agreed! template is a custom operation of deploymentSpec, while podTemplateSpec is another computation expression. I haven't found a way to merge them - do you know how?

@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Dec 17, 2019

@vshapenko In your example, there are no nested type instantiations. In Pulumi, every resource constructor (Something) accepts and argument bag (SomethingArgs), which may, in turn, contain sub-args (SomethingElseArgs). That's how cloud resources are modeled, but this leads to nesting problems.

I don't think Fabulous has anything akin to inputs-outputs either.

@Lanayx

This comment has been minimized.

Copy link

@Lanayx Lanayx commented Dec 17, 2019

@mikhailshilkov looks like it's possible, have you taken a look into http://www.fssnip.net/hf/title/DSL-for-constructing-HTML ?

@Szer

This comment has been minimized.

Copy link

@Szer Szer commented Dec 17, 2019

Also you could take a look at how Saturn deals with nested CE here

https://github.com/SaturnFramework/Saturn/blob/master/sample/Controller.Sample/Controller.Sample.fs

@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Dec 25, 2019

@Lanayx Thanks for the link, it's a good one.

Although I was able to construct a kubernetes deployment with the syntax from your first message (i.e. similar to HTML, no spec-spec redundant pairs), I'm not yet able to make it useful.

HTML is different in an important way: any container can be a parent of any other element. So, the code keeps track of the current path with a stack.

Pulumi components aren't homogenous: I want to enforce the exact layering structure, e.g. deployment spec -> template -> pod spec -> container. There should be exactly one element at the exact place in the hierarchy.

I can't find a way to enforce this yet. How should I link a pod spec to a template with a 1-on-1 linked relationship?

@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Dec 25, 2019

@Szer Do you mean this snippet?

controller {
    index (fun _ -> task {
        return {a = "hello"; b = "world"}
    })
}

It seems to be similar to my comment above and seems to have the same problems of rather complicated nesting (the use of index function, not a CE directly inside another CE).

@Lanayx

This comment has been minimized.

Copy link

@Lanayx Lanayx commented Dec 26, 2019

Yes, I can't the way to enforce restriction either. Found the closed proposal https://github.com/fsharp/fslang-suggestions/blob/d48c35ce216e2bff148937ec028ad61e5c273fdf/archive/suggestion-7199500-allow-nested-computation-expression-for-writing-b.md , probably current problem will be a good killer example for @dsyme :)

@Szer

This comment has been minimized.

Copy link

@Szer Szer commented Dec 27, 2019

@Szer Do you mean this snippet?

controller {
    index (fun _ -> task {
        return {a = "hello"; b = "world"}
    })
}

It seems to be similar to my comment above and seems to have the same problems of rather complicated nesting (the use of index function, not a CE directly inside another CE).

Here is an example of type checking with Nested CE
https://gist.github.com/Szer/be85a58ef239820dd80d75e1e119492f

@mikhailshilkov

This comment has been minimized.

Copy link
Collaborator Author

@mikhailshilkov mikhailshilkov commented Jan 8, 2020

Thanks @Szer! Based on your Yield-based examples, I managed to make the following snippet compile:

let d = deployment "mydep" {
    deploymentSpec {
        replicas 1
        selector {
            matchLabels appLabels
        }
        podTemplateSpec {
            metadata {
                labels appLabels
            }
            podSpec {
                container {
                    name "nginx"
                    image "nginx"
                    containerPort { 
                        containerPortValue 80
                    }
                }
            }
        }
    }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.