Volume resource types?

[mitchdenny]

Initially this PR is to trigger some conversation on how we want to handle the concept of volumes in Aspire and how that might relate to end-to-end deployment scenarios. Right now our volume support is limited to just inner loop support, but there are times where a developer may wish to make use of a containerized solution in real production environments where data persistence is required.

In platforms such as Kubernetes the concept of volumes would address this need which can then be mounted to containerized workloads. Cloud providers with various container workload solutions address volumes in different ways but generically there is the concept of a volume which can be mounted to a workload.

My proposal is that we introduce volumes as a first class concept within Aspire:

var builder = DistributedApplication.CreateBuilder(args);
var data = builder.AddVolume("data");
var mycontainer = builder.AddContainer("mycontainer", "myimage")
                                          .WithVolumeMount(data, "/path/in/volume", "/path/in/container");

The manifest would contain something like this:

{
  "resources": {
    "data": {
      "type": "volume.v0"
    },
    "mycontainer": {
      "type": "container.v0",
      "image": "myimage",
      "volumeMounts": [
        {
          "volume": "data",
          "volumePath": "/path/in/volume",
          "containerPath": "/path/in/container"
        }
    }
  }
}

Aspirate would create a persistent volume within the same namespace that the app is deployed so everything is self-contained. How something like azd would work when targeting a specific cloud provider gets a little more interesting. For persistent volumes you would probably want to use Azure Files. But for that to work we need to decide how to associate the volume with an Azure Storage account.

/cc @ellismg @davidfowl @DamianEdwards @prom3theu5

Microsoft Reviewers: Open in CodeFlow

[prom3theu5]

i think this is a great start.

I'm guessing this first pass is for persistence locations, so that we have the ability for running containers to persist past container lifetime?

Where do we go from here for populating volumes with data, that we want available at container start.

Thinking for the current sample scenarios:

• Db Init scripts
• Metrics
• Volume Mounts
  etc

Proposing to expand that volume resource with a data entry

This would allow bundling volumes in tooling as containers, which - for me at least in kube, would allow me to use as init-containers to populate kube volumes without having to worry about what CSI storage class is being used.

 "data": {
      "type": "volume.v0"
      "data": {
        "localPath": "/some/local/path",
        "remotePath": "/some/path/inside/the/volume"
       }
    },

Like so...

{
  "resources": {
    "configFiles": {
      "type": "volume.v0"
      "data": {
        "localPath": "/some/local/path",
        "remotePath": "/some/path/inside/the/volume/and/container"
       }
    },
    "someOtherData": {
      "type": "volume.v0"
       "data": {
        "localPath": "/some/local/path",
        "remotePath": "/some/path/inside/the/volume/and/container"
       }
    },
    "mycontainer": {
      "type": "container.v0",
      "image": "myimage",
      "volumeMounts": [
        "configFiles",
        "someOtherFiles"
        ]
    }
  }
}

With the above, when we build a container to use as an init container, the data is copied into the remotePath of the container image.
Essentially giving us:

{
"volume": "configFiles",
"volumePath": "configFiles.data.remotePath",
"containerPath": "configFiles.data.remotePath"
}

Because the volumePath and containerPath will always match here, we can drop the requirements on being explicit about that in the container volumeMounts section, and it can be inferred directly from the child resource, and just include an array of the child resource names?

Alternatively to this, we keep the containerPath, and infer only the volumePath from the volume resource remotePath entry?

[prom3theu5]

Thinking about it a little more we'd probably want to specify limits on the size of the volume as well

In both persistence and data supplication scenarios

[ellismg]

But for that to work we need to decide how to associate the volume with an Azure Storage account.

We could have an "implicit/default" one and then perhaps there could be some azure specific way to override? We do implicit resources today, like the ACR that container images get pushed to, a "default" storage account doesn't seem terrible?

[mitchdenny]

But for that to work we need to decide how to associate the volume with an Azure Storage account.

We could have an "implicit/default" one and then perhaps there could be some azure specific way to override? We do implicit resources today, like the ACR that container images get pushed to, a "default" storage account doesn't seem terrible?

A thought, that I'm not committed to but thought I would share is what if there was an optional parent on a volume? And we would then extend the Azure Storage APIs for Aspire to support creating a volume which is parented to it. It would work something like this:

var builder = DistributedApplication.CreateBuilder(args);
var files = builder.AddAzureStorage("mystorage").AddFiles("myfiles"); // AddFiles would be new.
var volume = files.AddVolume("myvolume", "/path/in/files");

[mitchdenny]

Other could providers could do similar things for theirs. A way of creating an associated volume but exerting control over where the data is hosted.

[ellismg]

A thought, that I'm not committed to but thought I would share is what if there was an optional parent on a volume?

Would you expect this to be present in the manifest (e.g. the same way that database resources are parented to database servers)? That seems appealing, but I wonder if instead you would want a way to specify this in a way that would allow multiple parents, depending on the cloud you are deploying to. For example, if you wanted to write an Aspire program that used persistence in this way and have it be deployable to both Azure/ACA and K8s, would you expect to have to change the app host depending on what cloud you were targeting (i.e. so we produce the files value in a different way?). Or would we imagine that in that case you'd call some "abstract" AddStorage thing that would give you a generalized File Like store, and on Azure this means Azure Storage, on AWS it means EFS (or something?) and on k8s it means something else?

[rozbo]

I don't understand why the manifest not include the volumes ? this is a very useful feature since we can genreate docker-compose.yaml from manifest...etc

[stbau04]

I fear that the phrase "volume resource" could be confusing for this approach. If i stumble over a volume resource in combination with docker (as it is used in the inner loop) i think of docker volumes instantly. Therefore i would expect an concept that is somewhat similar to docker volumes.

var builder = DistributedApplication.CreateBuilder(args);
var data = builder.AddVolume("data");
var mycontainer = builder.AddContainer("mycontainer", "myimage")
                                          .WithVolumeMount(data, "/path/in/volume", "/path/in/container");

This would mean that we create a volume resource where later a path in the resource is mounted as a volume to the docker container? As a source path is specifed, i would expect that i can use the same volume resource for multiple different volume mounts?

I think either we should either align with docker and have no source path (only one base directory that is mounted somewhere on the container) or we should call the volume resources something else (e..g. storage resource). Both would result in the phrase "volume" only being used for the actual mounts on containers and therefore no ambiguities. As no source path would in my opinion probably require another intermediate resource i would suggest that we call volume resources storage resources or something like that instead

[stbau04]

Would there still be bind mounts possible within volume resources?

Relevant for #1437 and #837