Implement Devfile volume support

[rajivnathan]

Devfile components allow specifying volumes. eg. See https://github.com/eclipse/che-devfile-registry/blob/master/devfiles/java-web-spring/devfile.yaml

• Implement functions to dynamically add volumes and mounts to fat pod's spec so that volumes can be mounted to containers that correspond to components with volumes.
• Create PVC resource if applicable (emptyDir or PVC)

See comment #2472 (comment) for details

[kadel]

/area devfile
/area storage

[maysunfaisal]

I can take a look at this,

/assign @maysunfaisal

[maysunfaisal]

I have a branch with changes for the Devfile volumes based on the poc branch https://github.com/rajivnathan/odo-fork/compare/kdo-poc-devfile...maysunfaisal:2472-pvc-1?expand=1. This assumes Devfile components can share volumes(see issue #2510) and assumes a default volume size of 1Gi(see issue #2511)

I can open PRs in the odo repo once the PRs for the parser and kclient functions are in.

[maysunfaisal]

Description

With the feature story for odo to consume devfiles(see https://github.com/eclipse/che-devfile-registry/tree/master/devfiles and epic #2470), we want odo push to create a single pod for all the devfile component containers and create the volumes associated with the components and add them to the devfile pod.

This issue deals with the volume support for the devfile components. On odo push, after the devfile has been parsed and the devfile pod deployment created containing all the devfile component containers, we should create all the PVCs as listed in the devfile. The PVCs are then added to the pod Volume and the volumes are mounted in the Pod Container's Mount Path.

For example consider the devfile component snippet:

components:
  -
    type: chePlugin
    id: redhat/java/latest
    memoryLimit: 1512Mi
  -
    type: dockerimage
    image: websphere-liberty:19.0.0.3-webProfile7
    alias: runtime
    env:
      - name: MAVEN_CONFIG  
        value: ""
    memoryLimit: 768Mi
    endpoints:
      - name: '8080/tcp'
        port: 8080
    mountSources: true
    volumes:
      - name: m3
        containerPath: /home/user/.m3runtime
      - name: m2
        containerPath: /home/user/.m2runtime
      - name: m2
        containerPath: /home/user/.m2runtime222
  -
    type: dockerimage
    alias: tools
    image: quay.io/eclipse/che-java8-maven:nightly
    env:
      - name: MAVEN_CONFIG  
        value: ""
      - name: JAVA_OPTS
        value: "-XX:MaxRAMPercentage=50.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
          -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
          -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
          -Duser.home=/home/user"
      - name: MAVEN_OPTS
        value: $(JAVA_OPTS)
    memoryLimit: 768Mi
    endpoints:
      - name: '8080/tcp'
        port: 8080
    mountSources: true
    volumes:
      - name: m2
        containerPath: /home/user/.m2tools
      - name: m3
        containerPath: /home/user/.m3tools
      - name: m4
        containerPath: /home/user/.m4tools

The devfile has a total of three unique volumes - m2, m3 and m4 and according to the snippet above, odo shall create three RWX RWO PVCs - m2, m3, m4.

The two containers runtime & tools above share the same volume m2 and m3. Thus, container runtime will mount the PVC m2 at /home/user/.m2runtime & /home/user/.m2runtime222 and mount the PVC m3 at /home/user/.m3runtime. Whereas, container tools will mount the PVC m2 at /home/user/.m2tools and so on.

If you would like to discuss about volume sharing across dockerimage components from devfile point of view, please have a look at the issue #2510 for detailed discussion.

Acceptance Criteria

The logic discussed above has been considered and broken down to the following major pieces in the checklist. A working POC for the devfile volume implementation, can be found at https://github.com/rajivnathan/odo-fork/compare/kdo-poc-devfile...maysunfaisal:2472-pvc-1?expand=1.

The PR numbers will be added against the checklist as the code logic make their way from the devfile volume POC branch to the odo repo. For checklist item(s), that do not have a PR number, please have a look at the POC branch for the implementation.

• Function to add PVC Volume to Pod Template Spec Add kclient functions for creating volumes #2546
• Function to add Volume Mounts to Pod Template Spec containers Add kclient functions for creating volumes #2546
• Function to create PVC resource Add kclient functions for creating volumes #2546
• Function to create PVC from Devfile Add Volume Storage Adapter for Devfiles #2651
• Kclient function logic to add PVC and Volume Mount to Pod Template Spec Add Volume Storage Adapter for Devfiles #2651

Functionality we should update/improve

Working on this issue, it is evident that the current implementation of devfile do not support volume size, which the config.yaml previously provided for odo. A separate issue has been opened in this regard for discussion - #2511

[kadel]

I have a feeling that we are skipping a step here.
You started working on volume support but we are missing the part where we create Pod (Deployment) with all the containers and commands as they are defined in Devfile.

Not sure if it makes sense to start working on volumes if we don't have containers that would use them.

[maysunfaisal]

@kadel yes, you are right. As @rajivnathan is waiting on the devfile parser, he had worked temporarily on the poc branch where he took the poc parser and implemented a first pass of devfile component pod creation. I took Rajiv's poc devfile branch and worked on the first pass of devfile volumes which is what I meant to indicate by my POC branch above in the comments - the branch https://github.com/rajivnathan/odo-fork/compare/kdo-poc-devfile...maysunfaisal:2472-pvc-1?expand=1 has the devfile volume first pass implementation which is based on Rajiv's poc devfile component pod implementation

The PR #2546 also just has the kclient implementations for the volume. There is currently no other volume logic in there. I am waiting for the Devfile component PR in the odo master branch before proceeding.