Task template

[achimnol]

Let's make the parameter sets for batch sessions (#42) to be reusable and referrable by task graph API (#43).

A template should include:

• The kernel image
• Resource specification
• The batch command to execute
• The vfolder mounts
• Bootstrap script (encoded as base64) → Support for user-defined boostrap script #77
• Timeout (maybe infinite)
• Git repository auto-clone parameters (repo URL, credentials, target commit/branch)

A template can be used in the following contexts:

• Just run a single session using the template
• Run a task graph composed of multiple, staged tasks from multiple templates
  • In this case, each session will have /home/work/input and /home/work/output directories which are auto-created vfolders by the smanager for the input/output connection between different stages.
  • It may also have the task sequence number as a special environment variable so that each task may determine the partition of input data to process or the parameters to use with its computation, for parallelized tasks in the same stage.

┆Issue is synchronized with this Asana task by Unito

[kyujin-cho]

Initial design of the template:

---
api_version: v1
kind: taskTemplate
metadata:
  name: template1234
  tag: example-tag
spec:
  kernel:
    environ:
      MYCONFIG: XXX
    git:
      branch: '19.09'
      commit: 10daee9e328876d75e6d0fa4998d4456711730db
      repository: https://github.com/lablup/backend.ai-agent
      destinationDir: /home/work/baiagent
    image: python:3.6-ubuntu18.04
    run:
      bootstrap: IyEvYmluL3NoCmNkICRIT01FCm1rZGlyIHRtcDEKZWNobyAiMTIzNDUiID4gdG1wMS90ZXN0LnR4dAo=
  resources:
    cpu: '2'
    mem: 4g
  mounts:
    hostpath-test: /home/work/hostDesktop
    test-vfolder: 
  sessionType: interactive

Value of mount object's item means where to mount vFolder; this value is (of course) optional.
All other items are as-is.

[achimnol]

I first thought to store the task template as a database record with concrete columns, but it would be better to store just the YAML as you suggested for forward compatibility.
Let's just add a task_templates table to the manager which contains the ownership information (user, group, domain) along with a minimal set of metadata (id, created_at, is_active) and a text column content which stores the YAML template. Please implement all CRUD operations as in the agents as REST APIs.

[achimnol]

We're finally done, but later I will revisit this to test more thoroughly.