Task Array (flexible feature-wise task assignment for dynamic modular learning)

[Annie-LiGHT]

PROBLEM

Currently in DISCO, tasks comprise a single model with an expected input of a restricted size.

• We previously proposed way to modularize training into a series of feature-specific (or modality-specific) modules i.e. MoDN: Modular Decision Support Networks
• This approach allows training on imperfectly interoperable (IIO) datasets i.e. where one client has a different feature set than another, but with an overlapping subset in common.

Like in the image, if the x-axis is number_of_features)

• This scenario can be thought of as "horizontally vertical" distributed learning (i.e. different rows AND different columns)
• MoDN proposes 1encoder/feature and 1decoder/label, thus producing an array of models of size [feature set + label set]
• This can also apply to non-MoDN models (for example "Predict COVID" as a meta-task with subtasks of ["using demographic features" or "using lab and demographic features"....etc.]

---

PROPOSED FEATURE

• Support a meta-task that comprises an array of sub-task modules (i.e. an array of models)
• Connecting data sets with a various number of IIO features to meta-task
• Assign sub-task modules to users according to that users' available features

Example

• In MoDN if there is 1encoder/feature in a tabular dataset, we want to
  - only have one "meta task" or task array for the set of modules (as opposed to a long list of tasks for feature-specific encoder)
  - only send models relevant to the user's feature set.
  - allow input from tabular datasets of various sizes and assign sub-tasks to features.