Address the possibility of differing task id variables in different groups

[elray1]

Suppose I try to define a task id config file that looks like this:

{
  "rounds": [
    {
      "round_id": "2021-03-07",
      "model_tasks": [
        {
          "task_ids": {
            "origin_date": {
              "required": ["2022-03-07"],
              "optional": null
            },
            "scenario_id": {
              "required": ["A-2021-03-05", "B-2021-03-05"],
              "optional": null
            },
            "location": {
              "required": null,
              "optional": {"$ref": "#/$defs/task_ids/location/us_all"}
            }
          },
          "output_types": { ... }
        },
        {
          "task_ids": {
            "origin_date": {
              "required": ["2022-03-07"],
              "optional": null
            },
             "scenario_id": {
              "required": ["C-2021-03-05", "D-2021-03-05"],
              "optional": null
            },
            "location": {
              "required": null,
              "optional": {"$ref": "#/$defs/task_ids/location/us_all"}
            }
            "age_group": {
              "required": null,
              "optional": ["0-5", "6-17", "18-65", "65-130"]
            }
          },
          "output_types": { ... }
        }
      ],
      "submissions_due": {
        "start": "2021-03-09",
        "end": "2021-04-09"
      }
    }
  ]
}

The set up is that there are two groups of tasks, one related to scenarios A and B and a second related to scenarios C and D. For scenarios C and D, age_group is relevant, but it is not relevant for scenarios A and B.

My questions are:

1. Is there any reason to allow this kind of behavior? Or should we enforce that within each round, the same task id variables occur in all task groups?
2. If we do allow it, what are we expecting to be in submission files? This should be documented.

[elray1]

If round_id_from_variable = true then we certainly need to require that the round_id variable appears in all task groups.

[nickreich]

My preference for now would not be to engineer specifically for the scenario where tasks are different by scenario.

I think we could essentially allow teams to work around this by having, in your example age_group be a set of non-required categories, of which some are relevant to scenarios A and B and one ("all ages" or something like that) is relevant to C and D.

[elray1]

I think heading in a similar direction as @nickreich, but with a different proposal -- what if we engineer so as to explicitly not to allow for a different set of task id variables in different task id groups within the same round? i.e., in an example like this, the hub would be required to add an age_group id variable to the task id group defining settings for scenarios A and B?

[nickreich]

[annakrystalli]

Open issue in hubUtils to check programmatically that all unique task_ids are present in all model_task sets within a round

[annakrystalli]

Quick question @nickreich & @elray1 , have we agreed that task_id groups in a given round must share the same task_ids or can we assume that if one is missing it should be:

1. "all" or
2. NA

[nickreich]

I'm ok with @elray1 's suggestion to enforce that all task_id variables in different task_id groups within the round be identical. Therefore, there is no "inferring" of a missing values of task_ids that are specified in one group but not another.

[elray1]

@annakrystalli can this issue be closed, or should we open a related issue in hubUtils?

[annakrystalli]

I think it's been addressed but I'll ship to hubUtils to investigate and confirm. 👍

[nickreich]

I think maybe this can be closed? but am going to ask @annakrystalli to confirm at some point.