You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
divviup/janus#1574 proposes adding a new task configuration parameter that is implementation-specific, and only applicable when a task's query type is fixed_size. This means we will be changing the aggregator API by adding a second field to the QueryType::FixedSize enum variant. The field will be of type Option<Duration>. (When absent, fixed size tasks will function as normal, and when present, fixed size tasks will group reports by time, with the given granularity, and assign them to batches arbitrarily within those groups) It is only of relevance to the leader as well, since the leader is responsible for assembling fixed size batches. If the leader and helper somehow disagreed about this parameter, the leader's view of it would win, and there wouldn't be any other ill effects.
This will affect divviup-api directly because of the API change (though I think I can make this backwards-compatible with #[serde(default)]) but it will also break an assumption of the database schema. Currently, the max_batch_size column stores both the task's query type (encoded through nullability) and the maximum batch size, if not null. After this change, there will be a second query type-specific parameter. We could add a new nullable integer column for the time bucketing duration, and let max_batch_size's nullability still indicate the task's query type, but it might be more robust to store the query type separately.
Looking further out, we will also need to be ready for other DAP implementations to not support this parameter, since it's implementation-specific. It's possible such an aggregator may support the aggregator API, but either return an error if the time bucketing parameter is present, or ignore the parameter entirely and successfully create the task.
The text was updated successfully, but these errors were encountered:
Regarding other aggregator implementations: I think this should effectively be a parameter that is only available to the user if Divvi Up is the Leader. We shouldn't expect other aggregators to support our implementation's features over-and-above what is required for DAP.
This seems like the sort of thing that could be addressed with an initial capabilities check on Aggregator creation, so aggregators could report a list of supported query types and vdafs (as well as confirming that the bearer token works and fetching the dap url and possibly even the allowed server role/s). As long as we're going to build out "some aggregators support different options" we might as well build it in a way that is extensible
divviup/janus#1574 proposes adding a new task configuration parameter that is implementation-specific, and only applicable when a task's query type is fixed_size. This means we will be changing the aggregator API by adding a second field to the
QueryType::FixedSize
enum variant. The field will be of typeOption<Duration>
. (When absent, fixed size tasks will function as normal, and when present, fixed size tasks will group reports by time, with the given granularity, and assign them to batches arbitrarily within those groups) It is only of relevance to the leader as well, since the leader is responsible for assembling fixed size batches. If the leader and helper somehow disagreed about this parameter, the leader's view of it would win, and there wouldn't be any other ill effects.This will affect divviup-api directly because of the API change (though I think I can make this backwards-compatible with
#[serde(default)]
) but it will also break an assumption of the database schema. Currently, themax_batch_size
column stores both the task's query type (encoded through nullability) and the maximum batch size, if not null. After this change, there will be a second query type-specific parameter. We could add a new nullable integer column for the time bucketing duration, and letmax_batch_size
's nullability still indicate the task's query type, but it might be more robust to store the query type separately.Looking further out, we will also need to be ready for other DAP implementations to not support this parameter, since it's implementation-specific. It's possible such an aggregator may support the aggregator API, but either return an error if the time bucketing parameter is present, or ignore the parameter entirely and successfully create the task.
The text was updated successfully, but these errors were encountered: