[QC-602] Batch processing in QC#779
Conversation
This allows to process partial data by QC Tasks and dump/merge results into a file. Then, the merged results can be picked up by the last parts of the QC chain (Checks, Aggregators, storage).
|
@Barthelemy This is essentially working and ready for a review, I only have to write tests now. |
|
I'll review it today. |
|
I am not sure it is related to tuning the test parameters #777 |
|
Weird. I did not touch I will also increase the timeout for |
Barthelemy
left a comment
There was a problem hiding this comment.
Good, the naming is ok, I don't have a better proposal.
The infrastructure generator is what it is ... :)
I tried to run the test but I get:
Error, monitor object of the QC Task could not be found.
|
The test could fail this way if you provide it with wrong UNIQUE_ID env var. It passes CI now, so I doubt this is a real problem. |
This allows to process partial data by QC Tasks and dump/merge results into a file.
Then, the merged results can be picked up by the last parts of the QC chain (Checks, Aggregators, storage).