You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The transformer is two containers as python 3 and the ATLAS runtime do not get along well. The first container writes the C++ code, and the second compiles and runs it. To communicate between the two everything is written to a Kubernetes PVC.
Why change
PVC's are generally a pain in the but and add another point of configuration. Don't use them unless there is a real reason. Given it takes ~100's of ms to write these files, caching is really not needed.
Change to
Two options were considered:
Write a zip of the files to a database
Write a zip of the files into the message that is sent to the runner container.
The latter seems to be the most simple solution. The files are quite small. Especially for something like ServiceX, where the queries are not building 100's of histograms. I expect the C++ zip to be well under 100K and probably more like 10K or so.
The text was updated successfully, but these errors were encountered:
This has been changed in the referenced repos now. While the code isn't perfect, it does work. It would be best to do a code review of this rabbit mq code at some point.
Currently
The transformer is two containers as python 3 and the ATLAS runtime do not get along well. The first container writes the C++ code, and the second compiles and runs it. To communicate between the two everything is written to a Kubernetes PVC.
Why change
PVC's are generally a pain in the but and add another point of configuration. Don't use them unless there is a real reason. Given it takes ~100's of ms to write these files, caching is really not needed.
Change to
Two options were considered:
The latter seems to be the most simple solution. The files are quite small. Especially for something like ServiceX, where the queries are not building 100's of histograms. I expect the C++ zip to be well under 100K and probably more like 10K or so.
The text was updated successfully, but these errors were encountered: