Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parallel processing for python3 #52

Merged
merged 5 commits into from
Nov 23, 2023

Conversation

IzaakWN
Copy link
Collaborator

@IzaakWN IzaakWN commented Nov 23, 2023

Fix parallel processing, which causes segmentation faults in python3.

This is caused due to a conflict between how python3 and ROOT handle objects in the memory. The segmentation fault happens when manually deleting a ROOT histogram that was piped from a different thread created by python.

The reason why I previously implementation the manual deletion of histograms, it because in the past I was getting a segmentation fault caused when too many histograms are created and left lingering in the memory. The main way to combat this was to clean the memory by manually deleting unneeded histograms, and closing/reopening the (input) ROOT files to which the histograms were assigned. I suspect it is not as much of a problem anymore in new python and ROOT versions, but we should keep this in mind if segmentation faults happen again. Hopefully switching the plotting routine to RDataFrame will automatically solve this memory issue (see Issue #51), because of different way it handles files, and because we do pipe them anymore via python.

@IzaakWN
Copy link
Collaborator Author

IzaakWN commented Nov 23, 2023

@oponcet & I validated the output. It solves the segmentation faults. However, the output histograms that are produced with parallel processing do not seem to display nicely by Visual Studios from the ROOT file for some reason. It looks fine in the TBrowser.

Will merge now.

@IzaakWN IzaakWN merged commit 84098d1 into cms-tau-pog:master Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant