Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Make and save predictions and do eval chip-by-chip #635
Currently, the predict command makes predictions for each chip in a scene, holding all the predictions in memory and then saves them to disk. This uses too much RAM for very large scenes. The same happens for the eval command. This PR makes it so that each prediction (for a chip) is written to disk immediately, and so that the eval is done one window at a time, and not over the whole scene.
The main change supporting this is making the
The vectorification process involves loading the whole prediction array into RAM, which means that won't work on very large scenes. If that is a priority, we can fix it in a separate PR.
4 times, most recently
Dec 24, 2018
@@ Coverage Diff @@ ## develop #635 +/- ## ========================================== Coverage ? 72.01% ========================================== Files ? 171 Lines ? 7878 Branches ? 0 ========================================== Hits ? 5673 Misses ? 2205 Partials ? 0
@@ Coverage Diff @@ ## develop #635 +/- ## =========================================== - Coverage 70.95% 70.76% -0.19% =========================================== Files 171 171 Lines 8000 8052 +52 =========================================== + Hits 5676 5698 +22 - Misses 2324 2354 +30
jamesmcclain left a comment •
Given the review instructions, there seems to be little for me to do other than run tests and inspect the code. I looked most carefully at 10d447a and 099c4d7 because they seem to be the most important parts of this multi-parter?
Meta-comment: multi-step changes should probably be in sequences of minimal PRs and unrelated changes (unless they are small) should probably be in separate PRs. I am sure that I am guilty of this, as well.
I agree. But in this case, I thought that the main commits (the first four) represented a cohesive set of changes that should be reviewed together, and that it would be more unwieldy to have a separate PR for each of them.