New neural-network (Neuquant inspired) color quantizer#371
Merged
Conversation
Wow, performance is *terrible*, even worse than I anticipated. The original algorithm is actually quite clever in avoiding floating-point math, but obviously it does this by using a (horrifying) amount of bitshifts. There's no good way to make these work in VB, at least for values determined at run-time, so I'm probably going to look at converting the algorithm to a simpler floating-point algorithm before proceeding any further.
- rewritten using floating-point math (instead of constant bitshifts which are troublesome to emulate in VB) - many more VB6-specific optimizations - no longer any need to unbias after constructing palette - proper Euclidean distance is now used for color-matching (instead of previous Manhattan distance), which brings performance *and* quality benefits - alpha channel now works correctly and is palettized too!
- progress bar now works while generating palette - algorithm now works reliably down to 2 colors (and it also works > 256 colors, but I don't plan to expose this in PD) - ongoing code clean-up and minor optimizations
I still need to make some UI changes in the Stylize > Palettize window to reflect these new features, but the core algorithm is now working extremely well. I've refactored a ton of code to get greatly improved VB6 performance, the code is now fully commented and restructured to actually make sense (and eliminate a bunch of magic numbers and unnecessary gyrations) and a few more bugs have been rectified to further improve quality. I'm really proud of how this has ultimately turned out, with better performance and quality than I expected going in.
Time for more aggressive micro-tuning across multiple PCs
30+ percent speed boost just from reworking variable declarations and array structure (UDT member access can be obnoxiously slow in VB, especially combined with arrays). I believe this is as fast as I can get the code for now. My next step is to reduce large image sizes prior to processing, which typically improves quantize quality (by reducing noisy inputs) while also providing large perf improvements.
- Effects > Stylize > Palettize UI has been overhauled to expose this new feature directly - All old FreeImage interop (to expose their quantizer(s)) has been deleted - Prior to quantizing with the new neural network engine, PD will downsample huge images to a reasonable size. This provides a large performance boost and also improves quality (by minimizing noisy inputs) - Certain 8-bit export paths have been migrated to this new quantizer. If dithering is used, any quantizer works well enough (because dithering hides mediocre palette selection) but if *no* dithering is used, this new neural network technique can produce a significantly better palette - and better compression as a result, since you won't need strong dithering to hide your meh palette. - Because some format exporters could benefit from this new feature, some export code has been mildly touched by this commit
Meant to do this in the previous commit; d'oh!
Static initialization costs have a disproportionately high benefit on old PCs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.