v0.96d - Bugfixes and pararell compression:
- Fixed keygeneration missmatch in nxdumptool tickets with elicense flag
- Fixed xci compressor for xci with tickets and titlekey starting by 0
- Fixed bug where ticket gets appended more than once in a multi-file with tickets
- Added json integrity check when updating nutdb. The update will take a little longer but the current json will be protected if the one in the repository is corrupted.
- Fixed issue where pykashi adds spaces when processing accented characters.
- Improved selector to list function.
- Added pararell compression as alternative to threads. Pararell compression compresses several files at the same time with a smaller cpu footprint.
In tests i managed to compress 20 files at the same time while keeping a 2-3Mb\s speed ratio per file with level 22 compression. This will depend on your computer or server.
Note: Current implementation has this issues:
1.- It compresses in batches so when one instance finishes it's place isn't taken by another file.
2.- It uses subprocesses. tqdm prints become a little wonky with subprocesses and ghost bars can appear out of position, specially with a high number of processes. To handle this the screen gets cleaned in 3second intervals and bar positions get preallocated. This reduces the wonkyness.
3- The wonkyness is just visual, it doesn't affect the correct processing of the files
Possible improvements to study:
I.- Separate filelist equally between instances so when one instance finishes processing a file another takes it's place.
II.- Let one thread update the progress bars exchanging the progress information between threads
For performance a function that arranges files by size has been added. That ensures files with similar sizes will be group on the same batch when possible