Conversation
commit 8f869a7edfcce5026f442ba0cb4b3ad5483848c9
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 16:24:33 2025 -0700
ignore obeys filesystem case sensitiviy
commit 749a4a9e59505c2e71970bc00e531548d614a946
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 16:24:06 2025 -0700
fix adding timestamps from inside archives
commit f40ac06f23061d77945f1aede85719e496b5c4a4
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 15:55:28 2025 -0700
move WalkSkipper back into walk dir
commit bfcf32ae57c669f236e31c2b08c542b2059f1447
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 15:54:11 2025 -0700
move walk out of __init__.py
commit b2dcea5f22e4d6fd2442b8fabd04351ea79563c1
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 15:51:03 2025 -0700
simplify using timestamps inwalk and archive. remove cruft
commit e4afb1692f34d6d30b74cd5f2aea56bf4e2a340e
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 15:37:26 2025 -0700
organize news
commit aea9ba29dbd14f0ea37211ed75585ed1b0fbfd4b
Author: AJ Slater <aj@slater.net>
Date: Thu Apr 24 15:35:20 2025 -0700
treestamps can skip archive without opening them
…ave memory repacking animated
archive gets an _optimize_in_place_on_disk flag container copy_skipped_files turns into a hydrate_optimized paths zip deletes files it would replace on disk and closes the file before appending the new files.
* add the scheduler first draft * Replace walk/dispatch with futures-based scheduler (#101) Swap multiprocessing.pool.Pool for concurrent.futures.ProcessPoolExecutor and route all work through a new Scheduler that manages a ContainerNode tree, three job types (Unpack/OptimizeLeaf/Repack), backpressure, and per-node failure rollback. Key changes: - walk.py: delete _finish_results, _handle_container, _walk_container; add _enqueue_children callback; walk() creates Scheduler and drains it - init.py: Pool → ProcessPoolExecutor - container.py: remove set_task/optimize_contents/_tasks; rename _hydrate_optimized_path_info → hydrate_optimized_path_info (public API for scheduler) - archive.py: always copy unchanged files during walk (archive must be open); adapt to new hydrate API; drop optimize_contents override - pdf.py: rename hydrate method to match new public API - scheduler.py: fix ContainerNode hashability (eq=False); use attribute access for config; respect handler's own _do_repack flag for animated image format conversion; delegate hydration to handler subclass - config: add fail_fast and fail_fast_container options - Bump treestamps ≥2.5.2 for pickleable GrovestampsConfig Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Add --fail-fast and --fail-fast-container CLI options (#102) Expose the two new scheduler failure modes on the command line: - --fail-fast: stop all optimization on the first error - --fail-fast-container: fail the entire top-level container when an inner repack fails Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * code cleanups" * change method ordering for clairity. minor comment edits * speling * move old_timestamps into walk as legacy_timestamps * refactor place format in plugins/base. make hanlder factory it's own class. consolidate report class and walk class. * consolidate _handler-factory fucntion * update deps * code cleanup pass * warn on digitial signature and encrypted pdf, do not error * update deps * use new treestamps methods * webp tools superclass * remove unused variable from pdf plugin * update deps * fix treestamps use and directory finishing, compacting * v6.1.0 bump news --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* bump version and news * rightyper typechecked * Fix rightyper-introduced type annotation errors Fixes 68 basedpyright errors and 51 ty errors introduced by the automated rightyper pass. Changes are annotation-only. - PathInfo.__init__ path_info parameter widened from None to PathInfo | None - Removed spurious self: "picopt.xxx.Y" annotations naming wrong modules - Simplified **kwargs bogus unions to Any for variadic super().__init__ passthroughs - config/handlers.py handler_stages/convert_chain/native types widened to proper base types - scheduler.py _trigger_fail_fast/_cancel_subtree widened Exception to BaseException | None - detect_format.py filtered image.info keys to str; suppressed dynamic mpinfo attr - pil_convertible _file_format image_class widened from BmpImageFile to Image - tar.py removed type[TarGzDetector] cls annotation violating LSP - tests: PosixPath -> Path; ts:None/config:None -> float|None/dict|None - Cleaned unused TYPE_CHECKING imports left behind after removing bad annotations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * correct typing changes * revert to python 3.10 and remove righttyper * update devenv * remove quoted self types * remove self types * format & fix --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix pickling of ContainerHandler when a timestamps file exists When a prior picopt run leaves a .picopt_treestamps.yaml, loading it via ruamel.yaml leaves the YAML instance holding a Reader with an open BufferedReader. Submitting an UnpackJob (or RepackJob) to the ProcessPoolExecutor then fails with "cannot pickle 'BufferedReader' instances" because the handler carries that Grovestamps through its _timestamps / _skipper attributes. Add __getstate__ on ContainerHandler that nulls both fields before pickling. The scheduler owns the real Grovestamps on the main thread; workers never actually used it (in-archive timestamp consumption happened in a pickled copy whose mutations never propagated back). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * remove clean_for_repack, superseded by __getstate__ The method nulled the same two fields __getstate__ now nulls at pickle time, and its one caller always ran on a handler that had already round-tripped through a worker — so the fields were already None. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.