When processing many files, there's currently no way to see progress or an ETA from the Pipeline class itself - only pyopia process's own CLI progress bar shows this, which isn't available to anyone using Pipeline programmatically (e.g. from a notebook or a custom script).
Proposal: an opt-in Pipeline.enable_progress_tracking(total_files, log_interval=5) method that logs progress (percent complete, elapsed time, ETA) every log_interval files as Pipeline.run() is called in a loop. No-op / zero overhead unless explicitly enabled, and safe to use with pyopia process --num-chunks since it just uses the standard logger (each chunk tracks its own progress independently).
When processing many files, there's currently no way to see progress or an ETA from the
Pipelineclass itself - onlypyopia process's own CLI progress bar shows this, which isn't available to anyone usingPipelineprogrammatically (e.g. from a notebook or a custom script).Proposal: an opt-in
Pipeline.enable_progress_tracking(total_files, log_interval=5)method that logs progress (percent complete, elapsed time, ETA) everylog_intervalfiles asPipeline.run()is called in a loop. No-op / zero overhead unless explicitly enabled, and safe to use withpyopia process --num-chunkssince it just uses the standard logger (each chunk tracks its own progress independently).