refactor: replace os.path with pathlib across libraries and workspaces

[Jammy2211]

Overview

Migrate all source code and workspaces from os.path.* (notably os.path.join)
to pathlib.Path. The codebase already uses pathlib extensively in newer code,
but ~99 legacy os.path.* references remain across 10 repos. This is a
mechanical refactor for consistency — no behaviour changes.

Plan

• Replace os.path.join(a, b) with Path(a) / b
• Replace os.path.exists / isdir / isfile with Path equivalents
• Replace os.path.dirname(os.path.realpath(__file__)) with Path(__file__).parent.resolve()
• Replace os.path.basename / split / splitext with .name / .parent / .suffix
• Wrap with str(...) only where a downstream API requires a string
• Drop now-unused import os lines
• Verify per-repo with unit tests; verify workspaces with /smoke_test

Detailed implementation plan

Affected Repositories

• PyAutoFit (primary)
• PyAutoConf
• PyAutoArray
• PyAutoGalaxy
• PyAutoLens
• autogalaxy_workspace
• autolens_workspace
• autolens_workspace_developer
• autofit_workspace_developer
• euclid_strong_lens_modeling_pipeline

Work Classification

Both (libraries first, workspaces follow)

Branch Survey

Repository	Current Branch	Dirty?
PyAutoConf	main	clean
PyAutoFit	main	clean
PyAutoArray	main	clean
PyAutoGalaxy	main	clean
PyAutoLens	main	clean
autogalaxy_workspace	main	88 changed (unrelated dataset regen by smoke-test-optimization)
autolens_workspace	main	148 changed (unrelated dataset regen by smoke-test-optimization)
autolens_workspace_developer	main	TBD
autofit_workspace_developer	main	TBD
euclid_strong_lens_modeling_pipeline	main	TBD

Suggested branch: feature/use-pathlib
Worktree root: ~/Code/PyAutoLabs-wt/use-pathlib/ (created later by /start_library)

Implementation Steps

1. Refactor each library in dependency order: PyAutoConf → PyAutoFit → PyAutoArray → PyAutoGalaxy → PyAutoLens. Run unit tests after each.
2. Refactor workspaces: autogalaxy_workspace, autolens_workspace, autolens_workspace_developer, autofit_workspace_developer, euclid_strong_lens_modeling_pipeline. Run /smoke_test to validate.
3. Resolve workspace dirty state in autogalaxy_workspace and autolens_workspace before cutting workspace branches (coordinate with smoke-test-optimization task).
4. Ship via /ship_library then /ship_workspace.

Key Files

• PyAutoConf/autoconf/directory_config.py — os.path.isdir checks
• PyAutoConf/autoconf/setup_colab.py — os.path.join for workspace paths
• PyAutoConf/autoconf/json_prior/generate.py — os.path.exists
• PyAutoConf/test_autoconf/test_fitsable.py — many os.path.join in test data
• PyAutoFit/autofit/non_linear/paths/directory.py — os.path.exists/isfile
• PyAutoFit/autofit/non_linear/paths/abstract.py — os.path.exists
• PyAutoFit/autofit/non_linear/search/**/search.py — checkpoint file checks
• PyAutoFit/autofit/mapper/model_mapper.py — os.path.dirname(os.path.realpath(__file__))
• PyAutoFit/autofit/tools/util.py — os.path.join, os.path.split, os.path.exists
• PyAutoFit/autofit/database/__init__.py — os.path.exists
• PyAutoArray/autoarray/util/dataset_util.py, autoarray/plot/utils.py, 4 test files
• PyAutoGalaxy/autogalaxy/plot/plot_utils.py, autogalaxy/util/plot_utils.py, 1 test file
• PyAutoLens/... — 19 references (largest by count)
• autolens_workspace/scripts/imaging/features/advanced/subhalo/sensitivity/slam_source_*.py
• autolens_workspace/scripts/interferometer/casa_reduction.py
• All docs/conf.py (sphinx boilerplate sys.path.insert(0, os.path.abspath(".")) — converted too, per "no more os.path")

Risks / edge cases

• External libraries that may need str (some FITS readers in older versions) — wrap with str() where needed.
• os.path.realpath resolves symlinks; Path.resolve() is equivalent in Python 3.6+.
• Test fixtures: pytest's tmp_path is already a Path, aligns naturally.
• The 88 + 148 dirty files in autogalaxy_workspace / autolens_workspace are dataset binaries from in-progress smoke-test-optimization — handle separately at workspace-phase time.

Original Prompt

Click to expand starting prompt

Most examples now use Pathlib, but there are still some os.path.join uses from
legacy.

Can you update all source code and workspaces to not use path.join at all
and always use Pathlib?

This should aspan all source code, workspaces, etc, no more os.path!

[Jammy2211]

Scope correction

Initial scan only counted os.path.* (99 refs). Re-scan including bare path.* from from os import path finds ~776 references across the same 10 repos:

Repo	os.path.*	bare path.*	Total
PyAutoConf	14	21	35
PyAutoFit	22	50	72
PyAutoArray	16	205	221
PyAutoGalaxy	6	139	145
PyAutoLens	19	133	152
autogalaxy_workspace	4	14	18
autolens_workspace	13	27	40
autolens_workspace_developer	3	53	56
autofit_workspace_developer	1	24	25
euclid_strong_lens_modeling_pipeline	1	11	12

Per the prompt's literal "no more os.path!", we're going for the full sweep — both forms.

[Jammy2211]

Smoke Test Results — 2026-05-07

Workspace	Passed	Failed	Skipped	Total
autofit_workspace	6	0	1	7
autogalaxy_workspace	6	0	0	6
autolens_workspace	7	0	0	7
autolens_workspace_test	11	0	1	12
euclid_strong_lens_modeling_pipeline	6	0*	0	6
TOTAL	36	0	2	38

* All 6 euclid scripts initially failed with WorkspaceVersionMismatchError (workspace at 2026.4.13.6 vs library at 2026.5.1.4). This is a pre-existing issue unrelated to the pathlib refactor. With PYAUTO_SKIP_WORKSPACE_VERSION_CHECK=1 set, all 6 pass cleanly. Recommend adding that to euclid_strong_lens_modeling_pipeline/config/build/env_vars.yaml defaults in a follow-up.

Library refactor verified — no regressions. The 5 library PRs are safe to merge.

[Jammy2211]

Library PRs Merged — 2026-05-07

PRs (all merged)

• refactor: replace os.path with pathlib PyAutoConf#104
• refactor: replace os.path with pathlib #1258
• refactor: replace os.path with pathlib PyAutoArray#300
• refactor: replace os.path with pathlib PyAutoGalaxy#388
• refactor: replace os.path with pathlib PyAutoLens#497

Status

Library refactor done. Workspace pathlib refactor still pending. Next: /start_workspace to convert ~149 references across autogalaxy_workspace, autolens_workspace, autolens_workspace_developer, autofit_workspace_developer, and euclid_strong_lens_modeling_pipeline.

Outstanding (separate fix)

The euclid pipeline currently fails the workspace-version check (workspace 2026.4.13.6 vs library 2026.5.1.4). Add PYAUTO_SKIP_WORKSPACE_VERSION_CHECK=1 to euclid_strong_lens_modeling_pipeline/config/build/env_vars.yaml defaults — this is not a regression from this PR.

[Jammy2211]

Shipped — 2026-05-07

PRs (all merged)

Library:

• refactor: replace os.path with pathlib PyAutoConf#104
• refactor: replace os.path with pathlib #1258
• refactor: replace os.path with pathlib PyAutoArray#300
• refactor: replace os.path with pathlib PyAutoGalaxy#388
• refactor: replace os.path with pathlib PyAutoLens#497

Workspace:

• refactor: replace os.path with pathlib in workspace scripts autogalaxy_workspace#59
• refactor: replace os.path with pathlib in workspace scripts autolens_workspace#128
• https://github.com/PyAutoLabs/autolens_workspace_developer/pull/50
• refactor: replace os.path with pathlib in workspace scripts Jammy2211/autofit_workspace_developer#15
• refactor: replace os.path with pathlib in workspace scripts euclid_strong_lens_modeling_pipeline#12

Summary

Replaced all os.path.* and bare path.* (from from os import path) with pathlib.Path across 5 libraries and 5 workspaces. ~700 references converted in total. 3,188 library unit tests pass; 36/36 workspace smoke tests pass. No public API changed — internal refactor only.

Session Notes

• Real scope was ~7× larger than the initial issue body claimed. Original count of ~99 missed bare path.* from from os import path. Updated the issue with the corrected breakdown mid-task.
• PlotPatch.__call__ now stringifies recorded paths (self.paths.append(str(path))). Library plotter code now passes Path to plt.savefig in some paths and str in others; uniformly storing str in the test fixture avoids fragility in dozens of plot-output assertions. This is the only test-fixture-level change.
• Sphinx docs/conf.py boilerplate updated: sys.path.insert(0, str(Path(".").resolve())) instead of os.path.abspath(".").
• CRLF line endings preserved. A few PyAutoArray / PyAutoGalaxy files have CRLF on main; intermediate Python scripts that touched them re-saved as LF, which inflated diffs by 5–10×. A targeted post-pass restored CRLF where needed before each commit.
• Sonnet delegation worked well for bulk mechanical conversion (3 of 4 library converters and the workspace shipper). Still Opus-finished a few files where multi-line path.join(...) blocks left trailing / artefacts that the regex couldn't clean — worth a note for future skill design.

Outstanding (separate fix, not regression)

The euclid pipeline currently fails its workspace-version check by default (workspace 2026.4.13.6 vs library 2026.5.1.4). With PYAUTO_SKIP_WORKSPACE_VERSION_CHECK=1 set, it passes cleanly. Recommend adding that env var to euclid_strong_lens_modeling_pipeline/config/build/env_vars.yaml defaults in a follow-up — pre-existing issue, not caused by this PR.

ensure_workspace_labels.sh does not cover Jammy2211/autofit_workspace_developer or PyAutoLabs/autolens_workspace_developer — the pending-release label was missing on the former and had to be created via gh api. Worth adding both to the helper.

Post-merge cleanup

• Worktree ~/Code/PyAutoLabs-wt/use-pathlib removed.
• All 5 library + 5 workspace feature/use-pathlib branches deleted (local + origin).
• Canonical libraries pulled main.
• Canonical autogalaxy_workspace and autolens_workspace pulls skipped — their working trees have uncommitted dataset-binary changes from the in-progress smoke-test-optimization task. The merged commits will land when that task ships its dirty state.