chore: bump dev dependencies to resolve Dependabot security alerts

[sudsali]

Summary

Updates development dependencies to modern versions, resolving 28 open Dependabot security alerts. Runtime dependencies (numpy, pandas, pyspark) are unchanged — this only affects the development environment.

Changes

Python version floor: >=3.8 → >=3.9 (3.8 is EOL since Oct 2024, CI only tests 3.9)

Dev dependency bumps:

Package	Before	After	Reason
pytest	^6.2.4	^8.0	Security (CVE in <9.0.3)
pytest-cov	^2.11.1	^5.0	Compatibility with pytest 8
coverage	^5.5	^7.0	Current stable
black	^21.5b1	^24.0	Security (CVE in <24.3.0, <26.3.1)
flake8	^3.9.2	^7.0	Current stable
flake8-docstrings	^1.6.0	^1.7	Minor bump
pre-commit	^2.12.1	^3.5	Current (^4.0 needs Python 3.10+)
pytest-rerunfailures	^9.1.1	^14.0	Current stable
twine	^3.4.1	^5.0	Current stable

Removed (unmaintained/unnecessary):

• pytest-runner — deprecated, replaced by direct pytest invocation
• pytest-flake8 — unmaintained, run flake8 directly
• safety — replaced by GitHub's Dependabot for vulnerability scanning

What this fixes

Resolves Dependabot's inability to update transitive dependencies (urllib3, cryptography, etc.) which were pinned by ancient dev deps. After this merge, Dependabot should be able to create successful update PRs.

What's NOT changed

• Runtime dependencies (numpy, pandas, pyspark) — no user impact
• PySpark version support (3.1-3.5) — unchanged
• Library functionality — zero code changes

Test plan

• CI passes (poetry install + pytest) on all PySpark matrix variants
• poetry lock resolves cleanly
• Dependabot alerts start resolving after merge

[github-actions]

No issues found.

---

Generated by AI (model: us.anthropic.claude-opus-4-6-v1, prompt: db2249a9) — may not be fully accurate. Reply if this doesn't help.

[github-actions]

No issues found and CI is passing. Auto-approved.

---

Generated by AI — human merge required.