Repository navigation
v0.9.0
obelize can do more for a library it has no pack for, and ships more packs. A pack can be about the
Python itself (datetime.utcnow(), pkg_resources, the modules Python 3.12 and 3.13 removed), hand
its files to a maintained tool (numpy/numpy-1-to-2 runs your own ruff), and flag a changed
signature only at the calls it can break. pack synth and upgrade use the last, and can take
claims from a migration guide that a model you set up has read and code has checked. A requirement
that names extras keeps them when its pin moves, and the GitHub Action can upgrade a dependency in
one job and open the pull request from another. Everything new is reached by a pack or a flag that
uses it: an explicit --pack run of the three bundled packs of 0.8.0 writes what it wrote (their
fixtures are unchanged and pass, the benchmark records below agree) and their pack hashes are
unchanged.
What does change for every repository is what a default run finds, since a default run uses every
pack and there are now seven. The three packs for the Python report what they cannot rewrite, so a
repository that only has from distutils.core import setup in its setup.py exits 0 from
scan --fail-on eligible --fail-on needs_review --fail-on unsupported in 0.8.0 and 4 now,
fix --apply exits 4 where it said "No pack applies", and the GitHub Action, whose default
fail-on is those three statuses, fails the job. The same holds for pkg_resources, cgi,
telnetlib, utcnow() and the other surfaces the packs name, and for np.float_, np.NaN and the
other names numpy/numpy-1-to-2 names. Pass --pack to run the packs you mean, or fail-on to the
statuses you accept (docs/CLI.md, UPGRADING). A default scan
also parses a file that holds a token of one more pack, once for each pack that names it: numpy,
pkg_resources, utcnow, utcfromtimestamp or the name of a module Python removed.
What is deliberately not done: obelize installs no tool (the ruff that numpy/numpy-1-to-2 runs is
yours, on your PATH), and ruff is the one fixer in the registry. --agent takes claude_code
only: a backend for codex exec was checked against its documentation on 2026-10-27 (CLI 0.164.0)
and not built, since it documents features.shell_tool and permission profiles but no setting that
leaves a model without tools, so a text-only run cannot be guaranteed
(ADR-076, which says what would change that).
upgrade --from still takes a release as the one in use only where no manifest declares the library.
Benchmark
I took Gate 1's scan and the three fix dry runs again with this release, over the same 80
repositories each at the same commits, so they read obelize 0.9.0. 317 of the 320 records are the
ones 0.8.0 gave, finding for finding and withheld row for withheld row. miguel-webplace/questiFy
of the Gemini frame is not served by GitHub now, so its record in the scan and in the dry run is an
error and that frame's shares are over 79 repositories. ZeroTang05/cyber-doctor of the PyPDF2
frame, an error in the 0.8.0 records, is served again and its record is the one 0.7.0 gave, so that
frame's shares are over 80 repositories again.
The dry run of obelize upgrade with no pack is new: four frames of twenty public repositories that
still use the old API of pydantic 1, SQLAlchemy 1.4, urllib3 1 and pandas 2, each planned with a pack
made from the two releases. Over those 80 repositories it writes no file: 36 stop before planning,
33 are held and 11 have no usage of a name the pack lists. It finds 944 usages: 78 are places a rule
would rewrite if nothing else were left in the repository (all in pydantic, rename_member), the
pack only flags 810 and 56 are held for a cause of its own, and none is in a file the dry run would
write. That is what a plan holds: no --apply was run on these repositories, so how often a move
passes their tests is not measured (the results).
Unsupported patterns
docs/KNOWN_ISSUES.md has a new section on the packs for the Python itself
and one on the tools a pack can run. What it names:
- None of the three packs for the Python has been run over public repositories, so how often they
write is not known. python/datetime-utcnowrewrites calls only:field(default_factory=datetime.utcnow)and a
SQLAlchemydefault=datetime.utcnoware reported and keep their file as it was. The value stays
naive, asutcnow()made it; a test that mocksdatetimein a module of its own is not seen and
fails after the rewrite.setuptools/pkg-resources-to-importlibwrites nothing and is gated at Python 3.10.python/removed-stdlib-modulesrenamespipes.quoteand reports the rest. It leaves outimp,
uu,nis,cryptandchunk, whose names sit insideimport,uuid,finish,encryptand
chunk.- A default scan, which runs every known pack, parses the files that hold
pkg_resourcesor the
name of a removed module once more for each of these packs, and those that spellutcnowor
utcfromtimestampfor the datetime pack. Over a synthetic repository of 4,000 files a default
scan took 2.3 s, against 2.5 s before the packs. - A comment inside a rewritten call moves above the statement that holds it.
numpy/numpy-1-to-2rewrites 23 names, reports the others the migration guide lists, and holds
the whole repository back for any one of those, the pin included. It does not see what NumPy 2
changes besides names (promotion, the default integer on Windows, the C API), nor strings. It moves
a requirement that excludes NumPy 2 (numpy==1.26.4,numpy<2) tonumpy>=2and leaves one that
admits it (numpy>=1.22) as written, but it does not judge the other pins beside it (a
pandas==1.5.3), so install the result. It was run over five public repositories and has no
Gate 1 measurement. A fixer that moves an import, renames a keyword or needs the target release a
repository states (ruff'sUP035andAIR301,django-upgrade,pyupgrade) is not registered
(ADR-074).--guidehas been run once on a public guide (pydantic 1 to 2) and measured nowhere else. What the
model drafts is its recall, not obelize's: a first prompt gave 13 claims from the guide and the
second 41 and 47 in two runs. A change in another distribution, a class body (class Config) and
what a function does have no name to hold a claim to, a table or a page built in the browser may
not be read, and a rewrite rests on the API diff and not on the guide, so your tests decide whether
it means what the old call did (the section on migration guides in KNOWN_ISSUES.md).- A name that a pack flags only at some calls is not listed where it is handed to another call
(map(pd.to_datetime, rows), a callback), whatever positional arguments the code that receives it
passes, and a deprecation of the whole name that happens to use the word of a changed parameter
lets a call by. A name that also left or moved, a function that changed kind and every method are
still listed wherever they are used (the section on upgrading a dependency in KNOWN_ISSUES.md). upgradewith no pack stopped 18 of those 80 repositories on a manifest that declares the library
loosely (pydantic,pydantic<2,^1.9.1) or below the release the pack was made from, with
legacy_version_unsupported, though--fromnames a release inside the pack's range: a declaration
decides, and--fromcounts only where none exists. Whether it should overrule a loose one is not
decided (ADR-072).- A repository's plan is written only when nothing in it is left, for a library that keeps its import
name, so one place the pack can only flag holds the rest of the repository: 32 of the 33 held
repositories had such a place. The section on upgrading a dependency in KNOWN_ISSUES.md has both.
The formats that changed are pack.yaml and pack.schema.json, which describes it: a side of a
migration is a distribution (package and version) or the Python (python), and not both, and
to.requires_python is required only with a distribution, a rewrite_pattern takes imports,
each of a closed list of modules, flag_only takes calls, external_fixer is a new kind (fixer,
symbols), manifest_dependency takes provides_extras, a change may carry a source of its own
whose type may be guide, which is never bundled, and a module of a shared pack may have no
prefilter token in its name when each symbol under it ends in one. The id of a pack made with a
guide ends with -with- and a digest of the guide's text, and guide.json is the new record beside
such a pack. run.json gains packs[].fixers and packs[].stated, and eight codes join the bail
vocabulary and the bails an import row may carry, which findings.json, plan.json,
proposal.json and run.json list (external_tool_missing, external_tool_version_unsupported,
external_tool_failed, external_tool_timeout, external_tool_size_limit,
external_fix_out_of_scope, external_fix_incomplete, external_fix_not_idempotent). The reasons
alias_collision and local_import are also given for a rewrite_pattern that cannot name a module
it imports, the text of runtime_unsupported for a pack whose new side is the Python names the pack,
and a finding's evidence, which was always null, may carry the words of
SCAN_VOCABULARY.md section 14 (findings.json follows).
Added
- A pack made from two releases can take what the library's authors say in prose.
pack synthand
upgradetake--guide <url|file>(up to three), and a model you set up in your own
configuration (the endpoint inmodel, else the agent inagent) reads the guide and drafts
claims: a name renamed, a method renamed, a call written another way, a name that needs a person.
Code keeps a claim only if its quote is in the guide, its names are in both releases, its pattern
is one a pack may hold and the API diff does not contradict it. A kept claim is a report that
quotes the guide, and a rewrite only where the quote says in a known shape that the new name takes
the place of the old one (a line of a table, an arrow, "oldis nownew", "usenewinstead of
old"), no word in its sentence limits or takes that back ("not", "but", "only"), and the diff
shows the old name removed or deprecated and the new one there, written the same way (not a
coroutine where the old was none); for a pattern, every call it writes must resolve in the new
release. Only the text of the guide goes to the model, in parts, with the names of the library and
its releases; nothing of your code does, the host or program is printed first, and with no model
set up, or under--offline, the option is skipped with one line and the guide is not read.
upgradereads the guide only when no pack is known. After two parts one after another that fail,
or 30 minutes, the model is not asked for the rest. The pack's folder gets aguide.jsonwith the
day the guide was read, the model's answers, the counts and what became of each claim, and
pack synth --guide-replaymakes the same pack again from it with no model, on any day. A file
must be a regular file with a text or HTML suffix, and a page is HTML by the type it is served
with. A guide is fetched as the index is, so a page that moved to another host is refused and the
message names the host. On pydantic 1.10.26 to 2.12.5 and its migration guide the model drafted 47
claims, 31 were kept and one of them,BaseModel.copytomodel_copy, is a rewrite, and the
answers fed back through the checks as they are now give the same result
(the run,
why). - A requirement that names extras (
mcp[cli]>=1.28.1) is no longer always left as it was. A pack
made bypack synthlists the extras the newer release's wheel gives asProvides-Extra
(manifest_dependency.provides_extras), and a requirement of that one distribution whose extras
are all listed has only the version after them replaced (mcp[cli]>=2.12.5,<3). An extra the
release does not list, a release with no wheel, a renamed distribution (PyPDF2[crypto]), a
constraints file, a hash, a continued line and the other forms where the edit would be more than
the version leave the line as it was, named in the summary as before
(the forms). The bundled packs list none, so what they
write is what it was. - The GitHub Action can upgrade a dependency.
mode: upgradewithdistributions: pydanticruns
obelize upgrade --apply --yes --branchfor each dependency in the checkout, with no credential
that writes, and uploads for each a git bundle of the branch it made, a patch, the trial's
evidence and a report (bundle-file,report-file,changedandexit-codeare its outputs, the
last the code of the first dependency that failed outright, else of the first not proven;
fail-on-unprovenfails the job on exit3or6, and any other code always fails it). It stops
if the event ispull_request_target,workflow_runorissue_comment, ifGH_TOKEN,
GITHUB_TOKENorACTIONS_ID_TOKEN_REQUEST_TOKENis set, or if the checkout kept a credential.
The pull request is opened by a second job that runs none of your code:action/upgrade.py publishtakes only the dependencies the workflow lists, checks that the bundle is one commit on
the commit the workflow ran on, editing only files the base has that are Python sources or
manifests (no.github, link, executable, deletion or new file), and then pushes a commit it makes
itself from that tree and opens the pull request.mode: scanis what it was
(Upgrading in CI,
why). - A change of kind
external_fixerruns a maintained tool over a file whose every place the scan
found is one of the names it lists, named by an id obelize holds (ruff:NPY201) and never by a
command, flag, path or release.
The tool is found among the absolute entries of yourPATHand started with an argv list, a few
environment variables and an empty directory, under a 60-second deadline and 4 MiB each way, and
only in a release the registry lists (ruff 0.6.2 to 0.17.3, each run over the fixtures; its
release is read asmajor.minor.patchand nothing else, again whenever the file is replaced). It
runs after the pack's other rules, in a session that is ended whatever happens, with its output
read no further than the limit, and what it prints is kept only if it changed no line the scan did
not report or a rule wrote, added only imports, blank lines and comments beside a replaced line,
none of whose names a function, class or comprehension of the file binds itself (the call would
read the local:UnboundLocalError), wrote no line wider than the pack's layout, changes nothing
run on its own output, and the scan of it finds none of the names it claims
(ruff leavesfrom numpy import float_standing, so such a file stays as it was). The run folder
records, per fixer, the release the tool printed, how it was started and the hashes of what it was
given and printed for each file (packs[].fixers);fixsaysnumpy/numpy-1-to-2 runs ruff (NPY201) on 3 files.ahead of the diff;pack validateprints what a fixer runs and the command
that installs it;scansays on stderr when what it found cannot be written for want of the tool;
pack testhas a check for the tool, and the checks that need it do not run, so a bundled pack
that fails only those exits7and is not reported as a defect of obelize
(ADR-074). - A bundled pack,
numpy/numpy-1-to-2: 23 names NumPy 2 removed, written by ruff as the name that
also exists on NumPy 1 (np.float_asnp.float64,np.NaNasnp.nan,np.cumproductas
np.cumprod), 75 more reported (every name of the main namespace that NumPy 1.26.4 has and 2.0.2
or 2.5.4 lacks is one or the other, 98 in all, among themmsort, 26 internal constants and the
exceptions that moved tonumpy.exceptions), and the dependency line. Each rewrite was measured
on NumPy 1.26.4 and 2.x, and a name whose replacement ruff writes in a way that changes the value
(np.NINF ** 2) is reported. A name written as a string (dtype='float_') is not found. - A
flag_onlychange can say which calls it flags a symbol for.callsmaps a symbol to a shape
ofkeywords,positional_fromandomits, and a call with a*argument that could pass a
position the shape lists, or a**argument that could pass a keyword or a name, is one. In a
pack that shares its module, a call that matches none is not listed, and neither is a name that
is imported, annotated or handed to a call (Column(Integer),map(f, rows)) unless that call
passes a keyword the shape lists (partial(f, removed=1)); a class named as a base, a name
assigned to another and a bare decorator are listed as before. Each call listed says what matched
in itsevidence(keyword timeout,omitted retries,position 2,spread), in the terminal
table and inREPORT.md. A symbol with no entry is flagged wherever it is used, so the bundled
packs and every pack written before read as they did. A shape can saybasesfor a class whose
bases changed: no call is one, and code that derives from it is.
pack synthwrites the shapes: from a function's or a class's parameter changes, from a class
whose bases changed and that is built with what it was, and from a deprecation whose message
names a parameter of the same name that changed (it is about the calls that pass it). A class
whose constructor comes from a base that changed, a name that also left or moved, a deprecation
that names no changed parameter, a function that changed kind, and every method, is flagged
wherever it is used. Over eight repositories of each of four migrations the places listed went
from 4, 163, 293 and 220 (4, 120, 195 and 219 distinct places) to 2, 98, 28 and 41 (urllib3,
pydantic, SQLAlchemy, pandas). - The comparison reads
lib.no_default,no_defaultandNoDefault.no_defaultas "not given", as
it does_Unset, so a default that became one is no change; a private name that does not say it
is one (_DEFAULT_TIMEOUT) is read as a value. pandas 3.1 had 2,872 of its 4,284 default changes
that name. A parameter that a new**kwargstakes is a removal all the same
(sessionmaker(autocommit=True)raises in SQLAlchemy 2.0), and the first parameter of a method is
its receiver whatever it is called, so__pydantic_self__is no parameter. - A side of a pack can be the Python:
from: {python: ">=3.12"}andto: {python: ">=3.2"}, where
to.pythonis whatrequires_pythonis for a distribution. A repository that declares a Python
outside it is blocked (runtime_unsupported), and one that declares none is not. A pack whose
new side is the Python moves no dependency line, and its old distribution's lines in a manifest
are context.obelize upgrade pkg_resourcessays it is no dependency and names the pack to run. - A
rewrite_patterncan spell names through the modules it lists underimports, such as
datetime.timezone, but only modules of a closed list of those that build or read a value
(datetime,json,re,importlib.metadataand 22 others, in
docs/PACK_SPEC.md), and a name one segment below one. A pack that listsos,
importlib,socketor a module nobody has looked at is refused when it loads.obelize pack validateprints the paths a pack may import, and so does the header offix. The rule names a
module as the file already does (import datetime as dt,from datetime import timezone) and
otherwise adds one import line below the last import above the earliest call; a name the file
binds to something else (including by acasecapture or adel), a file with a star import, or
a file that imports only inside functions, leaves the call for you. - A module of a shared pack may have no prefilter token in its name when each symbol under it ends
in one:python/datetime-utcnowlooks forutcnowandutcfromtimestamp. - Three bundled packs for the Python itself.
python/datetime-utcnowrewritesdatetime.utcnow()
anddatetime.utcfromtimestamp(x)tonow(timezone.utc)andfromtimestamp(x, timezone.utc)
with.replace(tzinfo=None), which keeps the value naive.setuptools/pkg-resources-to-importlib
reports every use ofpkg_resourceswith theimportlib.metadataorimportlib.resources
replacement and writes nothing.python/removed-stdlib-modulesrenamespipes.quoteto
shlex.quoteand reports the other modules Python 3.12 and 3.13 removed, with where each went.
Changed
obelize upgradefor a dependency that is already on the newest release, with no--toand no
pack that moves it to another distribution, says so and exits0with nothing written, where it
exited2("nothing to synthesize"). A pack that moves it to another distribution still runs.obelize --helpnames NumPy 1.x to 2.x among the supported migrations.- A template may contain
None,TrueandFalse, which it could not before: they were read as
names to import. obelize pack validateprints a side that is the Python asPython >=3.12.- A run in which no pack applies counts the files the packs it tried parsed:
files_parsedwas0
there, because only the packs a run kept were counted. obelize upgrade <module>for a module of the Python or of setuptools names the manifests it
looked at, andobelize upgrade setuptoolssays on stderr that the pack forpkg_resourcesis
run byobelize fix --pack, whichupgradedoes not do.- The paragraph on a repository whose declared Python a pack's new code does not run on names the
pack when the new side is the Python, so several packs in one run can be told apart. - The step
fix --applysuggests after a run that nothing verified no longer says to install what
the run wrote into the manifest when every pack of the run is one that moves no dependency line. obelize upgrade --from <release>takes that release for the one in use where no manifest
declares the dependency, so a repository that never names the library is planned instead of
stopped with "No manifest declares". A declaration still decides:pydantic>=1.8stops the plan
as before, and the message says that--fromwas not used. The run says what it took in the
summary and inREPORT.md, andrun.jsongainspacks[].stated(run.schema.jsonfollows). A
fix, ascanand anupgradewithout--fromare stopped as they were.
Fixed
obelize pack synth sqlalchemy --from 1.4.54 --to 2.0.55made no pack and exited1. Two names
had moved up out of two submodules, and the fixture that shows the move imported both, which the
rules leave alone because the package is then bound twice. The fixture now shows the names of
one submodule, and SQLAlchemy 1.4 to 2.0 and to 2.1 make a pack that passespack test.