Repository navigation
Code Analysis Python Relative Imports
Draft #1216, revision 48d31661fe941f7b1985c61484268e71fda6674e, extends the existing package-path correction to relative namespace imports. It is unmerged. The comparison parent is fab79183128c2b63d203a87a40063a327772d137; earlier absolute-import results remain in the package and namespace record.
A namespace package can span the repository root and src/. Previous relative lookup searched only the caller's physical directory, even though absolute lookup already combined those portions. Under the same root/src order this could miss a sibling or select the wrong file.
For example, ns/consumer.py imports from .deep import child. Both ns/deep/child.py and src/ns/deep/child.py exist, but only src/ns/deep has __init__.py. CPython selects the regular package in src; the previous analyzer selected the namespace child under the root. This is a target error, not just an unresolved count.
Expected paths were fixed before candidate scans and checked by isolated CPython 3.13.16 with exactly sys.path = [root, root/src]. Only tiny authored modules were executed.
| Probe | Previous result | Candidate / Python result |
|---|---|---|
| Root namespace caller imports sibling in src | Unknown | src/ns/right.py |
| Src namespace caller imports sibling in root | Unknown | ns/left.py |
| Parent-relative import crosses portions | Unknown | src/ns/right.py |
| Regular child package overrides namespace child | ns/deep/child.py |
src/ns/deep/child.py |
| Earlier module overrides later package, caller is in src | src/ns/dep/__init__.py |
ns/dep.py |
| Regular parent refuses a sibling from another root | Initializer dependency | Same initializer dependency |
The last row is a deliberate semantic boundary. Python raises ImportError for the missing attribute, but loads the initializer. Keeping that file dependency does not assert that the attribute exists. The evaluator initially compared the runtime failure with file-edge resolution, then corrected this interpretation before implementation; it is not counted as a sixth defect.
The Python import statement reference distinguishes attribute lookup from submodule loading. PEP 420 establishes namespace portions and regular-package precedence. Two unchanged CPython fixture modules were also mapped from portion1/portion2 to root/src, with two authored relative callers. Their values and selected files match Python; both previously missing references are recovered. This adapts an import-system regression fixture, not a new application incident.
The small relative-path helper reuses the component walk and captured directory index. It expands only an inferred namespace that includes the caller's directory, retaining root/src order. Regular packages, analyzed roots that are themselves packages and unknown external package contexts preserve their existing boundaries. Relative namespace specifiers also retain the actual number of leading dots.
Python extraction revision is 5. Affected generated task examples changed only revision/snapshot metadata. No parser, dependency, environment discovery or configurable search-root layer was added. The analyzer does not execute the target project.
Root/src remains a convention. Import hooks, arbitrary sys.path, compiled modules, runtime path edits, object identity and complete partial-initialization effects remain outside this model. Correct file targets do not establish runtime reachability or complete impact.
- One new positive regression fails on the exact parent; the regular/analyzed-root boundary control already passes there. All 235 module tests pass on official Node 22.23.2 / Python 3.13.16.
- Ansible collection scope (52 files / 108 imports), pluggy (31 / 235) and pytest (274 / 4,066) retain every import record. These are preservation controls, not new accuracy improvements in those applications.
- Six authored probes, one adapted CPython scope and the three repository controls preserve file, function, symbol and parse records. All have zero parse errors.
- All ten normal CLI results match the unpacked archive byte for byte. Repeated archive SHA256:
90f306c8ad2cb27728b82ac31831b3c42849bc2eff29124b7a4f8e37702fd01a. - A separate agent review found no actionable correctness issue and checked five additional small CPython scenarios. This is not human acceptance.
- The self-contained public-fetch reproduction below passed on the published commit, including all oracle, parent/candidate and archive comparisons. CI for the final head is checked separately in the PR.
Use Bash, Git, Node 22, npm, unzip and CPython 3.13. Analyzer package paths and the Ansible collection scope are fetched sparsely to avoid unrelated artifacts. Only analyzer dependencies are installed in temporary directories. Target application code is not executed; only the small authored and upstream-derived fixture modules are imported for the oracle.
Self-contained public reproduction
#!/usr/bin/env bash
set -euo pipefail
# Run with ARCHIFY_REF=<published candidate commit> and Python 3.13, Node, npm, git, unzip.
# Set BAUIFY_PYTHON to the CPython 3.13 executable if needed.
ARCHIFY_REF=${ARCHIFY_REF:-48d31661fe941f7b1985c61484268e71fda6674e}
[[ "$ARCHIFY_REF" =~ ^[0-9a-f]{40}$ ]] || { echo 'ARCHIFY_REF must be a full lowercase commit SHA' >&2; exit 1; }
PYTHON=${BAUIFY_PYTHON:-python3.13}
"$PYTHON" -c 'import sys; assert sys.implementation.name == "cpython" and sys.version_info[:2] == (3,13), sys.version'
WORK=$(mktemp -d "${TMPDIR:-/tmp}/archify-relative-public.XXXXXXXX")
echo "Results and checkouts: $WORK"
export SOURCES="$WORK/sources" EVIDENCE_ROOT="$WORK/results"
mkdir -p "$SOURCES" "$EVIDENCE_ROOT"
cat > "$WORK/public-fixtures.py" <<'ARCHIFY_FIXTURES_PY'
import importlib, json, os, pathlib, subprocess, sys
assert sys.version_info[:2] == (3, 13), f"CPython 3.13 required: {sys.version}"
assert sys.implementation.name == 'cpython'
root = pathlib.Path(os.environ['SOURCES'])
evidence = pathlib.Path(os.environ['EVIDENCE_ROOT'])
cases = json.loads(r'''[
{
"name": "root-to-src",
"caller": "ns.consumer",
"from": "ns/consumer.py",
"expected": "src/ns/right.py",
"files": {
"ns/consumer.py": "from . import right as selected\n",
"src/ns/right.py": ""
}
},
{
"name": "src-to-root",
"caller": "ns.consumer",
"from": "src/ns/consumer.py",
"expected": "ns/left.py",
"files": {
"src/ns/consumer.py": "from . import left as selected\n",
"ns/left.py": ""
}
},
{
"name": "parent-relative",
"caller": "ns.inner.consumer",
"from": "ns/inner/consumer.py",
"expected": "src/ns/right.py",
"files": {
"ns/inner/consumer.py": "from .. import right as selected\n",
"src/ns/right.py": ""
}
},
{
"name": "regular-child-over-namespace",
"caller": "ns.consumer",
"from": "ns/consumer.py",
"expected": "src/ns/deep/child.py",
"files": {
"ns/consumer.py": "from .deep import child as selected\n",
"ns/deep/child.py": "value=1\n",
"src/ns/deep/__init__.py": "",
"src/ns/deep/child.py": "value=2\n"
}
},
{
"name": "earlier-module",
"caller": "ns.consumer",
"from": "src/ns/consumer.py",
"expected": "ns/dep.py",
"files": {
"src/ns/consumer.py": "from . import dep as selected\n",
"ns/dep.py": "",
"src/ns/dep/__init__.py": ""
}
},
{
"name": "regular-parent-control",
"caller": "ns.consumer",
"from": "ns/consumer.py",
"expected": "ns/__init__.py",
"files": {
"ns/__init__.py": "",
"ns/consumer.py": "from . import absent as selected\n",
"src/ns/absent.py": ""
},
"oracleExpected": null
}
]
''')
(evidence / 'expected-cases.json').write_text(json.dumps(cases, indent=2) + '\n')
for case in cases:
folder = root / case['name']
for name, contents in case['files'].items():
target = folder / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(contents)
code = """import importlib, json, os, sys
sys.path = [os.getcwd(), os.path.join(os.getcwd(), 'src')]
try:
module = importlib.import_module(%r)
result = {'to': os.path.relpath(module.selected.__file__).replace(os.sep, '/')}
except (ModuleNotFoundError, ImportError) as error:
result = {'to': None, 'error': type(error).__name__}
print(json.dumps(result))
""" % case['caller']
result = json.loads(subprocess.check_output([sys.executable, '-I', '-S', '-B', '-c', code], cwd=folder, text=True))
assert result['to'] == case.get('oracleExpected', case['expected']), (case['name'], result)
print(case['name'], result)
folder = root / 'cpython-portions'
files = {'foo/one.py': "attr = 'portion1 foo one'\n", 'src/foo/two.py': "attr = 'portion2 foo two'\n", 'foo/consumer_one.py': 'from . import two as selected\n', 'src/foo/consumer_two.py': 'from . import one as selected\n'}
for name, contents in files.items():
target = folder / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(contents)
for caller, expected, value in [('foo.consumer_one', 'src/foo/two.py', 'portion2 foo two'), ('foo.consumer_two', 'foo/one.py', 'portion1 foo one')]:
code = """import importlib,json,os,sys
sys.path=[os.getcwd(),os.path.join(os.getcwd(),'src')]
m=importlib.import_module(%r).selected
print(json.dumps([os.path.relpath(m.__file__).replace(os.sep,'/'),m.attr]))
""" % caller
result = json.loads(subprocess.check_output([sys.executable, '-I', '-S', '-B', '-c', code], cwd=folder, text=True))
assert result == [expected, value], result
print(caller, result)
ARCHIFY_FIXTURES_PY
cat > "$WORK/public-verify.mjs" <<'ARCHIFY_VERIFY_MJS'
import fs from 'node:fs';
import path from 'node:path';
import assert from 'node:assert/strict';
import {spawnSync} from 'node:child_process';
import {createHash} from 'node:crypto';
const e=process.env.EVIDENCE_ROOT;
const w=process.env.SOURCES;
const cli=process.argv[2];
const label=process.argv[3] || 'candidate';
const prior=process.env.PARENT_CLI;
const samples=JSON.parse(fs.readFileSync(e+'/expected-cases.json')).map(c=>({...c,root:w+'/'+c.name,authored:true}));
samples.push({name:'cpython-portions',root:w+'/cpython-portions',upstream:true});
const real=JSON.parse(fs.readFileSync(process.env.REAL_CASES));
samples.push(...real);
const run=(cmd,args,opts={})=>{
const r=spawnSync(cmd,args,{encoding:'utf8',timeout:180000,maxBuffer:2*1024*1024,...opts});
assert.equal(r.status,0,r.stderr||r.stdout||String(r.error));return r.stdout;
};
const results=[];
const hash=b=>createHash('sha256').update(b).digest('hex');
for(const c of samples){
if(c.pin){assert.equal(run('git',['-C',c.root,'rev-parse','HEAD']).trim(),c.pin);assert.equal(run('git',['-C',c.root,'status','--porcelain','--untracked-files=no']),'');}
for(const [tag,tool]of label==='candidate' ? [['parent',prior],[label,cli]] : [[label,cli]]){
const out=e+'/'+c.name+'-'+tag;
const receipt=run(process.execPath,[tool,'code-analysis','check',c.root,'--language','py','--fail-on','never','--out',out,'--json']);
fs.writeFileSync(out+'.receipt.json',receipt);
}
const parent=JSON.parse(fs.readFileSync(e+'/'+c.name+'-parent/raw-facts.json'));
const bytes=fs.readFileSync(e+'/'+c.name+'-'+label+'/raw-facts.json'),facts=JSON.parse(bytes);
assert.equal(facts.repository.adapterRevision,5);
for(const key of ['files','functions','symbols','parse_errors'])assert.deepEqual(facts[key],parent[key],c.name+': '+key);
if(c.authored){
const edge=facts.imports.find(x=>x.from===c.from&&!x.implicit);
assert.equal(edge.to,c.expected,c.name);
if(c.name!=='regular-parent-control') {
const source=c.files[c.from], [,dots,mod,name]=source.match(/^from (\.*)(\w*) import (\w+)/);
assert.equal(edge.specifier,dots+(mod?mod+'.':'')+name,c.name+': relative depth');
}else assert.deepEqual(facts.imports,parent.imports,'preserve initializer dependency despite missing attribute');
}
if(c.upstream){
for(const [from,to]of [['foo/consumer_one.py','src/foo/two.py'],['src/foo/consumer_two.py','foo/one.py']]){
assert.equal(parent.imports.find(x=>x.from===from&&!x.implicit).resolved,false);
assert.equal(facts.imports.find(x=>x.from===from&&!x.implicit).to,to);
}
}
const changes={removed:parent.imports.filter(x=>!facts.imports.some(y=>JSON.stringify(x)===JSON.stringify(y))),added:facts.imports.filter(x=>!parent.imports.some(y=>JSON.stringify(x)===JSON.stringify(y)))};
if(c.pin){
const withoutSpecifier=xs=>xs.map(({specifier,...x})=>JSON.stringify(x)).sort();
assert.deepEqual(withoutSpecifier(facts.imports),withoutSpecifier(parent.imports),c.name+': review target or flag changes');
}
if(label==='archive')assert.equal(hash(bytes),hash(fs.readFileSync(e+'/'+c.name+'-candidate/raw-facts.json')),c.name+': source/archive parity');
const result={name:c.name,pin:c.pin,files:facts.files.length,imports:facts.imports.length,parseErrors:facts.parse_errors?.length||0,changes,archiveParity:label==='archive'};
results.push(result);
console.log(JSON.stringify({name:c.name,files:result.files,imports:result.imports,changedRecords:changes.added.length,archiveParity:result.archiveParity}));
}
fs.writeFileSync(e+'/'+label+'-results.json',JSON.stringify({node:process.version,results},null,2)+'\n');
ARCHIFY_VERIFY_MJS
# Fetch only specified revisions; filtered sparse checkout avoids unrelated large assets.
fetch_checkout() {
local url=$1 revision=$2 dest=$3 pattern=$4
git init -q "$dest"
git -C "$dest" remote add origin "$url"
git -C "$dest" config core.sparseCheckout true
printf '%s\n' "$pattern" > "$dest/.git/info/sparse-checkout"
git -C "$dest" -c http.version=HTTP/1.1 fetch --quiet --depth=1 --filter=blob:none origin "$revision"
git -C "$dest" checkout -q --detach FETCH_HEAD
test "$(git -C "$dest" rev-parse HEAD)" = "$revision"
}
fetch_checkout https://github.com/tt-a1i/archify.git fab79183128c2b63d203a87a40063a327772d137 "$WORK/parent" '/archify/'
fetch_checkout https://github.com/tt-a1i/archify.git "$ARCHIFY_REF" "$WORK/candidate" $'/archify/\n/archify.zip'
"$PYTHON" - "$WORK/candidate/archify.zip" <<'PY'
import hashlib, pathlib, sys
actual = hashlib.sha256(pathlib.Path(sys.argv[1]).read_bytes()).hexdigest()
expected = '90f306c8ad2cb27728b82ac31831b3c42849bc2eff29124b7a4f8e37702fd01a'
assert actual == expected, (actual, expected)
print('archive SHA256:', actual)
PY
mkdir "$WORK/archive"
unzip -q "$WORK/candidate/archify.zip" -d "$WORK/archive"
for dir in "$WORK/parent/archify/modules/code-analysis" "$WORK/candidate/archify/modules/code-analysis" "$WORK/archive/archify/modules/code-analysis"; do
(cd "$dir" && npm ci --ignore-scripts --no-audit --no-fund)
done
# These pinned upstream modules are reproduced verbatim; only the two callers are authored.
# CPython source: cbc944f4bc59639a444dd971c737788ba2283a91,
# Lib/test/test_importlib/namespace_pkgs/portion{1,2}/foo/{one,two}.py.
"$PYTHON" "$WORK/public-fixtures.py"
fetch_checkout https://github.com/ansible/ansible.git a7e289870529620963946ee00bbc374404d8ff4b "$WORK/ansible" '/test/integration/targets/collections/collection_root_user/'
fetch_checkout https://github.com/pytest-dev/pluggy.git 3d5417a4eff449e17e5bb3e1545b63cbf0e0c768 "$WORK/pluggy" '/*'
fetch_checkout https://github.com/pytest-dev/pytest.git 08017d9ada4c72a7b6a78f7873c238070e24c982 "$WORK/pytest" '/*'
export REAL_CASES="$WORK/real-cases.json" PARENT_CLI="$WORK/parent/archify/bin/archify.mjs"
"$PYTHON" - "$WORK" <<'PY'
import json, os, pathlib, sys
w = pathlib.Path(sys.argv[1])
cases = [dict(name='ansible-collections',root=str(w/'ansible/test/integration/targets/collections/collection_root_user'),pin='a7e289870529620963946ee00bbc374404d8ff4b'), dict(name='pluggy',root=str(w/'pluggy'),pin='3d5417a4eff449e17e5bb3e1545b63cbf0e0c768'), dict(name='pytest',root=str(w/'pytest'),pin='08017d9ada4c72a7b6a78f7873c238070e24c982')]
pathlib.Path(os.environ['REAL_CASES']).write_text(json.dumps(cases))
PY
node "$WORK/public-verify.mjs" "$WORK/candidate/archify/bin/archify.mjs" candidate
node "$WORK/public-verify.mjs" "$WORK/archive/archify/bin/archify.mjs" archive
echo "Verified; results retained at $WORK"