Repository navigation
Code Analysis Pandas Selection
This study asks three concrete questions about DataFrame.__getitem__: how scalar and list column selection differ, how boolean row masks align or fail, and why duplicate or MultiIndex columns change the result. The answers and 14 finite expectations were written from pinned source before running Archify. This is a source-reading study using the saved Python entry capability in Draft #1234, not a new product fix.
The source is pandas v2.2.3 at 0691c5cf90477d3503834d983f69350f250a6ff7, a historical version. Captures use Draft #1216 at 272f995b; saved questions use Draft #1234 at 29f73e9c. These experimental branches have not been merged into Labs, dev or main.
For ordinary unique flat columns, df['A'] takes the scalar fast path and produces a Series, while df[['A']] uses the strict column indexer and keeps a DataFrame. Missing requested labels raise KeyError. The one-item list does not qualify for scalar squeezing. This follows the actual conditions in __getitem__:4062–4128, including its helper calls; the return shape is a source interpretation, not an emitted call fact.
A boolean Series with an unequal index first triggers a warning and then aligns to the frame's row labels. Extra labels can still align successfully; a missing required label raises IndexingError. A wrong-length non-Series mask raises ValueError. Object arrays containing boolean-like values and missing entries can fail earlier in is_bool_indexer; nullable Boolean arrays instead treat missing entries as false in the selected validation path. These are distinct branches, not a universal rule that every missing boolean value errors. See row filtering, mask recognition, label alignment and array validation.
With flat columns A,A,B, selecting A returns a DataFrame, but selecting the unique B still returns a Series. With MultiIndex columns (X,A),(X,B),(Y,A), selecting X returns its subgroup DataFrame, while the unique full tuple (X,A) returns a Series. The MultiIndex path has its own helper and additional cases; empty-string placeholders, deeper partial tuples and broader duplicate/lexsort behavior are outside this finite study. See dispatch and _getitem_multilevel.
The normal Python CLI captured 1,510 files, 14,316 import records and 29,431 functions, with no Python parse errors. These are inventory counts, not adjudicated accuracy rates or coverage of pandas' Cython/native implementation. --fail-on never returned exit 0 while retaining 310 findings: 97 warnings and 213 informational findings. They were not individually evaluated in this study.
The saved question selected the unique DataFrame.__getitem__ range at lines 4062–4128 and verified current source freshness. Its containing file has 162 import records, including 142 resolved targets whose current captured files passed freshness checks. These are file dependencies, not the method's call targets. Python call coverage remains unavailable, with a null total and an empty record list. The source and unpacked archive produced identical question JSON after excluding only the workflow document path.
The 38,240,651-byte raw facts file has SHA256 c8660cb998c202b1432d8681949e386f91c0a5e7fa17babecfaf9c7bc206a5a5. Fresh public capture reproduced this hash. A successful question receipt identifies a reading entry point; it does not answer the three semantic questions above.
A later source audit selected only the two file-level warning findings, including the overview's first priority. It checked seven files and ten import sites against captured text: nine were genuinely unguarded module-scope imports, and a nearby TYPE_CHECKING control was correctly marked type-only. The other findings and the full 150-file strongly connected component were not independently adjudicated.
-
COUP-dfc0f62edb50: real static cycle, unproven execution order. Its displayed path goes frompandas/__init__.py:175to_normalize.py, then back throughimport pandas as pd. Its separateBooleanDtypepartial-initialization hypothesis followscore/api → algorithms → dtypes/cast → io/_util → pandas. These source edges exist. Butio/_util.py:7only imports the package object; graph traversal does not establish that a cached package initializer runs again or reads a missing name. Ordinary dotted imports also load parent packages before children. The current rule already calls this a static hypothesis and explicitly leaves load-time behavior unverified. Python's module-cache documentation explains why a file-graph path alone is insufficient. -
COUP-4429223b198a: package self-reference, uncaptured extension child.tslibs/__init__.py:39importsdtypesfrom its own package. The Python adapter records the requested package initializer when no captured.pychild exists. The actual child is built fromdtypes.pyx, outside this Python-source capture. The following direct child import at line 41 remains unresolved. The self-edge therefore records a package-file dependency; it does not locate the imported object's implementation or prove recursive initializer execution.
These are valid static facts with limited runtime-risk specificity, not demonstrated extraction defects. No new severity policy, import simulator or product patch follows from this check. This later audit parsed source and read CPython's import implementation; it did not rerun pandas, and the earlier successful finite runtime cases do not disprove static coupling.
A separate isolated environment installed binary wheels for pandas 2.2.3, NumPy 2.1.3, python-dateutil 2.9.0.post0, pytz 2024.2, tzdata 2024.2 and six 1.16.0. All six wheel hashes matched official PyPI metadata. pandas reported the exact pinned Git revision, and five inspected Python source files matched the checkout byte for byte. This does not establish identity of every compiled component.
All 14 prewritten cases matched: scalar/list selection, a missing label in a list, positional/wrong-length boolean masks, reordered/extra/missing Series labels, object-NA and nullable-Boolean masks, duplicate columns and the two selected MultiIndex routes. The probe executes this bounded pandas library behavior. Archify's capture itself does not execute the target. No pandas application, full test suite or source build was run; no live Archify installation changed.
Two independent GPT-6.1 Terra medium sessions received the same questions and pinned source. One also received saved question orientation. Neither received the expected answers, runtime observations or the other's response. Both answered the three main questions correctly, including the extra-label mask case and the distinction between repeated A and unique B. This single qualitative pair supplies no estimate of reader accuracy, speed, token cost or benefit from orientation.
Both added an incorrect unconditional claim that an all-true mask makes a shallow copy. The source says copy(deep=None); copy/view guarantees were explicitly outside the original questions. Following that call through NDFrame.copy to BlockManager.copy shows that None chooses shallow behavior when Copy-on-Write is enabled and deep behavior otherwise. Both helper files also matched the installed wheel byte for byte.
Two post-reader checks, kept separate from the original 14, used homogeneous integer frames and an ordinary BlockManager. Before a result write, NumPy reported no shared data memory when Copy-on-Write was disabled and shared memory when it was enabled; that write left the original frame unchanged in both cases. This supports the stated configuration distinction for these cases, not every dtype, manager or subclass. The error belongs to reader interpretation, not an incorrect Archify fact. The lesson is to verify consequential extra claims at the helper that owns them.
The original saved JSON is 169,999 bytes. The oriented reader encountered a truncated display and then read the source. A truncated display is not a complete reading of the artifact. Keep the full evidence file, inspect its entry and coverage first, then select dependencies after following the source's helper references.
The following executed projection preserves counts and marks the subset explicitly. On this capture, selecting common.py and indexing.py produced 2,562 bytes and two matching records. This is a documentation recipe, not a new analyzer API or a measured efficiency improvement.
"""Read a saved question without dumping its complete inventories into a tool reply."""
import json
import sys
from pathlib import Path
question = json.loads(Path(sys.argv[1]).read_text())
targets = set(sys.argv[2:])
imports = question['imports']
calls = question['calls']
scope = question['scope']
files = scope['facts']['files']
projection = {
'kind': question['kind'],
'stage': question['stage'],
'entry': question['entry'],
'calls': {key: calls[key] for key in ['coverage', 'reason', 'total', 'included', 'omitted']},
'scope': {'analysisRoot': scope['analysisRoot'],
'capturedLanguage': scope['facts']['language'],
'capturedFiles': len(files) if files is not None else None,
'git': scope['git']},
'imports': {'totalRecords': len(imports),
'resolvedRecords': sum(bool(item.get('resolved')) for item in imports),
'selectedTargets': sorted(targets),
'matchingRecords': [item for item in imports if item.get('to') in targets]},
'note': 'Projection only. Full evidence remains in the input file. File dependencies are not call targets.',
}
print(json.dumps(projection, indent=2))Save it as project-question.py, then run:
python3 project-question.py question-source.json pandas/core/common.py pandas/core/indexing.py > projected-question.jsonThe script below was executed from a directory containing only the script. It fetches fixed public revisions, checks the archive and raw-facts hashes, compares source/archive orientation, installs six verified binary wheels in a temporary venv, runs the 14 finite cases, then runs the two separate post-reader checks. Its SHA256 is a60efee8667385cfd5342d26189c55b681286838c1d5eb74fc7060deb0c7edc7.
Requirements: Git, Node 22.23.2, CPython 3.13.16 with venv/pip, unzip, shasum and network access to GitHub, npm and PyPI. Set PYTHON if the executable is not named python3.13. The successful run used macOS arm64; another platform needs compatible binary wheels for the pinned versions. The script fails rather than building pandas or NumPy from source.
The first replay attempt omitted the target Git origin and therefore produced different repository provenance and a different snapshot/hash despite equal captured arrays. That failed log was retained, checkout setup was corrected to include the canonical origin, and the full final script passed. Provenance was not stripped to force equality. The reader comparison is recorded above; the script does not rerun readers.
Executed standalone replay
#!/bin/sh
set -eu
# Public replay of static orientation and a separate, finite pandas wheel oracle.
test "$(node --version)" = v22.23.2 || { echo 'Node 22.23.2 required' >&2; exit 1; }
PYTHON=${PYTHON:-python3.13}
test "$("$PYTHON" -c 'import platform; print(platform.python_version())')" = 3.13.16 || { echo 'Python 3.13.16 required' >&2; exit 1; }
export BAUIFY_PYTHON="$PYTHON"
WORK=$(mktemp -d "${TMPDIR:-/tmp}/archify-pandas-replay.XXXXXX")
printf 'Workspace: %s\n' "$WORK"
pin() { git init -q "$WORK/$1"; git -C "$WORK/$1" remote add origin "https://github.com/$2.git"; git -C "$WORK/$1" fetch -q --no-tags --depth=1 origin "$3"; git -C "$WORK/$1" checkout -q --detach FETCH_HEAD; test "$(git -C "$WORK/$1" rev-parse HEAD)" = "$3"; }
pin producer tt-a1i/archify 272f995b9a0a38789c1d7d8c76e73ce62636fe5b
pin consumer tt-a1i/archify 29f73e9c30ee72b3f085a2b13caedcaa50d1d4da
pin pandas pandas-dev/pandas 0691c5cf90477d3503834d983f69350f250a6ff7
for a in producer consumer; do (cd "$WORK/$a/archify/modules/code-analysis" && npm ci --ignore-scripts --no-audit --no-fund); done
printf '%s %s\n' 3f0a03a6d523fdf100779a5addc8d57c8e00dda36d854502e3e00d0915e4b22f "$WORK/consumer/archify.zip" | shasum -a 256 -c -
mkdir "$WORK/archive"; unzip -q "$WORK/consumer/archify.zip" -d "$WORK/archive"
(cd "$WORK/archive/archify/modules/code-analysis" && npm ci --ignore-scripts --no-audit --no-fund)
mkdir "$WORK/runtime" "$WORK/wheels"
cd "$WORK"
cat > 'finite-cases.json' <<'PANDAS_EMBED_FINITE-CASES_JSON'
{
"status": "prewritten source expectations; NOT executed",
"baseFixture": {
"kind": "ordinary DataFrame",
"index": [
"r0",
"r1"
],
"columns": [
"A",
"B"
],
"rows": [
[
10,
100
],
[
20,
200
]
]
},
"cases": [
{
"expression": "df[\"A\"]",
"expected": "Series named A, values10,20, indexr0,r1"
},
{
"expression": "df[[\"A\"]]",
"expected": "DataFrame shape2x1, columns[A] in requested order"
},
{
"expression": "df[[\"missing\"]]",
"expected": "KeyError"
},
{
"expression": "df[[True,False]]",
"expected": "DataFrame one rowr0, both columns"
},
{
"expression": "df[[True]]",
"expected": "ValueError Item wrong length"
},
{
"expression": "df[Series([True,False],index=[\"r1\",\"r0\"])]",
"expected": "UserWarning reindex; DataFrame rowr1 after label alignment"
},
{
"expression": "df[Series([True,False],index=[\"r0\",\"extra\"])]",
"expected": "UserWarning then IndexingError, missing target rowr1"
},
{
"expression": "df[Series([True,False,True],index=[\"r0\",\"r1\",\"extra\"])]",
"expected": "UserWarning; successfully align r0,r1 and select rowr0, despite initial Series length3"
},
{
"expression": "df[array([True,None],dtype=object)]",
"expected": "ValueError from is_bool_indexer, before _getitem_bool_array"
},
{
"expression": "df[pd.array([True,pd.NA],dtype=\"boolean\")]",
"expected": "NA becomes false; select rowr0, no blanket NA-error assertion"
},
{
"expression": "duplicateColumns[\"A\"] where columns=[A,A,B]",
"expected": "DataFrame containing both A columns"
},
{
"expression": "duplicateColumns[\"B\"] where columns=[A,A,B]",
"expected": "Series; globally duplicate columns do not force DataFrame for unique selected B"
},
{
"expression": "multiColumns[\"X\"] where columns=[(X,A),(X,B),(Y,A)]",
"expected": "DataFrame subgroup with levels dropped to columnsA,B"
},
{
"expression": "multiColumns[(\"X\",\"A\")] on same MultiIndex",
"expected": "Series for full unique tuple"
}
]
}
PANDAS_EMBED_FINITE-CASES_JSON
cat > 'runtime/probe.py' <<'PANDAS_EMBED_RUNTIME_PROBE_PY'
import json, platform, sys, traceback, warnings
from pathlib import Path
import numpy as np
import pandas as pd
from importlib.metadata import version
here=Path(__file__).resolve().parent
frozen=json.loads((here.parent/'finite-cases.json').read_text())
df=pd.DataFrame([[10,100],[20,200]],index=['r0','r1'],columns=['A','B'])
dup=pd.DataFrame([[10,11,100],[20,21,200]],index=['r0','r1'],columns=['A','A','B'])
multi=pd.DataFrame([[10,11,100],[20,21,200]],index=['r0','r1'],columns=pd.MultiIndex.from_tuples([('X','A'),('X','B'),('Y','A')]))
cases=[lambda:df['A'],lambda:df[['A']],lambda:df[['missing']],lambda:df[[True,False]],lambda:df[[True]],lambda:df[pd.Series([True,False],index=['r1','r0'])],lambda:df[pd.Series([True,False],index=['r0','extra'])],lambda:df[pd.Series([True,False,True],index=['r0','r1','extra'])],lambda:df[np.array([True,None],dtype=object)],lambda:df[pd.array([True,pd.NA],dtype='boolean')],lambda:dup['A'],lambda:dup['B'],lambda:multi['X'],lambda:multi[('X','A')]]
assert len(cases)==len(frozen['cases'])==14
results=[]
for n,fn in enumerate(cases):
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter('always')
try:
result=fn()
record={'type':type(result).__name__,'shape':list(result.shape),'index':result.index.tolist(),'columns':result.columns.tolist() if isinstance(result,pd.DataFrame) else None,'name':result.name if isinstance(result,pd.Series) else None,'values':result.to_numpy().tolist()}
except Exception as error:
frames=traceback.extract_tb(error.__traceback__)
record={'type':'error','errorType':type(error).__name__,'message':str(error),'traceback':[{'file':x.filename,'line':x.lineno,'function':x.name,'source':x.line} for x in frames]}
record['warnings']=[{'type':type(w.message).__name__,'message':str(w.message)} for w in caught]
record['case']=n+1;record['expression']=frozen['cases'][n]['expression'];record['frozenExpected']=frozen['cases'][n]['expected']
results.append(record)
(here/'observations.json').write_text(json.dumps(results,indent=2,ensure_ascii=False)+'\n')
# Compare the recorded outcomes with the independently frozen matrix, without altering it.
checks=[
results[0]['type']=='Series' and results[0]['name']=='A' and results[0]['values']==[10,20] and results[0]['index']==['r0','r1'],
results[1]['type']=='DataFrame' and results[1]['shape']==[2,1] and results[1]['columns']==['A'],
results[2].get('errorType')=='KeyError',
results[3]['type']=='DataFrame' and results[3]['index']==['r0'] and results[3]['columns']==['A','B'],
results[4].get('errorType')=='ValueError' and 'Item wrong length' in results[4].get('message',''),
results[5]['index']==['r1'] and any(w['type']=='UserWarning' for w in results[5]['warnings']),
results[6].get('errorType')=='IndexingError' and any(w['type']=='UserWarning' for w in results[6]['warnings']),
results[7]['index']==['r0'] and any(w['type']=='UserWarning' for w in results[7]['warnings']),
results[8].get('errorType')=='ValueError' and any(x['function']=='is_bool_indexer' for x in results[8].get('traceback',[])) and not any(x['function']=='_getitem_bool_array' for x in results[8].get('traceback',[])),
results[9]['type']=='DataFrame' and results[9]['index']==['r0'],
results[10]['type']=='DataFrame' and results[10]['columns']==['A','A'],
results[11]['type']=='Series' and results[11]['name']=='B',
results[12]['type']=='DataFrame' and results[12]['columns']==['A','B'],
results[13]['type']=='Series' and list(results[13]['name'])==['X','A'],
]
identity=getattr(pd,'__git_version__',None)
summary={'python':sys.version,'platform':platform.platform(),'versions':{x:version(x) for x in ['pandas','numpy','python-dateutil','pytz','tzdata','six']},'pandasGitVersion':identity,'targetCommit':'0691c5cf90477d3503834d983f69350f250a6ff7','binarySourceIdentity':'exact commit reported' if identity=='0691c5cf90477d3503834d983f69350f250a6ff7' else 'gap: wheel did not report the pinned commit','matches':checks,'matchedCount':sum(checks),'caseCount':len(checks)}
(here/'runtime-summary.json').write_text(json.dumps(summary,indent=2)+'\n')
print(json.dumps(summary,indent=2))
if not all(checks): raise SystemExit('Observed discrepancies retained in observations.json')
PANDAS_EMBED_RUNTIME_PROBE_PY
node "$WORK/producer/archify/bin/archify.mjs" code-analysis check "$WORK/pandas" --language py --fail-on never --json --out "$WORK/capture" > "$WORK/check-receipt.json" 2> "$WORK/check-stderr.txt"
printf '%s %s\n' c8660cb998c202b1432d8681949e386f91c0a5e7fa17babecfaf9c7bc206a5a5 "$WORK/capture/raw-facts.json" | shasum -a 256 -c -
QUESTION='For an ordinary DataFrame how do column, boolean row and duplicate or MultiIndex selection differ?'
for source in consumer archive; do
node "$WORK/$source/archify/bin/archify.mjs" code-analysis task "$WORK/capture" --question "$QUESTION" --entry pandas/core/frame.py --function 'DataFrame.__getitem__' --root "$WORK/pandas" --format json --out "$WORK/question-$source.json" --json > "$WORK/question-$source-receipt.json" 2> "$WORK/question-$source-stderr.txt"
done
node --input-type=module <<'PANDAS_CHECK'
import fs from 'node:fs';import assert from 'node:assert/strict';
const read=p=>JSON.parse(fs.readFileSync(p,'utf8'));const f=read('capture/raw-facts.json');
assert.equal(f.files.length,1510);assert.equal(f.imports.length,14316);assert.equal(f.functions.length,29431);assert.equal((f.parse_errors??[]).length,0);
const a=read('question-consumer.json'), b=read('question-archive.json');
for(const x of [a,b]){assert.equal(x.entry.currentStatus,'ok');assert.equal(x.entry.factsStatus,'valid');assert.deepEqual(x.entry.location,{file:'pandas/core/frame.py',symbol:'DataFrame.__getitem__',startLine:4062,endLine:4128});assert.equal(x.imports.length,162);const resolved=x.imports.filter(i=>i.resolved);assert.equal(resolved.length,142);assert.ok(resolved.every(i=>i.target?.currentStatus==='ok'&&i.target?.factsStatus==='valid'&&!i.target.parseErrors.length));assert.equal(x.calls.coverage,'unavailable');assert.equal(x.calls.total,null);}
delete a.workflow;delete b.workflow;assert.deepEqual(a,b);console.log('Static: 1510 files, 14316 imports, 29431 functions, 0 parse errors; entry 4062–4128, 162 imports/142 fresh; call coverage unavailable; source/archive orientation equal except workflow.');
PANDAS_CHECK
"$PYTHON" -m venv "$WORK/venv"
"$WORK/venv/bin/python" -m pip download --only-binary=:all: --no-deps --dest "$WORK/wheels" pandas==2.2.3 numpy==2.1.3 python-dateutil==2.9.0.post0 pytz==2024.2 tzdata==2024.2 six==1.16.0
"$WORK/venv/bin/python" - <<'PANDAS_WHEELS'
import hashlib,json,pathlib,urllib.request,zipfile
versions={'pandas':'2.2.3','numpy':'2.1.3','python-dateutil':'2.9.0.post0','pytz':'2024.2','tzdata':'2024.2','six':'1.16.0'}
paths=list(pathlib.Path('wheels').glob('*.whl')); assert len(paths)==6
result=[]
for name,version in versions.items():
with urllib.request.urlopen(f'https://pypi.org/pypi/{name}/{version}/json',timeout=60) as response: meta=json.load(response)
found=[p for p in paths if p.name in {x['filename'] for x in meta['urls']}]; assert len(found)==1,(name,found)
p=found[0]; actual=hashlib.sha256(p.read_bytes()).hexdigest(); official=next(x['digests']['sha256'] for x in meta['urls'] if x['filename']==p.name);assert actual==official,(name,p.name)
result.append({'package':name,'version':version,'filename':p.name,'sha256':actual,'officialPyPI':True})
if name=='pandas':
files=['pandas/core/frame.py','pandas/core/common.py','pandas/core/indexing.py','pandas/core/indexers/utils.py','pandas/core/indexes/base.py']
with zipfile.ZipFile(p) as z:
for f in files: assert z.read(f)==pathlib.Path('pandas',f).read_bytes(),f
pathlib.Path('wheel-verification.json').write_text(json.dumps({'wheels':result,'pandasPythonFilesByteIdentical':files},indent=2)+'\n')
print('Six binary wheel hashes match official PyPI metadata; five inspected pandas Python files match pinned source.')
PANDAS_WHEELS
"$WORK/venv/bin/python" -m pip install --no-index --find-links "$WORK/wheels" --no-deps pandas==2.2.3 numpy==2.1.3 python-dateutil==2.9.0.post0 pytz==2024.2 tzdata==2024.2 six==1.16.0
(cd "$WORK/runtime" && "$WORK/venv/bin/python" probe.py)
"$WORK/venv/bin/python" - <<'PANDAS_RESULT'
import json
x=json.load(open('runtime/runtime-summary.json'));assert x['pandasGitVersion']=='0691c5cf90477d3503834d983f69350f250a6ff7';assert x['caseCount']==14 and x['matchedCount']==14 and all(x['matches']);print('Runtime: 14/14 finite cases; wheel reports pinned commit.')
PANDAS_RESULT
# Post-reader source-audit follow-up; separate from frozen 14-case matrix.
mkdir "$WORK/post-reader-follow-up"
cat > "$WORK/post-reader-follow-up/copy-follow-up.py" <<'PANDAS_POST_READER_COPY'
"""Post-reader follow-up, separate from the original frozen 14-case matrix."""
import json
from pathlib import Path
import numpy as np
import pandas as pd
assert pd.__version__ == '2.2.3'
assert pd.__git_version__ == '0691c5cf90477d3503834d983f69350f250a6ff7'
here = Path(__file__).resolve().parent
# Source expectations established before these two executions:
# frame.py:4151-4152 -> generic.py:6811 -> managers.py:571-577.
expected = {'origin': 'post-reader source audit; not the original frozen matrix',
'manager': 'BlockManager', 'sharesMemoryBeforeWrite': {False: False, True: True}}
(here / 'copy-follow-up-expected.json').write_text(json.dumps(expected, indent=2) + '\n')
results = []
for cow in [False, True]:
with pd.option_context('mode.copy_on_write', cow):
frame = pd.DataFrame({'A': [10, 20], 'B': [100, 200]})
assert type(frame._mgr).__name__ == 'BlockManager'
selected = frame[[True, True]]
shares = bool(np.shares_memory(frame.to_numpy(), selected.to_numpy()))
assert shares == expected['sharesMemoryBeforeWrite'][cow]
selected.iloc[0, 0] = 999
assert frame.iloc[0, 0] == 10
results.append({'copyOnWrite': cow, 'manager': type(frame._mgr).__name__,
'sharesMemoryBeforeWrite': shares,
'originalValueAfterSelectedWrite': int(frame.iloc[0, 0]),
'selectedValueAfterWrite': int(selected.iloc[0, 0])})
(here / 'copy-follow-up-results.json').write_text(json.dumps(results, indent=2) + '\n')
print(json.dumps(results, indent=2))
PANDAS_POST_READER_COPY
(cd "$WORK/post-reader-follow-up" && "$WORK/venv/bin/python" copy-follow-up.py)
"$WORK/venv/bin/python" - <<'PANDAS_POST_READER_CHECK'
import json
r=json.load(open('post-reader-follow-up/copy-follow-up-results.json'))
assert len(r)==2 and [x['copyOnWrite'] for x in r]==[False,True]
assert [x['sharesMemoryBeforeWrite'] for x in r]==[False,True]
assert all(x['manager']=='BlockManager' and x['originalValueAfterSelectedWrite']==10 and x['selectedValueAfterWrite']==999 for x in r)
print('Post-reader follow-up: both finite BlockManager/CoW cases matched; separate from frozen 14.')
PANDAS_POST_READER_CHECKSet WORK to the workspace path printed by the public replay above. Save the helper below as verify-pandas-cycles.py, then run python3.13 -I -B verify-pandas-cycles.py "$WORK". It reads existing source, captured JSON and Git metadata; it does not execute pandas or rerun extraction. The command passed against the already completed fresh public replay: two finding identities, seven files and ten source sites, plus the native-child boundary. It does not verify the entire 150-file component. Helper SHA256: 10b15bf0f002ce56418053fde230033aa23607407903d1706b79fc198e8bdf00.
Executed finite cycle-evidence check
#!/usr/bin/env python3
"""Verify two finite pandas v2.2.3 cycle examples after the public replay.
Usage: python3.13 -I -B verify-pandas-cycles.py WORK
WORK must contain the replay's pandas/ checkout and capture/ directory.
Reads source/JSON/Git metadata only; never imports pandas or writes into WORK.
This is a pinned evidence check, not a general cycle validator.
"""
import argparse
import ast
import hashlib
import json
import os
from pathlib import Path
import subprocess
import sys
TARGET = '0691c5cf90477d3503834d983f69350f250a6ff7'
RAW_SHA256 = 'c8660cb998c202b1432d8681949e386f91c0a5e7fa17babecfaf9c7bc206a5a5'
SNAPSHOT = 'f83d7b542dfce50f723d0d82406812e040e68b0512024f33b492efa9ece6ebb7'
IDS = ['COUP-dfc0f62edb50', 'COUP-4429223b198a']
# Displayed paths, binding-order candidate, and one nearby typing control only.
SITES = [
('pandas/__init__.py', 175, 'pandas/io/json/_normalize.py', False),
('pandas/io/json/_normalize.py', 20, 'pandas/__init__.py', False),
('pandas/io/json/_normalize.py', 21, 'pandas/__init__.py', False),
('pandas/__init__.py', 49, 'pandas/core/api.py', False),
('pandas/core/api.py', 23, 'pandas/core/algorithms.py', False),
('pandas/core/algorithms.py', 36, 'pandas/core/dtypes/cast.py', False),
('pandas/core/dtypes/cast.py', 86, 'pandas/io/_util.py', False),
('pandas/io/_util.py', 7, 'pandas/__init__.py', False),
('pandas/_libs/tslibs/__init__.py', 39, 'pandas/_libs/tslibs/__init__.py', False),
('pandas/core/dtypes/cast.py', 103, 'pandas/__init__.py', True),
]
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('work', type=Path, help='existing public replay WORK directory')
work = parser.parse_args().work.resolve()
root = work / 'pandas'
raw = (work / 'capture/raw-facts.json').read_bytes()
assert hashlib.sha256(raw).hexdigest() == RAW_SHA256, 'unexpected raw-facts hash'
facts = json.loads(raw)
repo = facts['repository']
assert repo['baseRevision'] == TARGET and repo['snapshotId'] == SNAPSHOT
assert (repo['language'], repo['adapter'], repo['adapterRevision']) == ('py', 'python@3.13.16', 5)
env = {**os.environ, 'GIT_OPTIONAL_LOCKS': '0'}
def git(*args):
return subprocess.run(['git', '-C', str(root), *args], check=True,
capture_output=True, text=True, env=env).stdout.strip()
assert git('rev-parse', 'HEAD') == TARGET, 'unexpected pandas checkout'
assert not git('status', '--short'), 'pandas checkout must remain clean'
findings = json.loads((work / 'capture/findings.json').read_text(encoding='utf-8'))
selected = [next(d for d in findings['diagnostics'] if d['id'] == ident) for ident in IDS]
for finding, count in zip(selected, [150, 1]):
assert finding['code'] == 'coupling/import-cycle'
assert finding['severity'] == 'warning' and finding['evidence']['kind'] == 'eager'
assert len(finding['subject']['files']) == count
candidate = selected[0]['evidence']['partialInitCandidate']
assert candidate == {
'entry': 'pandas/core/api.py', 'module': 'pandas.core.api', 'name': 'BooleanDtype',
'boundAt': 29, 'viaLine': 23, 'importer': 'pandas/__init__.py', 'line': 49,
'chain': ['pandas/core/api.py', 'pandas/core/algorithms.py',
'pandas/core/dtypes/cast.py', 'pandas/io/_util.py', 'pandas/__init__.py'],
}
assert selected[0]['evidence']['path'] == ['pandas/__init__.py', 'pandas/io/json/_normalize.py']
assert selected[1]['evidence']['path'] == ['pandas/_libs/tslibs/__init__.py']
captured = {f['path']: f for f in facts['files']}
texts, trees, parents = {}, {}, {}
rows = []
for file, line, target, type_only in SITES:
if file not in texts:
text = (root / file).read_text(encoding='utf-8')
assert text == captured[file]['sourceText'], f'changed captured text: {file}'
texts[file] = text
tree = trees[file] = ast.parse(text, filename=file)
parents[file] = {child: node for node in ast.walk(tree)
for child in ast.iter_child_nodes(node)}
nodes = [n for n in ast.walk(trees[file])
if isinstance(n, (ast.Import, ast.ImportFrom)) and n.lineno == line]
assert len(nodes) == 1, (file, line)
node = nodes[0]
chain = []
parent = parents[file].get(node)
while parent is not None:
item = {'kind': type(parent).__name__}
if isinstance(parent, ast.If):
item['test'] = ast.unparse(parent.test)
chain.append(item)
parent = parents[file].get(parent)
edges = [e for e in facts['imports'] if e['from'] == file and e['line'] == line
and e.get('to') == target and not e.get('implicit')]
assert len(edges) == 1, (file, line, target)
edge = edges[0]
assert edge['resolved'] and bool(edge.get('typeOnly')) == type_only
if not type_only:
assert chain == [{'kind': 'Module'}], (file, line, chain)
assert not edge.get('lazy') and not edge.get('conditional')
else:
assert any(a.get('test') == 'TYPE_CHECKING' for a in chain)
rows.append({'file': file, 'line': line, 'to': target, 'typeOnly': type_only,
'statement': ast.get_source_segment(texts[file], node)})
assert next(s for s in facts['symbols'] if s['file'] == 'pandas/core/api.py'
and s['name'] == 'BooleanDtype')['line'] == 29
child = 'pandas/_libs/tslibs/dtypes.pyx'
build = 'pandas/_libs/tslibs/meson.build'
assert (root / child).is_file() and child not in captured
assert not (root / 'pandas/_libs/tslibs/dtypes.py').exists()
assert "'dtypes': {'sources': ['dtypes.pyx']}" in (root / build).read_text(encoding='utf-8')
assert any(e['from'] == 'pandas/_libs/tslibs/__init__.py' and e['line'] == 41
and e['specifier'] == 'pandas._libs.tslibs.dtypes' and not e['resolved']
for e in facts['imports'])
print(json.dumps({
'status': 'ok', 'work': str(work), 'python': sys.version.split()[0],
'target': TARGET, 'rawFactsSha256': RAW_SHA256, 'snapshotId': SNAPSHOT,
'selectedFindingIds': IDS, 'reportedComponentSizes': [150, 1],
'inspectedPythonFiles': len(texts), 'verifiedImportSites': len(rows),
'unguardedSites': sum(not r['typeOnly'] for r in rows), 'typingControl': True,
'nativeChildExcluded': True, 'sites': rows,
'limits': ['Only these two findings and ten sites were checked; the whole 150-file SCC was not adjudicated.',
'Static dependencies and a binding-order hypothesis are not a runtime failure proof.',
'No pandas/fixture runtime, dependency installation, extraction, or product tests were run.'],
}, indent=2))
if __name__ == '__main__':
main()AI-assisted investigation and writing; source, runtime checks and interpretation limits are stated separately. See saved language questions and the validation index.