-
Notifications
You must be signed in to change notification settings - Fork 266
Description
Describe the bug
TdtRawIO._parse_header fails to find .sev files in multi-block TDT datasets on case-sensitive filesystems (Linux). Two bugs cause a silent fallback to .tev, resulting in all channels reading identical (wrong) data.
- Wrong search directory: The
pathvariable at
python-neo/neo/rawio/tdtrawio.py
Line 290 in c672dbd
| sev_filename = (path / sev_stem).with_suffix(".sev") |
never points to the block subdirectory. With default sortname="", it's a stale loop variable from line
python-neo/neo/rawio/tdtrawio.py
Lines 102 to 106 in c672dbd
| tankname = self.dirname.stem | |
| for path in self.dirname.iterdir(): | |
| if is_tdtblock(path): | |
| segment_names.append(path.stem) | |
With a sortname set, it gets reassigned to self.dirname (the tank root) at line
python-neo/neo/rawio/tdtrawio.py
Lines 165 to 175 in c672dbd
| # If there exists an external sortcode in ./sort/[sortname]/*.SortResult | |
| # (generated after offline sorting) | |
| if self.sortname != "": | |
| try: | |
| if self.tdt_block_mode == "multi": | |
| path = self.dirname | |
| else: | |
| path = self.dirname.parent | |
| for file in os.listdir(path / "sort" / self.sortname): | |
| if file.endswith(".SortResult"): | |
| sortresult_filename = path / "sort" / self.sortname / file |
In both cases, SEV files live in self.dirname / segment_name (e.g., tank/Block-1/), not where path points.
- Case mismatch: The multi-block path constructs filenames with lowercase _ch (ch1.sev),
python-neo/neo/rawio/tdtrawio.py
Lines 286 to 290 in c672dbd
| # path = self.dirname / segment_name | |
| if self.tdt_block_mode == "multi": | |
| # for multi block datasets the names of sev files are fixed | |
| sev_stem = f"{tankname}_{segment_name}_{stream_name}_ch{chan_id}" | |
| sev_filename = (path / sev_stem).with_suffix(".sev") |
but for the data that I have TDT writes it with uppercase _Ch (Ch1.sev). On Linux, the file is not found. The single-block code path already handles this with a [cC]h glob patter, the multi-block path just missed it.
Both bugs cause .sev lookup to fail silently. Neo falls back to .tev and all channels read identical wrong data. No error or warning is raised.
To Reproduce
I don't have a publicly shareable multi-block dataset with .sev files yet, but the bug is visible from the code. The existing test datasets don't cover this path: aep_05 is multi-block but has no .sev files; dataset_0_single_block has .sev files but goes through the (correct) single-block code path.
I can also contribute a trimmed multi-block test dataset with .sev files to ephy_testing_data once I have explicit permission from the data provider, so this code path gets regression-tested.
Expected behaviour
Each column of the returned array should contain distinct per-channel data read from the corresponding.sevfile.
Environment:
- OS: Linux (Debian 13, case-sensitive ext4)
- Python version: 3.12
- Neo version: 0.14.3
- NumPy version: 1.26
Additional context
I've verified a minimal fix locally. Will open a PR with it soon.