Skip to content

TdtRawIO: multi-block SEV files not found (wrong path + case mismatch) #1814

@ree-gupta

Description

@ree-gupta

Describe the bug
TdtRawIO._parse_header fails to find .sev files in multi-block TDT datasets on case-sensitive filesystems (Linux). Two bugs cause a silent fallback to .tev, resulting in all channels reading identical (wrong) data.

  1. Wrong search directory: The path variable at

sev_filename = (path / sev_stem).with_suffix(".sev")

never points to the block subdirectory. With default sortname="", it's a stale loop variable from line

tankname = self.dirname.stem
for path in self.dirname.iterdir():
if is_tdtblock(path):
segment_names.append(path.stem)

With a sortname set, it gets reassigned to self.dirname (the tank root) at line

# If there exists an external sortcode in ./sort/[sortname]/*.SortResult
# (generated after offline sorting)
if self.sortname != "":
try:
if self.tdt_block_mode == "multi":
path = self.dirname
else:
path = self.dirname.parent
for file in os.listdir(path / "sort" / self.sortname):
if file.endswith(".SortResult"):
sortresult_filename = path / "sort" / self.sortname / file

In both cases, SEV files live in self.dirname / segment_name (e.g., tank/Block-1/), not where path points.

  1. Case mismatch: The multi-block path constructs filenames with lowercase _ch (ch1.sev),

# path = self.dirname / segment_name
if self.tdt_block_mode == "multi":
# for multi block datasets the names of sev files are fixed
sev_stem = f"{tankname}_{segment_name}_{stream_name}_ch{chan_id}"
sev_filename = (path / sev_stem).with_suffix(".sev")

but for the data that I have TDT writes it with uppercase _Ch (Ch1.sev). On Linux, the file is not found. The single-block code path already handles this with a [cC]h glob patter, the multi-block path just missed it.

Both bugs cause .sev lookup to fail silently. Neo falls back to .tev and all channels read identical wrong data. No error or warning is raised.

To Reproduce
I don't have a publicly shareable multi-block dataset with .sev files yet, but the bug is visible from the code. The existing test datasets don't cover this path: aep_05 is multi-block but has no .sev files; dataset_0_single_block has .sev files but goes through the (correct) single-block code path.

I can also contribute a trimmed multi-block test dataset with .sev files to ephy_testing_data once I have explicit permission from the data provider, so this code path gets regression-tested.

Expected behaviour
Each column of the returned array should contain distinct per-channel data read from the corresponding.sevfile.

Environment:

  • OS: Linux (Debian 13, case-sensitive ext4)
  • Python version: 3.12
  • Neo version: 0.14.3
  • NumPy version: 1.26

Additional context

I've verified a minimal fix locally. Will open a PR with it soon.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions