Error with flongle flow cell splitting

[Dcwrenn]

Hello ReadFish folks,
We've run into an issue while trying to run a ReadFish experiment on a flongle. When attempting to split the flow cell into two regions we get the following error:

| Traceback (most recent call last):
|   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 378, in generate_flowcell
|     arr = np.array(np.split(arr, split, axis=axis))
|   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/numpy/lib/shape_base.py", line 864, in split
|     raise ValueError(
| ValueError: array split does not result in an equal division
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
|   File "<cattrs generated structure readfish._config.Conf>", line 33, in structure_Conf
|     return __cl(
|   File "<attrs generated init readfish._config.Conf>", line 14, in __init__
|     self.__attrs_post_init__()
|   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 284, in __attrs_post_init__
|     split_channels = generate_flowcell(self.channels, len(self.regions) or 1)
|   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 384, in generate_flowcell
|     raise ValueError("The flowcell cannot be split evenly")
| ValueError: The flowcell cannot be split evenly

When we've adjusted our TOML to only have one region we get the following error:

Traceback (most recent call last):
  File "/home/dcwrenn/miniconda3/envs/readfish/bin/readfish", line 8, in <module>
    sys.exit(main())
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_cli_base.py", line 61, in main
    raise SystemExit(args.func(parser, args, extras))
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/entry_points/targets.py", line 624, in run
    logger.info(conf.describe_experiment())
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 468, in describe_experiment
    {draw_flowcell_split(self.channels, split, index=index)}"""
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 326, in draw_flowcell_split
    row, col = get_coords(pos, flowcell_size)
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 211, in get_coords
    raise ValueError("channel cannot be below 0 or above flowcell_size")
ValueError: channel cannot be below 0 or above flowcell_size

This is a new issue since updating to the newest MinKNOW (v.24.02.16) and ReadFish (v.2024.2.0) versions. We get no errors when using the same TOML with a MinION flow cell. readfish validate reports no issue with the TOML itself and outputs two clear regions. Our TOML with 2 regions is below. Thank you for your help! 🐟

# Basecaller configuration
[caller_settings.dorado]
config = "dna_r10.4.1_e8.2_400bps_5khz_fast.cfg"
address = "ipc:///tmp/.guppy/5555"

# Aligner Configuration
[mapper_settings.mappy]
fn_idx_in = "/data/dynamic_pipeline/dart01_20240604h/readfish/references/2024-06-04_14-14-42_reference.mmi"
n_threads = 4

[[regions]]
name = "control"
control = true
min_chunks = 1
max_chunks = 2
targets = []
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "stop_receiving"
multi_off = "stop_receiving"
no_seq = "stop_receiving"
no_map = "stop_receiving"
above_max_chunks = "stop_receiving"
below_min_chunks = "stop_receiving"

[[regions]]
name = "dynamic01"
control = false
min_chunks = 1
max_chunks = 2
targets = []
single_on = "unblock"
multi_on = "unblock"
single_off = "stop_receiving"
multi_off = "stop_receiving"
no_seq = "proceed"
no_map = "proceed"
above_max_chunks = "unblock"
below_min_chunks = "proceed"

[github-actions]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[mattloose]

HI,

Thanks for raising this - we will have a look. We tend not to run on flongles for a variety of reasons.

Will post here when the issue is resolved.

Matt

[mattloose]

OK - we've found the issue - it is a bug due to the unusual size of the flongle flowcell.

We will fix this and implement in the next release of readfish. If you want to manually make this work for your flongle, please edit this line in your local copy of the code:

readfish/src/readfish/_utils.py

Line 284 in 8e5210e

flowcell_size: int, split: int = 1, axis: int = 1, index: int = 0

and change
axis: int = 1
to
axis: int = 0

The reason the validate command works is that it does not the flowcell shape - this is only done when you run with a live flowcell.

[Dcwrenn]

Hi Matt,
Thank you for your quick response. Apologies for my delay in getting back to you - we've just been able to try the manual fix you suggested. We are still getting errors after adjusting to axis: int = 0. The new errors we are seeing are below:

Traceback (most recent call last):
  File "<cattrs generated structure readfish._config.Conf>", line 33, in structure_Conf
    return __cl(
  File "<attrs generated init readfish._config.Conf>", line 14, in __init__
    self.__attrs_post_init__()
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 284, in __attrs_post_init__
    split_channels = generate_flowcell(self.channels, len(self.regions) or 1)
  File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 384, in generate_flowcell
    raise ValueError("The flowcell cannot be split evenly")
ValueError: The flowcell cannot be split evenly

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/home/dcwrenn/miniconda3/envs/readfish/bin/readfish", line 8, in <module>
  |     sys.exit(main())
  |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_cli_base.py", line 61, in main
  |     raise SystemExit(args.func(parser, args, extras))
  |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/entry_points/targets.py", line 616, in run
  |     conf = Conf.from_file(args.toml, read_until_client.channel_count, logger=logger)
  |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 376, in from_file
  |     return cls.from_dict(dict_, channels)
  |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 394, in from_dict
  |     return conv.structure(dict_, cls)
  |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/cattrs/converters.py", line 332, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure readfish._config.Conf>", line 36, in structure_Conf
  |     except Exception as exc: raise __c_cve('While structuring ' + 'Conf', [exc], __cl)
  | cattrs.errors.ClassValidationError: While structuring Conf (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 378, in generate_flowcell
    |     arr = np.array(np.split(arr, split, axis=axis))
    |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/numpy/lib/shape_base.py", line 864, in split
    |     raise ValueError(
    | ValueError: array split does not result in an equal division
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "<cattrs generated structure readfish._config.Conf>", line 33, in structure_Conf
    |     return __cl(
    |   File "<attrs generated init readfish._config.Conf>", line 14, in __init__
    |     self.__attrs_post_init__()
    |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_config.py", line 284, in __attrs_post_init__
    |     split_channels = generate_flowcell(self.channels, len(self.regions) or 1)
    |   File "/home/dcwrenn/miniconda3/envs/readfish/lib/python3.10/site-packages/readfish/_utils.py", line 384, in generate_flowcell
    |     raise ValueError("The flowcell cannot be split evenly")
    | ValueError: The flowcell cannot be split evenly
    +------------------------------------

[Adoni5]

Hi @Dcwrenn, apologies that should read:

readfish/src/readfish/_utils.py

Lines 331 to 333 in 8e5210e

	def generate_flowcell(
	flowcell_size: int, split: int = 1, axis: int = 1, odd_even: bool = False
	) -> list[list[int]]:

axis: int = 1
to
axis: int = 0

What we've changed above will let us draw an ASCII split flow cell to the CLI, so that is also required!
Cheers.
Rory

[Dcwrenn]

@Adoni5 Apologies again for the delayed response - we are back up and running! We ended up using a combination of the solution provided in this issue and the solution provided in issue 365.

For others that may encounter the same problem we first made a clean install of readfish using pip install git+https://github.com/LooseLab/readfish@issue365/flongle-draw-split and then changed axis: int = 1 to axis: int = 0 in lines 287 and 349 of _utils.py

Thank you for all of your help!