Standard set of data-loaders for training and making predictions for DNA sequence-based models.
Clone or download
Avsecz Merge pull request #20 from kipoi/remove_some_deps
remove h5py and related dependencies
Latest commit 37daca6 Nov 1, 2018

README.md

kipoiseq

CircleCI Coverage status

Standard set of data-loaders for training and making predictions for DNA sequence-based models.

All dataloaders in kipoiseq.dataloaders decorated with @kipoi_dataloader (SeqIntervalDl and StringSeqIntervalDl) are compatible Kipoi models and can be directly used when specifying a new model in model.yaml:

...
default_dataloader:
  defined_as: kipoiseq.dataloaders.SeqIntervalDl
  default_args:
    auto_resize_len: 1000 # override default args in SeqIntervalDl
    
dependencies:
  pip:
    - kipoiseq
...

Installation

pip install kipoiseq

Getting started

from kipoiseq.dataloaders import SeqIntervalDl

dl = SeqIntervalDl.init_example()  # use the provided example files
# your own files
dl = SeqIntervalDl("intervals.bed", "genome.fa")

len(dl)  # length of the dataset

dl[0]  # get one instance. # returns a dictionary: 
# dict(inputs=<one-hot-encoded-array>, 
#      targets=<additional columns in the bed file>, 
#      metadata=dict(ranges=GenomicRanges(chr=, start, end)...

all = dl.load_all()  # load the whole dataset

# load batches of data
it = dl.batch_iter(32, num_workers=8)  # load batches of data in parallel using 8 workers
# returns a dictionary with all three keys: inputs, targets, metadata

it = dl.batch_train_iter(32, num_workers=8)
# returns a tuple: (inputs, targets), can be used directly with keras' `model.fit_generator`

More info:

How to write your own data-loaders