Spring cleanup for v3

[jakobnissen]

BioSequences is a large package. Probably larger than it needs to be. It seems like a hold-over from the olden days when BioJulia was much more monolithic similar to e.g. Biopython.

In my opinion, it's easier and cleaner if BioSequences only contain the sequence types itself, and the most basic processing on them. Nothing fancy.

Are there things we can move out of BioSequences for v3? Here are my suggestions. If anyone thinks any of this belongs in this package, please leave a comment

Functionality to remove

• src/composition.jl
• src/demultiplexer.jl
• src/minhash.jl
• src/nmask.jl
• src/refseq
• ConditionIterator
• VoidAlphabet
• CharAlphabet

Functionality to migrate to other packages

• K-mer composition
• Barcode demultiplexing
• K-mer minhashing

Dependencies to remove
These are either not used, or can be removed with trivial changes

• BioGenerics
• Printf
• Combinatorics
• IndexableBitVectors

A few comments to these

composition and minhash

These kmer techniques really belong in another package. There are so many cool kmer bithacks, and so much to do with kmers. We can't possibly do it all in this package.
I'd much rather have a handful of kmer iterators in this package, and leave it at that. Other packages can then build on the kmer iterators and sequences from here.

NMask and RefSeq

Do people really use these? It seems to me one might as well use LongDNASeq or its 2-bit equivalent. Theoretically, there is some space saving possible by using the NMask. Practically speaking, I doubt we have the dev manpower to maintain yet another longsequence-like sequence.

Demultiplexer

This seems very application-specific, and should probably be part of some NGS-related package.

ConditionIterator

This just seems like a re-implementation of Iterators.Filter, with much less functionality.

VoidAlphabet and CharAlphabet

I can't see the point of VoidAlphabet at all. CharAlphabet is useful for testing the generic interfaces, so I propose we move it from src to test.

[kescobo]

This all makes sense to me. Regarding Kmers - @benjward had some other kmer-like type (skipmer maybe?), and there's also https://github.com/BioJulia/KmerAnalysis.jl and https://github.com/BioJulia/NTupleKmers.jl

This maybe warrants a different discussion, but there are A LOT of packages in this org. As part of spring cleaning, we may want to go through and add deprecated or inactive badges to some of them, and probably archive them too

[jakobnissen]

Good call. Skimming through the packages, many are unmaintained. It hurts the org's reputation when users find non-archived, yet unmaintained packages.
However, I don't feel comfortable archiving other people's packages unless they've gone inactive. We should get @benjward in on this (and @bicycle1885 on this, if he is reachable).

[pdimens]

I agree with splitting. It would be useful to, as you've suggested, pop out the kmer things into it's own package, like KmerLib.jl or something of that sort. I like the idea of non-monolithic packages, and kmers have such ubiquitous utility that it would be a lot easier to keep that self-contained and import it into other packages as needed.

[kescobo]

Started a discussion about org cleanup here: https://github.com/orgs/BioJulia/teams/developers/discussions/1

[kescobo]

Oh, that might be only accessible to people on the developer's team 🤔

[TransGirlCodes]

Kmers have been removed, and will be migrating to Kmers.jl I've got a branch over there with this ongoing. I think I'll have it depend on v3 BioSequences right off the bat.

[TransGirlCodes]

On NMask and RefSeq, personally, I've never used them. And when it comes to storing and working with genome assemblies in BioJulia, I plan on it being graph centric with node sequences buffered and fetched on the fly from disk (as I do with read datastores) so a specific reference genome type is probably not really needed going forward. In my graph representation, N's will not be nessecery as such gaps are represented by links between sequences with non negative distances.

[CiaranOMara]

Is not NMask and RefSeq used by TwoBit? In which case I use them.

[TransGirlCodes]

Yes it may be.

[TransGirlCodes]

Perhaps TwoBit can implement its own sequence type, for those who use it and absolutely need it. In the same way Kmers are being moved out to Kmers.jl

[TransGirlCodes]

All of the tasks in the issue have been ticked. @jakobnissen Is this ready to close/resolve?