FolkTuneFinder ABC Tools
Tools for working with ABC Notation files (http://abcnotation.com) created whilst making FolkTuneFinder.com. This is just a bit of fun to try out Rust language. It might go nowhere.
- Well documented, fully tested and generally friendly to a newcomer to the codebase.
- Parser should be absolutely as friendly as possible, providing hints on error.
- All useful parts should be available as a Rust library for other people to build on.
- Parser should be decoupled from utils.
- Tools should be self-contained in a single repository, with minimal dependencies, and easy to distribute.
- Successfully parse all valid files in the FolkTuneFinder corpus.
- High test coverage.
- All entities commented.
- Automatic developerment.
- No warnings.
- Formatted with rustfmt for consistency.
This will be a general purpose ABC tool. It may provide a range of functionality:
- Verify and clean ABC files.
- Rudimentary visualisation via export to SVG.
- Similarity clustering.
- MIDI output.
- Handle Windows newline characters.
- Handle escape sequences for LaTeX accents.
- Handle escaped closing square brackets for inline fields.
- Run over entire folktunefinder.com corpus and make sure all parse errors are well-known (i.e. no UnknownErorrs).
- Uniformalize line endings (based on stats or configurable).
- Strip and normalize whitespace around headers.
- Sort headers.
- Lowest common denomenator in time signature.
- Shortest possible notation for notes (/ and /2, dotted rhythm).
- Remove empty text fields.
- Normalize to Unicode or to escape sequence ASCII.
- 4/4 => C etc
$ cat test_resources/so-far.abc | target/debug/abctool check There was 1 error! | M: > ^-- I've got to the end of the ABC tune before I'm ready. I was in the middle of reading a time signature | There was 1 error! | M:3 > ^-- I expected to find a slash for the time signature. | There was 1 error! | M:3/ > ^-- I expected to find a number here. | There was 1 error! | M:3/4 | > ^-- I expected to find a header, but found something else. | There were 2 errors! | M:23456789012/1234567890 > ^-- This number is longer than I expected. | T:Hello | M:1111 > ^-- I expected to find a slash for the time signature. | T:This | There were 2 errors! | M:23456789012/1234567890 > ^-- This number is longer than I expected. | T:Hello | M:1111 > ^-- I expected to find a slash for the time signature. | T:This | There was 1 error! | M: > ^-- I've got to the end of the ABC tune before I'm ready. I was in the middle of reading a time signature
$ cat test_resources/butterfly.abc | target/debug/abctool typeset
With minims and crotchets:
Scan Tune Database
A database of ABC tunes is stored in a cache. They are read from the filesystem in the directory specified by the
BASE evironment variable. Files can be anywhere in the directory hierarchy, but should each have distinct numerical names, such as
To update the tune database:
BASE=/path/to/abcs cargo run scan
The tunecache file will be stored at
/path/to/abcs/tunecache. When new tunes are added, run re-scan. Only new files will be added. It is a simple concatenation of the files into one blob, with tune IDs and length data. Because reading hundreds of thousands of files is slow, database-oriented functions work from this cache.
Serve up ABC, typeset SVG, and in future, perform search:
HTTP_BIND=0.0.0.0:3000 BASE=~/tune-db cargo run server
BASE- where are the ABC tunes? e.g. /tmp/tunes
HTTP_BIND- http bind address and port for server? e.g. 0.0.0.0:8000
DEBUG_MAX_ID- limit tune top id to this value. Selects a subset for profiling, debugging, etc.
On a random Macbook air, full scan of 200,000 tunes and error reporting:
- DEBUG: 3m5.416s
- RELEASE: 0m47.434s = ~4x speedup
Install Tarpaulin for test coverage.
cargo install cargo-tarpaulin
This is open source software, and has the "MIT License", see LICENSE file.
You are very welcome to use this software in accordance with the license. If you do, I would be very grateful if you let me know and give credit where appropriate!