fIcy: an icecast/shoutcast stream grabber suite
fIcy is a small icecast/shoutcast stream grabber suite for use under shell environment. Its goal is to automatically rip a stream into user customisable files. It will work with ICY compatible streams, allowing you to either to save the stream to disk or to pipe the output to a media player, or even both. fIcy, among other uses, is ideal for batch/unattended recording of radio programs and stream debugging.
The fIcy package includes:
- fIcy itself, a stream separator/multiplexer,
- fResync, a fast MPEG-resyncing utility,
- fPls, a playlist frontend for fIcy.
fIcy [options] <server [port [path]]|url>The main program. Takes directly a stream url and dumps the tracks on the specified file/s and standard output, depending on the settings.
fPls [options] <file|url> [fIcy options]Playlist manager. Reads a playlist (local or remote) and manages fIcy retries/timeouts/errors, forwarding the specified flags.
fResync [options] fileMPEG resyncing utility. Re-aligns head frame headers on dumped or broken files. Usually needed for embedded hardware decoders or editing software.
-d Do not dump the output to stdout. Useful when only ripping. -E num Enumerate files when song title [metadata] changes, starting at num. When 0, fIcy will try to find the highest unused file number automatically. Uses -o as a prefix. -h Help -c Do not clobber files. -A Append to existing files instead of overwriting. -m Use song title [metadata] when naming files. Uses -o as prefix. -n If the file exists create new one with .n as suffix. -p When dumping to stdout consider writing errors as transient (that is: flush the output buffer until stdout is ready). Useful when you pipe the output to a media player and want to kill it while not interrupting the rip. -o file Dump the output to file or use file as a prefix (depending on other settings). Hint: to dump without a prefix use "./". -s suffix Use sfx as a suffix for new files. Hint: the .mp3/.m4a extension is NOT implicit. -t Display song title [metadata] while ripping. -r Remove/don't save partial chunks. This will skip the first chunk and remove the last one upon termination which are (supposedly) incomplete. To use in combination with -m or -E. -q file Append "file name" sequence to file. The file name is written upon file completition. This may be used to trigger events and rejoin splitted parts with an external tool without -E. fResync will use this file in the future. -x regex Save only files whose title (NOT filename) matches against this (or one of these) extended regular expressions. Multiple -x can be specified on the command line to form OR conditions. Dump unaffected. Can be combined with -X. -X regex Do NOT save files whose title matches against this extended regular expression. Same semantics as -x. -I file Load include/exclude REs from file. Each line must be prefixed with + or - to indicate whether it's a positive or negative expression (-xX). -f expr Filter titles through the specified coprocessor expression. The raw title is passed to the expression (doesn't include any additional prefixes/suffixes). As the result will be used internally, some limitations apply. Read carefully the Filtering section. -F file Filter titles through the specified coprocessor script. Same semantics as -f, but the expressions are loaded from a file instead. Conflicts with -f. -C path Specify the path of the external title rewriting coprocessor. Defaults to "sed". The executable must support the '-e', '-f' flags and operate through stdin/out, like "sed". -M time Maximum recording time. See Notes. -i time Maximum network idle time. Stops recording after the specified amount of time is passed without network activity. Defaults to 0 (default tcp timeout). -a file Read authentication credentials from file (the file must contain a line of the form user:password). Note that only the Basic HTTP authentication scheme is supported. -l num Redirect follow limit. Defaults to 1. 0 disables redirection entirely.
-b By default fResync maps the entire file into memory when operating. However this can create problems on loaded systems with large files or when simulating. This reverts to a buffered I/O mode. This flag is also implicit when simulating. -s Simulate the process. Print on the standard output the starting sync offset and stream length, but don't modify the source file. -v Verbose. -n frames Require/decode at least n valid consecutive frames to validate the sync offset. Defaults to 6. -m len Maximum frame length. Defaults to 1597. fResync uses this value to determine the maximal region of the file to be checked.
-P path Specify a different name or full path for the fIcy executable (defaults to "fIcy"). -v Verbose. -R max Specifies the maximal number of retries to do for each stream upon connection/read failure. -L max Specifies the maximal number of loops to do for the entire playlist (-1 for infinite). -T time Wait time to pause after each failure. -M time Maximum cumulative recording time. See Notes. -i time Maximum network idle time. Same as fIcy's when loading a playlist via http. Forwarded to fIcy. -a file Read authentication credentials from file. Same as fIcy's when loading a playlist via http. The credentials are automatically forwarded to fIcy, but you can override them when needed. -l num Redirect follow limit. Same as fIcy's when loading a playlist via http. Forwarded to fIcy. -d file Run as a daemon, redirecting messages to file. fIcy's -d option is enforced. As the process is chdir-ed to the root directory you also have to specify absolute paths for all options, including fIcy's ones.
Rip a station until stopped:
mkdir files fPls -L-1 http://netradio.invalid/listen.pls -s .mp3 -o files/ -cmrdt
Use fIcy to manage reconnections and display titles while playing:
fPls http://example.com:8080/listen.pls -t | mpg123 -
Connect directly to the stream with server:port and /path:
fIcy -s .mp3 -o ./ -md 22.214.171.124 8080 /path/to/stream
Rip an .mp3 stream while playing, but allows the player to be restarted later by using a named fifo (note that you can re/open "fifo" with any player):
$ mkfifo fifo $ fIcy -p ... > fifo $ mpg123 fifo
Record your favourite program "XYZ" usually on-air between 16:30-17:00:
at 16:30 fPls -M 30m http://example.com/listen.pls -o program.mp3 -x XYZ ^D
Cleanup a ripped and/or damaged mp3 file:
Audio-oriented rebuffering tool. Ideal for lousy streams.
Frame-level mp3 cutting tool.
We would like to remind you that saving streams containing copyrighted material without explicit consent is ILLEGAL. For stream administrators, please see our statement in the FAQ_.
The output files produced by fIcy may miss audio framing information and headers since the separation does not consider the audio data. For this reason, your player 'may' (but should not) fail to reproduce the dump or output some initial noise: this is expected. fResync can be used to cleanup MPEG files after processing.
You can also use other tools such as mpgedit for cutting the file in arbitrary positions without diminishing the quality. Assuming that your song spans across three files (use -q to know which ones), that's how to proceed:
cat 1.mp3 2.mp3 3.mp3 > temp.mp3 && xmpgedit temp.mp3
Do not resync the files if you're going to post-process them this way: fResync would remove at least one boundary frame on each file, while other tools could also insert extra empty frames to silence the decoder!
The -M flag supported by both fIcy and fPls accepts a time specification in seconds, HH:MM or N minutes/hours/days. Also beware that -M specified in fPls means cumulative recording time (time accumulates across retries/timeouts), while -M specified in fIcy means single stream recording time (recording stops at the first error or when the specified time has elapsed).
Most online radio stations tend to put banners in the title that will be shown in the player, and eventually result in the filename. To overcome to this (and more), fIcy offers the possibility to rewrite each title through a normal sed script via the "-fF" flags. A real sed coprocess is used along the execution so all of sed's power is available, but some limitations apply:
- Each line of input should result in one output line, and ONE ONLY.
- Two consecutive identical titles will result in the second one being ignored (thus NOT splitting the stream). Consider this rule, as removing carefully the banner could result in a better separation.
- The resulting title will still apply for -xXI as usual.
- Please note that the title is filtered, not the filename (which may still have some characters removed/modified). Use -tv to see what is actually sent to the filter.
You can actually use any executable that works as a stream editor by specifying the path with '-C'. The executable must support the '-e' (inline expression) and '-f' (script file) flags or, at least, ignore them. This allows for any script or custom executable to be used when a "sed" script is considered inadequate.
As an example, suppose your titles look like this:
Artist - Title (radiobanner)
You can write a sed expression or script containing:
to remove the trailing part. This facility can also be used to uniform file names, invert Artist/Title positions and so on. Clever use of the pattern space can also be used to merge albums. sed alone can be used to debug expressions, eg:
echo "test title" | sed -e 'expr'
Refer to the sed(1) manual for a complete list of commands you can use.
fIcy comes with a very simple Makefile that should work on any system using gcc and GNU make. Documentation is generated from these files using rst2html.
All standard environment flags are supported, including DESTDIR/PREFIX for relocated installation. If you need to use a different compiler (for example on OpenBSD), you can call make as follows:
make CXX=eg++ PREFIX=/usr install
instead of changing manually the Makefile. Please note that, when using gcc, at least g++ >= 3 is required to compile fIcy.
Also note that, for "fPls" to work, fIcy must be already installed (be in "PATH") or a full fIcy path must be specified with -P.
If you feel to discuss improvements and suggestions, and/or test beta releases before announcement you can subscribe to ficy-users by either sending an empty email to <firstname.lastname@example.org>, using GMane (group "gmane.comp.audio.ficy.users") or by contacting the author at <email@example.com>. The archives are accessible via web through https://firstname.lastname@example.org/ or via news directly.
"fIcy" can be found at https://www.thregr.org/~wavexx/software/fIcy/
fIcy's GIT repository is publicly accessible at: