The Wub Machine
The Wub Machine is a web/console app that automatically remixes music into Dubstep and Electro-House.
Check it out in action at the.wubmachine.com.
If you're interested, it's relatively simple to use this to create your own remixers and your own remix websites. Go nuts. :)
- The Remixer
- The Web Frontend
The Wub Machine has only been tested on OS X and Ubuntu, but should theoretically work anywhere the dependencies are installed. The installer script is currently a bit overzealous and installs a bunch of stuff, but should work. You'll need an API key from the Echo Nest to setup the Wub Machine, which you should then set as an environment variable ($ECHO_NEST_API_KEY). The Wub Machine itself has a large list of dependencies, which the install.sh tries to setup:
I've gone to great lengths (well, ~50 lines) to make the Wub Machine as light as possible on system resources. To remix a song x MB in size and y minutes long, expect:
- about 60MB * y in memory usage.
- at most 60MB * y in temporary disk space
- up to x MB of network bandwidth
In short, my 2-minute, 4MB test song uses up about 120MB of memory, about 100MB of disk space, and on first run, 4MB of upload bandwidth.
To remix from the command line, do:
python -m remixers/dubstep <filename.[mp3|m4a|mp4|wav]>
or if you want to try ElectroHouse, do:
python -m remixers/electrohouse <filename.[mp3|m4a|mp4|wav]>
The resulting file will be placed in the same folder as the original.
To integrate with a larger app, simply do something like:
from dubstep import Dubstep class MyProject(): def wubwub( self ): remixer = Dubstep( self, "song.mp3", "output.mp3", ".mp3", "some-kinda-uid", self.log ) remixer.deleteOriginal = False remixer.start() def log( self, update ): print "Hey look, progress from the remixer!" print update m = MyProject() m.wubwub()
The Remix superclass uses a separate thread to monitor progress, and spawns a new process from that thread to do the heavy lifting. If you want to create a new remixer, you can modify the Dubstep class to remix however you want.
The Web Frontend
The web frontend uses Tornado, Tornadio (Socket.IO), MySQL, and a couple other things to allow people to remix over the web. It has built-in SoundCloud sharing, a neat monitoring/stats page, and plenty of tested features. However, it's still buggy, of course. To get it up and running 100%, you'll need an API key from SoundCloud, which you can then place in config.yml.
Start the web frontend by doing:
There's also a very, very rough version of a daemonizer... if you want to start it and forget about it, run:
python server.py start
To stop it, try running:
python server.py stop
...although that probably won't work 100% of the time. In case of emergency, do:
kill -9 `cat ./server.py.pid`
...and if that doesn't work, then find the pid with:
ps aux | grep [p]ython
and kill the process.
The web frontend makes use of a
config.yml, to dictate config variables at and during runtime. During runtime, you say? Why, yes!
config.py is a little bit of python metaprogramming that reloads each config variable as needed, only when it's changed. Simple.
What variables can you find in config.yml, you ask? Allow me to explain:
app_nameis the public name of the app. i.e.: the Wub Machine, when running on my server, is called
the Wub Machine. This is used in various places around the code.
Remix Queue Settings (can be changed on the fly)
maximum_concurrent_remixesis, well, the maximum number of remixes that should be running at any one time. Note that this is not strictly enforced while the remixer is running, but rather at each entrypoint to the queue. Certain race conditions may allow more than this number to be remixed at once. Ideally, set this to the number of cores you have on your machine. An entry-level Linode box can run 4 beautifully.
maximum_waiting_remixesdictates how many remixes should be allowed to wait in the queue. If this limit is reached, the homepage will refuse uploads for all new page loads. Pages that have already loaded are still allowed to add to the queue, and any pages that are closed while waiting will have their remixes deleted from the queue.
hourly_remix_limitis the number of remixes allowed in a given hour. Note that this is defined as the past 60 minutes, not as since the top of the hour.
Timeouts (all in seconds)
cleanup_timeoutis the time between periodic cleans that delete remixes, uploads, and artwork.
remix_timeoutis the maximum time a remix can take before being stopped/killed and deleted.
wait_timeoutis the maximum time someone can wait for a remix before it is automatically deleted.
watch_timeoutis the amount of time a browser has to open a progress socket (via Socket.io) after uploading a file. Otherwise, it is removed from the queue and deleted.
monitor_limitis the number of items to display upon initial load of the monitor page.
monitor_time_limitis the amount of time (in seconds) displayed on the graph.
nginxdictates if the app is running behind an Nginx proxy. Tornadio should have support for this, but I couldn't get it to work, so I rolled my own.
database_connect_stringis the SqlAlchemy database connection string for your DB. Cannot be changed at runtime.
log_fileis the name of the file to log all requests/warnings/errors/info to. This can be pretty verbose.
log_nameis the name the server file should be referred to in the log file. (i.e.:
log_formatis the format string of Python LogRecord attributes to be used.
echo_database_queriesis passed into SqlAlchemy's engine constructor to show database queries in the log file. This gets horribly verbose. Cannot be changed at runtime.
allowed_file_extensionsis a list of allowed file extensions, including periods. Limited to ['.mp3', '.m4a', '.mp4', '.wav'] by the Remix API at the moment.
socket_io_portis what Socket.IO/Tornadio binds to, defaults to
socket_extra_sepis the separator between the Socket.IO URL and a resource ID (
UID), defaults to
monitor_resourceis the name of the Monitor resource, defaults to
progress_resourceis the name of the user-facing song progress resource, defaults to
soundcloud_consumeris your SoundCloud Consumer API key
soundcloud_secretis your SoundCloud secret API key (not sure if this is actually used anywhere, tbh)
soundcloud_redirectis the URL for SoundCloud to redirect to after OAuth login
soundcloud_app_idis the numeric app ID of your registered SoundCloud app - used to filter songs uploaded to SoundCloud.
soundcloud_timeoutis the number of seconds before a SoundCloud upload times out.
soundcloud_descriptionis the description tag automatically appended to any track shared to SoundCloud.
soundcloud_tag_listis a list of tags to be set on any shared tracks. Useful for filtering later.
soundcloud_sharing_noteis the note used when sharing a track through SoundCloud to FB/Twitter.
Caveats & Other Notes
To edit the HTML markup, you can find the HTML templates in /templates. The SoundCloud widget on the homepage is mostly hardcoded, and the JS for that page is included in it... not very nice. But pretty fast. There are rough edges all over this thing, so please feel free to contribute. A lot of documentation is just not included.