RTLEARNER (or Real-Time Learner) is a set of example applications for fast dataset creation and real-time classification tasks in MaxMSP and the Python machine learning library Sci-Kit Learn. These two environments were chosen because of their wide adoption by artists and their unique strengths in the domains of real-time signal processing and statistical analysis. MaxMSP has long been the programming environment of choice for computer music practitioners, containing within it a wide variety of real-time composable audio processing units. Alternatively, Sci-Kit Learn has emerged as the goto framework for machine learning tasks, and provides with it a number of popular ML (machine learning) algorithms for classification in a highly approachable yet optimized and configurable format.
RTLEARNER implements a bi-directional UDP client/server architecture to act as a go-between for the MaxMSP and Python environment. MaxMSP handles audio input and output streams, feature extraction, dataset creation, and passes feature sets to machine learning models hosted by the Python environment. The Sci-Kit Learn library makes predictions on feature sets in real-time and sends them back to MaxMSP for musical adaptation. The max and python files do their best to provide commonly used programming patterns and tools for legibility and adaptability.
Machine Learning tools such as FluCoMa, CataRT, Magenta Studio, and ml.* offer classification, prediction, and even generation processes in Max, all within a singular software environment. The trade off for this elegance is less composability in the underlying model parameters and less exposure to the lingua franca of machine learning, Python. For most, the aforementioned tools are extremely powerful systems for classification and regression techniques, for the more curious programmer, RTLEARNER provides systems for going deeper into the black box.
-
Both Windows and Mac are supported.
-
Python 3.9 is recommended. This was tested using a virtual environment using anaconda.
-
Python dependencies can be downloaded using the Python environment's library manager or using the following terminal commands.
pip install sklearn, numpy, pandas, python-osc -
Install MaxMSP.
-
The RTLEARNER max patch uses a number of FluCoMa Max objects for feature extraction. The FluCoMa objects can be downloaded here, and will soon be available in the Max package manager. These objects may be swapped out for other analysis objects such as CNMAT's Zsa Descriptors which can be found in the Max package manager (though the patches included here do not depend on them).
RTLEARNER was initially developed during an independent study with Professor Johanna Devaney in the Sonic Arts MFA at Brooklyn College.
Gradient visualization liberated and altered with love from MADBUTTER/Emmanuel Jourdan found here.
Some of RTLEARNER's design patterns were inspired by the FluCoMa project. A huge thanks goes out to Pierre Tremblay and Ted Moore for their spring 2022 workshop.