240E - Audio Programming Series: Music Information Retrieval

Final projects

Thu 20 March 2014 by Andres Cabrera

Owen Campbell, MIR Visualizer

The goal of my project was to extend an existing real time music visualization system I had written using OpenFrameworks. The original visualizer was controlled entirely by the RMS of the input signal, so I wanted to incorporate a number of the other low-level audio features we covered in class in order to add more depth to the visuals. I used the OpenFrameworks addon for Aubio written by Paul Reimer https://github.com/paulreimer/ofxAudioFeatures, which despite its lack of documentation still helped me integrate Aubio with my existing code. Aubio offers a suite of higher level feature detection functions as well, including onset and pitch. However, because of the way my existing code was structured, I chose to stick to the low level features for now.

My strategy for expanding the number of different control parameters was to break down the behavior of the visualizer and reassign what had previously been controlled with different thresholds and short- and long-term histories of the RMS. So for each feature I stored all the same information I had been collecting for RMS for overall spectral energy, high frequency content, and the flux, spread, skewness, kurtosis, and slope of the spectrum. These features update on every FFT frame, synced to the audio callback in OpenFrameworks. A large part of the still ongoing process of using these features is determining useful ranges for their output and encapsulating that information in a way that makes it easy to swap features between different characteristics in the visuals. These visual features I'm mapping the audio features to so far are the lifetime and size of the particles, a spawning control, a trigger to scatter the particles, and color, opacity and 'texture' features which I haven't fully implemented. The texture feature determines the number of sides of each particle, shifting them from triangles to squares to pentagons all the way up to full circles. However, the OpenFrameworks addon I used to achieve this effect https://github.com/openframeworks/openFrameworks/tree/master/addons/ofxVectorGraphics does not seem to support alpha blending, which is a key improvement on the asethetic of the original visualizer.

http://nbviewer.ipython.org/gist/owengc/9da4715e2cfa718605d5

Anis Haron, Lick detection in an improvised passage

  1. a 'lick' can be understood as a known sequence of notes used in solos and melodic lines in popular genres such as Jazz and Rock
  2. a lick is a form of imitation, or a quote – with the goal of developing the phrase and building personal style and vocabulary
  3. usually quoted from the greats within the genre
  4. project will be approached similar to the methods of speech recognition – using HMM
  5. main issue -- database, and variation of licks
  6. licks could be played in any key, at any speed and variations

http://tinyurl.com/q7kzqqy

Aaron Demby Jones, MIR tool for algorithmic composition

Music information retrieval (MIR) typically starts with a particular audio file or collection of audio files from which the user would like to extract data (e.g. tempo, mood, genre, etc.). Typically, a number of interesting mathematical procedures are employed in this task. However, these same procedures have not generally been considered from the perspective of generating audio content. In this project, I propose a novel approach to algorithmic composition, derived from techniques of MIR.

http://nbviewer.ipython.org/gist/aadjones/9769985

https://drive.google.com/file/d/0B7tBXGWPlLhqakQ4M0hGVUVSQi1UMUtWaVFydk5LRUNnVWZj/edit?usp=sharing

Robert Miller, Shifty Looping-X: Expressive, meter-aware, non-repeating rhythmic looping

Shifty Looping-x, a creative application of MIR, uses multi-feature audio segmentation and beat tracking to extend Matthew Wright’s original implementation and development of the shifty looping technique. Traditional looping is advantageous because a few seconds of music can germinate into an arbitrarily larger sample; yet, looping quickly becomes dull because of its inherent repetition. As a result, Wright developed a new technique that maintains the advantageous of looping while addressing the repetitiveness. Using a Max/MSP patch and onset detector object, the program finds and suggests possible points within a given sound sample that could serve as start and stop points for a loop. The duration of these suggested loops are an integer number of bars, thus maintaining the metrical integrity of the sample. Once the program suggests loops, the user then auditions the candidates and discards any that may potentially dismember musical coherence. Because the program generally suggests an excessive number of loops, auditioning these loops quickly becomes tedious and obtrusive to the compositional process. Therefore, I propose to incorporate segmentation and texture window analysis to the implementation to improve the programs ability to find good loops and create a more autonomous system.

Instead of in a Max/MSP patch, the program will run with C++ using Allocore and Essentia, the latter powering the program’s MIR functionality. I will use Tzanetakis and Cook’s scheme for segmentation, but I will replace zero-crossing rate with running cross-correlation (or frame-by-frame autocorrelation). The other features are spectral centroid, spectral rolloff, spectral flux, and root-mean square energy. A significant and simultaneous changes within these features will be used in the decision making process of setting a loop point. The beat locations will also be factored in the decision of where to place a loop point. The distance between start and stop positions will be an integer number of bars away.

The main challenges with this project include creating an automated decision maker and controlling for the different types of music that may be passed to it. I will deliver a program (source) that runs via the command line, or time permitting a simple GUI application built using GLV. The program will be accompanied by a README and a few audio samples demonstrating the program.

https://github.com/milrob/shiftyLooping

Hafiz Muhammad, NICM: Raag Analysis

  1. Time on Pitch & HMM http://nbviewer.ipython.org/gist/muhammadhafiz/6e716182653e2f2e35c2
  2. Raag Analysis http://nbviewer.ipython.org/gist/muhammadhafiz/062a21c9caa3cae5cb2a
  3. Spectral difference-sitar onset VS tabla onset http://nbviewer.ipython.org/gist/muhammadhafiz/60719fc11964655b8690

Sean Cyrus Phillips, Classification of Animal Vocalizations

This project attemps to classify sounds as animal or non-animal based on low level audio features. http://nbviewer.ipython.org/urls/raw.githubusercontent.com/soundbysean/240E_iPython/master/240E_Final_Total_Set.ipynb?create=1

Comments