MAT 240A Spring 2017
When: W F noon-2pm
Where: Elings 2809
Instructor: Karl Yerkes Ph.D. 
TA: Qiu Weihao
http://mat.ucsb.edu/240/A
http://lists.create.ucsb.edu/mailman/listinfo/240

This course is an introduction to the many facets of audio programming. Audio programming can take many shapes, and requires different tools and practices for artistic projects, scientific research or industrial applications. This course will present an overview of these different aspects of audio programming while introducing and exploring key concepts that permeate most audio programming projects.

We explore a diverse palette of tools, covering the most common applications and challenges in audio programming. Our tools include high level and domain-specific languages useful for prototyping multimedia works (e.g., Pd, Max, SuperCollider, and ChucK), data science and engineering environments used in acoustics, psychoacoustics and DSP research (e.g., IPython Notebook, FAUST, MATLAB), and low level languages and libraries necessary for performant, portable, and maintainable applications (e.g., C/C++, RtAudio, Gamma).

Although the course cannot hope to explore the presented languages and topics in all their detail and complexity, its aim is to give the student the ability to make decisions about the environment and tools for their particular projects and target platforms, having a clear understanding of the challenges and constraints present when working with audio. It should also provide a starting platform for further exploration and learning into particular tools and techniques.

Topics

  • Digital Audio, sampling and representation.
  • Events and asynchronous data in audio contexts.
  • File and Realtime I/O concepts and APIs.
  • Concurrency (or lack thereof) in audio

Computing contexts

  • Single powerful computer (e.g., your laptop)
  • Mobile, less-powerful computer (e.g., your phone)
  • Embedded system (e.g., Raspberry Pi or Teensy)
  • Several Distributed/Networked computers (e.g., CREATE ensemble, NETFLIX)
  • Virtual machine / Browser environment (e.g., Gibber, WebAudio)
  • Offline (as opposed to Real-time)

We explore what characteristics each of these contexts share and how they differ.

Not Topics

  • Spectral tranformations
  • Music Information Retrieval
  • Audio Synthesis Techniques
  • Sonification
  • Audio Effects
  • Spatial Audio

As these topics are covered in later courses (namely 240 B-F), we will not cover them in this course. That said, we may incidentally touch on some of these in the course of our work.

Software

We will use many tools in this course. Look to the Install page for help with software installations.

  • Python
  • Csound
  • Javascript (WebAudio and Gibber)
  • Pd
  • Max/MSP
  • Jack/JackTrip
  • C/C++

Schedule

Week Topics
1 - Defining the scope of Digital Audio Programming and of this course
- Introduction to audio programming using data science and engineering tools
- Audio as a list of samples that can be handled as vectors, and mathematical operations on audio vectors
2 - Using a domain specific language for music: Csound
- Musical theory, concepts and ideas in code
- History of Computer Music Languages
3 - Introduction to Javascript and the WebAudio API
4 - Interactive music systems
- Live coding and interactive prototyping in Gibber and ChucK
Understanding and working with a dynamic audio chain
5 - Dataflow languages: Pd and Max/MSP
- Working with event based programming
6 - Programming in C
- Using build systems and Makefiles for audio projects
- Overview of File I/O APIs
7 - Real-time I/O and I/O APIs
- Callback versus blocking interfaces
- Considerations for real-time audio programming
8 - Asynchronous events and callbacks versus event polling
9 - C++ and Object oriented programming in relation to audio
- Using source control for version tracking and collaboration
- Introduction to Gamma and AlloSystem
10 - Plugin APIs overview
- Pd/Max “externals”


Final project

The final project should implement some original idea by the student using any of the systems reviewed (or another similar system). The project should concern some topic of interest to the student and should tackle some difficult idea or project.

Grading

  • 40% Homework
  • 40% Final project
  • 10% Theory assignment
  • 10% Class participation

References and Resources

There is no single textbook for this course, however the following resources may be useful at different stages: