CODE & FORM is a blog supporting the coding and teaching activities of Marius Watz. It contains documentation of workshops, as well as code for various projects. Email tips about relevant links or pieces of code to marius-at-unlekker-net.
Old workshop materials are still online, and can be found here.
It was great to play with Alexander and develop our collaboration further, his cinematic soundscapes are a treat to work with. Audience responses seem to indicate that we’re doing something right, so hopefully we’ll find more chances to play together in the near future.
Kabinett poster by Diogo Valerio / photos from performance at Generator.x 2.0
To use Sonia and SoniaHelper for sound input, download Processing - soundlibs.zip. Create a folder called “libraries” inside your sketchbook folder and put the contents of the archive inside it. When you restart Processing you should see the new libraries under the “Sketch > Import library” menu”.
Last week I went to Monkeytown, a restaurant / performance venue in Williamsburg, New York. One of the acts was Loud Objects, a performance project that features live soldering of pre-programmed sound generating chips. Working on an overhead so that the audience could enjoy the action, the performers soldered connections between the various chips, creating a semi-controlled soundscape of scrapes and glitches.
I just uploaded a short video to YouTube, not all that informative but it gives an idea. Supposedly this kind of thing is more common in Japan, but I’ve never seen it before. Definitely an interesting twist on physical computing…
I just bought a new toy for live performance: The Doepfer Drehbank MIDI controller. Sporting 64 rotary knobs, it should satisfy all my needs for live control of obscure parameters. In fact, I doubt I’ll ever come up with that many parameters, but having the option is nice.
Compared to the FaderFox LV-1 which I’ve been using so far, the tradeoff is control vs. portability. The Drehbank is fairly hefty, weighing at least 2 kg and measuring maybe 40 x 12 x 6 cm. If I already had a lot of performance gear that would probably be too much, but since I mostly only use one laptop and a controller it’s not too bad.
Plugging the controller into Processing was painless. The knobs are solid, with good resistance so that you can get a smooth turning motion. There’s even just enough room between the dials to allow a small white space for labelling. My only concern is about the power plug, which sticks out a bit far and would seem prone to coming unplugged if the unit is moved. But seeing as the unit is so heavy I won’t be moving it much anyway.
The Drehbank is going out of production, and Doepfer is selling off its remaining stock at the relatively low price of €299. Their site say that it’s sold out, but when I sent them an email to check they still had some left. There is also the smaller Pocket Control with 16 dials.
I just did some new live visuals for a performance with Alexander Rishaug. Alexander’s music tends towards the ambient, with rich textures and large soundscapes that develop slowly in time. Check out his Myspace for sound samples and bio. Alexander was the producer of the Generator.x concert tour for Rikskonsertene and was also responsible for hooking me up with Phonophani, so we have some good history together.
Minim, a new sound library by Damien Di Fede was just posted on processing.org. Sound has never been the strongest point of Processing, and with ESS and Sonia already in existence, that makes Minim the third attempt at providing sound functionality. But to be fair it should be said that Java on the whole has never been that good at sound, with Sun notoriously neglecting the media APIs.
Like ESS, Minim is based on the JavaSound API, but at first glance it seems like a more complete solution than either ESS or Sonia. It allows stereo input from either line-in or file (the others only do mono), it has beat detection and there is a class hierarchy for realtime sound synthesis and filtering. And there is more good news for those using Processing in Eclipse or other Java IDEs - Minim comes with full source, released under a GNU license. This should be mandatory for libraries, but sadly isn’t.
I haven’t tested Minim yet, but it certainly seems very promising. For my own purposes (realtime sound responsive visuals), the stereo option as well as the beat detection could come in very handy. I recently created new visual set for a gig at Lovebytes, if I had known about Minim I might have tried it then.
AudioFileIn: Mono and Stereo playback of WAV, AIFF, AU, SND, and MP3 files.
AudioFileOut: Mono and Stereo audio recording either buffered or direct to disk.
AudioInput: Mono and Stereo input monitoring.
AudioOutput: Mono and Stereo sound synthesis.
AudioSignal: A simple interface for writing your own sound synthesis classes.
Comes with all the standard waveforms, a pink noise generator and a white noise generator. Additionally, you can extend the Oscillator class for easy implementation of your own periodic waveform.
AudioEffect: A simple interface for writing your own audio effects.
Comes with low pass, high pass, band pass, and notch filters. Additionally, you can extend the IIRFilter class for easy implementation of your own IIR filters.
Easy to attach signals and effects to AudioInputs and AudioOutputs. All the mixing and processing is taken care of for you.
Provides an FFT class for doing spectrum analysis.
Provides a BeatDetect class for doing beat detection.
My thanks to Damien Di Fede for contributing what looks like a very useful addition to the Processing arsenal.
Daniel Dihardja has just released a new Processing library called jm-Etude. It is a wrapper for the Jmusic Java library, which provides a framework for musical composition. In the Processing tradition, jm-Etude simplifies this further, setting up a structure where “scores”, “parts” and “phrases” together make up compositions in the form of “Etudes”. The library uses notes as the molecular unit, and MIDI is used for sound playback.
jm-Etude should be useful for a classically structured approach to music, but also to create realtime generative compositions. It nicely fills gaps left by Ess and Sonia, which focus on playing audio files or simpler waveform synthesis. Hopefully Dihardja will add examples showing how to use other sounds than the built-in MIDI instruments, as they sound notoriously cheesy.
See the Jmusic javadoc for an impression of what the underlying library can do.
Update: Daniel has explained a little about future plans for custom instruments:
[...] jm-Etude is not heavy MIDI based, it is just that for now the only playback function which is implemented is the jMusic midi playback. In jMusic the music data can be played and exported as MIDI or AUDIO and there are also libraries to build and use custom instruments, but I haven’t implemented them in the wrapper yet, because I’m still working on the MIDI export which will be a feature in the next update (is still a bit buggy now).
Toxi has posted a useful comment on the nature of FFTs over on the Processing forum. It should be of interest to those who have been looking at the SoniaHelper library.
[...] Even if you’re thinking of analyzing only the range of formants of human voices you can work with ONE frequency band, but please don’t mix this up with the spectrum bins returned by the FFT… In most cases speaking about a band of frequencies will refer to a whole collection of bins. Escpecially if you want to target/analyze the behaviour of a specific frequency bin, you’ll need as many bins as possible to start with. In theory a spectrum of only one bin would cover the whole frequency range 0 Hz < f < 1/2 the sampling freq...
Since the FFT is attempting to break down the signal into a series of sinoids, at least 2 samples are required in order to work out the contributing frequencies of this signal (It's mathematically impossible to do it with one only).
Running danger of becoming too mathematical, the number of frequency bins required directly relates to the number of samples analyzed. In other words, the fewer bins you want in your spectrum the shorter (in time) the chunk of signal analyzed and so the poorer the quality/precision of the computed spectrum as well. For most applications a spectrum of anything under 128 bins will be pretty unusable since the related time window is too short (<3ms) and you might be better off just working with the average volume of the samples by using Ess's equivalent of Sonia's LiveInput.getLevel() method.
Update: SoniaHelper should be renamed FFTHelper and integrated into unlekkerLib, but that hasn’t happened yet. The best way to use it right now is to download the code examples from the VJ workshop I did at Cimatics 2007. That has the code you need as well as some examples of practical usage.
I’ve put together a hack for normalizing the FFT analysis values produced by Sonia’s LiveInput.getSpectrum(). It looks at the maximum values produced, and scales them according to the most recent maximum values. That way, quiet sections will produce relative differences according to the current maximum value, so that both quiet and loud sections give dynamic results.
It also allows you to dampen the values so that the rate of change in values can be controlled. This can be used to make the values change appropriately according to the quality of the music, making it easier to use them for visuals that match the music.
Source code
You initialize SoniaHelper like this:
// SoniaHelper(int _n, int _nbands,boolean doAvg)
// Start up Sonia
LiveInput.start(spectrumLength);
// ...or start up Ess
Ess.start(this);
myChannel=new Channel("hello.aiff");
myChannel.setupFFT(512);
// init SoniaHelper
ffthelper=new SoniaHelper(spectrumLength,32,false);
ffthelper.setMaxLimits(200,2000);
damperval=0.1f;
ffthelper.setDamper(damperval);
Update SoniaHelper inside draw() by passing spectral values from Sonia or Ess:
