Hi! You've found the place where I leave tutorial-style notes for various projects and rant about broken computers and how they fail.
Sep 21, 2017
I wanted a pretty light visualizer for playing music, and already had an Arduino Uno fitted with an Adafruit Neopixel shield. All the tutorials I found need dedicated sound detection hardware to do this, so I’ve decided to do it in software, with the following reasoning:
A quick search revealed the existence of the
gstreamer library, featuring
an element called ‘spectrum’, which is meant to analyze the frequency spectrum
of an audio signal. Awesome, right? The docs come with an example application
which appears to extract the magnitude and phase for different frequency bins. It’s a good place to start. Install it:
apt install libgstreamer1.0-dev
The output of the example application:
gcc `pkg-config --cflags --libs glib-2.0` `pkg-config --cflags --libs gstreamer-1.0` example.c -o example.out ./example.out New spectrum message, endtime 0:00:07.200000000 band 0 (freq 1000): magnitude -52.714016 dB phase 1.557018 band 1 (freq 3000): magnitude -36.372467 dB phase 0.512789 band 2 (freq 5000): magnitude -16.085241 dB phase 0.393575 band 3 (freq 7000): magnitude -14.310267 dB phase -0.382180 band 4 (freq 9000): magnitude -28.397552 dB phase -0.445975 band 5 (freq 11000): magnitude -51.487675 dB phase -1.262673 band 6 (freq 13000): magnitude -52.313831 dB phase 1.529878 band 7 (freq 15000): magnitude -51.878403 dB phase 1.557018
This example uses a dummy source, and should be changed to point to your soundcard instead. On Debian systems, pulseaudio can be used as an audio source for gstreamer.
Install the relevant plugin:
sudo apt install gstreamer1.0-pulseaudio
List your pulseaudio sources:
pactl list short sources 0 alsa_output.pci-0000_00_1b.0.analog-stereo.monitor module-alsa-card.c s16le 2ch 48000Hz SUSPENDED 1 alsa_input.pci-0000_00_1b.0.analog-stereo module-alsa-card.c s16le 2ch 48000Hz SUSPENDED
In the example script, point the audio source to ‘pulserc’, and set the device to correctly point to your sound card output (unless you want to process all microphone input, case in which the second line can be ommitted).
src = gst_element_factory_make ("pulsesrc", "src"); g_object_set (G_OBJECT (src), "device", "alsa_output.pci-0000_00_1b.0.analog-stereo.monitor", NULL);
Full script available on Github
Now this bit is working nicely, it’s time to isolate the values we want to send to the Arduino. By default, the script prints magnitude and phase.
By examining the output of the script for a while with various types of music I decided to track momentary changes in magnitude. This is done for 8 frequency bands, as the Neopixel shield is an 8 by 5 matrix. The magnitude variation is from one sample to the next is small, rarely exceeding +/- 5 (how convenient!).
What could go here is a more complicated visualisation function based on both magnitude and frequency.
We are currently getting magnitude values from our modified example script, time to get process these numbers and send them across to the serial line. I installed python-serial:
apt install python3-serial
…and wrote a basic script which takes the output from our C script, does maths and delivers numbers to the serial line. The thing to note is the baud rate, which needs to be set high otherwise the Arduino lags behind the beat:
import serial import subprocess ser = serial.Serial('/dev/ttyUSB0', 115200) def read_o(): process=subprocess.Popen(["./pulse_freq.out"],stdout=subprocess.PIPE, universal_newlines=True) for stdout_line in iter(process.stdout.readline, ""): yield stdout_line values_before = [0,0,0,0,0,0,0,0] for item in read_o(): computed_values = [0,0,0,0,0,0,0,0] values_now = item.split(" ") for i in range(0,8): val = abs(int(float(values_now[i]) - float(values_before[i]))) computed_values[i] = val if val < 5 else 5 value = "".join(str(v) for v in computed_values) value += '\n' #print (value) ser.write(bytes(value,'utf-8')) values_before = item.split(" ")
Yes, I know, this should be done in C. Later.
Finally, writing the Arduino sketch, which was as simple as reading from the serial line and turn lights on and off in a column. Neopixels are individually addressable, and you can drive them easily with the Neopixel library.
Results, when music is playing and the python script is running:
Looks cool and syncs very well with the music.