NeuralPi is a project where a Raspberry Pi mini PC can be used to emulate any guitar pedal, the Ibanez TS9 in this case. This project mainly consists of Raspberry Pi 4, Hifi Berry ADC + DAC, Elk Audio OS hardware and NeuralPi VST3 plugin.
Why Guitar Pedals?
The pursuit of every musician is to compose a novel tune. Thus, guitar pedals are used by musicians to find creative ways to enhance their sound. Guitar pedals essentially alter the sound made by the electric guitar in a specific and controlled way, to generate sound effects.
A few examples of commonly used guitar pedals being reverb, delay, wah, tremolo and distortion, each one adding a unique effect to the sound. The below audio clips demonstrate the effect of reverb in the original tune.
Acoustic guitar sound without effect
Acoustic guitar sound with reverb effect
Hardware and Software setup
The detailed instructions for hardware assembly, Elk Audio OS setup can be found here.
Talking about the AI inference part, for the dataset, samples from the original TS9 pedal from the HT40 speaker (set on overdrive channel at 100% gain) were recorded by SM57 dynamic microphone. The samples were then trained on three neural network models: WaveNet, Stateless LSTM, and Stateful LSTM. Their final results were classified based on three attributes and the model with the best aggregate score, the Stateful LSTM was chosen.
The neural network used for this project is a Stateful LSTM. The table below shows the performance of all potential neural networks, with Stateful LSTM, scored highest.
Also, an improvement over the conventional LSTM model which produces sound with a lot of static, the plugin code optimizes it by implementing the model through an interface called RTNeutral.
The implementation through RTNeutral optimizes the CPU usage (for running the trained model) from 99% to 16%. The final step is to run the Real-time plugin on Elk Audio OS with JUCE framework.
Hence, with only 16% of CPU usage on RPi4, other effects such as cab simulation (impulse response), reverb, delay or flange can be added. Also, Elk Audio OS is specially designed for low latency audio, hence there is no interference from other processes in the plugin performance. This sums up the basic idea of the project and its performance analysis.
Also, since the project is open-source, enthusiasts can tinker with the code to emulate other guitar pedals as well. NeuralPi allows adding new models from a remote computer. To know more about the NeuralPi VST3 plugin, please visit the GitHub page.
Hrushikesh Vazurkar is an alumni of VNIT Nagpur, CSE Department. He is passionate about competitive programming, web development and AI.
