Audio log mel spectrogram is common used in many deep learning model. In this tutorial, we will introduce you how to compute it from a raw audio.
Read data from audio
Here, we will read audio data from a wav file using python librosa library.
The Difference Between scipy.io.wavfile.read() and librosa.load() in Python – Python Tutorial
Here is the example code:
import librosa import numpy as np np.set_printoptions(threshold=np.inf) audio_file = 'videoInvite.wav' sr = 8000 audio, fs = librosa.load(audio_file, sr = sr, mono = True) print(audio[0:20])
Run this code, we will see:
[ 3.0961567e-06 -2.1818676e-06 2.8186564e-06 1.0201638e-05 -4.9446317e-06 -4.2198621e-06 4.2476267e-07 7.9612337e-06 9.0552930e-06 -4.1868593e-06 -2.4259425e-06 4.1954286e-06 7.3456872e-06 1.7391468e-06 -6.7588239e-06 3.7534046e-06 3.1059151e-06 -6.5793356e-06 1.5550982e-06 3.1163402e-06]
Compute audio mel-spectrogram
We can use librosa.feature.melspectrogram() function to compute audio mel-spectrogram. To understand this function, you can read:
Compute and Display Audio Mel-spectrogram in Python – Python Tutorial
melspectrum = librosa.feature.melspectrogram(y=audio, sr=sr, hop_length= 512, window='hann', n_mels=80) print(melspectrum[0:5,0:10])
Then, we will find melspectrum is:
[ 3.0961567e-06 -2.1818676e-06 2.8186564e-06 1.0201638e-05 -4.9446317e-06 -4.2198621e-06 4.2476267e-07 7.9612337e-06 9.0552930e-06 -4.1868593e-06 -2.4259425e-06 4.1954286e-06 7.3456872e-06 1.7391468e-06 -6.7588239e-06 3.7534046e-06 3.1059151e-06 -6.5793356e-06 1.5550982e-06 3.1163402e-06] [[1.84848536e-09 4.47172033e-09 1.23510420e-08 2.69899156e-05 4.22528130e-04 3.90061119e-04 1.13106653e-04 8.93045799e-05 3.84244631e-05 5.31111837e-05] [3.74597153e-09 5.28659427e-09 7.71775799e-09 2.74749004e-06 2.42373062e-05 6.17956757e-05 9.28348454e-05 5.55870429e-05 2.58068376e-05 4.59296134e-05] [2.06403472e-09 5.17794518e-09 5.25394528e-09 1.22987717e-06 3.89917586e-05 8.19252236e-05 4.52492277e-05 4.63025244e-05 2.42831338e-05 1.96055444e-05] [1.45449641e-09 3.63059183e-09 8.71321149e-09 1.17916386e-06 7.66097437e-05 2.09815320e-04 6.19430575e-05 2.38208031e-05 6.31134799e-06 8.03145304e-06] [1.89813743e-09 3.58354257e-09 1.14143885e-08 2.46488753e-06 9.43929626e-05 2.89292540e-04 1.60610536e-04 9.87687672e-05 1.35738592e-05 3.52506468e-05]]
Compute audio log mel-spectrogram
We will create a function to compute. Here is the example code:
def logmel(x, C=1, clip_val=1e-5):
"""
PARAMS
------
C: compression factor
"""
return np.log(np.clip(x, a_min=clip_val, a_max = np.amax(x)) * C)
log_melspectrum = logmel(melspectrum)
print(log_melspectrum[0:5,0:10])
Run this code, we will find the log mel spectrogram of this audio is:
[[-11.512925 -11.512925 -11.512925 -10.520047 -7.7692547 -7.849207 -9.087179 -9.323458 -10.166817 -9.843123 ] [-11.512925 -11.512925 -11.512925 -11.512925 -10.627618 -9.691677 -9.284689 -9.797561 -10.564871 -9.9884 ] [-11.512925 -11.512925 -11.512925 -11.512925 -10.152161 -9.409703 -10.0033245 -9.980314 -10.625729 -10.839698 ] [-11.512925 -11.512925 -11.512925 -11.512925 -9.476787 -8.469283 -9.689295 -10.644951 -11.512925 -11.512925 ] [-11.512925 -11.512925 -11.512925 -11.512925 -9.268044 -8.148072 -8.736528 -9.222729 -11.207365 -10.253027 ]]
We can input it to our ai model to train.