Compute Audio Log Mel Spectrogram Feature: A Step Guide – Python Audio Processing

By | May 12, 2022
0 Comment

Audio log mel spectrogram is common used in many deep learning model. In this tutorial, we will introduce you how to compute it from a raw audio.

Read data from audio

Here, we will read audio data from a wav file using python librosa library.

The Difference Between scipy.io.wavfile.read() and librosa.load() in Python – Python Tutorial

Here is the example code:

import librosa
import numpy as np
np.set_printoptions(threshold=np.inf)
audio_file = 'videoInvite.wav'

sr = 8000
audio, fs = librosa.load(audio_file, sr = sr, mono = True)
print(audio[0:20])

Run this code, we will see:

[ 3.0961567e-06 -2.1818676e-06  2.8186564e-06  1.0201638e-05
 -4.9446317e-06 -4.2198621e-06  4.2476267e-07  7.9612337e-06
  9.0552930e-06 -4.1868593e-06 -2.4259425e-06  4.1954286e-06
  7.3456872e-06  1.7391468e-06 -6.7588239e-06  3.7534046e-06
  3.1059151e-06 -6.5793356e-06  1.5550982e-06  3.1163402e-06]

Compute audio mel-spectrogram

We can use librosa.feature.melspectrogram() function to compute audio mel-spectrogram. To understand this function, you can read:

Compute and Display Audio Mel-spectrogram in Python – Python Tutorial

melspectrum = librosa.feature.melspectrogram(y=audio, sr=sr, hop_length= 512, window='hann', n_mels=80)
print(melspectrum[0:5,0:10])

Then, we will find melspectrum is:

[ 3.0961567e-06 -2.1818676e-06  2.8186564e-06  1.0201638e-05
 -4.9446317e-06 -4.2198621e-06  4.2476267e-07  7.9612337e-06
  9.0552930e-06 -4.1868593e-06 -2.4259425e-06  4.1954286e-06
  7.3456872e-06  1.7391468e-06 -6.7588239e-06  3.7534046e-06
  3.1059151e-06 -6.5793356e-06  1.5550982e-06  3.1163402e-06]
[[1.84848536e-09 4.47172033e-09 1.23510420e-08 2.69899156e-05
  4.22528130e-04 3.90061119e-04 1.13106653e-04 8.93045799e-05
  3.84244631e-05 5.31111837e-05]
 [3.74597153e-09 5.28659427e-09 7.71775799e-09 2.74749004e-06
  2.42373062e-05 6.17956757e-05 9.28348454e-05 5.55870429e-05
  2.58068376e-05 4.59296134e-05]
 [2.06403472e-09 5.17794518e-09 5.25394528e-09 1.22987717e-06
  3.89917586e-05 8.19252236e-05 4.52492277e-05 4.63025244e-05
  2.42831338e-05 1.96055444e-05]
 [1.45449641e-09 3.63059183e-09 8.71321149e-09 1.17916386e-06
  7.66097437e-05 2.09815320e-04 6.19430575e-05 2.38208031e-05
  6.31134799e-06 8.03145304e-06]
 [1.89813743e-09 3.58354257e-09 1.14143885e-08 2.46488753e-06
  9.43929626e-05 2.89292540e-04 1.60610536e-04 9.87687672e-05
  1.35738592e-05 3.52506468e-05]]

Compute audio log mel-spectrogram

We will create a function to compute. Here is the example code:

def logmel(x, C=1, clip_val=1e-5):
    """
    PARAMS
    ------
    C: compression factor
    """
    return np.log(np.clip(x, a_min=clip_val, a_max = np.amax(x)) * C)

log_melspectrum = logmel(melspectrum)
print(log_melspectrum[0:5,0:10])

Run this code, we will find the log mel spectrogram of this audio is:

[[-11.512925  -11.512925  -11.512925  -10.520047   -7.7692547  -7.849207
   -9.087179   -9.323458  -10.166817   -9.843123 ]
 [-11.512925  -11.512925  -11.512925  -11.512925  -10.627618   -9.691677
   -9.284689   -9.797561  -10.564871   -9.9884   ]
 [-11.512925  -11.512925  -11.512925  -11.512925  -10.152161   -9.409703
  -10.0033245  -9.980314  -10.625729  -10.839698 ]
 [-11.512925  -11.512925  -11.512925  -11.512925   -9.476787   -8.469283
   -9.689295  -10.644951  -11.512925  -11.512925 ]
 [-11.512925  -11.512925  -11.512925  -11.512925   -9.268044   -8.148072
   -8.736528   -9.222729  -11.207365  -10.253027 ]]

We can input it to our ai model to train.

Leave a Reply