F0 is also called fundamental frequency, it can be used to distinguish the emotion of different persons. For example, paper F0-CONTOURS IN EMOTIONAL SPEECH gives us an exmaple.
We can find different emotion has different F0 distribution.
In this tutorial, we will introduce how to extract F0 from an audio in python.
How to extract F0 in python?
We can use python librosa to extract.
There are two functions to extract F0 in librosa, they are: librosa.pyin() and librosa.yin().
We will compare them.
We can find: librosa.pyin() compute F0 using probabilistic YIN, however, librosa.yin() get F0 using YIN.
As to return values, we also can find:
| librosa.pyin() | librosa.yin() |
| f0: np.ndarray [shape=(…, n_frames)]
time series of fundamental frequencies in Hertz. |
f0: np.ndarray [shape=(…, n_frames)]
time series of fundamental frequencies in Hertz. |
| voiced_flag: np.ndarray [shape=(…, n_frames)]
time series containing boolean flags indicating whether a frame is voiced or not. |
|
| voiced_prob: np.ndarray [shape=(…, n_frames)]
time series containing the probability that a frame is voiced. |
We will an example to show you how to use them.
Read a wav audio
Here is the example code:
import librosa import numpy as np # wav_file = r'F:\1221306.wav' wav_data, sr = librosa.load(wav_file, sr=8000, mono=True) print(wav_data.shape)
The sample rate of this wav file is 8000, single-channel.
Run this code, we will get:
(510025,)
Compute FO using librosa.yin()
Then we can extract F0 using code below:
#extract F0 using yin
f0 = librosa.yin(wav_data, fmin = librosa.note_to_hz('C2'), fmax= librosa.note_to_hz('C7'))
print(f0.shape)
print(f0)
In this code, we should notice:
librosa.note_to_hz(‘C2’) = ~65 Hz
librosa.note_to_hz(‘C7’) = ~2093 Hz
Run this code, we will get:
(997,) [1146.45969137 1146.04461105 1146.73431302 668.79892066 615.45648497 562.96476058 558.14046971 ...... 544.65753385 760.37344188 2094.17439589 2084.58654002 2205. 887.35019783 884.74519291 885.03654623 715.84000869]
Why is the shape of fo is 997?
Because hop_length is None, then
if hop_length is None:
hop_length = frame_length // 4
Here frame_length = 2048, hop_length = 512.
So 510025/512 + 1 = 997
We also should notice sr=22050, we have not used the sample rate of wav file.
If we use sr = 8000, we will get:
f0 = librosa.yin(wav_data, sr = 8000, fmin = librosa.note_to_hz('C2'), fmax= librosa.note_to_hz('C7'))
print(f0.shape)
We will get:
(997,) [ 415.94909437 415.79849834 416.04873035 242.64813448 223.29486983 204.25025327 202.49994366 200.51790152 200.35671551 419.35011289 418.36068785 198.91673928 ....... 184.81714292 184.20936447 185.28837089 196.6593469 428.93107994 474.25675725 477.35569135 438.61124584 157.25481533 157.23045796 197.60817555 275.87245057 759.79116404 756.31257688 763.0352532 321.94111486 320.99598836 1690.94812045 259.7151959 ]
Display F0 feature
We can use matplotlib to display F0, here is an example:
times = librosa.times_like(f0) import matplotlib.pyplot as plt fig, ax = plt.subplots() ax.set(title='YIN fundamental frequency estimation') ax.plot(times, f0, label='f0', color='cyan', linewidth=3) ax.legend(loc='upper right') plt.show()
We will see:
We also can extract F0 using librosa.pyin(), here is an example code:
f0, vid, vpd = librosa.pyin(wav_data, sr = 8000, fmin = librosa.note_to_hz('C2'), fmax= librosa.note_to_hz('C7'))
print(f0.shape)
print(f0)
print(vid.shape)
print(vpd.shape)
We will get:
(997,)
[ nan 415.30469758 415.30469758 242.69934088 223.84553226
204.08500573 202.90956259 200.57894748 200.57894748 420.13030572
417.71053322 199.42369775 191.52112393 181.81906999 175.625645
173.60841241 172.60850154 ...... .6955272 181.81906999
206.45635927 207.65234879 463.47885582 611.56401673 633.1307752
727.27627998 361.54373842 237.15608027 255.65002913 252.7136381
249.8109744 234.43211036 174.61411572 175.625645 263.14114678
217.47308448 218.73289324 429.95038611 517.24107876 260.11871293
218.73289324 168.66611791 152.89100418 156.464662 163.86452094
164.81377846 231.73942791 432.44106336 206.45635927 183.93170582
184.99721136 197.13312122 429.95038611 474.31216054 477.05982293
437.46578648 312.929324 157.37105257 197.13312122 nan
nan nan nan nan nan
nan nan]
(997,)
(997,)
We will find there exist some nan value in F0, we should replace them.
f0 = np.nan_to_num(f0)
Then we also can display f0. We will see:
