Auditory Feedback Modulation
Auditory Feedback Modulation (AFM) is a real-time approach for investigating how speakers respond to changes in the auditory feedback of their own voice. During speech or sustained phonation, the acoustic signal is captured by a microphone, modified in real time, and presented back to the speaker through headphones.
AFM setup.
By systematically manipulating selected characteristics of the auditory feedback, AFM makes it possible to study the interaction between auditory perception and vocal motor control. Changes in the perceived voice can induce compensatory adjustments in the speaker's vocal production, providing insights into the mechanisms underlying voice control and voice quality. The resulting changes in vocal production can be captured and quantified from the acoustic signal.
To investigate these responses experimentally, the AFM concept was implemented as a real-time signal processing chain in KiRAT. This processing chain can be seen in the main window of KiRAT under medical signal processing – logopedic signal processing – objective speech analysis - feedback and is visualized in the screenshot below.
Screenshot of signal processing chain in KiRAT.
The AFM processing is based on a simplified model of speech production. The speech signal can be separated into an excitation component and the acoustic filtering effects of the vocal tract. Linear predictive coding (LPC) is used to estimate the vocal-tract characteristics and obtain the corresponding residual signal.
The residual signal obtained from the LPC analysis is used as the basis for the auditory feedback modulation. Additional noise is introduced into the residual signal to modify the perceived voice quality and create characteristics such as a more hoarse or breathy voice.
To adapt the added noise to the speech signal, four different noise-shaping approaches were implemented:
- Windowing based on pitch frequency
- Inverted windowing based on pitch frequency
- Shaping based on the residual signal envelope
- Pitch-based windowing combined with a constant noise component
The amount of added noise is controlled using the signal-to-noise ratio (SNR), allowing the strength of the modulation to be adjusted and compared across different conditions.
In addition to noise modulation, spectral decay is applied to control the distribution of the added noise across frequencies. A spectral decay of 2 dB per octave was used in the implemented system. Higher spectral decay values result in a more subdued and muted perception of the modified voice.
Signal processing overview.
The AFM demo can be operated through a user interface that provides different options for conducting and analyzing auditory feedback modulation experiments. During a live experiment, the selected modulation condition is applied while the current status and progress of the measurement are displayed.
In addition, the analysis view provides a visualization of the recorded speech signal and the extracted acoustic features. This allows the vocal response to the applied auditory perturbation to be observed and evaluated over time.
Previously recorded measurements can also be loaded and processed. The resulting data and analysis parameters can be visualized and summarized, allowing individual measurements as well as multiple recordings to be compared.
Screenshot of user interface of KiRAT.