NEURAL ACOUSTIC MULTIPOLE SPLATTING
FOR ROOM IMPULSE RESPONSE SYNTHESIS

Geonwoo Baek and Jung-Woo Choi

School of Electrical Engineering, KAIST

[ arXiv ]

Abstract

Room Impulse Response (RIR) prediction at arbitrary receiver positions is essential for practical applications such as spatial audio ren- dering. We propose Neural Acoustic Multipole Splatting (NAMS), which synthesizes RIRs at unseen receiver positions by learning the positions of neural acoustic multipoles and predicting their emitted signals and directivities using a neural network. Representing sound fields through a combination of multipoles offers sufficient flexibility to express complex acoustic scenes while adhering to physical constraints such as the Helmholtz equation. We also introduce a pruning strategy that starts from a dense splatting of neural acoustic multipoles and progressively eliminates redundant ones during training. Experiments conducted on both real and synthetic datasets indicate that the proposed method surpasses previous approaches on most metrics while maintaining rapid inference. Ablation studies reveal that multipole splatting with pruning achieves better performance than the monopole model with just 20% of the poles.

Framework

overall_architecture

Datasets

MeshRIR
MeshRIR
MeshRIR
MeshRIR
Apartment566
Apartment566
Apartment566
Apartment566
Apartment566
Apartment566
Apartment716
Apartment716
Apartment716
Apartment716
Apartment716
Apartment716

Multipoles

MeshRIR
MeshRIR
Intialized multipoles
MeshRIR
Trained multipoles
MeshRIR
Apartment566
Apartment566
Intialized multipoles
Apartment566
Trained multipoles
Apartment566
Apartment716
Apartment716
Intialized multipoles
Apartment716
Trained multipoles
Apartment716

RIR samples

MeshRIR
Configuration 1
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram
Configuration 2
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram
Apartment566
Configuration 1
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram
Configuration 2
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram
Apartment716
Configuration 1
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram
Configuration 2
Room
GT
Waveform
NAMS
Waveform
AVR
Waveform
NAF
Waveform
GT
Spectrum
NAMS
Spectrum
AVR
Spectrum
NAF
Spectrum
GT
Spectrogram
NAMS
Spectrogram
AVR
Spectrogram
NAF
Spectrogram