一、引言:声音是XR沉浸感的另一半

视觉给了我们XR世界的"形",而音频给了它"魂"。没有空间音频,虚拟世界只是一张平面的画;有了空间音频,用户能听到身后有人走动、头顶有飞机飞过、隔壁房间传来闷响,沉浸感才能真正成立。

空间音频的核心体验:

  • 方位感:能分辨声音来自前后左右上下
  • 距离感:近处声音清晰响亮,远处声音微弱模糊
  • 运动感:声音随头部转动实时变化(头追踪音频)
  • 环境感:房间混响让空间听起来真实
  • 遮挡感:墙壁挡住的声音变闷变小

第二代骁龙XR2+平台(XR2+ Gen2)集成了专门的Hexagon DSP音频处理子系统,支持HRTF卷积、多声源混音、房间混响等全部在DSP上完成,CPU零占用,延迟<5ms。本文将系统讲解空间音频原理和实战开发。

二、空间音频渲染管线

在这里插入图片描述

2.1 完整音频处理链

声源输入(多路PCM)
    ↓
距离模型(Distance Model) → 衰减曲线 + 多普勒效应
    ↓
HRTF滤波 → 头相关传递函数卷积(方位+仰角)
    ↓
房间混响(Reverb) → 早期反射 + 后期混响(RT60)
    ↓
遮挡/阻塞(Occlusion) → 射线检测 + 低通滤波
    ↓
双耳混合(Binaural Mix) → L/R声道叠加
    ↓
头部追踪更新 → IMU实时更新声源相对方位
    ↓
后处理(EQ/压缩/ANC)
    ↓
DAC输出 → 耳机/扬声器

在这里插入图片描述

2.2 HRTF头相关传递函数

HRTF(Head-Related Transfer Function)描述了声音从空间某点传播到人耳鼓膜的过程中,头部、耳廓、躯干对声音的滤波效应。每个人的耳廓形状不同,HRTF也不同。

"""
HRTF空间音频处理核心实现
基于SOFA格式HRTF数据库
"""
import numpy as np
from scipy import signal
from typing import Tuple, Optional
import os


class HRTFFilter:
    """HRTF滤波器"""
    
    def __init__(self, sofa_path: str = None, sample_rate: int = 48000):
        self.sample_rate = sample_rate
        self.hrir_l = None  # 左耳脉冲响应
        self.hrir_r = None  # 右耳脉冲响应
        self.azimuths = None
        self.elevations = None
        self.radii = None
        
        if sofa_path and os.path.exists(sofa_path):
            self.load_sofa(sofa_path)
        else:
            # 使用简化HRTF模型(实际项目请加载真实SOFA数据)
            self._init_default_hrtf()
    
    def _init_default_hrtf(self, num_az: int = 36, num_el: int = 18):
        """初始化简化HRTF模型(用于演示)"""
        # 方位角和仰角网格
        self.azimuths = np.linspace(-180, 180, num_az, endpoint=False)
        self.elevations = np.linspace(-90, 90, num_el)
        self.radii = np.array([1.0])
        
        # 简化HRIR: 用ITD(双耳时间差) + ILD(双耳声级差)模型
        filter_len = 128
        self.hrir_l = np.zeros((num_az, num_el, 1, filter_len))
        self.hrir_r = np.zeros((num_az, num_el, 1, filter_len))
        
        for ai, az in enumerate(self.azimuths):
            for ei, el in enumerate(self.elevations):
                az_rad = np.radians(az)
                el_rad = np.radians(el)
                
                # ITD模型 (头半径约8.75cm)
                head_radius = 0.0875
                speed_of_sound = 343.0
                itd = -head_radius / speed_of_sound * (np.sin(el_rad) + np.cos(az_rad))
                itd_samples = int(itd * self.sample_rate)
                
                # ILD模型 (高频阴影效应)
                ild = 5.0 * np.sin(az_rad)  # dB
                
                # 生成简化HRIR
                t = np.arange(filter_len) / self.sample_rate
                # 左耳
                delay_l = max(0, itd_samples)
                self.hrir_l[ai, ei, 0] = self._make_hrir(
                    filter_len, delay_l / self.sample_rate, -ild/2)
                # 右耳
                delay_r = max(0, -itd_samples)
                self.hrir_r[ai, ei, 0] = self._make_hrir(
                    filter_len, delay_r / self.sample_rate, ild/2)
    
    def _make_hrir(self, length: int, delay_sec: float, gain_db: float) -> np.ndarray:
        """生成简化HRIR脉冲"""
        t = np.arange(length) / self.sample_rate - delay_sec
        # 简单低通+延迟模型
        hrir = np.exp(-t * 200) * np.sinc(t * 2000) * (10 ** (gain_db / 20))
        hrir[t < 0] = 0
        return hrir / (np.max(np.abs(hrir)) + 1e-8)
    
    def load_sofa(self, path: str):
        """加载SOFA格式HRTF数据库(需安装python-sofa)"""
        try:
            import netCDF4
            # 实际项目中解析SOFA格式
            # Data.IR: [M, R, N] 测量数×接收器×采样点
            # SourcePosition: [M, 3] 方位角/仰角/距离
            # 此处为占位
            print(f"[HRTF] 请加载真实SOFA数据库: {path}")
            self._init_default_hrtf()
        except ImportError:
            print("[HRTF] netCDF4未安装,使用默认HRTF模型")
            self._init_default_hrtf()
    
    def get_hrir(self, azimuth: float, elevation: float, distance: float = 1.0) -> Tuple[np.ndarray, np.ndarray]:
        """
        获取指定方位的HRIR(双线性插值)
        
        Args:
            azimuth: 方位角(-180~180度), 0=正前方, 90=右侧
            elevation: 仰角(-90~90度), 0=水平, 90=正上方
            distance: 距离(米)
        
        Returns:
            (left_hrir, right_hrir)
        """
        # 方位角归一化
        az = azimuth % 360
        if az > 180: az -= 360
        
        # 最近邻插值(简化)
        ai = np.argmin(np.abs(self.azimuths - az))
        ei = np.argmin(np.abs(self.elevations - np.clip(elevation, -90, 90)))
        
        hl = self.hrir_l[ai, ei, 0].copy()
        hr = self.hrir_r[ai, ei, 0].copy()
        
        # 距离衰减
        dist_gain = min(1.0, 1.0 / max(distance, 0.1))
        # 距离相关的高频衰减(空气吸收)
        hl *= dist_gain
        hr *= dist_gain
        
        return hl, hr
    
    def interpolate_hrir(self, azimuth: float, elevation: float) -> Tuple[np.ndarray, np.ndarray]:
        """双线性插值获取HRIR"""
        az = ((azimuth + 180) % 360) - 180
        el = np.clip(elevation, -90, 90)
        
        # 找到四个邻居
        ai = np.searchsorted(self.azimuths, az) - 1
        ei = np.searchsorted(self.elevations, el) - 1
        ai = np.clip(ai, 0, len(self.azimuths) - 2)
        ei = np.clip(ei, 0, len(self.elevations) - 2)
        
        az0, az1 = self.azimuths[ai], self.azimuths[ai+1]
        el0, el1 = self.elevations[ei], self.elevations[ei+1]
        
        t = (az - az0) / (az1 - az0 + 1e-8)
        u = (el - el0) / (el1 - el0 + 1e-8)
        
        # 双线性插值
        hl = ((1-t)*(1-u)*self.hrir_l[ai, ei, 0] +
              t*(1-u)*self.hrir_l[ai+1, ei, 0] +
              (1-t)*u*self.hrir_l[ai, ei+1, 0] +
              t*u*self.hrir_l[ai+1, ei+1, 0])
        hr = ((1-t)*(1-u)*self.hrir_r[ai, ei, 0] +
              t*(1-u)*self.hrir_r[ai+1, ei, 0] +
              (1-t)*u*self.hrir_r[ai, ei+1, 0] +
              t*u*self.hrir_r[ai+1, ei+1, 0])
        
        return hl, hr


class SpatialAudioSource:
    """空间音频源"""
    
    def __init__(self, position: np.ndarray = None, audio_data: np.ndarray = None,
                 sample_rate: int = 48000):
        self.position = position or np.array([0.0, 0.0, 1.0])
        self.velocity = np.zeros(3)
        self.audio_data = audio_data  # 单声道PCM
        self.sample_rate = sample_rate
        self.volume = 1.0
        self.loop = False
        self.playing = False
        self._play_pos = 0
        self.reverb_send = 0.3  # 混响发送量
    
    def play(self, audio_data: np.ndarray = None, loop: bool = False):
        if audio_data is not None:
            self.audio_data = audio_data
        self.loop = loop
        self.playing = True
        self._play_pos = 0
    
    def stop(self):
        self.playing = False
    
    def get_samples(self, num_samples: int, listener_pos: np.ndarray,
                    listener_rot: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
        """
        获取空间化后的立体声样本
        
        Args:
            num_samples: 需要的采样点数
            listener_pos: 听者位置
            listener_rot: 听者旋转矩阵(3x3)
        
        Returns:
            (left_channel, right_channel)
        """
        if not self.playing or self.audio_data is None:
            return np.zeros(num_samples), np.zeros(num_samples)
        
        # 计算相对位置(听者坐标系)
        rel_pos = self.position - listener_pos
        rel_pos_local = listener_rot.T @ rel_pos  # 转换到听者局部坐标系
        
        # 计算方位角和仰角
        dist = np.linalg.norm(rel_pos_local)
        az = np.degrees(np.arctan2(rel_pos_local[0], -rel_pos_local[2]))  # 前=-Z, 右=+X
        el = np.degrees(np.arcsin(rel_pos_local[1] / (dist + 1e-8)))
        
        # 多普勒效应
        speed_of_sound = 343.0
        rel_vel = listener_rot.T @ self.velocity
        doppler_shift = 1.0 + np.dot(rel_vel, rel_pos_local / (dist + 1e-8)) / speed_of_sound
        doppler_shift = np.clip(doppler_shift, 0.9, 1.1)
        
        return dist, az, el, doppler_shift

2.3 空间音频渲染引擎

"""
空间音频渲染引擎
整合HRTF、混响、遮挡、距离模型
"""
import numpy as np
from scipy import signal


class SpatialAudioRenderer:
    """空间音频渲染器"""
    
    def __init__(self, sample_rate: int = 48000, block_size: int = 256):
        self.sample_rate = sample_rate
        self.block_size = block_size
        
        self.hrtf = HRTFFilter(sample_rate=sample_rate)
        self.sources = []  # type: List[SpatialAudioSource]
        
        # 听者状态
        self.listener_pos = np.zeros(3)
        self.listener_rot = np.eye(3)
        
        # 混响(简化Schroeder混响)
        self.reverb = SchroederReverb(sample_rate)
        
        # 输出缓冲
        self._out_l = np.zeros(block_size)
        self._out_r = np.zeros(block_size)
        self._reverb_l = np.zeros(block_size)
        self._reverb_r = np.zeros(block_size)
        
        # 卷积状态(overlap-save)
        self._conv_state_l = {}
        self._conv_state_r = {}
    
    def add_source(self, source: SpatialAudioSource) -> int:
        self.sources.append(source)
        return len(self.sources) - 1
    
    def set_listener_pose(self, position: np.ndarray, rotation: np.ndarray):
        """更新听者位姿(头部追踪回调)"""
        self.listener_pos = position.copy()
        self.listener_rot = rotation.copy()
    
    def render_block(self) -> Tuple[np.ndarray, np.ndarray]:
        """渲染一个音频块"""
        self._out_l.fill(0)
        self._out_r.fill(0)
        self._reverb_l.fill(0)
        self._reverb_r.fill(0)
        
        for idx, src in enumerate(self.sources):
            if not src.playing:
                continue
            
            # 获取源相对方位
            rel_pos = src.position - self.listener_pos
            rel_local = self.listener_rot.T @ rel_pos
            dist = np.linalg.norm(rel_local)
            
            if dist < 0.1: dist = 0.1
            
            az = np.degrees(np.arctan2(rel_local[0], -rel_local[2]))
            el = np.degrees(np.arcsin(np.clip(rel_local[1] / dist, -1, 1)))
            
            # 获取源音频数据
            mono = src.get_samples(self.block_size)
            
            # 距离衰减(反平方律)
            dist_gain = min(1.0, 1.0 / dist)
            # 高频空气吸收
            air_absorb = np.exp(-dist * 0.1)
            
            # 获取HRTF
            hl, hr = self.hrtf.get_hrir(az, el, dist)
            
            # 卷积(使用FFT快速卷积)
            convolved_l = signal.fftconvolve(mono, hl, mode='same') * dist_gain * src.volume
            convolved_r = signal.fftconvolve(mono, hr, mode='same') * dist_gain * src.volume
            
            # 干声混合
            self._out_l += convolved_l
            self._out_r += convolved_r
            
            # 湿声(混响)
            self._reverb_l += convolved_l * src.reverb_send
            self._reverb_r += convolved_r * src.reverb_send
        
        # 混响处理
        rev_l, rev_r = self.reverb.process(self._reverb_l, self._reverb_r)
        self._out_l += rev_l
        self._out_r += rev_r
        
        # 限制器(防削波)
        peak = max(np.max(np.abs(self._out_l)), np.max(np.abs(self._out_r)))
        if peak > 0.95:
            self._out_l *= 0.95 / peak
            self._out_r *= 0.95 / peak
        
        return self._out_l.copy(), self._out_r.copy()


class SchroederReverb:
    """Schroeder混响器(并行梳状滤波器 + 全通滤波器)"""
    
    def __init__(self, sample_rate: int, rt60: float = 0.8):
        self.sr = sample_rate
        self.rt60 = rt60
        
        # 4个并行梳状滤波器
        comb_delays = [int(0.0297 * sr), int(0.0371 * sr),
                       int(0.0411 * sr), int(0.0437 * sr)]
        g = 0.805  # 反馈增益
        
        self.combs_l = [CombFilter(d, g) for d in comb_delays]
        self.combs_r = [CombFilter(d + 5, g) for d in comb_delays]
        
        # 2个串联全通滤波器
        ap_delays = [int(0.005 * sr), int(0.0017 * sr)]
        self.aps_l = [AllPassFilter(d, 0.7) for d in ap_delays]
        self.aps_r = [AllPassFilter(d + 3, 0.7) for d in ap_delays]
    
    def process(self, in_l: np.ndarray, in_r: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
        out_l = np.zeros_like(in_l)
        out_r = np.zeros_like(in_r)
        
        for comb in self.combs_l:
            out_l += comb.process(in_l)
        for comb in self.combs_r:
            out_r += comb.process(in_r)
        
        out_l /= len(self.combs_l)
        out_r /= len(self.combs_r)
        
        for ap in self.aps_l:
            out_l = ap.process(out_l)
        for ap in self.aps_r:
            out_r = ap.process(out_r)
        
        return out_l * 0.5, out_r * 0.5


class CombFilter:
    """梳状滤波器"""
    def __init__(self, delay: int, feedback: float):
        self.buf = np.zeros(delay + 100)
        self.delay = delay
        self.fb = feedback
        self.idx = 0
    
    def process(self, x: np.ndarray) -> np.ndarray:
        y = np.zeros_like(x)
        for i in range(len(x)):
            d = self.buf[(self.idx - self.delay) % len(self.buf)]
            y[i] = d
            self.buf[self.idx] = x[i] + self.fb * d
            self.idx = (self.idx + 1) % len(self.buf)
        return y


class AllPassFilter:
    """全通滤波器"""
    def __init__(self, delay: int, feedback: float):
        self.buf = np.zeros(delay + 100)
        self.delay = delay
        self.fb = feedback
        self.idx = 0
    
    def process(self, x: np.ndarray) -> np.ndarray:
        y = np.zeros_like(x)
        for i in range(len(x)):
            d = self.buf[(self.idx - self.delay) % len(self.buf)]
            y[i] = -self.fb * x[i] + d
            self.buf[self.idx] = x[i] + self.fb * d
            self.idx = (self.idx + 1) % len(self.buf)
        return y

三、Unity中空间音频实战

3.1 Unity原生AudioSource配置

// SpatialAudioSetup.cs - Unity空间音频快速配置
using UnityEngine;
using UnityEngine.Audio;

[RequireComponent(typeof(AudioSource))]
public class SpatialAudioSetup : MonoBehaviour
{
    [Header("空间音频设置")]
    [Range(0, 5)]
    public float dopplerLevel = 1.0f;
    [Range(0, 360)]
    public float spread = 0f;
    public AudioRolloffMode rolloffMode = AudioRolloffMode.Logarithmic;
    public float minDistance = 1f;
    public float maxDistance = 50f;
    
    [Header("XR设置")]
    public bool headTracking = true;
    public bool useHRTF = true;
    
    private AudioSource _audioSource;
    
    void Awake()
    {
        _audioSource = GetComponent<AudioSource>();
        ConfigureSpatialAudio();
    }
    
    void ConfigureSpatialAudio()
    {
        // 启用3D空间化
        _audioSource.spatialize = true;
        _audioSource.spatialBlend = 1.0f;  // 完全3D
        
        // 距离设置
        _audioSource.rolloffMode = rolloffMode;
        _audioSource.minDistance = minDistance;
        _audioSource.maxDistance = maxDistance;
        
        // 多普勒
        _audioSource.dopplerLevel = dopplerLevel;
        
        // 空间扩散
        _audioSource.spread = spread;
        
        // XR插件设置
        // 在Project Settings > Audio中选择XR Spatializer插件
        // 如: Meta XR Audio / Oculus Spatializer / Resonance Audio
        Debug.Log($"[SpatialAudio] 配置完成: {gameObject.name}");
    }
    
    void Update()
    {
        // 如果使用XR头部追踪,AudioListener跟随Main Camera
        // Unity XR Plugin Management会自动处理
    }
}

3.2 动态3D音效管理器

// SpatialAudioManager.cs - 3D音效管理器
using UnityEngine;
using System.Collections.Generic;

public class SpatialAudioManager : MonoBehaviour
{
    public static SpatialAudioManager Instance { get; private set; }
    
    [Header("音频池")]
    public int poolSize = 20;
    public GameObject audioSourcePrefab;
    public AudioMixerGroup sfxMixerGroup;
    public AudioMixerGroup musicMixerGroup;
    
    private List<AudioSource> _audioPool = new List<AudioSource>();
    private Transform _listenerTransform;
    
    [Header("环境音效")]
    public AudioClip[] ambientSounds;
    [Range(0, 1)]
    public float ambientVolume = 0.3f;
    
    void Awake()
    {
        Instance = this;
        _listenerTransform = Camera.main.transform;
        InitializePool();
    }
    
    void InitializePool()
    {
        for (int i = 0; i < poolSize; i++)
        {
            var go = new GameObject($"SpatialAudio_{i}");
            go.transform.SetParent(transform);
            var src = go.AddComponent<AudioSource>();
            src.playOnAwake = false;
            src.spatialize = true;
            src.spatialBlend = 1.0f;
            src.rolloffMode = AudioRolloffMode.Logarithmic;
            src.minDistance = 0.5f;
            src.maxDistance = 30f;
            src.outputAudioMixerGroup = sfxMixerGroup;
            _audioPool.Add(src);
        }
    }
    
    public AudioSource Play3D(AudioClip clip, Vector3 position, float volume = 1f,
                              float pitch = 1f, float minDist = 1f, float maxDist = 30f,
                              bool loop = false)
    {
        var src = GetAvailableSource();
        if (src == null) return null;
        
        src.transform.position = position;
        src.clip = clip;
        src.volume = volume;
        src.pitch = pitch;
        src.minDistance = minDist;
        src.maxDistance = maxDist;
        src.loop = loop;
        src.Play();
        
        if (!loop)
            StartCoroutine(ReturnToPool(src, clip.length / pitch));
        
        return src;
    }
    
    public AudioSource PlayAttached(AudioClip clip, Transform target, float volume = 1f)
    {
        var src = GetAvailableSource();
        if (src == null) return null;
        
        src.transform.SetParent(target);
        src.transform.localPosition = Vector3.zero;
        src.clip = clip;
        src.volume = volume;
        src.spatialBlend = 1f;
        src.Play();
        
        if (!src.loop)
            StartCoroutine(ReturnToPool(src, clip.length, true));
        
        return src;
    }
    
    System.Collections.IEnumerator ReturnToPool(AudioSource src, float delay, bool detach = false)
    {
        yield return new WaitForSeconds(delay + 0.1f);
        src.Stop();
        if (detach)
            src.transform.SetParent(transform);
        src.gameObject.SetActive(true);
    }
    
    AudioSource GetAvailableSource()
    {
        foreach (var src in _audioPool)
        {
            if (!src.isPlaying)
            {
                src.gameObject.SetActive(true);
                return src;
            }
        }
        return null; // 池满
    }
    
    // 便捷方法
    public void PlayFootstep(Vector3 position, AudioClip[] variations)
    {
        var clip = variations[Random.Range(0, variations.Length)];
        Play3D(clip, position, volume: 0.5f, pitch: Random.Range(0.9f, 1.1f));
    }
    
    public void PlayCollision(Vector3 position, float impactForce)
    {
        // 根据碰撞力度调节音量
        float vol = Mathf.Clamp01(impactForce / 10f);
        // 碰撞音效...
    }
}

四、空间音频开发流程

方位不准

有延迟

爆音/杂音

质量达标

空间音频开发

音频系统配置

选择Spatializer插件
Meta/Resonance/Google

配置AudioMixer分组
SFX/Music/Ambient/Voice

设置采样率48kHz
缓冲大小256samples

声源制作

准备单声道音频素材
立体声无法正确空间化

设置3D参数
距离/多普勒/扩散

配置AudioSource
SpatialBlend=1

高级效果

房间混响Reverb
根据场景空间大小

遮挡/阻塞Occlusion
射线检测+低通滤波

门户/区域Portal
房间之间声音传递

材质响应
不同表面反射不同

头部追踪

AudioListener跟随Camera

HRTF随头部旋转实时更新

旋转平滑避免跳变

性能优化

限制同时播放声源数
最多20-30个3D声源

距离裁剪
超过maxDistance停止

优先级管理
重要声音优先

DSP处理在Hexagon上
CPU零占用

音频质量验证

校准HRTF参数/检查坐标系

减小缓冲/使用硬件DSP

检查增益/添加限制器

空间音频完成

五、音频遮挡实现

// AudioOcclusion.cs - 音频遮挡效果
using UnityEngine;

[RequireComponent(typeof(AudioSource))]
public class AudioOcclusion : MonoBehaviour
{
    [Header("遮挡设置")]
    public LayerMask occlusionLayers;
    [Range(0, 1)]
    public float occlusionVolume = 0.1f;  // 被遮挡时的音量
    public float occlusionLPCutoff = 1000f;  // 被遮挡时低通截止频率
    public float transitionSpeed = 5f;
    
    private AudioSource _audioSource;
    private AudioLowPassFilter _lowPass;
    private Transform _listener;
    private float _targetVolume = 1f;
    private float _targetLP = 20000f;
    
    void Awake()
    {
        _audioSource = GetComponent<AudioSource>();
        _lowPass = gameObject.AddComponent<AudioLowPassFilter>();
        _lowPass.cutoffFrequency = 20000f;
    }
    
    void Update()
    {
        if (_listener == null)
            _listener = Camera.main.transform;
        if (_listener == null) return;
        
        Vector3 toListener = _listener.position - transform.position;
        float dist = toListener.magnitude;
        
        // 射线检测遮挡
        bool occluded = Physics.Raycast(
            transform.position, toListener.normalized,
            dist, occlusionLayers);
        
        // 检查射线数量(多点检测更准确)
        if (!occluded)
        {
            // 从声源顶部/底部/两侧发射射线
            Vector3[] offsets = {
                transform.right * 0.3f,
                -transform.right * 0.3f,
                transform.up * 0.3f,
            };
            foreach (var offset in offsets)
            {
                if (Physics.Raycast(transform.position + offset, toListener.normalized,
                    dist, occlusionLayers))
                {
                    occluded = true;
                    break;
                }
            }
        }
        
        // 设置目标值
        _targetVolume = occluded ? occlusionVolume : 1f;
        _targetLP = occluded ? occlusionLPCutoff : 20000f;
        
        // 平滑过渡
        _audioSource.volume = Mathf.Lerp(_audioSource.volume,
            _audioSource.volume * _targetVolume, Time.deltaTime * transitionSpeed);
        _lowPass.cutoffFrequency = Mathf.Lerp(_lowPass.cutoffFrequency,
            _targetLP, Time.deltaTime * transitionSpeed);
    }
}

六、性能优化

空间音频的DSP处理虽然不在CPU上,但如果声源数量过多、距离裁剪不当,仍然会耗尽DSP算力和内存带宽。以下是关键优化策略。

6.1 声源数量限制

骁龙XR2+ Gen2的Hexagon DSP可以同时处理32+个空间声源(HRTF卷积+混响),但这并不意味着应用中可以无限制地添加3D声源。实际开发中应控制在20个以内为安全线,留出余量给系统服务(麦克风处理、语音识别等)。

// AudioSourcePool.cs - 声源数量控制与优先级管理
using UnityEngine;
using System.Collections.Generic;
using System.Linq;

public class AudioSourcePool : MonoBehaviour
{
    public static AudioSourcePool Instance { get; private set; }
    
    [Header("池配置")]
    public int maxSimultaneousSources = 20;  // 最大同时播放声源数
    public int poolSize = 30;                 // 预分配池大小
    
    [Header("优先级")]
    public int voicePriority = 100;      // 语音最高
    public int combatPriority = 80;       // 战斗音效
    public int ambientPriority = 20;     // 环境音最低
    
    private List<PooledSource> _activeSources = new List<PooledSource>();
    private Queue<AudioSource> _availableSources = new Queue<AudioSource>();
    
    struct PooledSource
    {
        public AudioSource source;
        public int priority;
        public float startTime;
    }
    
    void Awake()
    {
        Instance = this;
        for (int i = 0; i < poolSize; i++)
        {
            var go = new GameObject($"AudioPool_{i}");
            go.transform.SetParent(transform);
            var src = go.AddComponent<AudioSource>();
            src.playOnAwake = false;
            src.spatialize = true;
            src.spatialBlend = 1f;
            src.rolloffMode = AudioRolloffMode.Logarithmic;
            _availableSources.Enqueue(src);
        }
    }
    
    public AudioSource Play3D(AudioClip clip, Vector3 pos, int priority,
                               float volume = 1f, float minDist = 1f,
                               float maxDist = 30f, bool loop = false)
    {
        // 池满时:踢掉最低优先级的非循环声源
        if (_activeSources.Count >= maxSimultaneousSources)
        {
            EvictLowestPriority(priority);
        }
        
        AudioSource src = GetAvailableSource();
        if (src == null) return null;
        
        src.transform.position = pos;
        src.clip = clip;
        src.volume = volume;
        src.minDistance = minDist;
        src.maxDistance = maxDist;
        src.loop = loop;
        src.Play();
        
        _activeSources.Add(new PooledSource
        {
            source = src,
            priority = priority,
            startTime = Time.time,
        });
        
        return src;
    }
    
    void EvictLowestPriority(int newPriority)
    {
        // 找到优先级最低且可以被踢掉的声源
        var candidates = _activeSources
            .Where(s => !s.source.loop && s.priority < newPriority)
            .OrderBy(s => s.priority)
            .ThenBy(s => s.startTime)
            .ToList();
        
        if (candidates.Count > 0)
        {
            var victim = candidates[0];
            victim.source.Stop();
            _activeSources.Remove(victim);
            _availableSources.Enqueue(victim.source);
        }
    }
    
    AudioSource GetAvailableSource()
    {
        // 清理已播放完的
        _activeSources.RemoveAll(s =>
        {
            if (!s.source.isPlaying && !s.source.loop)
            {
                _availableSources.Enqueue(s.source);
                return true;
            }
            return false;
        });
        
        if (_availableSources.Count > 0)
            return _availableSources.Dequeue();
        return null;
    }
    
    public int GetActiveSourceCount() => _activeSources.Count;
}

6.2 距离裁剪

超出最大听觉范围的声源应该完全停止DSP处理,而不只是音量衰减到零。Unity的AudioSource有maxDistance属性,当声源超过此距离时会自动停止输出,但HRTF卷积仍在执行。更好的做法是动态启停声源。

// AudioDistanceCulling.cs - 距离裁剪管理
using UnityEngine;

[RequireComponent(typeof(AudioSource))]
public class AudioDistanceCulling : MonoBehaviour
{
    public float cullDistance = 25f;     // 超过此距离完全停用
    public float resumeDistance = 22f;   // 低于此距离恢复(滞回避免频繁切换)
    public float checkInterval = 0.2f;   // 检查间隔(不需要每帧检查)
    
    private AudioSource _source;
    private Transform _listener;
    private float _checkTimer;
    private bool _isCulled = false;
    
    void Awake()
    {
        _source = GetComponent<AudioSource>();
        _source.rolloffMode = AudioRolloffMode.Logarithmic;
        _source.maxDistance = cullDistance;
    }
    
    void Update()
    {
        _checkTimer += Time.deltaTime;
        if (_checkTimer < checkInterval) return;
        _checkTimer = 0;
        
        if (_listener == null)
            _listener = Camera.main?.transform;
        if (_listener == null) return;
        
        float dist = Vector3.Distance(transform.position, _listener.position);
        
        if (_isCulled && dist < resumeDistance)
        {
            _isCulled = false;
            _source.enabled = true;
            // 不自动Play,由外部控制
        }
        else if (!_isCulled && dist > cullDistance)
        {
            _isCulled = true;
            _source.enabled = false;  // 完全停用,不占DSP资源
        }
    }
    
    void OnDrawGizmosSelected()
    {
        Gizmos.color = _isCulled ? Color.red : Color.green;
        Gizmos.DrawWireSphere(transform.position, cullDistance);
        Gizmos.color = Color.yellow;
        Gizmos.DrawWireSphere(transform.position, resumeDistance);
    }
}

6.3 DSP硬件加速与延迟控制

骁龙XR2+ Gen2的Hexagon DSP音频子系统支持HRTF卷积、多声源混音、房间混响全部在DSP上完成。关键配置参数如下:

参数推荐值说明
采样率48kHzXR标准,兼顾质量和带宽
缓冲大小256 samples5.3ms延迟,平衡延迟与CPU开销
HRTF滤波器长度128 taps足够精度,不过长占用DSP
同时声源上限20-32个超出后优先级管理
混响RT600.3-1.2s根据房间大小调节
混响发送量0.2-0.4干湿比平衡
// AudioPerformanceConfig.cs - 硬件音频配置
using UnityEngine;
using UnityEngine.Audio;

public class AudioPerformanceConfig : MonoBehaviour
{
    [Header("DSP配置")]
    public int dspBufferSize = 256;       // 256 samples @ 48kHz = 5.3ms
    public int sampleRate = 48000;
    public int maxVoices = 24;            // DSP最大声源数
    
    [Header("AudioMixer配置")]
    public AudioMixer mainMixer;
    
    void Awake()
    {
        // 设置DSP缓冲大小(越小延迟越低,但CPU开销越大)
        // 必须在Awake中设置,运行时无法修改
        AudioSettings.Reset(AudioSettings.outputSampleRate);
        
        var config = AudioSettings.GetConfiguration();
        config.sampleRate = sampleRate;
        config.dspBufferSize = dspBufferSize;
        config.numRealVoices = maxVoices;
        config.numVirtualVoices = maxVoices * 2;
        AudioSettings.Reset(config.sampleRate);
        AudioSettings.SetConfiguration(config);
        
        // XR音频推荐设置
        AudioSettings.speakerMode = AudioSpeakerMode.Stereo;  // 双耳渲染
        
        float actualLatency = (float)dspBufferSize / sampleRate * 1000f;
        Debug.Log($"[Audio] DSP: {dspBufferSize} samples @ {sampleRate}Hz "
                  $"=> {actualLatency:F1}ms延迟, 最大{maxVoices}声源");
    }
    
    void Update()
    {
        // 监控DSP负载
        float dspLoad = AudioSettings.GetDSPProcessorChannelCount() > 0 ? 
            (float)maxVoices / AudioSettings.GetDSPBufferInfo() : 0f;
        
        // 实际监控应使用平台API:
        // Meta XR: OVRPlugin.GetAudioOutLatency()
        // 通用: AudioSettings.GetConfiguration().dspBufferSize
    }
}

6.4 音频性能优化清单

优化项方法效果
声源数量池化管理+优先级踢除DSP负载降低30-50%
距离裁剪超范围完全停用AudioSource减少无效HRTF卷积
缓冲大小256 samples(5.3ms)平衡延迟与开销
混响预计算离线烘焙Impulse Response运行时省去实时卷积
音频压缩使用Vorbis而非PCM节省50-70%内存
单声道素材3D声源必须单声道立体声无法正确空间化
LOD音频远距离切换低采样率版本减少带宽
虚拟声源超过限制的声源标记virtual不占DSP,仅记状态

七、总结

空间音频是XR沉浸感不可或缺的一半。本文系统介绍了:

  1. HRTF原理:头相关传递函数、方位角/仰角/距离三维定位
  2. 音频渲染管线:距离模型→HRTF卷积→混响→遮挡→双耳渲染→头部追踪
  3. 核心算法实现:HRTF滤波器、Schroeder混响器、空间音频渲染引擎
  4. Unity实战:AudioSource配置、3D音频对象池、音频遮挡
  5. 性能优化:声源数量限制、距离裁剪、DSP硬件加速

骁龙XR2+ Gen2的Hexagon DSP可以同时处理32+个空间声源,CPU零占用,端到端延迟<5ms,为沉浸式XR音频体验提供了强大的硬件支撑。

下一篇我们将探讨手势识别与眼动追踪交互开发。

Logo

邀请您加入社区

更多推荐