mirror of
https://gitlab.freedesktop.org/pipewire/pipewire.git
synced 2026-07-06 00:06:33 -04:00
module-rtp: Add audio codec support to audio.c and replace opus.c with it
Opus was integrated as a completely separate code path to the PCM audio processing found in audio.c. This is actually not ideal, since the only part that actually is Opus specific is the part that en- and decodes from and to PCM. The rest is 1:1 the same PCM handling. For this reason, it is much better to instead add audio codec support to audio.c, meaning that the code in there can now encode PCM audio right before sending it out as RTP, and decode incoming packets to PCM right before actually processing the decoded audio data. This significantly modifies how stream.c initializes the PCM audio path, since the audio codec feature is new. It now treats the Opus subtype as an audio codec selector instead of a selector for an entirely alternate code path (like how MIDI integration remains entirely separate). Since audio codecs usually require their frames to be decoded in order, this also integrates the RTP jitter buffer in the RTP module. Opus is now integrated as such a codec in audio.c. When it is selected, incoming packets in rtp_audio_receive() are first inserted into the jitter buffer. That buffer then outputs packets in order, and then, these packets are decoded to PCM. The rest of the processing chain goes as usual. A similar route is used for when the jitter buffer signals packet loss to be able to apply PLC. For encoding, it is similar (except that no jitter buffer is involved); in rtp_audio_flush_packets(), when Opus is active, the PCM data is rerouted to be fed to Opus for encoding, and the Opus output is then placed into the iovec array instead of the original PCM. This also improves overall Opus support; it supports S16 PCM data in addition to F32 data, correctly checks the ptime, sample rate etc. for Opus compatibility, computes an ideal bitrate, allows for manual bitrate selection and encoding complexity adjustment (via the new stream properties "opus.encoder.bitrate" and "opus.encoder.complexity"), sets several other Opus CTLs to fixed values, supports the Opus restricted-lowdelay mode (sacrifices Speech code paths for lower latency, enabled by setting the "opus.encoder.restricted-lowdelay" stream property to true), and also uses Opus' PLC in case of packet loss. The audio codec interface is designed such that adding other codecs in the future is easily doable. New integrations need to implement the function pointers found in the rtp_audio_codec structure, and expose an instance of such a custom rtp_audio_codec structure instance (see the get_rtp_opus_codec() implementation for an example).
This commit is contained in:
parent
31bb82e116
commit
6b524fd596
12 changed files with 1461 additions and 471 deletions
|
|
@ -199,7 +199,8 @@ PW_LOG_TOPIC(mod_topic, "mod." NAME);
|
|||
"( sess.min-ptime=<minimum packet time in milliseconds, default:2> ) " \
|
||||
"( sess.max-ptime=<maximum packet time in milliseconds, default:20> ) " \
|
||||
"( sess.media=<string, the media type audio|midi|opus, default audio> ) " \
|
||||
"( audio.format=<format, default:"DEFAULT_RAW_AUDIO_FORMAT"> ) " \
|
||||
"( audio.format=<format, default:"DEFAULT_RAW_AUDIO_FORMAT " for media type audio, " \
|
||||
DEFAULT_OPUS_AUDIO_FORMAT " for media type opus, not used for MIDI> ) " \
|
||||
"( audio.rate=<sample rate, default:"SPA_STRINGIFY(DEFAULT_RATE)"> ) " \
|
||||
"( audio.channels=<number of channels, default:"SPA_STRINGIFY(DEFAULT_CHANNELS)"> ) " \
|
||||
"( audio.position=<channel map, default:"DEFAULT_POSITION"> ) " \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue