mirrors/pulseaudio
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/pulseaudio/pulseaudio.git synced 2025-11-05 13:29:57 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
pulseaudio/src/modules/echo-cancel/echo-cancel.h

190 lines
7.7 KiB
C
Raw Normal View History

echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/***
This file is part of PulseAudio.
Copyright 2010 Arun Raghavan <arun.raghavan@collabora.co.uk>
PulseAudio is free software; you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation; either version 2.1 of the License,
or (at your option) any later version.
PulseAudio is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
update FSF addresses to FSF web page FSF addresses used in PA sources are no longer valid and rpmlint generates numerous warnings during packaging because of this. This patch changes all FSF addresses to FSF web page according to the GPL how-to: https://www.gnu.org/licenses/gpl-howto.en.html Done automatically by sed-ing through sources.
2014-11-26 14:14:51 +01:00
along with PulseAudio; if not, see <http://www.gnu.org/licenses/>.
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
***/
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
#ifndef fooechocancelhfoo
#define fooechocancelhfoo
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif
#include <pulse/sample.h>
#include <pulse/channelmap.h>
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
#include <pulsecore/core.h>
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#include <pulsecore/macro.h>
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#include <speex/speex_echo.h>
echo-cancel: Add speex preprocessing This allows the selective enabling of speex' preprocessing algorithms before running the echo-canceller -- for now this includes automatic gain control, noise suppression and echo suppression. It's all off by default for now, though at some point in the near future we might want to enable at least denoising by default. The denoising works pretty well, though we might want to add a way to tweak the noise-suppression knob that libspeex provides. The AGC option is just a stop-gap -- we need a real AGC mechanism that tweaks the source volume rather than doing this in software. The speex documentation mentions VAD and dereverb, but it appears that these are not complete yet. We don't do all this in a separate module from module-echo-cancel to avoid the overhead of adding another virtual source. It makes more sense to make a separate virtual source module that can be used for cases where preprocessing is useful but AEC is not (for e.g. noise suppression for fan noise in a recording application). Another reason to keep this integrated with the AEC module is that the echo suppression bits use the speex echo canceller state. This does leak some information about the AEC implementation into module-echo-cancel, but this is unavoidable.
2011-05-19 13:29:49 +05:30
#include <speex/speex_preprocess.h>
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
#include "adrian.h"
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* Common data structures */
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
typedef struct pa_echo_canceller_msg pa_echo_canceller_msg;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
typedef struct pa_echo_canceller_params pa_echo_canceller_params;
struct pa_echo_canceller_params {
union {
echo-cancel: Extend null implementation to arbitary sample specs The new null implementation works with arbitrary sample specs for source and sink. In particular, it handles a different number of channels for source and sink. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:05 +01:00
struct {
echo-cancel: Enable different sample specs for rec and out stream Enable advanced AEC methods to use different specs (i.e., number of channels) for rec and out stream. A typical application is beam forming resp. multi-channel AEC, which takes multiple record channels to produce an echo-canceled output stream. This commit alters the EC API as follows: the EC's init() used to get source and sink's sample spec/channel map. The new interface renamed source to rec and sink to play and additionally passes sample spec and channel map of the out stream. The new parameter names of init() {rec,play,out}_{ss,map} are more intuitive and also resemble to the parameter names known from run(). Both rec_{ss,map} and out_{ss,map} are initialized as we knew it from source_{ss,map} before being passed to init(). The previous EC implementations only require trivial changes, i.e., setting rec_{ss,map} to out_{ss,map} at the end of init() in case that out_{ss,map} is modified in init().
2013-02-18 16:31:03 +01:00
pa_sample_spec out_ss;
echo-cancel: Extend null implementation to arbitary sample specs The new null implementation works with arbitrary sample specs for source and sink. In particular, it handles a different number of channels for source and sink. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:05 +01:00
} null;
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
struct {
SpeexEchoState *state;
echo-cancel: Move speex preprocessing out of the main module I initially included put the Speex preprocessing assuming that we'd want to use the digital gain control and noise suppression from Speex for all echo cancelling implementations. In practice, we're probably going to get entire implementations all processing in one package (WebRTC, custom modules from various vendors, etc.). This moves out this preprocessing and related knobs into the speex implementation, which serves to clean out all implementation-specific details from the module-echo-cancel core.
2011-08-25 17:47:05 +05:30
SpeexPreprocessState *pp_state;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
} speex;
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#ifdef HAVE_ADRIAN_EC
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
struct {
uint32_t blocksize;
AEC *aec;
} adrian;
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#endif
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
#ifdef HAVE_WEBRTC
struct {
/* This is a void* so that we don't have to convert this whole file
* to C++ linkage. apm is a pointer to an AudioProcessing object */
void *apm;
echo-cancel: Fix webrtc canceller when rec channels != play channels The calculations around how many samples were sent to the canceller engine was not updated when we started supporting different channel counts for playback and capture.
2016-02-17 19:47:08 +05:30
unsigned int blocksize; /* in frames */
echo-cancel: webrtc canceller supports different in/out channel counts Needed for upcoming beamforming code.
2016-02-17 19:47:10 +05:30
pa_sample_spec rec_ss, play_ss, out_ss;
echo-cancel: Use webrtc's deinterleaved API This is required to have unequal channel counts on capture in and out streams, which is needed for beamforming to work. The deinterleaved API only works with floating point samples.
2016-02-17 19:47:12 +05:30
float *rec_buffer[PA_CHANNELS_MAX], *play_buffer[PA_CHANNELS_MAX]; /* for deinterleaved buffers */
echo-cancel: Allow enabling tracing output from the webrtc canceller
2016-02-17 19:47:00 +05:30
void *trace_callback;
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool agc;
echo-cancel: Start capture at a sane volume if we're doing webrtc AGC This is required to make sure the capture output has sufficient energy for the AGC to do its job.
2016-02-17 19:47:02 +05:30
bool first;
echo-cancel: Use anonymous unions for echo canceller params Makes this part of the code just a little less verbose.
2016-02-17 19:47:06 +05:30
unsigned int agc_start_volume;
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
} webrtc;
#endif
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* each canceller-specific structure goes here */
echo-cancel: Use anonymous unions for echo canceller params Makes this part of the code just a little less verbose.
2016-02-17 19:47:06 +05:30
};
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Set this if canceller can do drift compensation. Also see set_drift()
* below */
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool drift_compensation;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
};
typedef struct pa_echo_canceller pa_echo_canceller;
struct pa_echo_canceller {
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Initialise canceller engine. */
Fix alignment issues introduced by the pa_bool -> bool conversion
2013-06-28 01:03:47 +02:00
bool (*init) (pa_core *c,
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
pa_echo_canceller *ec,
echo-cancel: Enable different sample specs for rec and out stream Enable advanced AEC methods to use different specs (i.e., number of channels) for rec and out stream. A typical application is beam forming resp. multi-channel AEC, which takes multiple record channels to produce an echo-canceled output stream. This commit alters the EC API as follows: the EC's init() used to get source and sink's sample spec/channel map. The new interface renamed source to rec and sink to play and additionally passes sample spec and channel map of the out stream. The new parameter names of init() {rec,play,out}_{ss,map} are more intuitive and also resemble to the parameter names known from run(). Both rec_{ss,map} and out_{ss,map} are initialized as we knew it from source_{ss,map} before being passed to init(). The previous EC implementations only require trivial changes, i.e., setting rec_{ss,map} to out_{ss,map} at the end of init() in case that out_{ss,map} is modified in init().
2013-02-18 16:31:03 +01:00
pa_sample_spec *rec_ss,
pa_channel_map *rec_map,
pa_sample_spec *play_ss,
pa_channel_map *play_map,
pa_sample_spec *out_ss,
pa_channel_map *out_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes,
echo-cancel: Let AEC module determine source/sink spec Since the source and sink specification will need to be determined by the AEC algorithm (can it handle multi-channel audio, does it work with a fixed sample rate, etc.), we negotiate these using inout parameters at initialisation time. There is opportunity to make the sink-handling more elegant. Since the sink data isn't used for playback (just processing), we could pass through the data as-is and resample to the required spec before using in the cancellation algorithm. This isn't too important immediately, but would be nice to have.
2010-09-06 22:23:51 +05:30
const char *args);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* You should have only one of play()+record() or run() set. The first
* works under the assumption that you'll handle buffering and matching up
* samples yourself. If you set run(), module-echo-cancel will handle
* synchronising the playback and record streams. */
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine 'nframes' playback frames. */
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
void (*play) (pa_echo_canceller *ec, const uint8_t *play);
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine 'nframes' record frames. nframes processed frames are
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
* returned in out. */
void (*record) (pa_echo_canceller *ec, const uint8_t *rec, uint8_t *out);
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine nframes playback and record frames, with a reasonable
* effort at keeping the two in sync. nframes processed frames are
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
* returned in out. */
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void (*run) (pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Optional callback to set the drift, expressed as the ratio of the
* difference in number of playback and capture samples to the number of
* capture samples, for some instant of time. This is used only if the
* canceller signals that it supports drift compensation, and is called
* before record(). The actual implementation needs to derive drift based
* on point samples -- the individual values are not accurate enough to use
* as-is. */
/* NOTE: the semantics of this function might change in the future. */
void (*set_drift) (pa_echo_canceller *ec, float drift);
/* Free up resources. */
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
void (*done) (pa_echo_canceller *ec);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Structure with common and engine-specific canceller parameters. */
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
pa_echo_canceller_params params;
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
/* msgobject that can be used to send messages back to the main thread */
pa_echo_canceller_msg *msg;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
};
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
/* Functions to be used by the canceller analog gain control routines */
echo-cancel: Convert AGC API to deal with pa_volume_t It is expected that the underlying AGC mechanism will likely provide a single volume for the source rather than a per-channel volume. Dealing with per-channel volumes just adds complexity with regards to the actual volume setting (depending on whether volume sharing is enabled or not, we would set the volume on the source output of the virtual source, and their sample specs may be different). Using a single volume allows us to sidestep this problem entirely.
2016-02-25 17:58:38 +05:30
pa_volume_t pa_echo_canceller_get_capture_volume(pa_echo_canceller *ec);
void pa_echo_canceller_set_capture_volume(pa_echo_canceller *ec, pa_volume_t volume);
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
echo-cancel: Add function pa_echo_canceller_blocksize_power2() computes EC block size in frames (rounded down to nearest power-of-2) based on sample rate and milliseconds move code from speex AEC implementation to module-echo-cancel such that functionality can be reused by other AEC implementations Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2013-02-13 17:26:51 +01:00
/* Computes EC block size in frames (rounded down to nearest power-of-2) based
* on sample rate and milliseconds. */
uint32_t pa_echo_canceller_blocksize_power2(unsigned rate, unsigned ms);
modules: Add null/dummy echo canceller I find a dummy/passthrough implementation useful for AEC debugging Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-07-24 10:16:56 +02:00
/* Null canceller functions */
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool pa_null_ec_init(pa_core *c, pa_echo_canceller *ec,
Fix alignment issues introduced by the pa_bool -> bool conversion
2013-06-28 01:03:47 +02:00
pa_sample_spec *rec_ss, pa_channel_map *rec_map,
pa_sample_spec *play_ss, pa_channel_map *play_map,
pa_sample_spec *out_ss, pa_channel_map *out_map,
uint32_t *nframes, const char *args);
modules: Add null/dummy echo canceller I find a dummy/passthrough implementation useful for AEC debugging Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-07-24 10:16:56 +02:00
void pa_null_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
void pa_null_ec_done(pa_echo_canceller *ec);
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* Speex canceller functions */
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool pa_speex_ec_init(pa_core *c, pa_echo_canceller *ec,
Fix alignment issues introduced by the pa_bool -> bool conversion
2013-06-28 01:03:47 +02:00
pa_sample_spec *rec_ss, pa_channel_map *rec_map,
pa_sample_spec *play_ss, pa_channel_map *play_map,
pa_sample_spec *out_ss, pa_channel_map *out_map,
uint32_t *nframes, const char *args);
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void pa_speex_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
void pa_speex_ec_done(pa_echo_canceller *ec);
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#ifdef HAVE_ADRIAN_EC
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
/* Adrian Andre's echo canceller */
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool pa_adrian_ec_init(pa_core *c, pa_echo_canceller *ec,
Fix alignment issues introduced by the pa_bool -> bool conversion
2013-06-28 01:03:47 +02:00
pa_sample_spec *rec_ss, pa_channel_map *rec_map,
pa_sample_spec *play_ss, pa_channel_map *play_map,
pa_sample_spec *out_ss, pa_channel_map *out_map,
uint32_t *nframes, const char *args);
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void pa_adrian_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
void pa_adrian_ec_done(pa_echo_canceller *ec);
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#endif
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
#ifdef HAVE_WEBRTC
/* WebRTC canceller functions */
PA_C_DECL_BEGIN
Remove pa_bool_t and replace it with bool. commands used for this (executed from the pulseaudio/src directory): find . -regex '\(.*\.[hc]\|.*\.cc\|.*\.m4\)' -not -name 'macro.h' \ -a -not -name 'reserve.[ch]' -a -not -name 'reserve-monitor.[ch]' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'glib-mainloop.c' -a -not -name 'gkt-test.c' \ -a -not -name 'poll-win32.c' -a -not -name 'thread-win32.c' \ -a -not -name 'dllmain.c' -a -not -name 'gconf-helper.c' \ -exec sed -i -e 's/\bpa_bool_t\b/bool/g' \ -e 's/\bTRUE\b/true/g' -e 's/\bFALSE\b/false/g' {} \; and: sed -i -e '181,194!s/\bpa_bool_t\b/bool/' \ -e '181,194!s/\bTRUE\b/true/' -e \ '181,194!s/\bFALSE\b/false/' pulsecore/macro.h
2013-06-27 19:28:09 +02:00
bool pa_webrtc_ec_init(pa_core *c, pa_echo_canceller *ec,
Fix alignment issues introduced by the pa_bool -> bool conversion
2013-06-28 01:03:47 +02:00
pa_sample_spec *rec_ss, pa_channel_map *rec_map,
pa_sample_spec *play_ss, pa_channel_map *play_map,
pa_sample_spec *out_ss, pa_channel_map *out_map,
uint32_t *nframes, const char *args);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
void pa_webrtc_ec_play(pa_echo_canceller *ec, const uint8_t *play);
void pa_webrtc_ec_record(pa_echo_canceller *ec, const uint8_t *rec, uint8_t *out);
void pa_webrtc_ec_set_drift(pa_echo_canceller *ec, float drift);
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
void pa_webrtc_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
void pa_webrtc_ec_done(pa_echo_canceller *ec);
PA_C_DECL_END
#endif
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
#endif /* fooechocancelhfoo */
Reference in a new issue Copy permalink