mirrors/pulseaudio
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/pulseaudio/pulseaudio.git synced 2025-11-05 13:29:57 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
pulseaudio/src/modules/echo-cancel/echo-cancel.h

175 lines
7 KiB
C
Raw Normal View History

echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/***
This file is part of PulseAudio.
Copyright 2010 Arun Raghavan <arun.raghavan@collabora.co.uk>
PulseAudio is free software; you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation; either version 2.1 of the License,
or (at your option) any later version.
PulseAudio is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with PulseAudio; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
USA.
***/
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
#ifndef fooechocancelhfoo
#define fooechocancelhfoo
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif
#include <pulse/sample.h>
#include <pulse/channelmap.h>
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
#include <pulsecore/core.h>
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#include <pulsecore/macro.h>
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
#include <speex/speex_echo.h>
echo-cancel: Add speex preprocessing This allows the selective enabling of speex' preprocessing algorithms before running the echo-canceller -- for now this includes automatic gain control, noise suppression and echo suppression. It's all off by default for now, though at some point in the near future we might want to enable at least denoising by default. The denoising works pretty well, though we might want to add a way to tweak the noise-suppression knob that libspeex provides. The AGC option is just a stop-gap -- we need a real AGC mechanism that tweaks the source volume rather than doing this in software. The speex documentation mentions VAD and dereverb, but it appears that these are not complete yet. We don't do all this in a separate module from module-echo-cancel to avoid the overhead of adding another virtual source. It makes more sense to make a separate virtual source module that can be used for cases where preprocessing is useful but AEC is not (for e.g. noise suppression for fan noise in a recording application). Another reason to keep this integrated with the AEC module is that the echo suppression bits use the speex echo canceller state. This does leak some information about the AEC implementation into module-echo-cancel, but this is unavoidable.
2011-05-19 13:29:49 +05:30
#include <speex/speex_preprocess.h>
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
#include "adrian.h"
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* Common data structures */
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
typedef struct pa_echo_canceller_msg pa_echo_canceller_msg;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
typedef struct pa_echo_canceller_params pa_echo_canceller_params;
struct pa_echo_canceller_params {
union {
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
struct {
SpeexEchoState *state;
echo-cancel: Move speex preprocessing out of the main module I initially included put the Speex preprocessing assuming that we'd want to use the digital gain control and noise suppression from Speex for all echo cancelling implementations. In practice, we're probably going to get entire implementations all processing in one package (WebRTC, custom modules from various vendors, etc.). This moves out this preprocessing and related knobs into the speex implementation, which serves to clean out all implementation-specific details from the module-echo-cancel core.
2011-08-25 17:47:05 +05:30
SpeexPreprocessState *pp_state;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
} speex;
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#ifdef HAVE_ADRIAN_EC
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
struct {
uint32_t blocksize;
AEC *aec;
} adrian;
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#endif
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
#ifdef HAVE_WEBRTC
struct {
/* This is a void* so that we don't have to convert this whole file
* to C++ linkage. apm is a pointer to an AudioProcessing object */
void *apm;
uint32_t blocksize;
pa_sample_spec sample_spec;
echo-cancel: Hook up WebRTC analog gain control
2011-11-04 16:38:30 +05:30
pa_bool_t agc;
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
} webrtc;
#endif
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* each canceller-specific structure goes here */
} priv;
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Set this if canceller can do drift compensation. Also see set_drift()
* below */
pa_bool_t drift_compensation;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
};
typedef struct pa_echo_canceller pa_echo_canceller;
struct pa_echo_canceller {
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Initialise canceller engine. */
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
pa_bool_t (*init) (pa_core *c,
pa_echo_canceller *ec,
echo-cancel: Let AEC module determine source/sink spec Since the source and sink specification will need to be determined by the AEC algorithm (can it handle multi-channel audio, does it work with a fixed sample rate, etc.), we negotiate these using inout parameters at initialisation time. There is opportunity to make the sink-handling more elegant. Since the sink data isn't used for playback (just processing), we could pass through the data as-is and resample to the required spec before using in the cancellation algorithm. This isn't too important immediately, but would be nice to have.
2010-09-06 22:23:51 +05:30
pa_sample_spec *source_ss,
pa_channel_map *source_map,
pa_sample_spec *sink_ss,
pa_channel_map *sink_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes,
echo-cancel: Let AEC module determine source/sink spec Since the source and sink specification will need to be determined by the AEC algorithm (can it handle multi-channel audio, does it work with a fixed sample rate, etc.), we negotiate these using inout parameters at initialisation time. There is opportunity to make the sink-handling more elegant. Since the sink data isn't used for playback (just processing), we could pass through the data as-is and resample to the required spec before using in the cancellation algorithm. This isn't too important immediately, but would be nice to have.
2010-09-06 22:23:51 +05:30
const char *args);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* You should have only one of play()+record() or run() set. The first
* works under the assumption that you'll handle buffering and matching up
* samples yourself. If you set run(), module-echo-cancel will handle
* synchronising the playback and record streams. */
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine 'nframes' playback frames. */
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
void (*play) (pa_echo_canceller *ec, const uint8_t *play);
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine 'nframes' record frames. nframes processed frames are
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
* returned in out. */
void (*record) (pa_echo_canceller *ec, const uint8_t *rec, uint8_t *out);
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
/* Feed the engine nframes playback and record frames, with a reasonable
* effort at keeping the two in sync. nframes processed frames are
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
* returned in out. */
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void (*run) (pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Optional callback to set the drift, expressed as the ratio of the
* difference in number of playback and capture samples to the number of
* capture samples, for some instant of time. This is used only if the
* canceller signals that it supports drift compensation, and is called
* before record(). The actual implementation needs to derive drift based
* on point samples -- the individual values are not accurate enough to use
* as-is. */
/* NOTE: the semantics of this function might change in the future. */
void (*set_drift) (pa_echo_canceller *ec, float drift);
/* Free up resources. */
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
void (*done) (pa_echo_canceller *ec);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
/* Structure with common and engine-specific canceller parameters. */
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
pa_echo_canceller_params params;
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
/* msgobject that can be used to send messages back to the main thread */
pa_echo_canceller_msg *msg;
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
};
echo-cancel: Add infrastructure for cancellers to do AGC This adds some infrastructure for canceller implementations to also perform acoustic gain control. Cancellers now have a couple of new API calls that allow them to get/set capture volume. This is made slightly complex by the fact that cancellation happens in thread context while most volume mangling needs to be done in main context. To deal with this, while getting the volume we save source volume updates as they are propagated to thread context and use this cached value for queries. To set the volume, we send an async message to main context and let that set the source volume.
2011-11-04 16:17:26 +05:30
/* Functions to be used by the canceller analog gain control routines */
void pa_echo_canceller_get_capture_volume(pa_echo_canceller *ec, pa_cvolume *v);
void pa_echo_canceller_set_capture_volume(pa_echo_canceller *ec, pa_cvolume *v);
modules: Add null/dummy echo canceller I find a dummy/passthrough implementation useful for AEC debugging Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-07-24 10:16:56 +02:00
/* Null canceller functions */
pa_bool_t pa_null_ec_init(pa_core *c, pa_echo_canceller *ec,
pa_sample_spec *source_ss, pa_channel_map *source_map,
pa_sample_spec *sink_ss, pa_channel_map *sink_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes, const char *args);
modules: Add null/dummy echo canceller I find a dummy/passthrough implementation useful for AEC debugging Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-07-24 10:16:56 +02:00
void pa_null_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
void pa_null_ec_done(pa_echo_canceller *ec);
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#ifdef HAVE_SPEEX
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
/* Speex canceller functions */
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
pa_bool_t pa_speex_ec_init(pa_core *c, pa_echo_canceller *ec,
echo-cancel: Let AEC module determine source/sink spec Since the source and sink specification will need to be determined by the AEC algorithm (can it handle multi-channel audio, does it work with a fixed sample rate, etc.), we negotiate these using inout parameters at initialisation time. There is opportunity to make the sink-handling more elegant. Since the sink data isn't used for playback (just processing), we could pass through the data as-is and resample to the required spec before using in the cancellation algorithm. This isn't too important immediately, but would be nice to have.
2010-09-06 22:23:51 +05:30
pa_sample_spec *source_ss, pa_channel_map *source_map,
pa_sample_spec *sink_ss, pa_channel_map *sink_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes, const char *args);
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void pa_speex_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Split out speex code from the core module This splits out the echo-cancelling core from the PA-specific bits to allow us to plug in other echo-cancellation engines.
2010-09-06 15:51:20 +05:30
void pa_speex_ec_done(pa_echo_canceller *ec);
build-sys: Make speex library optional make speex library dependency optional, this affects the resampler and the echo canceller module this patch supersedes an earlier patch proposal and addresses the following comments: * fix order of pa_echo_canceller_method_t enum and ec_table (Frederic) * the default resampler is speex if available as before, otherwise ffmpeg (Arun) * does not touch the Adrian EC implementation (see separate patch) (Arun)
2011-12-11 16:07:41 +01:00
#endif
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#ifdef HAVE_ADRIAN_EC
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
/* Adrian Andre's echo canceller */
echo-cancel: Add SSE optimisation to the adrian module Optimises the core inner-product function, which takes the most CPU. The SSE-optimised bits of the adrian echo canceller only if the CPU that PA is running on actually supports SSE.
2010-09-21 20:42:32 +05:30
pa_bool_t pa_adrian_ec_init(pa_core *c, pa_echo_canceller *ec,
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
pa_sample_spec *source_ss, pa_channel_map *source_map,
pa_sample_spec *sink_ss, pa_channel_map *sink_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes, const char *args);
echo-cancel: Mark immutable parameters as const in vfunc Marks the recording and playback streams as const in the pa_echo_canceller->run method for clarity.
2010-09-07 15:07:39 +05:30
void pa_adrian_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
echo-cancel: Add alternative echo-cancellation implementation This adds Andre Adrian's AEC implementation from his intercom project (http://andreadrian.de/intercom/) as an alternative to the speex echo cancellation routines. Since the implementation was in C++ and not in the form of a library, I have converted the code to C and made a local copy of the implementation. The implementation actually works on floating point data, so we can tweak it to work with both integer and floating point samples (currently we just use S16LE).
2010-09-07 13:43:58 +05:30
void pa_adrian_ec_done(pa_echo_canceller *ec);
echo-cance: Make Adrian canceller optional makes the Adrian echo canceller implementation optional at compile time this patch supersedes an earlier patch proposal and addresses the following comments: * separate patch from speex dependency rework (Arun) * check that at least one EC implementation is available (Arun) * properly align yes/no in configure summary for Adrian (Frederic)
2011-12-11 16:07:42 +01:00
#endif
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
#ifdef HAVE_WEBRTC
/* WebRTC canceller functions */
PA_C_DECL_BEGIN
pa_bool_t pa_webrtc_ec_init(pa_core *c, pa_echo_canceller *ec,
pa_sample_spec *source_ss, pa_channel_map *source_map,
pa_sample_spec *sink_ss, pa_channel_map *sink_map,
echo-cancel: Enable different blocksizes for sink and source In order to support different blocksizes for source and sink (e.g, for 4-to-1 beamforming/echo canceling which involves 4 record channels and 1 playback channel) the AEC API is altered: The blocksize for source and sink may differ (due to different sample specs) but the number of frames that are processed in one invokation of the AEC implementation's run() function is the same for the playback and the record stream. Consequently, the AEC implementation's init() function initalizes 'nframes' instead of 'blocksize' and the source's and sink's blocksizes are derived from 'nframes'. The old API also caused code duplication in each AEC implementation's init function for the compution of the blocksize, which is eliminated by the new API. Signed-off-by: Stefan Huber <s.huber@bct-electronic.com> Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
2012-12-20 11:33:04 +01:00
uint32_t *nframes, const char *args);
echo-cancel: Plug in WebRTC drift compensation This adds the ability for echo cancellers to provide their own drift compensation, and hooks in the appropriate bits to implement this in the WebRTC canceller. We do this by introducing an alternative model for the canceller. So far, the core engine just provided a run() method which was given blocksize-sized chunks of playback and record samples. The new model has the engine provide play() and record() methods that can (in theory) be called by the playback and capture threads. The latter would actually do the processing required. In addition to this a set_drift() method may be provided by the implementation. PA will provide periodic samples of the drift to the engine. These values need to be aggregated and processed over some time, since the point values vary quite a bit (but generally fit a linear regression reasonably accurately). At some point of time, we might move the actual drift calculation into PA and change the semantics of this function. NOTE: This needs further testing before being deemed ready for wider use.
2011-10-07 16:28:11 +05:30
void pa_webrtc_ec_play(pa_echo_canceller *ec, const uint8_t *play);
void pa_webrtc_ec_record(pa_echo_canceller *ec, const uint8_t *rec, uint8_t *out);
void pa_webrtc_ec_set_drift(pa_echo_canceller *ec, float drift);
echo-cancel: Add the WebRTC echo canceller This adds the WebRTC echo canceller as another module-echo-cancel backend. We're exposing both the full echo canceller as well as the mobile echo control version as modargs. Pending items: 1. The mobile canceller doesn't seem to work at the moment. 2. We still need to add bits to hook in drift compensation (to support sink and source from different devices). The most controversial part of this patch would probably be the mandatory build-time dependency on a C++ compiler. If the optional --enable-webrtc-aec is set, then there's also a dependency on libstdc++.
2011-09-19 13:41:13 +05:30
void pa_webrtc_ec_run(pa_echo_canceller *ec, const uint8_t *rec, const uint8_t *play, uint8_t *out);
void pa_webrtc_ec_done(pa_echo_canceller *ec);
PA_C_DECL_END
#endif
echo-cancel: Add multiple include protection for header
2011-08-24 10:34:19 +05:30
#endif /* fooechocancelhfoo */
Reference in a new issue Copy permalink