It is expected that the underlying AGC mechanism will likely provide a
single volume for the source rather than a per-channel volume. Dealing
with per-channel volumes just adds complexity with regards to the
actual volume setting (depending on whether volume sharing is enabled or
not, we would set the volume on the source output of the virtual source,
and their sample specs may be different).
Using a single volume allows us to sidestep this problem entirely.
This is required to have unequal channel counts on capture in and out
streams, which is needed for beamforming to work. The deinterleaved API
only works with floating point samples.
The calculations around how many samples were sent to the canceller
engine was not updated when we started supporting different channel
counts for playback and capture.
FSF addresses used in PA sources are no longer valid and rpmlint
generates numerous warnings during packaging because of this.
This patch changes all FSF addresses to FSF web page according to
the GPL how-to: https://www.gnu.org/licenses/gpl-howto.en.html
Done automatically by sed-ing through sources.
Enable advanced AEC methods to use different specs (i.e., number of
channels) for rec and out stream. A typical application is beam forming
resp. multi-channel AEC, which takes multiple record channels to produce
an echo-canceled output stream.
This commit alters the EC API as follows: the EC's init() used to get
source and sink's sample spec/channel map. The new interface renamed
source to rec and sink to play and additionally passes sample spec and
channel map of the out stream. The new parameter names of init()
{rec,play,out}_{ss,map} are more intuitive and also resemble to the
parameter names known from run(). Both rec_{ss,map} and out_{ss,map} are
initialized as we knew it from source_{ss,map} before being passed to
init(). The previous EC implementations only require trivial changes,
i.e., setting rec_{ss,map} to out_{ss,map} at the end of init() in case
that out_{ss,map} is modified in init().
computes EC block size in frames (rounded down to nearest power-of-2) based
on sample rate and milliseconds
move code from speex AEC implementation to module-echo-cancel such that
functionality can be reused by other AEC implementations
Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
The new null implementation works with arbitrary sample specs for source
and sink. In particular, it handles a different number of channels for
source and sink.
Signed-off-by: Stefan Huber <s.huber@bct-electronic.com>
Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
In order to support different blocksizes for source and sink (e.g, for
4-to-1 beamforming/echo canceling which involves 4 record channels and 1
playback channel) the AEC API is altered:
The blocksize for source and sink may differ (due to different sample
specs) but the number of frames that are processed in one invokation of
the AEC implementation's run() function is the same for the playback and
the record stream. Consequently, the AEC implementation's init()
function initalizes 'nframes' instead of 'blocksize' and the source's
and sink's blocksizes are derived from 'nframes'. The old API also
caused code duplication in each AEC implementation's init function for
the compution of the blocksize, which is eliminated by the new API.
Signed-off-by: Stefan Huber <s.huber@bct-electronic.com>
Acked-by: Peter Meerwald <p.meerwald@bct-electronic.com>
makes the Adrian echo canceller implementation optional at compile time
this patch supersedes an earlier patch proposal and addresses the following
comments:
* separate patch from speex dependency rework (Arun)
* check that at least one EC implementation is available (Arun)
* properly align yes/no in configure summary for Adrian (Frederic)
make speex library dependency optional, this affects the resampler
and the echo canceller module
this patch supersedes an earlier patch proposal and addresses the following
comments:
* fix order of pa_echo_canceller_method_t enum and ec_table (Frederic)
* the default resampler is speex if available as before, otherwise ffmpeg (Arun)
* does not touch the Adrian EC implementation (see separate patch) (Arun)
This adds some infrastructure for canceller implementations to also
perform acoustic gain control. Cancellers now have a couple of new API
calls that allow them to get/set capture volume.
This is made slightly complex by the fact that cancellation happens in
thread context while most volume mangling needs to be done in main
context. To deal with this, while getting the volume we save source
volume updates as they are propagated to thread context and use this
cached value for queries. To set the volume, we send an async message to
main context and let that set the source volume.
This adds the ability for echo cancellers to provide their own drift
compensation, and hooks in the appropriate bits to implement this in the
WebRTC canceller.
We do this by introducing an alternative model for the canceller. So
far, the core engine just provided a run() method which was given
blocksize-sized chunks of playback and record samples. The new model has
the engine provide play() and record() methods that can (in theory) be
called by the playback and capture threads. The latter would actually do
the processing required.
In addition to this a set_drift() method may be provided by the
implementation. PA will provide periodic samples of the drift to the
engine. These values need to be aggregated and processed over some time,
since the point values vary quite a bit (but generally fit a linear
regression reasonably accurately). At some point of time, we might move
the actual drift calculation into PA and change the semantics of this
function.
NOTE: This needs further testing before being deemed ready for wider use.
This adds the WebRTC echo canceller as another module-echo-cancel
backend. We're exposing both the full echo canceller as well as the
mobile echo control version as modargs.
Pending items:
1. The mobile canceller doesn't seem to work at the moment.
2. We still need to add bits to hook in drift compensation (to support
sink and source from different devices).
The most controversial part of this patch would probably be the
mandatory build-time dependency on a C++ compiler. If the optional
--enable-webrtc-aec is set, then there's also a dependency on libstdc++.
I initially included put the Speex preprocessing assuming that we'd want
to use the digital gain control and noise suppression from Speex for all
echo cancelling implementations. In practice, we're probably going to
get entire implementations all processing in one package (WebRTC, custom
modules from various vendors, etc.).
This moves out this preprocessing and related knobs into the speex
implementation, which serves to clean out all implementation-specific
details from the module-echo-cancel core.
The echo suppress attenuation value was being incorrectly modified.
Fixed and added 2 arguments to change the attenuation of the residual
echo filter. Default values of the speex preprocessor will be used when
omitted.
This allows the selective enabling of speex' preprocessing algorithms
before running the echo-canceller -- for now this includes automatic
gain control, noise suppression and echo suppression. It's all off by
default for now, though at some point in the near future we might want
to enable at least denoising by default.
The denoising works pretty well, though we might want to add a way to
tweak the noise-suppression knob that libspeex provides.
The AGC option is just a stop-gap -- we need a real AGC mechanism that
tweaks the source volume rather than doing this in software.
The speex documentation mentions VAD and dereverb, but it appears that
these are not complete yet.
We don't do all this in a separate module from module-echo-cancel to
avoid the overhead of adding another virtual source. It makes more sense
to make a separate virtual source module that can be used for cases
where preprocessing is useful but AEC is not (for e.g. noise suppression
for fan noise in a recording application).
Another reason to keep this integrated with the AEC module is that the
echo suppression bits use the speex echo canceller state. This does leak
some information about the AEC implementation into module-echo-cancel,
but this is unavoidable.
Optimises the core inner-product function, which takes the most CPU. The
SSE-optimised bits of the adrian echo canceller only if the CPU that PA
is running on actually supports SSE.
Since all algorithms will need to specify a block size (the amount of
data to be processed together), we make this a common parameter and have
the implementation set it at initialisation time.
This adds Andre Adrian's AEC implementation from his intercom project
(http://andreadrian.de/intercom/) as an alternative to the speex echo
cancellation routines. Since the implementation was in C++ and not in
the form of a library, I have converted the code to C and made a local
copy of the implementation.
The implementation actually works on floating point data, so we can
tweak it to work with both integer and floating point samples (currently
we just use S16LE).
Since the source and sink specification will need to be determined by
the AEC algorithm (can it handle multi-channel audio, does it work with
a fixed sample rate, etc.), we negotiate these using inout parameters at
initialisation time.
There is opportunity to make the sink-handling more elegant. Since the
sink data isn't used for playback (just processing), we could pass
through the data as-is and resample to the required spec before using in
the cancellation algorithm. This isn't too important immediately, but
would be nice to have.
