audiocovert: improve dither setup

The quantize is the amount of bits we want to keep from the original
signal, subtract the amount of bits for noise. Clamp this to 0 (all
noise).
Calculate the scale factor better with powf() and avoid overflows.

Fixes #2479
This commit is contained in:
Wim Taymans 2022-06-28 10:11:46 +02:00
parent 67c2202044
commit b7e26002be
3 changed files with 9 additions and 11 deletions

View file

@ -34,10 +34,10 @@
struct dither {
uint32_t quantize;
#define DITHER_METHOD_NONE 0
#define DITHER_METHOD_RECTANGULAR 2
#define DITHER_METHOD_TRIANGULAR 3
#define DITHER_METHOD_SHAPED_5 4
#define DITHER_METHOD_NONE 0
#define DITHER_METHOD_RECTANGULAR 2
#define DITHER_METHOD_TRIANGULAR 3
#define DITHER_METHOD_SHAPED_5 4
uint32_t method;
uint32_t n_channels;
uint32_t cpu_flags;