security: fix integer overflow in DSF file buffer allocation

Memory Safety: High

When parsing a DSF audio file, blocksize and channels are read as
uint32_t from untrusted file data and multiplied together for the
buffer allocation. A malicious file could set these to values whose
product overflows, resulting in a small allocation followed by
out-of-bounds writes when the buffer is filled.

Add overflow checking before the multiplication and validate that
neither value is zero. Also use calloc(channels, blocksize) instead
of calloc(1, blocksize * channels) to let calloc perform its own
internal overflow check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Wim Taymans 2026-04-23 16:44:10 +02:00
parent 440f24f35f
commit 2ccb8a7d88

View file

@ -3,6 +3,7 @@
/* SPDX-License-Identifier: MIT */ /* SPDX-License-Identifier: MIT */
#include <errno.h> #include <errno.h>
#include <stdint.h>
#include <sys/mman.h> #include <sys/mman.h>
#include <sys/stat.h> #include <sys/stat.h>
#include <unistd.h> #include <unistd.h>
@ -95,7 +96,10 @@ static int read_fmt(struct dsf_file *f)
if (size > s) if (size > s)
f_skip(f, size - s); f_skip(f, size - s);
f->buffer = calloc(1, f->info.blocksize * f->info.channels); if (f->info.blocksize == 0 || f->info.channels == 0 ||
f->info.channels > SIZE_MAX / f->info.blocksize)
return -EINVAL;
f->buffer = calloc(f->info.channels, f->info.blocksize);
if (f->buffer == NULL) if (f->buffer == NULL)
return -errno; return -errno;