security: fix integer overflow in Bluetooth codec codesize calculations

Memory Safety: High

Several Bluetooth audio codec implementations calculate codesize by
multiplying samples * channels * sizeof(sample_type) without overflow
checks. The parameters come from Bluetooth codec negotiation, which is
influenced by the remote peer. If the multiplication overflows, codesize
wraps to a small value, causing subsequent buffer size checks to pass
while the actual data processing operates on the full (larger) sample
count, leading to heap buffer overflows.

Affected codecs: LC3 (BAP), LC3plus (A2DP), Opus (A2DP), Opus-G (A2DP).

Add overflow checks before each codesize multiplication to ensure the
result fits in the target integer type, returning -EINVAL on overflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Wim Taymans 2026-04-23 18:50:14 +02:00
parent 00413a3263
commit e3e1c4d214
4 changed files with 30 additions and 2 deletions

View file

@ -413,10 +413,15 @@ static void *codec_init(const struct media_codec *codec, uint32_t flags,
}
this->e.samples = lc3plus_enc_get_input_samples(this->enc);
this->e.codesize = this->e.samples * this->channels * sizeof(int32_t);
spa_assert(this->e.samples <= LC3PLUS_MAX_SAMPLES);
if (this->e.samples > INT_MAX / (int)sizeof(int32_t) / SPA_MAX(this->channels, 1)) {
res = -EINVAL;
goto error;
}
this->e.codesize = this->e.samples * this->channels * sizeof(int32_t);
this->e.bitrate = this->bitrate;
this->e.next_bitrate = this->bitrate;