alsa-ucm: Fix possible segfault from recursion due to too many devices

While trying to figure out device subsets that have aren't internally
contain conflicting devices, we walk through all possible subsets and
check each set if it satisfies ConflictingDevices/SupportedDevices
listed in UCM configuration. For a better user experience, we want to
skip subsets that are fully included in another valid subset we will
also generate.

The iterate_device_subsets() function that achieves the former is
intentionally in iterative form to avoid a stack overflow, since it will
walk through 2^n sets. However, the iterate_maximal_device_subsets()
function that skips incomplete sets is in recursive form, as I had
assumed tail-call optimization would take care of the potential problem.

Convert iterate_maximal_device_subsets() to an iterative form, because
the recursion seems to trigger a segfault with more than 16 devices on
PulseAudio. It doesn't seem to happen on PipeWire, but it's better to
not leave it to luck.

Link: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/838
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
This commit is contained in:
Alper Nebi Yasak 2024-12-04 17:34:45 +03:00 committed by Wim Taymans
parent f3a7f2c28e
commit e726c5f6db

View file

@ -1425,22 +1425,20 @@ static pa_idxset *iterate_device_subsets(pa_idxset *devices, void **state) {
static pa_idxset *iterate_maximal_device_subsets(pa_idxset *devices, void **state) {
uint32_t idx;
pa_alsa_ucm_device *dev;
pa_idxset *subset;
pa_idxset *subset = NULL;
pa_assert(devices);
pa_assert(state);
subset = iterate_device_subsets(devices, state);
if (!subset)
return subset;
/* Skip this group if it's incomplete, by checking if we can add any
* other device. If we can, this iteration is a subset of another
* group that we already returned or eventually return. */
PA_IDXSET_FOREACH(dev, devices, idx) {
if (!pa_idxset_contains(subset, dev) && devset_supports_device(subset, dev)) {
pa_idxset_free(subset, NULL);
return iterate_maximal_device_subsets(devices, state);
while (subset == NULL && (subset = iterate_device_subsets(devices, state))) {
/* Skip this group if it's incomplete, by checking if we can add any
* other device. If we can, this iteration is a subset of another
* group that we already returned or eventually return. */
PA_IDXSET_FOREACH(dev, devices, idx) {
if (subset && !pa_idxset_contains(subset, dev) && devset_supports_device(subset, dev)) {
pa_idxset_free(subset, NULL);
subset = NULL;
}
}
}