If the sample finished playing before we finished the roundtrip to
get the sink_index, it will be destroyed. When the roundtrip completes,
it will try to use invalid memoryy and crash.
Make sure we destroy all pending replies before destroying the sample
to avoid this problem.
Fixes#2151