Instead of requiring the upstream node to resubmit the delayed
samples, keep the samples ourselves. The benefit is probably too
small to measure but it simplifies things a lot.
Takes into account the number of skipped samples when deciding whether
we should copy the remaining samples in the history or not.
Fixes dropping audio issue when resampling from high frequencies
such as 48KHz to low requencies such as 8KHz.