Remove the separate error event and merge with the done event with error code. This then also marks the sample as done when in error so that it gets cleaned up properly. Fixes #486
Also handle errors.
Decrease the actual ref counter, instead of the pointer value.