Don't close an -1 fd in clear_data.
If we let the client allocate buffer, set our fd and data to invalid
values. If the client decides to renegotiate before we get the buffer
data we might otherwise try to clear the mem_id (default 0) or
close the fd (also default 0).
Fixes#5162