When setting the Latency parameter on one side of the converter, set
it also on the other size. We should actually implement propagating
the latency through all the elements of the converter later.
Implement latency handling on fmtconvert.
merger and splitter change latency on all ports when on port changes.
All this makes the configured and exposed latencies visible on all
ports from adapter.