Downstream elements accessing dmabufs from the CPU currently need to map
and unmap buffers on every frame.
While the kernel's page cache avoids most overhead, this still requires a
round-trip through the kernel and possibly non-negligible work in
`dma_buf_mmap()`. Keeping the buffers mapped avoids that without causing
additional syncronization work, as the later should only happen when
`DMA_BUF_IOCTL_SYNC` is triggered in `gst_dmabuf_mem_map()`.
A common scenario where this matters is clients using cameras. The
downstream elements in question may not be aware of dmabufs - e.g.
`videoconvert` - or fail to import the dmabuf and fall back to import
from memory - e.g. `glupload`.
Notes:
- GstShmAllocator implicitly does this already.
- We could also do this in the MemFd case, however I'm less convinced
about the trade-offs.
When a link enters the "ERROR" state, it is scheduled for destruction in
`module-link-factory.c:link_state_changed()`, which queues `destroy_link()`
to be executed on the context's work queue.
However, if the link is destroyed by means of `pw_impl_link_destroy()`
directly after that, then `link_destroy()` unregisters the associated
`pw_global`'s event hook, resulting in `global_destroy()` not being called
when `pw_impl_link_destroy()` proceeds to call `pw_global_destroy()` some
time later. This causes the scheduled async work to not be cancelled. When
it runs later, it will trigger a use-after-free since the `link_data` object
is directly tied to the `pw_impl_link` object.
For example, if the link is destroyed when the client disconnects:
==259313==ERROR: AddressSanitizer: heap-use-after-free on address 0x7ce753028af0 at pc 0x7f475354a565 bp 0x7ffd71501930 sp 0x7ffd71501920
READ of size 8 at 0x7ce753028af0 thread T0
#0 0x7f475354a564 in destroy_link ../src/modules/module-link-factory.c:253
#1 0x7f475575a234 in process_work_queue ../src/pipewire/work-queue.c:67
#2 0x7b47504e7f24 in source_event_func ../spa/plugins/support/loop.c:1011
[...]
0x7ce753028af0 is located 1136 bytes inside of 1208-byte region [0x7ce753028680,0x7ce753028b38)
freed by thread T0 here:
#0 0x7f475631f79d in free /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:51
#1 0x7f4755594a44 in pw_impl_link_destroy ../src/pipewire/impl-link.c:1742
#2 0x7f475569dc11 in do_destroy_link ../src/pipewire/impl-port.c:1386
#3 0x7f47556a428b in pw_impl_port_for_each_link ../src/pipewire/impl-port.c:1673
#4 0x7f475569dc3e in pw_impl_port_unlink ../src/pipewire/impl-port.c:1392
#5 0x7f47556a02d8 in pw_impl_port_destroy ../src/pipewire/impl-port.c:1453
#6 0x7f4755634f79 in pw_impl_node_destroy ../src/pipewire/impl-node.c:2447
#7 0x7b474f722ba8 in client_node_resource_destroy ../src/modules/module-client-node/client-node.c:1253
#8 0x7f47556d7c6c in pw_resource_destroy ../src/pipewire/resource.c:325
#9 0x7f475545f07d in destroy_resource ../src/pipewire/impl-client.c:627
#10 0x7f47554550cd in pw_map_for_each ../src/pipewire/map.h:222
#11 0x7f4755460aa4 in pw_impl_client_destroy ../src/pipewire/impl-client.c:681
#12 0x7b474fb0658b in handle_client_error ../src/modules/module-protocol-native.c:471
[...]
Fix this by cancelling the work queue item in `link_destroy()`, which should
always run, regardless of the ordering of events.
Fixes#4691
When the port is renamed, we queue a PORT_RENAME callback with an arg1
as 1. Before emitting the event we check the registered state of the
port but we want to suppress the event if the port was *not* registered,
ie. when the registered state != 1 (arg1).
Improve the spa_ump_to_midi function so that it can consume multiple UMP
messages and produce multiple midi messages.
Some UMP messages (like program changes) need to be translated into up
to 3 midi messages. Do this byt adding a state to the function and by
making it consume the input bytes, just like the spa_ump_from_midi
function.
Adapt code to this new world. This is a little API break..
cd68819feb added code to follow the state
change of the node because at the time it the Start code was async and
it was better to complete it before emitting the state.
9b80855821 however made the Start command
sync again and so we can safely emit the running state in the Start.
The effect is that the Running state change is emitted first and then
the node eventfd is added to the data loop that can then call the process
callback. Having the process callback happen before the RUNNING state
change is unexpected and racy.
The idea of fd2db174c1 was to allow for
property updates after the global was registered but this check was not
removed to actually make this happen.
Fixes#4851
Follow various other elements like glupload and gtk4paintablesink and
print the negotiated caps at a higher priority than debug. A small
quality of life improvement to facilitate debugging.
When clearing the buffers, remove the stale pointers and file descriptors
as accidental reuse of those is problematic and potentially difficult to
diagnose. Do this for every data plane.
Also clear the node's `buffer` structures to remove any references to
the provided `spa_buffer` objects and related metadata.
The current handling of `SPA_DATA_MemPtr` is not entirely correct because
its handling of multi-planar buffers is not appropriate: it leaks memory
mappings because it overwrites `buffer::ptr` for each plane.
Since this data type should not really be in use in normal deployments,
let's remove it for now.
After e0e8bf083 ("spa: libcamera: source: create eventfd before starting camera"),
things are still not entirely correct. This change ensures that if starting
the camera fails, then the runtime state, most importantly the ring buffer
of completed requests is restored to its initial state.
Furthermore, it is also ensured that `impl::active` can never be observed
by the data thread while it is being changed, which is achieved by setting
it before/after adding/removing the event source.
While concurrent `spa_loop_add_source()` invocations work with the current
main epoll-based implementation, this is not guaranteed, so lock the loop.
Similarly to how it is done elsewhere and for removal already.
pipewire assumes that the buffers are removed from a port when its format is
changed even without an explicit call to `port_use_buffer()`.
So if the format is not just tested, clear the buffers, as well as the libcamera
requests and buffers because they are also recreated when a new format is set.
This matches the behaviour of the v4l2 plugin. Furthermore, this change also
removes the call to `spa_libcamera_use_buffers()` because that function does
nothing. And finally this change necessitates that the current format is always
reset (when not testing), not just before reaching `spa_libcamera_set_format()`.
Remove the code that is supposed to compare the current and the to-be-set
format for returning early if they match. This is removed:
* v4l2 also does not have it either;
* it needs more consideration (there are not properly handled fields);
* it would make later changes somewhat more complicated.
Since `impl::requestComplete()` runs in an internal libcamera thread, extra care
would need to be taken to validate all accesses to common data structures. For
example, the function might call `spa_libcamera_buffer_recycle()`, which accesses
`impl::ctrls`, which would be unsafe because it could read or modified at the same
time on the data thread. So move the processing of requests to the data loop.
It is possible that multiple requests complete before the data loop can run
`libcamera_on_fd_events()`. However, previously only the earliest item was
processed from the ring buffer, meaning that in those cases request processing
could be lagging request completion by multiple requests.
Fix that by getting the number of available requests and processing them all.
Presently, the ring buffer of completed requests is only cleared
when the buffers are removed from the port. This is not entirely
correct since pause/start commands do not clear the buffers but
they stop the camera. As a consequence, it is possible that some
completed requests stay in the ring buffer, causing them to be
mistakenly processed when the camera is started next.
So reset the ring buffer after the camera is stopped, the same time
as the queue of free buffers is cleared.
A `libcamera::Request` is directly tied to the camera, not any ports ("streams")
of it. So move the request completion ring buffer into the `impl` struct.
This is a prerequisite for supporting multiple libcamera streams (~ exposing
multiple ports on the node) in the future.
The request will be needed later, so store that directly in the
completion ring buffer for easy access.
This is fine even though `impl::requestComplete()` calls `Request::reuse()`
because only the request cookie is used in `libcamera_on_fd_events()` and
that remains constant during the lifetime of a request object.
The handling of `impl::pendingRequests` is a bit problematic because,
for example, during startup, if `spa_libcamera_buffer_recycle()` observes
that `impl::active == false`, then it will try to append to `impl::pendingRequests`,
which is being iterated in the main thread in `spa_libcamera_stream_on()`.
That is not allowed on an `std::deque`.
So instead remove it altogether, and simply queue all requests when starting.
After `libcamera::Camera::stop()` returns, every request is guaranteed not
to be used by libcamera, and they can be freely reused, so this is safe to do.
This also removes the need for calling `spa_libcamera_buffer_recycle()` when
the buffers are set/unset on a port since that function no longer changes
anything apart from updating `buffer::flags` but `spa_libcamera_alloc_buffers()`
is modified appropriately to take care of that. And in the case of
`spa_libcamera_clear_buffers()` clearing the flag is not required because the
next `spa_libcamera_alloc_buffers()` call will reset reset the flags.
Currently only a single stream is used. This makes it easy to associate
each request with the appropriate frame buffer when it is created. In
turn, this makes reusing requests a bit simpler and a bit more efficient.
So do that: add buffers to requests when they are allocated in `allocBuffers()`.
At the moment the libcamera buffer allocation is completely tied to format
negotiation. `freeBuffers()` undoes `allocBuffers()`. And `allocBuffers()`
is called as part of `spa_libcamera_set_format()`. Therefore `freeBuffers()`
should be called independent of the state of the buffers on any port.
Otherwise unsetting the format while there are no buffers on the port will
cause the libcamera requests and buffers not to be released correctly. Similarly,
removing the buffers from a port would clear the libcamera requests and buffers,
making the node unusable without setting a new format.
`freeBuffers()` should undo exactly what `allocBuffers()` does. However,
it currently also clears `impl::pendingRequests`, but that is filled
by `spa_libcamera_buffer_recycle()` during `spa_libcamera_alloc_buffers()`.
So remove the clearing of `impl::pendingRequests` from `freeBuffers()` and
move it directly into `spa_libcamera_clear_buffers()`.
Debian supports many architectures, and it is relatively easy to work
with multiarch packages, and finally `meson env2mfile` supports
generating cross files with the `--debarch` option.
Previously only native builds were done in the CI, so use debian to
build pipewire for multiple architectures.
Some packages are unfortunately not multiarch compatible, so a separate
container is built for every architecture.
Check if the "TracerPid" field in `/proc/self/status` is non-zero. This does
not need libcap, and does not depend on capabilities, and it also works under
qemu user emulation.
