Downstream elements accessing dmabufs from the CPU currently need to map
and unmap buffers on every frame.
While the kernel's page cache avoids most overhead, this still requires a
round-trip through the kernel and possibly non-negligible work in
`dma_buf_mmap()`. Keeping the buffers mapped avoids that without causing
additional syncronization work, as the later should only happen when
`DMA_BUF_IOCTL_SYNC` is triggered in `gst_dmabuf_mem_map()`.
A common scenario where this matters is clients using cameras. The
downstream elements in question may not be aware of dmabufs - e.g.
`videoconvert` - or fail to import the dmabuf and fall back to import
from memory - e.g. `glupload`.
Notes:
- GstShmAllocator implicitly does this already.
- We could also do this in the MemFd case, however I'm less convinced
about the trade-offs.
Follow various other elements like glupload and gtk4paintablesink and
print the negotiated caps at a higher priority than debug. A small
quality of life improvement to facilitate debugging.
Make a new body.h file with some functions to deal with pod and their
body. Make the iter.h functions use mostly this.
Rework the parser so that it only uses body.h functions. With the separation
of pod+body, we can read and verify the pod once and then use the
verified copy to handle the rest of the body safely.
We do this because iter.h only works in pods in memory that doesn't change
because it is vulnerable to modifications of the data after verifying it.
The new parser is not vulnerable to this and will not cause invalid
memory access when used on shared memory. There is however no need for
atomic operations to read the headers, whever is read is either valid
and useable of invalid and rejected.
See #4822
When _probe() is called, take a ref to the newly created devices instead
if sinking the floating ref, since gst_clear_object() is called when
core is disconnected. Otherwise the devices will be freed before the
caller gets them.
Fixes the following assert in the caller:
g_object_is_floating: assertion 'G_IS_OBJECT (object)' failed
Or sometimes a segfault with the backtrace:
0 g_type_check_instance_is_fundamentally_a (type_instance=type_instance@entry=0x116c1b0, fundamental_type=fundamental_type@entry=80) at /usr/src/debug/glib-2.0/2.84.0/gobject/gtype.c:3918
1 0xb6d40cc6 in g_object_is_floating (_object=0x116c1b0) at /usr/src/debug/glib-2.0/2.84.0/gobject/gobject.c:3843
2 0xb6bc4c74 in gst_device_provider_get_devices (provider=0x109ba00) at /usr/src/debug/gstreamer1.0/1.24.12/gst/gstdeviceprovider.c:426
The change from 1 to 8 was done without justification in the commit
message and possibly for debug purposes. Unfortunately it breaks
negotiation with the libcamera virtual pipeline, which defaults to
4 buffers.
Set the the value to 1 again as successful negotiation - even with an
unusually low number of buffers - is usually more desirable than an
error.
Fixes: 98b7dc7c0 ("gst: don't do set_caps from the pipewire callback")
Get the clock pointer using the io_changed stream event.
and update the clock before triggering the process
The clock needs to be updated in the data loop thread
and before triggering the process so move the calls to
`pw_stream_trigger_process` from gstreamer thread context
into the data loop thread context by invoking a callback
and update the clock inside the data loop callback
before the trigger
In case of encoded video we get n_planes as 0 from the video info so
passing that as n_datas is failing during the buffer negotiation. Make
sure to use an appropriate value based on whether we have raw video or
not.
Co-authored-by: Taruntej Kanakamalla <taruntej@asymptotic.io>
When PW source is used with something like Camera and the camera is
disconnected, all buffers are removed and stream will be paused.
When using PW sink with source, the sink side pipeline can go to EOS.
This again results in all the buffers being removed and stream being
paused on the source side. PW source side pipeline can also crash if
the sink was in the middle of frame copying a buffer to render which
got removed.
Handle this scenario by sending a flush-start event at the start of
buffer removal and flush-stop at the end followed by an end of stream
or pipeline error depending on user selection.
For a pipeline like below, we might want to dynamically switch the audio
source.
gst-launch-1.0 -e pipewiresrc autoconnect=false ! queue ! audioconvert ! autoaudiosink
On switching to a different audio source, any one of driver, quantum
or clock rate might change which changes the return `result` value of
gst_pipewire_clock_get_internal_time.
This can result in the basesrc create function incorrectly waiting in
gst_clock_id_wait. We post clock lost message to fix this. In the case
of gst-launch, it will set the pipeline to PAUSED and then PLAYING to
to force a new clock and a new base_time distribution.
Without the clock lost message, the following can be seen
before re-linking to a different source
0:00:30.887602864 79499 0x7fffe8000d40 DEBUG GST_CLOCK gstsystemclock.c:1158:gst_system_clock_id_wait_jitter_unlocked:<pipewireclock0> entry 0x7fffd803fad0 time 0:00:17.024565416 now 0:00:17.024109144 diff (time-now) 456272
after re-linking to a different source
0:00:45.790843245 79499 0x7fffe8000d40 DEBUG GST_CLOCK gstsystemclock.c:1158:gst_system_clock_id_wait_jitter_unlocked:<pipewireclock0> entry 0x7fffd803fad0 time 0:00:31.927694059 now 0:00:17.066883864 diff (time-now) 14860810195
With the clock lost message, the following can be seen
before re-linking to a different source
0:01:09.336533552 89461 0x7fffe8000d40 DEBUG GST_CLOCK gstsystemclock.c:1158:gst_system_clock_id_wait_jitter_unlocked:<pipewireclock0> entry 0x7fffd803fad0 time 0:00:58.198536772 now 0:00:58.197444926 diff (time-now) 1091846
after re-linking to a different source
0:01:21.659827958 89461 0x7fffe8000d40 DEBUG GST_CLOCK gstsystemclock.c:1158:gst_system_clock_id_wait_jitter_unlocked:<pipewireclock0> entry 0x7fffd803fad0 time 0:28:24.853517646 now 0:28:24.853527204 diff (time-now) -9558
Note the difference in `time` and `now` fields of the above log message.
This is easy to reproduce by using a pipewiresink as the audio source
with a pipeline like below, as one of the sources during switching.
gst-launch-1.0 -e audiotestsrc wave=ticks ! audioconvert ! audio/x-raw,format=F32LE,rate=48000,channels=1 !
pipewiresink stream-properties="props,media.class=Audio/Source,node.description=pwsink" client-name=pwsink
Applications need to handle the GST_MESSAGE_CLOCK_LOST message in their
bus handlers.
Let's make sure we own the memory in buffers, so that we can be
resilient to the PW link going away. This currently maintains the status
quo of copying data into the pipewirepool for sending to the remote end,
but moves the allocation of buffers so that ownership is maintained by
the sink in all cases.
There are some tricky corners, especially with bufferpool vs. buffers
param negotiation -- bufferpool parameters can be negotiated in
GStreamer before the link even comes up, so we try to adapt the buffers
param to use the negotiated value. For now, that is more brittle than
tying those two aspects together. We can revisit this if we can find a
way to tie pipeline state and link state more closely.
Co-authored-by: Arun Raghavan <arun@asymptotic.io>
A number of changes for correctness.
1) We expose the actualy min and max values we support in the
allocation query.
2) We don't support max_buffers as 0, as unlimited buffers is not an
option
3) In ParamBuffers, we request the max_buffers from bufferpool config,
as we cannot dynamically allocate buffers
We need to make sure the memory sizes are correctly initialised so the
meta makes sense, and we don't copy the meta from the input buffer as
that doesn't make sense given we have our own meta already.
Setting the default size to 0 and outside of the min/max range now means
that there is no suggestion for the size and it should use the
suggestion of the peer.
Counter-intuitive as it seems, when we are driving the clock, we can't
also provide a clock from PipeWire to the pipeline -- we need the
pipeline to drive the graph.
So we make the mode control whether we provide a clock or not.
When using PW source, one might want to dynamically link PW source to
a different source. Setting possible_caps to NULL prevents the caps
intersect from returning a successful result on format change. Do not
set possible_caps to NULL as we get that from peer caps which should
stay the same ideally for the duration of pipeline run. That allows
re-linking PW source any number of times with a pipeline like below.
gst-launch-1.0 pipewiresrc autoconnect=false ! queue ! video/x-raw,format=YUY2 ! videoconvert ! xvimagesink
The above pipeline can be made to switch between a camera source and a
screen capture source like wf-recorder.
Note that this fix only improves the status quo and won't work if the
peer caps change due to a re-negotiation.
We might end up in a situation where depending on the pipeline,
intersect might not give us fixated caps.
Possible example of such a pipeline can be below.
gst-launch-1.0 -e pipewiresrc target-object=<path> ! audioconvert !
audio/x-raw,format=S16LE,rate=48000,channels=2 ! lamemp3enc !
filesink location=test.mp3
This results in non-fixated caps like below when intersecting caps from
format param and possible_caps which depends on what we have downstream
in the pipeline.
audio/x-raw, layout=(string)interleaved, format=(string)S16LE, rate=(int)48000, channels=(int)2, channel-mask=(bitmask)0x0000000000000003;
audio/x-raw, layout=(string)interleaved, format=(string)S16LE, rate=(int)48000, channels=(int)2
To fix this, fixate the caps explicitly.
The device passed to gst_device_provider_device_add() is transfer:floating, so
we need increase its ref, otherwise the pointer we keep internally will be a
dangling ref.
Also gst_device_provider_device_remove() doesn't actually release the device, so
we have to do it ourselves.
Fixes#4616
In case negotiation is first attempted with unfixed caps, bufferpool support was
unconditionally disabled. Then at a second caps negotiation attempt it wasn't
restored according to the property value.
In case of the video, if the buffer to be rendered is from upstream and
not from the pipewirepool, map the memory into video frames and copy the
frames instead of doing a buffer copy.
Avoid splitting of buffers in the case of video, because that might break
the frame layout, especially planar formats, for the applications which
use pipewiresink as a camera source to capture video.
Deprecate the `always-copy` property in pipewiresrc and use the new
property `use-bufferpool` in both pipewiresrc and pipewiresink to
specify whether to use the gstpipewirepool or not.
The gstpipewirepool is used for video by default unless `use-bufferpool`
is set to `false` and it is not used for audio by default unless
`use-bufferpool` is set to `true`
During negotiation, ensure that possible_caps exists before calling
gst_caps_intersect_full(). Sometimes possible_caps seem to be NULL which
cause the GST_IS_CAPS assert to fail.
The reason for the occasional NULL possible_caps during initial
negotiation is unknown, but this is also possible during renegotiation.
The handle_format_change() may be triggered before the next create(),
win which case we have not yet discovered in the streaming thread that
we are in the NOT_NEGOTIATED state. We should guard against this
possibility as well.
