mirrors/pipewire
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/pipewire/pipewire.git synced 2025-11-03 09:01:54 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
pipewire/spa/plugins/support/loop.c

1477 lines
37 KiB
C
Raw Normal View History

treewide: use SPDX tags to specify copyright information SPDX tags make the licensing information easy to understand and clear, and they are machine parseable. See https://spdx.dev for more information.
2023-02-08 18:12:00 +01:00
/* Spa */
/* SPDX-FileCopyrightText: Copyright © 2018 Wim Taymans */
/* SPDX-License-Identifier: MIT */
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <signal.h>
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
#include <threads.h>
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
#include <stdatomic.h>
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Reorganise SPA tree Reorganise the SPA includes to make it more extensible later Simplify the naming of the buffer and meta params
2017-11-10 13:36:14 +01:00
#include <spa/support/loop.h>
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
#include <spa/support/system.h>
Reorganise SPA tree Reorganise the SPA includes to make it more extensible later Simplify the naming of the buffer and meta params
2017-11-10 13:36:14 +01:00
#include <spa/support/log.h>
#include <spa/support/plugin.h>
#include <spa/utils/list.h>
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
#include <spa/utils/cleanup.h>
#include <spa/utils/atomic.h>
names: add standard factory name definitions Define a set of standard factory names and document what they contain. This makes it possible to change the implementation by mapping the factory-name to a different shared library.
2019-06-21 13:31:34 +02:00
#include <spa/utils/names.h>
loop: rate limit xrun messages When the reader thread locks up for some reason, avoid excessive logs about the invoke queue being filled. See #3532
2023-09-30 09:29:20 +02:00
#include <spa/utils/ratelimit.h>
node: improve async handling Remove the done and error callbacks. The error callback is in an error message. The done callback is replace with spa_pending. Make enum_params take a callback and data for the results. This allows us to push the results one after another to the app and avoids ownership issues of the passed data. We can then extend this to handle the async case by doing a _wait call with a spa_pending+callback+data that will be called when the _enum_params returns and async result. Add a sync method. All methods can now return SPA_RESULT_IS_ASYNC return values and you can use spa_node_wait() to register a callback when they complete with optional extra parameters. This makes it easier to sync and handle the reply. Make helper methods to simulate the sync enum_params behaviour for sync nodes. Let the transport generate the sequence number for pw_resource_sync() and pw_proxy_sync(). That way we don't need to keep track of numbers ourselves and we can match the reply to the request easily.
2019-02-20 17:51:05 +01:00
#include <spa/utils/result.h>
Remove dynamic types Do not use dynamic types anymore. The reason is that it's difficult: - to maintain a shared type database over a network. - the extra overhead when translating between processes and for maintaining the translation tables. - race conditions in translating in RT-threads, this is a problem because we want to make event streams. We now have simple enums with types and extension points for all types. This is also nicer to use in general. We don't need the mapper anymore or pass strings around as types. There is a parallel type info system to get more info about ids and enums and their hierarchy. It can also be used for debugging.
2018-08-23 17:47:57 +02:00
#include <spa/utils/type.h>
Reorganise SPA tree Reorganise the SPA includes to make it more extensible later Simplify the naming of the buffer and meta params
2017-11-10 13:36:14 +01:00
#include <spa/utils/ringbuffer.h>
treewide: replace strcmp() == 0 with spa_streq() This change is only done in source files for now, header files will be done separately.
2021-05-18 11:36:13 +10:00
#include <spa/utils/string.h>
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: export log topic enumerations
2023-12-23 21:07:45 +02:00
SPA_LOG_TOPIC_DEFINE_STATIC(log_topic, "spa.loop");
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
#undef SPA_LOG_TOPIC_DEFAULT
#define SPA_LOG_TOPIC_DEFAULT &log_topic
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: export log topic enumerations
2023-12-23 21:07:45 +02:00
support: use defines for alignment
2022-01-03 11:14:15 +01:00
#define MAX_ALIGN 8
#define ITEM_ALIGN 8
#define DATAS_SIZE (4096*8)
loop: handle remove while dispatching better Keep the array of dispatched sources around in the loop. When a source is removed while dispatching, set the data to NULL so that we don't try to deref the source again or call its function. Fixes #2114
2022-02-08 10:09:06 +01:00
#define MAX_EP 32
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
/* the number of concurrent queues for invoke. This is also the number
* of threads that can concurrently invoke. When there are more, the
* retry timeout will be used to retry. */
#define QUEUES_MAX 128
#define DEFAULT_RETRY (1 * SPA_USEC_PER_SEC)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
/** \cond */
struct invoke_item {
size_t item_size;
spa_invoke_func_t func;
uint32_t seq;
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
uint32_t count;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
void *data;
cleanups Follow the data/size argument order convention to make the api more consistent.
2017-11-20 15:26:44 +01:00
size_t size;
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
bool block;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
void *user_data;
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
static int loop_signal_event(void *object, struct spa_source *source);
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
struct queue;
#define IDX_INVALID ((uint16_t)0xffff)
union tag {
struct {
uint16_t idx;
uint16_t count;
} t;
uint32_t v;
};
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct impl {
struct spa_handle handle;
struct spa_loop loop;
struct spa_loop_control control;
struct spa_loop_utils utils;
struct spa_log *log;
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
struct spa_system *system;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct spa_list source_list;
Revert "loop: remove destroy list" This reverts commit c474846c42967c44db069a23b76a29da6f496f33. In addition, `s->loop` is also checked before dispatching a source. The destroy list is needed in the presence of threads. The issue is that a source may be destroyed between `epoll_wait()` returning and thread loop lock being acquired. If this source is active, then a use-after-free will be triggered when the thread loop acquires the lock and starts dispatching the sources. thread 1 thread 2 ---------- ---------- loop_iterate spa_loop_control_hook_before // release lock pw_thread_loop_lock spa_system_pollfd_wait // assume it returns with source A pw_loop_destroy_source(..., A) // frees storage of A pw_thread_loop_unlock spa_loop_control_hook_after // acquire the lock for (...) { struct spa_source *s = ep[i].data; s->rmask = ep[i].events; // use-after-free if `s` refers to // the previously freed `A` Fixes #2147
2022-02-18 18:36:36 +01:00
struct spa_list destroy_list;
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
struct spa_list free_list;
Move listeners to spa and rename to hooks The listeners are generally useful in spa, move it there and rename to hooks. Implement loop hooks with it. Fix some cleanup issues in stream and remote
2017-08-08 16:56:29 +02:00
struct spa_hook_list hooks_list;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: handle queue overflow better When a queue overflows we place the queue back in the stack and try again. Because it's at the top of the stack we take exactly the same queue and keep on looping forever if the other thread is blocked for some reason. Instead, mark the queue as overflowed and only place it back in the stack when we have flushed it. This avoids a deadlock when the main-thread invokes on the data loop and blocks and when the data loop invokes on the main-thread and overflows the queue.
2024-11-14 15:58:09 +01:00
struct spa_ratelimit rate_limit;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
int retry_timeout;
spa: some more invoke -> locked calls
2025-03-10 17:46:28 +01:00
bool prio_inherit;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
union tag head;
uint32_t n_queues;
struct queue *queues[QUEUES_MAX];
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_mutex_t lock;
pthread_cond_t cond;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_cond_t accept_cond;
int n_waiting;
int n_waiting_for_accept;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
int poll_fd;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
pthread_t thread;
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
int enter_count;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
int recurse;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
struct spa_source *wakeup;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
uint32_t count;
uint32_t flush_count;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
unsigned int polling:1;
};
struct queue {
struct impl *impl;
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
uint16_t idx;
uint16_t next;
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
int ack_fd;
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
bool close_fd;
struct queue *overflow;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct spa_ringbuffer buffer;
loop: align buffer and invoke_items to 8 bytes To avoid unaligned access messages from sanitizer See #497
2020-12-21 20:46:13 +01:00
uint8_t *buffer_data;
support: use defines for alignment
2022-01-03 11:14:15 +01:00
uint8_t buffer_mem[DATAS_SIZE + MAX_ALIGN];
loop: handle queue overflow better When a queue overflows we place the queue back in the stack and try again. Because it's at the top of the stack we take exactly the same queue and keep on looping forever if the other thread is blocked for some reason. Instead, mark the queue as overflowed and only place it back in the stack when we have flushed it. This avoids a deadlock when the main-thread invokes on the data loop and blocks and when the data loop invokes on the main-thread and overflows the queue.
2024-11-14 15:58:09 +01:00
};
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl {
struct spa_source source;
struct impl *impl;
struct spa_list link;
union {
spa_source_io_func_t io;
spa_source_idle_func_t idle;
spa_source_event_func_t event;
spa_source_timer_func_t timer;
spa_source_signal_func_t signal;
} func;
spa: support: loop: move struct members Move the boolean members of `struct source_impl` to the end of the struct. This changes the size of the struct from 104 bytes to 96 bytes on x86-64. Before: struct source_impl { struct spa_source source; /* 0 48 */ struct impl * impl; /* 48 8 */ struct spa_list link; /* 56 16 */ /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */ _Bool close; /* 72 1 */ /* XXX 7 bytes hole, try to pack */ union { spa_source_io_func_t io; /* 80 8 */ spa_source_idle_func_t idle; /* 80 8 */ spa_source_event_func_t event; /* 80 8 */ spa_source_timer_func_t timer; /* 80 8 */ spa_source_signal_func_t signal; /* 80 8 */ } func; /* 80 8 */ _Bool enabled; /* 88 1 */ /* XXX 7 bytes hole, try to pack */ struct spa_source * fallback; /* 96 8 */ /* size: 104, cachelines: 2, members: 7 */ /* sum members: 90, holes: 2, sum holes: 14 */ /* last cacheline: 40 bytes */ }; After: struct source_impl { struct spa_source source; /* 0 48 */ struct impl * impl; /* 48 8 */ struct spa_list link; /* 56 16 */ /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */ union { spa_source_io_func_t io; /* 72 8 */ spa_source_idle_func_t idle; /* 72 8 */ spa_source_event_func_t event; /* 72 8 */ spa_source_timer_func_t timer; /* 72 8 */ spa_source_signal_func_t signal; /* 72 8 */ } func; /* 72 8 */ struct spa_source * fallback; /* 80 8 */ _Bool close; /* 88 1 */ _Bool enabled; /* 89 1 */ /* size: 96, cachelines: 2, members: 7 */ /* padding: 6 */ /* last cacheline: 32 bytes */ };
2022-02-19 15:12:31 +01:00
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
struct spa_source *fallback;
spa: support: loop: move struct members Move the boolean members of `struct source_impl` to the end of the struct. This changes the size of the struct from 104 bytes to 96 bytes on x86-64. Before: struct source_impl { struct spa_source source; /* 0 48 */ struct impl * impl; /* 48 8 */ struct spa_list link; /* 56 16 */ /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */ _Bool close; /* 72 1 */ /* XXX 7 bytes hole, try to pack */ union { spa_source_io_func_t io; /* 80 8 */ spa_source_idle_func_t idle; /* 80 8 */ spa_source_event_func_t event; /* 80 8 */ spa_source_timer_func_t timer; /* 80 8 */ spa_source_signal_func_t signal; /* 80 8 */ } func; /* 80 8 */ _Bool enabled; /* 88 1 */ /* XXX 7 bytes hole, try to pack */ struct spa_source * fallback; /* 96 8 */ /* size: 104, cachelines: 2, members: 7 */ /* sum members: 90, holes: 2, sum holes: 14 */ /* last cacheline: 40 bytes */ }; After: struct source_impl { struct spa_source source; /* 0 48 */ struct impl * impl; /* 48 8 */ struct spa_list link; /* 56 16 */ /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */ union { spa_source_io_func_t io; /* 72 8 */ spa_source_idle_func_t idle; /* 72 8 */ spa_source_event_func_t event; /* 72 8 */ spa_source_timer_func_t timer; /* 72 8 */ spa_source_signal_func_t signal; /* 72 8 */ } func; /* 72 8 */ struct spa_source * fallback; /* 80 8 */ _Bool close; /* 88 1 */ _Bool enabled; /* 89 1 */ /* size: 96, cachelines: 2, members: 7 */ /* padding: 6 */ /* last cacheline: 32 bytes */ };
2022-02-19 15:12:31 +01:00
bool close;
bool enabled;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
/** \endcond */
loop: rate limit xrun messages When the reader thread locks up for some reason, avoid excessive logs about the invoke queue being filled. See #3532
2023-09-30 09:29:20 +02:00
static inline uint64_t get_time_ns(struct spa_system *system)
{
struct timespec ts;
spa_system_clock_gettime(system, CLOCK_MONOTONIC, &ts);
return SPA_TIMESPEC_TO_NSEC(&ts);
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static int loop_add_source(void *object, struct spa_source *source)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: cleanups
2019-05-23 15:11:49 +02:00
source->loop = &impl->loop;
support: also protect against recursive invocations Add an extra private field to the source to store the pollevent of the current iteration. This changes ABI but it seems an embedded source is not used outside of our own plugins and the unit test doesn't test this ABI case. Whenever a source is removed, we can set the data field of the pollevent to NULL so that it won't be handled in any iteration anymore. Avoid dispatching the same event multiple times when doing recursive iterations. Add some more unit tests for this. Fixes #2114
2022-02-08 17:21:10 +01:00
source->priv = NULL;
spa: support: loop: initialize rmask Set `rmask` to zero when a source is added to, or removed from the loop.
2022-02-19 09:40:27 +01:00
source->rmask = 0;
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
return spa_system_pollfd_add(impl->system, impl->poll_fd, source->fd, source->mask, source);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static int loop_update_source(void *object, struct spa_source *source)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(source->loop == &impl->loop);
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
return spa_system_pollfd_mod(impl->system, impl->poll_fd, source->fd, source->mask, source);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
static void detach_source(struct spa_source *source)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
support: also protect against recursive invocations Add an extra private field to the source to store the pollevent of the current iteration. This changes ABI but it seems an embedded source is not used outside of our own plugins and the unit test doesn't test this ABI case. Whenever a source is removed, we can set the data field of the pollevent to NULL so that it won't be handled in any iteration anymore. Avoid dispatching the same event multiple times when doing recursive iterations. Add some more unit tests for this. Fixes #2114
2022-02-08 17:21:10 +01:00
struct spa_poll_event *e;
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
source->loop = NULL;
source->rmask = 0;
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
support: also protect against recursive invocations Add an extra private field to the source to store the pollevent of the current iteration. This changes ABI but it seems an embedded source is not used outside of our own plugins and the unit test doesn't test this ABI case. Whenever a source is removed, we can set the data field of the pollevent to NULL so that it won't be handled in any iteration anymore. Avoid dispatching the same event multiple times when doing recursive iterations. Add some more unit tests for this. Fixes #2114
2022-02-08 17:21:10 +01:00
if ((e = source->priv)) {
/* active in an iteration of the loop, remove it from there */
e->data = NULL;
source->priv = NULL;
loop: handle remove while dispatching better Keep the array of dispatched sources around in the loop. When a source is removed while dispatching, set the data to NULL so that we don't try to deref the source again or call its function. Fixes #2114
2022-02-08 10:09:06 +01:00
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
}
static int remove_from_poll(struct impl *impl, struct spa_source *source)
{
spa_assert(source->loop == &impl->loop);
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
return spa_system_pollfd_del(impl->system, impl->poll_fd, source->fd);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
static int loop_remove_source(void *object, struct spa_source *source)
{
spa: support: loop: assert loop is not polling when source is removed `spa_source`s whose backing storage is not managed by the loop cannot be safely removed while the loop is polling. Assert that it does not happen.
2022-03-04 16:53:26 +01:00
struct impl *impl = object;
int res = remove_from_poll(impl, source);
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
detach_source(source);
return res;
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
static void loop_queue_destroy(void *data)
{
struct queue *queue = data;
struct impl *impl = queue->impl;
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
if (queue->close_fd)
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
spa_system_close(impl->system, queue->ack_fd);
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
if (queue->overflow)
loop_queue_destroy(queue->overflow);
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
spa_log_info(impl->log, "%p destroyed queue %p idx:%d", impl, queue, queue->idx);
free(queue);
}
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
static struct queue *loop_create_queue(void *object, bool with_fd)
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
{
struct impl *impl = object;
struct queue *queue;
int res;
queue = calloc(1, sizeof(struct queue));
if (queue == NULL)
return NULL;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
queue->idx = IDX_INVALID;
queue->next = IDX_INVALID;
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
queue->impl = impl;
queue->buffer_data = SPA_PTR_ALIGN(queue->buffer_mem, MAX_ALIGN, uint8_t);
spa_ringbuffer_init(&queue->buffer);
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
if (with_fd) {
if ((res = spa_system_eventfd_create(impl->system,
SPA_FD_EVENT_SEMAPHORE | SPA_FD_CLOEXEC)) < 0) {
spa_log_error(impl->log, "%p: can't create ack event: %s",
impl, spa_strerror(res));
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
goto error;
}
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
queue->ack_fd = res;
queue->close_fd = true;
while (true) {
uint16_t idx = SPA_ATOMIC_LOAD(impl->n_queues);
if (idx >= QUEUES_MAX) {
/* this is pretty bad, there are QUEUES_MAX concurrent threads
* that are doing an invoke */
spa_log_error(impl->log, "max queues %d exceeded!", idx);
res = -ENOSPC;
goto error;
}
queue->idx = idx;
if (SPA_ATOMIC_CAS(impl->queues[queue->idx], NULL, queue)) {
SPA_ATOMIC_INC(impl->n_queues);
break;
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
}
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
spa_log_info(impl->log, "%p created queue %p idx:%d %p", impl, queue, queue->idx,
(void*)pthread_self());
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
return queue;
error:
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
loop_queue_destroy(queue);
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
errno = -res;
return NULL;
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
static inline struct queue *get_queue(struct impl *impl)
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
{
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
union tag head, next;
head.v = SPA_ATOMIC_LOAD(impl->head.v);
while (true) {
struct queue *queue;
if (SPA_UNLIKELY(head.t.idx == IDX_INVALID))
return NULL;
queue = impl->queues[head.t.idx];
next.t.idx = queue->next;
next.t.count = head.t.count+1;
if (SPA_LIKELY(__atomic_compare_exchange_n(&impl->head.v, &head.v, next.v,
0, __ATOMIC_ACQ_REL, __ATOMIC_RELAXED))) {
spa_log_trace(impl->log, "%p idx:%d %p", queue, queue->idx, (void*)pthread_self());
return queue;
}
}
return NULL;
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
static inline void put_queue(struct impl *impl, struct queue *queue)
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
{
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
union tag head, next;
spa_log_trace(impl->log, "%p idx:%d %p", queue, queue->idx, (void*)pthread_self());
head.v = SPA_ATOMIC_LOAD(impl->head.v);
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
while (true) {
queue->next = head.t.idx;
next.t.idx = queue->idx;
next.t.count = head.t.count+1;
if (SPA_LIKELY(__atomic_compare_exchange_n(&impl->head.v, &head.v, next.v,
0, __ATOMIC_ACQ_REL, __ATOMIC_RELAXED)))
break;
loop: refcount the queues The loop in the TSS gets an extra refcount and is unreffed when the TSS destroy is called. We can then also ref the queue during the function callback. When the queue (thread) was destroyed during the callback, ignore the result and continue with the next queues. See #4356
2024-10-21 17:47:31 +02:00
}
}
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
static inline int32_t item_compare(struct invoke_item *a, struct invoke_item *b)
loop: flush items in invoke from thread When we do invoke from the loop thread, first process the pending invoke items before handling ours. This can be used to sync the invoke queue before cleanup.
2019-08-16 15:04:27 +02:00
{
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
return (int32_t)(a->count - b->count);
}
static void flush_all_queues(struct impl *impl)
{
uint32_t flush_count;
loop: flush items in invoke from thread When we do invoke from the loop thread, first process the pending invoke items before handling ours. This can be used to sync the invoke queue before cleanup.
2019-08-16 15:04:27 +02:00
int res;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
flush_count = SPA_ATOMIC_INC(impl->flush_count);
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
while (true) {
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
struct queue *cqueue, *queue = NULL;
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
struct invoke_item *citem, *item = NULL;
uint32_t cindex, index;
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
spa_invoke_func_t func;
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
bool block;
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
uint32_t i, n_queues;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
n_queues = SPA_ATOMIC_LOAD(impl->n_queues);
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
for (i = 0; i < n_queues; i++) {
/* loop over all queues and overflow queues */
for (cqueue = impl->queues[i]; cqueue != NULL;
cqueue = SPA_ATOMIC_LOAD(cqueue->overflow)) {
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (spa_ringbuffer_get_read_index(&cqueue->buffer, &cindex) <
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
(int32_t)sizeof(struct invoke_item))
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
continue;
loop: handle queue overflow better When a queue overflows we place the queue back in the stack and try again. Because it's at the top of the stack we take exactly the same queue and keep on looping forever if the other thread is blocked for some reason. Instead, mark the queue as overflowed and only place it back in the stack when we have flushed it. This avoids a deadlock when the main-thread invokes on the data loop and blocks and when the data loop invokes on the main-thread and overflows the queue.
2024-11-14 15:58:09 +01:00
citem = SPA_PTROFF(cqueue->buffer_data,
cindex & (DATAS_SIZE - 1), struct invoke_item);
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (item == NULL || item_compare(citem, item) < 0) {
item = citem;
queue = cqueue;
index = cindex;
}
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
}
}
if (item == NULL)
break;
loop: flush items in invoke from thread When we do invoke from the loop thread, first process the pending invoke items before handling ours. This can be used to sync the invoke queue before cleanup.
2019-08-16 15:04:27 +02:00
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
spa_log_trace_fp(impl->log, "%p: flush item %p", queue, item);
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
/* first we remove the function from the item so that recursive
* calls don't call the callback again. We can't update the
* read index before we call the function because then the item
* might get overwritten. */
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
func = spa_steal_ptr(item->func);
loop: release queue lock before calling invoke function We don't actually need to hold the lock while calling the invoke function, we only need the lock to protect the list of queues.
2024-07-30 12:04:42 +02:00
if (func) {
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
item->res = func(&impl->loop, true, item->seq, item->data,
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
item->size, item->user_data);
loop: release queue lock before calling invoke function We don't actually need to hold the lock while calling the invoke function, we only need the lock to protect the list of queues.
2024-07-30 12:04:42 +02:00
}
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
/* if this function did a recursive invoke, it now flushed the
* ringbuffer and we can exit */
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (flush_count != SPA_ATOMIC_LOAD(impl->flush_count))
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
break;
loop: don't write from multiple threads We can only write from one thread to the ringbuffer so bypass the ringbuffer when doing in-thread invoke. Only flush the current items so that out-of-thread items don't get inserted.
2022-12-08 08:01:40 +01:00
index += item->item_size;
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
block = item->block;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
spa_ringbuffer_read_update(&queue->buffer, index);
loop: flush items in invoke from thread When we do invoke from the loop thread, first process the pending invoke items before handling ours. This can be used to sync the invoke queue before cleanup.
2019-08-16 15:04:27 +02:00
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
if (block && queue->ack_fd != -1) {
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
if ((res = spa_system_eventfd_write(impl->system, queue->ack_fd, 1)) < 0)
loop: don't assert in cleanup Just issue a warning instead of asserting. Firefox does strange things to the fds that make things crash otherwise.
2022-11-08 15:45:55 +01:00
spa_log_warn(impl->log, "%p: failed to write event fd:%d: %s",
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
queue, queue->ack_fd, spa_strerror(res));
loop: flush items in invoke from thread When we do invoke from the loop thread, first process the pending invoke items before handling ours. This can be used to sync the invoke queue before cleanup.
2019-08-16 15:04:27 +02:00
}
}
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
}
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
static int
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
loop_queue_invoke(void *object,
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
spa_invoke_func_t func,
uint32_t seq,
Use remote to join nodes on remote graphs Make a new method that can take a local node and run it in a remote pipeline. This basically replaces all functionality of the streams and more. Add 2 examples for exporting a sink and a v4l2 node Make some more things const Cleanups Make it possible to do things when the node needs scheduling. The default node will schedule the local pipeline but the remote node might also schedule the remote pipeline.
2017-07-25 19:52:31 +02:00
const void *data,
cleanups Follow the data/size argument order convention to make the api more consistent.
2017-11-20 15:26:44 +01:00
size_t size,
loop: add block flag to invoke Remove async shutdown and block instead.
2017-06-26 10:41:19 +02:00
bool block,
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
void *user_data)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
struct queue *queue = object, *orig = queue, *overflow;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
struct impl *impl = queue->impl;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct invoke_item *item;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
int res;
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
int32_t filled;
uint32_t avail, idx, offset, l0;
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
bool in_thread;
pthread_t loop_thread, current_thread = pthread_self();
loop: don't write from multiple threads We can only write from one thread to the ringbuffer so bypass the ringbuffer when doing in-thread invoke. Only flush the current items so that out-of-thread items don't get inserted.
2022-12-08 08:01:40 +01:00
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
again:
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
loop_thread = impl->thread;
in_thread = (loop_thread == 0 || pthread_equal(loop_thread, current_thread));
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
filled = spa_ringbuffer_get_write_index(&queue->buffer, &idx);
loop: sleep and retry when the invoke queue is full When the invoke ringbuffer is full, sleep a little and try again. Add an option to set the retty timeout, setting this to 0 restores the old behaviour of returning -EPIPE. Most callers don't check the return values and might assume the invoke call is queued or executed, which could cause crashes or leaks. When the queue overruns, it's better to log a warning and hope that the problem is resolved soon. We might abort or return the error to the caller later if we want to break the retry loop. See !1887
2024-02-05 19:40:05 +01:00
spa_assert_se(filled >= 0 && filled <= DATAS_SIZE && "queue xrun");
avail = (uint32_t)(DATAS_SIZE - filled);
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (avail < sizeof(struct invoke_item))
loop: sleep and retry when the invoke queue is full When the invoke ringbuffer is full, sleep a little and try again. Add an option to set the retty timeout, setting this to 0 restores the old behaviour of returning -EPIPE. Most callers don't check the return values and might assume the invoke call is queued or executed, which could cause crashes or leaks. When the queue overruns, it's better to log a warning and hope that the problem is resolved soon. We might abort or return the error to the caller later if we want to break the retry loop. See !1887
2024-02-05 19:40:05 +01:00
goto xrun;
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
offset = idx & (DATAS_SIZE - 1);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: Fix crash because of overflow Also check if there is enough space to write the payload bytes. We check if there is enough space for the invoke_item structure first. Then we calculate how much bytes we need to use for the payload but we fail to check if we can actually write that much data, risking overwriting existing data from the ringbuffer and causing a crash later when we try to jump to invalid memory. Add some more comments.
2021-07-19 09:53:23 +02:00
/* l0 is remaining size in ringbuffer, this should always be larger than
* invoke_item, see below */
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
l0 = DATAS_SIZE - offset;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
item = SPA_PTROFF(queue->buffer_data, offset, struct invoke_item);
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
item->func = func;
item->seq = seq;
loop: flush items in the order they were added Add a count to each invoke item that is updated with an increasing loop atomic counter. Flush items from the queues based on their count so that items are flushed in the order they were added even if they were added to different queues.
2024-05-08 12:21:54 +02:00
item->count = SPA_ATOMIC_INC(impl->count);
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
item->size = size;
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
item->block = in_thread ? false : block;
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
item->user_data = user_data;
loop: support recursive loop flush Always append the item to the ringbuffer, even if we are invoking from the thread itself. This ensure all items are always invoked in the right order. If we invoke from the thread, flush all items of the ringbuffer and return. Make sure to set the callback to NULL before invoking so that recursive invoke doesn't call it again. When while flushing the items we get a recursive invoke, detect this with a counter and return immediately.
2022-12-07 21:18:16 +01:00
item->res = 0;
support: use defines for alignment
2022-01-03 11:14:15 +01:00
item->item_size = SPA_ROUND_UP_N(sizeof(struct invoke_item) + size, ITEM_ALIGN);
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
spa_log_trace(impl->log, "%p: add item %p filled:%d block:%d", queue, item, filled, block);
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
loop: improve invoke avail check First calculate the size of the aligned payload and then check if we can fit this aligned payload in the remaining space in the ringbuffer. Otherwise we might be able to fit the item + payload in the remaining space but then place the alignment bytes at the begginning, which would break alignment of the next invoke_item struct.
2021-07-19 10:12:15 +02:00
if (l0 >= item->item_size) {
loop: Fix crash because of overflow Also check if there is enough space to write the payload bytes. We check if there is enough space for the invoke_item structure first. Then we calculate how much bytes we need to use for the payload but we fail to check if we can actually write that much data, risking overwriting existing data from the ringbuffer and causing a crash later when we try to jump to invalid memory. Add some more comments.
2021-07-19 09:53:23 +02:00
/* item + size fit in current ringbuffer idx */
spa/buffer: rename SPA_MEMBER to SPA_PTROFF SPA_MEMBER is misleading, all we're doing here is pointer+offset and a type-casting the result. Rename to SPA_PTROFF which is more expressive (and has the same number of characters so we don't need to re-indent).
2021-05-06 13:41:44 +10:00
item->data = SPA_PTROFF(item, sizeof(struct invoke_item), void);
loop: Fix crash because of overflow Also check if there is enough space to write the payload bytes. We check if there is enough space for the invoke_item structure first. Then we calculate how much bytes we need to use for the payload but we fail to check if we can actually write that much data, risking overwriting existing data from the ringbuffer and causing a crash later when we try to jump to invalid memory. Add some more comments.
2021-07-19 09:53:23 +02:00
if (l0 < sizeof(struct invoke_item) + item->item_size) {
/* not enough space for next invoke_item, fill up till the end
* so that the next item will be at the start */
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
item->item_size = l0;
loop: Fix crash because of overflow Also check if there is enough space to write the payload bytes. We check if there is enough space for the invoke_item structure first. Then we calculate how much bytes we need to use for the payload but we fail to check if we can actually write that much data, risking overwriting existing data from the ringbuffer and causing a crash later when we try to jump to invalid memory. Add some more comments.
2021-07-19 09:53:23 +02:00
}
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
} else {
loop: Fix crash because of overflow Also check if there is enough space to write the payload bytes. We check if there is enough space for the invoke_item structure first. Then we calculate how much bytes we need to use for the payload but we fail to check if we can actually write that much data, risking overwriting existing data from the ringbuffer and causing a crash later when we try to jump to invalid memory. Add some more comments.
2021-07-19 09:53:23 +02:00
/* item does not fit, place the invoke_item at idx and start the
* data at the start of the ringbuffer */
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
item->data = queue->buffer_data;
support: use defines for alignment
2022-01-03 11:14:15 +01:00
item->item_size = SPA_ROUND_UP_N(l0 + size, ITEM_ALIGN);
loop: always place the invoke item in the queue Always place the invoke item in the queue and then either signal the other thread or flush the queue when not already flushing.
2020-11-16 15:16:20 +01:00
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (avail < item->item_size)
loop: sleep and retry when the invoke queue is full When the invoke ringbuffer is full, sleep a little and try again. Add an option to set the retty timeout, setting this to 0 restores the old behaviour of returning -EPIPE. Most callers don't check the return values and might assume the invoke call is queued or executed, which could cause crashes or leaks. When the queue overruns, it's better to log a warning and hope that the problem is resolved soon. We might abort or return the error to the caller later if we want to break the retry loop. See !1887
2024-02-05 19:40:05 +01:00
goto xrun;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
loop: align buffer and invoke_items to 8 bytes To avoid unaligned access messages from sanitizer See #497
2020-12-21 20:46:13 +01:00
if (data && size > 0)
memcpy(item->data, data, size);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
spa_ringbuffer_write_update(&queue->buffer, idx + item->item_size);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
if (in_thread) {
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
put_queue(impl, orig);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
/* when there is no thread running the loop we flush the queues from
* this invoking thread but we need to serialize the flushing here with
* a mutex */
if (loop_thread == 0)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_mutex_lock(&impl->lock);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
flush_all_queues(impl);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
if (loop_thread == 0)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_mutex_unlock(&impl->lock);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
res = item->res;
} else {
loop_signal_event(impl, impl->wakeup);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: don't usleep when queue is full When the queue is full, before this patch we used to go into usleep in the hope that the other thread will run and empty the queue and that we can retry after the usleep. This however does not always work because the other thread might be waiting for the thread that does the invoke call and we lock forever. Therefore we should always try to make progress in some way. Instead of waiting, allocate an (or use the previously allocated) overflow queue and write to that one. We can chain multiple overflow queues together as many as we need (but we might want to bound that as well). The loop.retry-timeout property is now deprecated. See #4114
2024-08-06 12:05:11 +02:00
if (block && queue->ack_fd != -1) {
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
uint64_t count = 1;
loop: blocking wait Release the locks before waiting.
2018-04-19 22:01:18 +02:00
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
if ((res = spa_system_eventfd_read(impl->system, queue->ack_fd, &count)) < 0)
spa_log_warn(impl->log, "%p: failed to read event fd:%d: %s",
queue, queue->ack_fd, spa_strerror(res));
port: work on port commands
2017-11-14 15:44:48 +01:00
loop: improve in_thread handling of invoke queue Because we now have a dedicated queue per thread, we can simply add our invoke item to the queue and then flush all the queues when we are running in the thread of the loop. This simplifies some things and removes potential out-of-order messages that got queued while flushing.
2024-04-29 15:56:00 +02:00
res = item->res;
}
else {
if (seq != SPA_ID_INVALID)
res = SPA_RESULT_RETURN_ASYNC(seq);
else
res = 0;
}
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
put_queue(impl, orig);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
return res;
loop: sleep and retry when the invoke queue is full When the invoke ringbuffer is full, sleep a little and try again. Add an option to set the retty timeout, setting this to 0 restores the old behaviour of returning -EPIPE. Most callers don't check the return values and might assume the invoke call is queued or executed, which could cause crashes or leaks. When the queue overruns, it's better to log a warning and hope that the problem is resolved soon. We might abort or return the error to the caller later if we want to break the retry loop. See !1887
2024-02-05 19:40:05 +01:00
xrun:
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
/* we overflow, make a new queue that shares the same fd
* and place it in the overflow array. We hold the queue so there
* is only ever one writer to the overflow field. */
overflow = queue->overflow;
if (overflow == NULL) {
overflow = loop_create_queue(impl, false);
if (overflow == NULL)
return -errno;
overflow->ack_fd = queue->ack_fd;
SPA_ATOMIC_STORE(queue->overflow, overflow);
}
queue = overflow;
goto again;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
Improve loop callbacks Pass just one data item to the callbacks. Add properties to port. Add user data to link Handle autolink with multiple ports More work on jack support
2017-08-11 19:16:30 +02:00
static void wakeup_func(void *data, uint64_t count)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
struct impl *impl = data;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
flush_all_queues(impl);
}
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
static int loop_invoke(void *object, spa_invoke_func_t func, uint32_t seq,
const void *data, size_t size, bool block, void *user_data)
{
struct impl *impl = object;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
struct queue *queue;
int res = 0, suppressed;
uint64_t nsec;
while (true) {
queue = get_queue(impl);
if (SPA_UNLIKELY(queue == NULL))
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
queue = loop_create_queue(impl, true);
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if (SPA_UNLIKELY(queue == NULL)) {
if (SPA_UNLIKELY(errno != ENOSPC))
return -errno;
/* there was no space for a new queue. This means QUEUE_MAX
* threads are concurrently doing an invoke. We can wait a little
* and retry to get a queue */
if (impl->retry_timeout == 0)
return -EPIPE;
loop: handle queue overflow better When a queue overflows we place the queue back in the stack and try again. Because it's at the top of the stack we take exactly the same queue and keep on looping forever if the other thread is blocked for some reason. Instead, mark the queue as overflowed and only place it back in the stack when we have flushed it. This avoids a deadlock when the main-thread invokes on the data loop and blocks and when the data loop invokes on the main-thread and overflows the queue.
2024-11-14 15:58:09 +01:00
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
nsec = get_time_ns(impl->system);
if ((suppressed = spa_ratelimit_test(&impl->rate_limit, nsec)) >= 0) {
spa_log_warn(impl->log, "%p: out of queues, retrying (%d suppressed)",
impl, suppressed);
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
usleep(impl->retry_timeout);
} else {
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
res = loop_queue_invoke(queue, func, seq, data, size, block, user_data);
loop: add overflow queues again Add the overflow queues again. We can easily iterate atomically over the overflow queues and flush them. Overflowing a queue is quite common when heavy swapping is done and should never cause a lockup, so allocate new queues as we need them. We can share the eventfd with the main queue to avoid wastings fds. The limit on the number of queues is then only for when concurrent threads want to invoke things, so 128 is plenty enough.
2024-11-14 17:31:23 +01:00
break;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
}
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
return res;
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
}
loop: add method to run a function with the lock Convert some _invoke to _locked
2025-03-10 17:24:13 +01:00
static int loop_locked(void *object, spa_invoke_func_t func, uint32_t seq,
const void *data, size_t size, void *user_data)
{
struct impl *impl = object;
int res;
pthread_mutex_lock(&impl->lock);
res = func(&impl->loop, false, seq, data, size, user_data);
pthread_mutex_unlock(&impl->lock);
return res;
}
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static int loop_get_fd(void *object)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
return impl->poll_fd;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static void
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
loop_add_hook(void *object,
struct spa_hook *hook,
const struct spa_loop_control_hooks *hooks,
void *data)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
hooks: add and use _fast callback function Add a _fast callback function that skips the version and method check. We can use this in places where performance is critical when we do the check out of the critical loops. Make all system methods _fast calls. We expect them to exist and have the right version. If we add new versions we can make them slow.
2023-05-06 00:24:05 +02:00
spa_return_if_fail(SPA_CALLBACK_CHECK(hooks, before, 0));
spa_return_if_fail(SPA_CALLBACK_CHECK(hooks, after, 0));
Move listeners to spa and rename to hooks The listeners are generally useful in spa, move it there and rename to hooks. Implement loop hooks with it. Fix some cleanup issues in stream and remote
2017-08-08 16:56:29 +02:00
spa_hook_list_append(&impl->hooks_list, hook, hooks, data);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static void loop_enter(void *object)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
pthread_t thread_id = pthread_self();
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_lock(&impl->lock);
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
if (impl->enter_count == 0) {
spa_return_if_fail(impl->thread == 0);
impl->thread = thread_id;
impl->enter_count = 1;
} else {
spa_return_if_fail(impl->enter_count > 0);
thread: use pthread_equal to compare thread ids
2023-04-04 12:43:25 +02:00
spa_return_if_fail(pthread_equal(impl->thread, thread_id));
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
impl->enter_count++;
}
move some trace to trace_fp
2023-05-05 17:41:13 +02:00
spa_log_trace_fp(impl->log, "%p: enter %p", impl, (void *) impl->thread);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static void loop_leave(void *object)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
pthread_t thread_id = pthread_self();
spa_return_if_fail(impl->enter_count > 0);
thread: use pthread_equal to compare thread ids
2023-04-04 12:43:25 +02:00
spa_return_if_fail(pthread_equal(impl->thread, thread_id));
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
move some trace to trace_fp
2023-05-05 17:41:13 +02:00
spa_log_trace_fp(impl->log, "%p: leave %p", impl, (void *) impl->thread);
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
loop: clean polling flag when leaving the loop When we leave the last recursive enter of the loop, clear the polling flag. It might be possible that it was not cleared because the loop might have been killed with pthread_kill. In any case, the _leave calls need to be made in this case as well. This fixes issues when jack clients stop because it triggers and assert because the polling flag is still active when the object is cleared. See !1171
2022-03-07 10:27:22 +01:00
if (--impl->enter_count == 0) {
support: loop_enter/leave hardening This commit adds a counter for loop_enter/leave and checks: - consecutive enter are used on the same thread - leave is used on the same thread as enter - at destruction, the enter_count must be 0
2022-02-08 10:58:36 +01:00
impl->thread = 0;
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
flush_all_queues(impl);
loop: clean polling flag when leaving the loop When we leave the last recursive enter of the loop, clear the polling flag. It might be possible that it was not cleared because the loop might have been killed with pthread_kill. In any case, the _leave calls need to be made in this case as well. This fixes issues when jack clients stop because it triggers and assert because the polling flag is still active when the object is cleared. See !1171
2022-03-07 10:27:22 +01:00
impl->polling = false;
}
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_unlock(&impl->lock);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
support: add support for checking loop context Add check for running the the loop context and thread. Add checks in filter and stream to avoid doing things when not run from the context main-loop because this can crash things when doing IPC from concurrent threads.
2023-03-31 18:40:57 +02:00
static int loop_check(void *object)
{
struct impl *impl = object;
pthread_t thread_id = pthread_self();
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
int res;
/* we are in the thread running the loop */
if (impl->thread == 0 || pthread_equal(impl->thread, thread_id))
return 1;
/* if lock taken by something else, error */
if ((res = pthread_mutex_trylock(&impl->lock)) != 0)
return -res;
/* we could take the lock, check if we actually locked it somewhere */
res = impl->recurse > 0 ? 1 : -EPERM;
pthread_mutex_unlock(&impl->lock);
return res;
support: add support for checking loop context Add check for running the the loop context and thread. Add checks in filter and stream to avoid doing things when not run from the context main-loop because this can crash things when doing IPC from concurrent threads.
2023-03-31 18:40:57 +02:00
}
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
static int loop_lock(void *object)
{
struct impl *impl = object;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
int res;
if ((res = pthread_mutex_lock(&impl->lock)) == 0)
impl->recurse++;
return -res;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
}
static int loop_unlock(void *object)
{
struct impl *impl = object;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
int res;
spa_return_val_if_fail(impl->recurse > 0, -EIO);
impl->recurse--;
if ((res = pthread_mutex_unlock(&impl->lock)) != 0)
impl->recurse++;
return -res;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
}
Revert "spa: loop: Change get_time() timeout to unsigned" This reverts commit c515f9bf8effdda78f4bfa091a8b35b6e7552c1a. The PW APIs use int64_t (partly because SPA_NSEC_PER_SEC is an LL), and we don't want to change already public API.
2025-06-03 15:20:15 +05:30
static int loop_get_time(void *object, struct timespec *abstime, int64_t timeout)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
{
if (clock_gettime(CLOCK_REALTIME, abstime) < 0)
return -errno;
abstime->tv_sec += timeout / SPA_NSEC_PER_SEC;
abstime->tv_nsec += timeout % SPA_NSEC_PER_SEC;
if (abstime->tv_nsec >= SPA_NSEC_PER_SEC) {
abstime->tv_sec++;
abstime->tv_nsec -= SPA_NSEC_PER_SEC;
}
return 0;
}
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
static int loop_wait(void *object, const struct timespec *abstime)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
{
struct impl *impl = object;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
int res;
impl->n_waiting++;
impl->recurse--;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
if (abstime)
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
res = pthread_cond_timedwait(&impl->cond, &impl->lock, abstime);
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
else
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
res = pthread_cond_wait(&impl->cond, &impl->lock);
impl->recurse++;
impl->n_waiting--;
return -res;
}
static int loop_signal(void *object, bool wait_for_accept)
{
struct impl *impl = object;
spa: loop: Fix potential uninitialised result
2025-06-06 12:47:09 +05:30
int res = 0;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
if (impl->n_waiting > 0)
if ((res = pthread_cond_broadcast(&impl->cond)) != 0)
return -res;
if (wait_for_accept) {
impl->n_waiting_for_accept++;
while (impl->n_waiting_for_accept > 0) {
if ((res = pthread_cond_wait(&impl->accept_cond, &impl->lock)) != 0)
return -res;
}
}
return res;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
}
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
static int loop_accept(void *object)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
{
struct impl *impl = object;
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
impl->n_waiting_for_accept--;
return -pthread_cond_signal(&impl->accept_cond);
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
}
support: add support for checking loop context Add check for running the the loop context and thread. Add checks in filter and stream to avoid doing things when not run from the context main-loop because this can crash things when doing IPC from concurrent threads.
2023-03-31 18:40:57 +02:00
Revert "loop: remove destroy list" This reverts commit c474846c42967c44db069a23b76a29da6f496f33. In addition, `s->loop` is also checked before dispatching a source. The destroy list is needed in the presence of threads. The issue is that a source may be destroyed between `epoll_wait()` returning and thread loop lock being acquired. If this source is active, then a use-after-free will be triggered when the thread loop acquires the lock and starts dispatching the sources. thread 1 thread 2 ---------- ---------- loop_iterate spa_loop_control_hook_before // release lock pw_thread_loop_lock spa_system_pollfd_wait // assume it returns with source A pw_loop_destroy_source(..., A) // frees storage of A pw_thread_loop_unlock spa_loop_control_hook_after // acquire the lock for (...) { struct spa_source *s = ep[i].data; s->rmask = ep[i].events; // use-after-free if `s` refers to // the previously freed `A` Fixes #2147
2022-02-18 18:36:36 +01:00
static inline void process_destroy(struct impl *impl)
{
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
struct source_impl *s;
spa_list_consume(s, &impl->destroy_list, link) {
spa_list_remove(&s->link);
detach_source(&s->source);
spa_list_append(&s->impl->free_list, &s->link);
}
Revert "loop: remove destroy list" This reverts commit c474846c42967c44db069a23b76a29da6f496f33. In addition, `s->loop` is also checked before dispatching a source. The destroy list is needed in the presence of threads. The issue is that a source may be destroyed between `epoll_wait()` returning and thread loop lock being acquired. If this source is active, then a use-after-free will be triggered when the thread loop acquires the lock and starts dispatching the sources. thread 1 thread 2 ---------- ---------- loop_iterate spa_loop_control_hook_before // release lock pw_thread_loop_lock spa_system_pollfd_wait // assume it returns with source A pw_loop_destroy_source(..., A) // frees storage of A pw_thread_loop_unlock spa_loop_control_hook_after // acquire the lock for (...) { struct spa_source *s = ep[i].data; s->rmask = ep[i].events; // use-after-free if `s` refers to // the previously freed `A` Fixes #2147
2022-02-18 18:36:36 +01:00
}
spa: support: loop: handle cancellation better Register a pthread cleanup handler to guarantee that `spa_source::{priv, rmask}` are cleared even if the thread is cancelled while the loop is dispatching. This is necessary, otherwise `spa_source::priv` could point to the stack of the cancelled thread, which will lead to problems like this later: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 144 e->data = NULL; [Current thread is 1 (LWP 5274)] (gdb) p e $1 = (struct spa_poll_event *) 0x7f845e297820 (gdb) bt #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 #1 0x00007f846b0276ad in free_source (s=0x7f845f435f60) at ../spa/plugins/support/loop.c:359 #2 0x00007f846b02a453 in loop_destroy_source (object=0x7f845f3af478, source=0x7f845f435f60) at ../spa/plugins/support/loop.c:786 #3 0x00007f846b02a886 in impl_clear (handle=0x7f845f3af478) at ../spa/plugins/support/loop.c:859 #4 0x00007f846b172f40 in unref_handle (handle=0x7f845f3af450) at ../src/pipewire/pipewire.c:211 #5 0x00007f846b173579 in pw_unload_spa_handle (handle=0x7f845f3af478) at ../src/pipewire/pipewire.c:346 #6 0x00007f846b15a761 in pw_loop_destroy (loop=0x7f845f434e30) at ../src/pipewire/loop.c:159 #7 0x00007f846b135d8e in pw_data_loop_destroy (loop=0x7f845f434cb0) at ../src/pipewire/data-loop.c:166 #8 0x00007f846b12c31c in pw_context_destroy (context=0x7f845f41c690) at ../src/pipewire/context.c:485 #9 0x00007f846b3ddf9e in jack_client_close (client=0x7f845f3c1030) at ../pipewire-jack/src/pipewire-jack.c:3481 ...
2022-06-01 16:36:56 +02:00
struct cancellation_handler_data {
struct spa_poll_event *ep;
int ep_count;
};
static void cancellation_handler(void *closure)
{
const struct cancellation_handler_data *data = closure;
for (int i = 0; i < data->ep_count; i++) {
struct spa_source *s = data->ep[i].data;
if (SPA_LIKELY(s)) {
s->rmask = 0;
s->priv = NULL;
}
}
}
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
static int loop_iterate_cancel(void *object, int timeout)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
support: also protect against recursive invocations Add an extra private field to the source to store the pollevent of the current iteration. This changes ABI but it seems an embedded source is not used outside of our own plugins and the unit test doesn't test this ABI case. Whenever a source is removed, we can set the data field of the pollevent to NULL so that it won't be handled in any iteration anymore. Avoid dispatching the same event multiple times when doing recursive iterations. Add some more unit tests for this. Fixes #2114
2022-02-08 17:21:10 +01:00
struct spa_poll_event ep[MAX_EP], *e;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int i, nfds;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: add `polling` flag Store whether or not the loop is currently polling, i.e. calling `spa_system_pollfd_wait()`.
2022-02-19 11:44:04 +01:00
impl->polling = true;
loop: simplify before and after events Because the signal can't be removed from the callback we can simply iterate backwards and then forwards. The first added hook (the unlock/lock pair) is called last before going into the poll and first when leaving. This executes all other callbacks inside a locked situation. And removing them with the lock is not going to cause problems.
2020-09-16 13:20:19 +02:00
spa_loop_control_hook_before(&impl->hooks_list);
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_unlock(&impl->lock);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: get array size using macro
2022-02-21 17:28:43 +01:00
nfds = spa_system_pollfd_wait(impl->system, impl->poll_fd, ep, SPA_N_ELEMENTS(ep), timeout);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_lock(&impl->lock);
loop: simplify before and after events Because the signal can't be removed from the callback we can simply iterate backwards and then forwards. The first added hook (the unlock/lock pair) is called last before going into the poll and first when leaving. This executes all other callbacks inside a locked situation. And removing them with the lock is not going to cause problems.
2020-09-16 13:20:19 +02:00
spa_loop_control_hook_after(&impl->hooks_list);
spa: support: loop: add `polling` flag Store whether or not the loop is currently polling, i.e. calling `spa_system_pollfd_wait()`.
2022-02-19 11:44:04 +01:00
impl->polling = false;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: handle cancellation better Register a pthread cleanup handler to guarantee that `spa_source::{priv, rmask}` are cleared even if the thread is cancelled while the loop is dispatching. This is necessary, otherwise `spa_source::priv` could point to the stack of the cancelled thread, which will lead to problems like this later: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 144 e->data = NULL; [Current thread is 1 (LWP 5274)] (gdb) p e $1 = (struct spa_poll_event *) 0x7f845e297820 (gdb) bt #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 #1 0x00007f846b0276ad in free_source (s=0x7f845f435f60) at ../spa/plugins/support/loop.c:359 #2 0x00007f846b02a453 in loop_destroy_source (object=0x7f845f3af478, source=0x7f845f435f60) at ../spa/plugins/support/loop.c:786 #3 0x00007f846b02a886 in impl_clear (handle=0x7f845f3af478) at ../spa/plugins/support/loop.c:859 #4 0x00007f846b172f40 in unref_handle (handle=0x7f845f3af450) at ../src/pipewire/pipewire.c:211 #5 0x00007f846b173579 in pw_unload_spa_handle (handle=0x7f845f3af478) at ../src/pipewire/pipewire.c:346 #6 0x00007f846b15a761 in pw_loop_destroy (loop=0x7f845f434e30) at ../src/pipewire/loop.c:159 #7 0x00007f846b135d8e in pw_data_loop_destroy (loop=0x7f845f434cb0) at ../src/pipewire/data-loop.c:166 #8 0x00007f846b12c31c in pw_context_destroy (context=0x7f845f41c690) at ../src/pipewire/context.c:485 #9 0x00007f846b3ddf9e in jack_client_close (client=0x7f845f3c1030) at ../pipewire-jack/src/pipewire-jack.c:3481 ...
2022-06-01 16:36:56 +02:00
struct cancellation_handler_data cdata = { ep, nfds };
pthread_cleanup_push(cancellation_handler, &cdata);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
/* first we set all the rmasks, then call the callbacks. The reason is that
* some callback might also want to look at other sources it manages and
* can then reset the rmask to suppress the callback */
for (i = 0; i < nfds; i++) {
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
struct spa_source *s = ep[i].data;
spa: support: loop: assert source's loop when dispatching
2022-03-06 14:56:29 +01:00
spa_assert(s->loop == &impl->loop);
support: also protect against recursive invocations Add an extra private field to the source to store the pollevent of the current iteration. This changes ABI but it seems an embedded source is not used outside of our own plugins and the unit test doesn't test this ABI case. Whenever a source is removed, we can set the data field of the pollevent to NULL so that it won't be handled in any iteration anymore. Avoid dispatching the same event multiple times when doing recursive iterations. Add some more unit tests for this. Fixes #2114
2022-02-08 17:21:10 +01:00
s->rmask = ep[i].events;
/* already active in another iteration of the loop,
* remove it from that iteration */
if (SPA_UNLIKELY(e = s->priv))
e->data = NULL;
s->priv = &ep[i];
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
if (SPA_UNLIKELY(!spa_list_is_empty(&impl->destroy_list)))
process_destroy(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
for (i = 0; i < nfds; i++) {
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
struct spa_source *s = ep[i].data;
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
if (SPA_LIKELY(s && s->rmask))
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
s->func(s);
spa: support: loop: reset rmask after dispatch Reset the `rmask` of the sources to zero after dispatching the callbacks. This way the sources are always as up-to-date as possible.
2022-02-19 09:42:49 +01:00
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
spa: support: loop: handle cancellation better Register a pthread cleanup handler to guarantee that `spa_source::{priv, rmask}` are cleared even if the thread is cancelled while the loop is dispatching. This is necessary, otherwise `spa_source::priv` could point to the stack of the cancelled thread, which will lead to problems like this later: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 144 e->data = NULL; [Current thread is 1 (LWP 5274)] (gdb) p e $1 = (struct spa_poll_event *) 0x7f845e297820 (gdb) bt #0 0x00007f846b025be2 in detach_source (source=0x7f845f435f60) at ../spa/plugins/support/loop.c:144 #1 0x00007f846b0276ad in free_source (s=0x7f845f435f60) at ../spa/plugins/support/loop.c:359 #2 0x00007f846b02a453 in loop_destroy_source (object=0x7f845f3af478, source=0x7f845f435f60) at ../spa/plugins/support/loop.c:786 #3 0x00007f846b02a886 in impl_clear (handle=0x7f845f3af478) at ../spa/plugins/support/loop.c:859 #4 0x00007f846b172f40 in unref_handle (handle=0x7f845f3af450) at ../src/pipewire/pipewire.c:211 #5 0x00007f846b173579 in pw_unload_spa_handle (handle=0x7f845f3af478) at ../src/pipewire/pipewire.c:346 #6 0x00007f846b15a761 in pw_loop_destroy (loop=0x7f845f434e30) at ../src/pipewire/loop.c:159 #7 0x00007f846b135d8e in pw_data_loop_destroy (loop=0x7f845f434cb0) at ../src/pipewire/data-loop.c:166 #8 0x00007f846b12c31c in pw_context_destroy (context=0x7f845f41c690) at ../src/pipewire/context.c:485 #9 0x00007f846b3ddf9e in jack_client_close (client=0x7f845f3c1030) at ../pipewire-jack/src/pipewire-jack.c:3481 ...
2022-06-01 16:36:56 +02:00
pthread_cleanup_pop(true);
Revert "loop: remove destroy list" This reverts commit c474846c42967c44db069a23b76a29da6f496f33. In addition, `s->loop` is also checked before dispatching a source. The destroy list is needed in the presence of threads. The issue is that a source may be destroyed between `epoll_wait()` returning and thread loop lock being acquired. If this source is active, then a use-after-free will be triggered when the thread loop acquires the lock and starts dispatching the sources. thread 1 thread 2 ---------- ---------- loop_iterate spa_loop_control_hook_before // release lock pw_thread_loop_lock spa_system_pollfd_wait // assume it returns with source A pw_loop_destroy_source(..., A) // frees storage of A pw_thread_loop_unlock spa_loop_control_hook_after // acquire the lock for (...) { struct spa_source *s = ep[i].data; s->rmask = ep[i].events; // use-after-free if `s` refers to // the previously freed `A` Fixes #2147
2022-02-18 18:36:36 +01:00
loop: return number of dispatched fds and update docs
2018-06-01 11:16:53 +02:00
return nfds;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
static int loop_iterate(void *object, int timeout)
{
struct impl *impl = object;
struct spa_poll_event ep[MAX_EP], *e;
int i, nfds;
impl->polling = true;
spa_loop_control_hook_before(&impl->hooks_list);
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_unlock(&impl->lock);
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
nfds = spa_system_pollfd_wait(impl->system, impl->poll_fd, ep, SPA_N_ELEMENTS(ep), timeout);
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
pthread_mutex_lock(&impl->lock);
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
spa_loop_control_hook_after(&impl->hooks_list);
impl->polling = false;
/* first we set all the rmasks, then call the callbacks. The reason is that
* some callback might also want to look at other sources it manages and
* can then reset the rmask to suppress the callback */
for (i = 0; i < nfds; i++) {
struct spa_source *s = ep[i].data;
s->rmask = ep[i].events;
/* already active in another iteration of the loop,
* remove it from that iteration */
if (SPA_UNLIKELY(e = s->priv))
e->data = NULL;
s->priv = &ep[i];
}
if (SPA_UNLIKELY(!spa_list_is_empty(&impl->destroy_list)))
process_destroy(impl);
loop: clear rmask after dispatching all sources To make the unit tests work again.
2023-05-05 18:36:08 +02:00
for (i = 0; i < nfds; i++) {
struct spa_source *s = ep[i].data;
if (SPA_LIKELY(s && s->rmask))
s->func(s);
}
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
for (i = 0; i < nfds; i++) {
struct spa_source *s = ep[i].data;
if (SPA_LIKELY(s)) {
s->rmask = 0;
s->priv = NULL;
}
}
return nfds;
}
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
static struct source_impl *get_source(struct impl *impl)
{
struct source_impl *source;
if (!spa_list_is_empty(&impl->free_list)) {
source = spa_list_first(&impl->free_list, struct source_impl, link);
spa_list_remove(&source->link);
spa_zero(*source);
} else {
source = calloc(1, sizeof(struct source_impl));
}
if (source != NULL) {
source->impl = impl;
spa_list_insert(&impl->source_list, &source->link);
}
return source;
}
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
static void source_io_func(struct spa_source *source)
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
spa_log_trace_fp(s->impl->log, "%p: io %08x", s, source->rmask);
s->func.io(source->data, source->fd, source->rmask);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static struct spa_source *loop_add_io(void *object,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
int fd,
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
uint32_t mask,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
bool close, spa_source_io_func_t func, void *data)
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl *source;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
source = get_source(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (source == NULL)
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.func = source_io_func;
source->source.data = data;
source->source.fd = fd;
source->source.mask = mask;
source->close = close;
source->func.io = func;
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
if ((res = loop_add_source(impl, &source->source)) < 0) {
if (res != -EPERM)
goto error_exit_free;
/* file fds (stdin/stdout/...) give EPERM in epoll. Those fds always
* return from epoll with the mask set, so we can handle this with
* an idle source */
source->source.rmask = mask;
source->fallback = spa_loop_utils_add_idle(&impl->utils,
mask & (SPA_IO_IN | SPA_IO_OUT) ? true : false,
(spa_source_idle_func_t) source_io_func, source);
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_trace(impl->log, "%p: adding fallback %p", impl,
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
source->fallback);
}
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return &source->source;
error_exit_free:
free(source);
errno = -res;
error_exit:
return NULL;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
static int loop_update_io(void *object, struct spa_source *source, uint32_t mask)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
struct impl *impl = object;
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
int res;
spa: support: loop: print previous mask when updating Print the previous event mask in `loop_update_io()` as well as the new one.
2022-02-19 19:38:08 +01:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(s->impl == object);
spa: support: loop: assert source type
2022-02-19 19:17:42 +01:00
spa_assert(source->func == source_io_func);
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa: support: loop: print previous mask when updating Print the previous event mask in `loop_update_io()` as well as the new one.
2022-02-19 19:38:08 +01:00
spa_log_trace(impl->log, "%p: update %08x -> %08x", s, source->mask, mask);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->mask = mask;
spa: support: loop: print previous mask when updating Print the previous event mask in `loop_update_io()` as well as the new one.
2022-02-19 19:38:08 +01:00
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
if (s->fallback)
res = spa_loop_utils_enable_idle(&impl->utils, s->fallback,
mask & (SPA_IO_IN | SPA_IO_OUT) ? true : false);
else
res = loop_update_source(object, source);
return res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static void source_idle_func(struct spa_source *source)
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
s->func.idle(source->data);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
static int loop_enable_idle(void *object, struct spa_source *source, bool enabled)
loop: cleanups
2019-05-23 15:11:49 +02:00
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
int res = 0;
loop: cleanups
2019-05-23 15:11:49 +02:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(s->impl == object);
spa: support: loop: assert source type
2022-02-19 19:17:42 +01:00
spa_assert(source->func == source_idle_func);
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if (enabled && !s->enabled) {
if ((res = spa_system_eventfd_write(s->impl->system, source->fd, 1)) < 0)
loop: don't assert in cleanup Just issue a warning instead of asserting. Firefox does strange things to the fds that make things crash otherwise.
2022-11-08 15:45:55 +01:00
spa_log_warn(s->impl->log, "%p: failed to write idle fd:%d: %s",
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
source, source->fd, spa_strerror(res));
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
} else if (!enabled && s->enabled) {
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
uint64_t count;
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if ((res = spa_system_eventfd_read(s->impl->system, source->fd, &count)) < 0)
loop: don't assert in cleanup Just issue a warning instead of asserting. Firefox does strange things to the fds that make things crash otherwise.
2022-11-08 15:45:55 +01:00
spa_log_warn(s->impl->log, "%p: failed to read idle fd:%d: %s",
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
source, source->fd, spa_strerror(res));
loop: cleanups
2019-05-23 15:11:49 +02:00
}
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
s->enabled = enabled;
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
return res;
loop: cleanups
2019-05-23 15:11:49 +02:00
}
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static struct spa_source *loop_add_idle(void *object,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
bool enabled, spa_source_idle_func_t func, void *data)
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl *source;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
source = get_source(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (source == NULL)
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit;
if ((res = spa_system_eventfd_create(impl->system, SPA_FD_CLOEXEC | SPA_FD_NONBLOCK)) < 0)
goto error_exit_free;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.func = source_idle_func;
source->source.data = data;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
source->source.fd = res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->close = true;
source->source.mask = SPA_IO_IN;
source->func.idle = func;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
if ((res = loop_add_source(impl, &source->source)) < 0)
goto error_exit_close;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (enabled)
loop: cleanups
2019-05-23 15:11:49 +02:00
loop_enable_idle(impl, &source->source, true);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return &source->source;
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit_close:
spa_system_close(impl->system, source->source.fd);
error_exit_free:
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
free(source);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
errno = -res;
error_exit:
return NULL;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static void source_event_func(struct spa_source *source)
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
loop: initialize some variables Just in case the read fails.
2021-03-27 19:23:34 +01:00
uint64_t count = 0;
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: improve error handling from fds When we try to read one of the events and there was an error, don't signal the callback. If the error is something else than EAGAIN log a warning. Especially for timerfd, EAGAIN can happen when the timer changed while polling. This can happen when running the profiler because it polls and updates the timer from different threads.
2022-12-01 19:52:23 +01:00
if ((res = spa_system_eventfd_read(s->impl->system, source->fd, &count)) < 0) {
if (res != -EAGAIN)
spa_log_warn(s->impl->log, "%p: failed to read event fd:%d: %s",
source, source->fd, spa_strerror(res));
return;
}
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
s->func.event(source->data, count);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static struct spa_source *loop_add_event(void *object,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa_source_event_func_t func, void *data)
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl *source;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
source = get_source(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (source == NULL)
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit;
if ((res = spa_system_eventfd_create(impl->system, SPA_FD_CLOEXEC | SPA_FD_NONBLOCK)) < 0)
goto error_exit_free;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.func = source_event_func;
source->source.data = data;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
source->source.fd = res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.mask = SPA_IO_IN;
source->close = true;
source->func.event = func;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
if ((res = loop_add_source(impl, &source->source)) < 0)
goto error_exit_close;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return &source->source;
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit_close:
spa_system_close(impl->system, source->source.fd);
error_exit_free:
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
free(source);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
errno = -res;
error_exit:
return NULL;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
static int loop_signal_event(void *object, struct spa_source *source)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(s->impl == object);
spa: support: loop: assert source type
2022-02-19 19:17:42 +01:00
spa_assert(source->func == source_event_func);
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if (SPA_UNLIKELY((res = spa_system_eventfd_write(s->impl->system, source->fd, 1)) < 0))
loop: don't assert in cleanup Just issue a warning instead of asserting. Firefox does strange things to the fds that make things crash otherwise.
2022-11-08 15:45:55 +01:00
spa_log_warn(s->impl->log, "%p: failed to write event fd:%d: %s",
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
source, source->fd, spa_strerror(res));
improve error handling Make sure we free properties we take ownership of. Add some more return values to functions that can fail.
2019-06-19 18:15:04 +02:00
return res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static void source_timer_func(struct spa_source *source)
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
loop: initialize some variables Just in case the read fails.
2021-03-27 19:23:34 +01:00
uint64_t expirations = 0;
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if (SPA_UNLIKELY((res = spa_system_timerfd_read(s->impl->system,
loop: improve error handling from fds When we try to read one of the events and there was an error, don't signal the callback. If the error is something else than EAGAIN log a warning. Especially for timerfd, EAGAIN can happen when the timer changed while polling. This can happen when running the profiler because it polls and updates the timer from different threads.
2022-12-01 19:52:23 +01:00
source->fd, &expirations)) < 0)) {
if (res != -EAGAIN)
spa_log_warn(s->impl->log, "%p: failed to read timer fd:%d: %s",
source, source->fd, spa_strerror(res));
return;
}
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
s->func.timer(source->data, expirations);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static struct spa_source *loop_add_timer(void *object,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa_source_timer_func_t func, void *data)
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl *source;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
source = get_source(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (source == NULL)
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit;
if ((res = spa_system_timerfd_create(impl->system, CLOCK_MONOTONIC,
SPA_FD_CLOEXEC | SPA_FD_NONBLOCK)) < 0)
goto error_exit_free;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.func = source_timer_func;
source->source.data = data;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
source->source.fd = res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.mask = SPA_IO_IN;
source->close = true;
source->func.timer = func;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
if ((res = loop_add_source(impl, &source->source)) < 0)
goto error_exit_close;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return &source->source;
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit_close:
spa_system_close(impl->system, source->source.fd);
error_exit_free:
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
free(source);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
errno = -res;
error_exit:
return NULL;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static int
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
loop_update_timer(void *object, struct spa_source *source,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct timespec *value, struct timespec *interval, bool absolute)
{
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct itimerspec its;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int flags = 0, res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(s->impl == object);
spa: support: loop: assert source type
2022-02-19 19:17:42 +01:00
spa_assert(source->func == source_timer_func);
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa_zero(its);
sprinkly SPA_LIKELY/UNLIKELY around
2020-03-16 12:52:28 +01:00
if (SPA_LIKELY(value)) {
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
its.it_value = *value;
} else if (interval) {
loop: fix update_timer handling of solo repeat argument I believe the intent here is that if a `interval` is provided but `value` is unset, then `value` should default to `period` so the timer first fires after one `interval`. Since `interval` is always a relative duration, `value` should be interpreted as a relative duration, not an absolute one.
2024-06-28 16:49:59 -07:00
// timer initially fires after one interval
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
its.it_value = *interval;
loop: fix update_timer handling of solo repeat argument I believe the intent here is that if a `interval` is provided but `value` is unset, then `value` should default to `period` so the timer first fires after one `interval`. Since `interval` is always a relative duration, `value` should be interpreted as a relative duration, not an absolute one.
2024-06-28 16:49:59 -07:00
absolute = false;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
sprinkly SPA_LIKELY/UNLIKELY around
2020-03-16 12:52:28 +01:00
if (SPA_UNLIKELY(interval))
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
its.it_interval = *interval;
sprinkly SPA_LIKELY/UNLIKELY around
2020-03-16 12:52:28 +01:00
if (SPA_LIKELY(absolute))
support: abstract some system functions Make a new API to hide some the implementation of eventfd, timerfd and signalfd along with clock and read/write/ioctl/close functions. We would like to have plugins use the abstractions so that we can switch them to something else when needed.
2019-06-04 17:07:34 +02:00
flags |= SPA_FD_TIMER_ABSTIME;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
if (SPA_UNLIKELY((res = spa_system_timerfd_settime(s->impl->system, source->fd, flags, &its, NULL)) < 0))
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
return 0;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static void source_signal_func(struct spa_source *source)
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
loop: initialize some variables Just in case the read fails.
2021-03-27 19:23:34 +01:00
int res, signal_number = 0;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: improve error handling from fds When we try to read one of the events and there was an error, don't signal the callback. If the error is something else than EAGAIN log a warning. Especially for timerfd, EAGAIN can happen when the timer changed while polling. This can happen when running the profiler because it polls and updates the timer from different threads.
2022-12-01 19:52:23 +01:00
if ((res = spa_system_signalfd_read(s->impl->system, source->fd, &signal_number)) < 0) {
if (res != -EAGAIN)
spa_log_warn(s->impl->log, "%p: failed to read signal fd:%d: %s",
source, source->fd, spa_strerror(res));
return;
}
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
s->func.signal(source->data, signal_number);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static struct spa_source *loop_add_signal(void *object,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
int signal_number,
spa_source_signal_func_t func, void *data)
{
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
struct impl *impl = object;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
struct source_impl *source;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
source = get_source(impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
if (source == NULL)
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit;
if ((res = spa_system_signalfd_create(impl->system,
signal_number, SPA_FD_CLOEXEC | SPA_FD_NONBLOCK)) < 0)
goto error_exit_free;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.func = source_signal_func;
source->source.data = data;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
source->source.fd = res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
source->source.mask = SPA_IO_IN;
source->close = true;
source->func.signal = func;
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
if ((res = loop_add_source(impl, &source->source)) < 0)
goto error_exit_close;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
return &source->source;
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit_close:
spa_system_close(impl->system, source->source.fd);
error_exit_free:
improve error handling Set errno for functions returning NULL if relevant. Propagate errno and result codes better. Handle more error cases.
2019-06-07 10:11:23 +02:00
free(source);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
errno = -res;
error_exit:
return NULL;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static void loop_destroy_source(void *object, struct spa_source *source)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
struct source_impl *s = SPA_CONTAINER_OF(source, struct source_impl, source);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: add some invariant assertions
2022-02-19 14:12:44 +01:00
spa_assert(s->impl == object);
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
spa_log_trace(s->impl->log, "%p ", s);
loop: handle file fd with eventfd epoll return EPERM for file fds like stdin/stdout because they are always ready for IO. use an idle source to handle these cases.
2020-04-14 18:05:45 +02:00
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if (s->fallback)
loop_destroy_source(s->impl, s->fallback);
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
else
remove_from_poll(s->impl, source);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: support: loop: rename variables It may be a little confusing that both the loop object and the `source_impl` objects are referred to with variables named `impl`. For this reason, rename all source_impl objects named `impl` to `s`.
2022-02-21 16:24:34 +01:00
if (source->fd != -1 && s->close) {
spa_system_close(s->impl->system, source->fd);
Move listeners to spa and rename to hooks The listeners are generally useful in spa, move it there and rename to hooks. Implement loop hooks with it. Fix some cleanup issues in stream and remote
2017-08-08 16:56:29 +02:00
source->fd = -1;
}
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
spa_list_remove(&s->link);
if (!s->impl->polling) {
detach_source(source);
spa_list_insert(&s->impl->free_list, &s->link);
} else
spa: support: loop: fix use-after-free when loop is reentered The core of the issue is the following: what happens if an active source is destroyed before it could be dispatched? For loop-managed sources (`struct source_impl`) this was addressed by storing all destroyed sources in a list, and only freeing them after dispatching has been finished. (0eb73f0f06a2e40362990080007db295141f6f1f) This approach works for both strictly single-threaded and `pw_thread_loop` loops assuming the loop is not reentered. However, if the loop is reentered, there can still be issues. Assume that in one iteration sources A and B are active, and returned from the system call, and source B is destroyed before the loop starts dispatching. Consider what happens when "A" is dispatched first, and it reenters the loop with timeout 0. Imagine there are no new events, so `loop_iterate()` will immediately return, but it will first destroy everything in the destroy list (this is done at the end of `loop_iterate()`). And herein lies the problem. In the previous iteration, there exists a `spa_poll_event` object which points to source "B", but that has just been destroyed at the end of the recursive iteration. This will trigger a use-after-free once the previous iteration inspects it. Fix that by processing the destroy list right after first processing the returned `spa_poll_event` objects, and "detach" the source from the loop and its iterations in `process_destroy()` before the source is destroyed. See #2114 #2147
2022-02-19 13:31:01 +01:00
spa_list_insert(&s->impl->destroy_list, &s->link);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static const struct spa_loop_methods impl_loop = {
SPA_VERSION_LOOP_METHODS,
.add_source = loop_add_source,
.update_source = loop_update_source,
.remove_source = loop_remove_source,
.invoke = loop_invoke,
loop: add method to run a function with the lock Convert some _invoke to _locked
2025-03-10 17:24:13 +01:00
.locked = loop_locked,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
static const struct spa_loop_control_methods impl_loop_control_cancel = {
SPA_VERSION_LOOP_CONTROL_METHODS,
.get_fd = loop_get_fd,
.add_hook = loop_add_hook,
.enter = loop_enter,
.leave = loop_leave,
.iterate = loop_iterate_cancel,
.check = loop_check,
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
.lock = loop_lock,
.unlock = loop_unlock,
.get_time = loop_get_time,
.wait = loop_wait,
.signal = loop_signal,
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
.accept = loop_accept,
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
};
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static const struct spa_loop_control_methods impl_loop_control = {
SPA_VERSION_LOOP_CONTROL_METHODS,
.get_fd = loop_get_fd,
.add_hook = loop_add_hook,
.enter = loop_enter,
.leave = loop_leave,
.iterate = loop_iterate,
support: add support for checking loop context Add check for running the the loop context and thread. Add checks in filter and stream to avoid doing things when not run from the context main-loop because this can crash things when doing IPC from concurrent threads.
2023-03-31 18:40:57 +02:00
.check = loop_check,
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
.lock = loop_lock,
.unlock = loop_unlock,
.get_time = loop_get_time,
.wait = loop_wait,
.signal = loop_signal,
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
.accept = loop_accept,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
static const struct spa_loop_utils_methods impl_loop_utils = {
SPA_VERSION_LOOP_UTILS_METHODS,
.add_io = loop_add_io,
.update_io = loop_update_io,
.add_idle = loop_add_idle,
.enable_idle = loop_enable_idle,
.add_event = loop_add_event,
.signal_event = loop_signal_event,
.add_timer = loop_add_timer,
.update_timer = loop_update_timer,
.add_signal = loop_add_signal,
.destroy_source = loop_destroy_source,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
Make interface types a string This is more in line with wayland and it allows us to create new interfaces in modules without having to add anything to the type enum. It also removes some lookups to map type_id to readable name in debug.
2019-12-19 13:15:10 +01:00
static int impl_get_interface(struct spa_handle *handle, const char *type, void **interface)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
struct impl *impl;
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
spa_return_val_if_fail(handle != NULL, -EINVAL);
spa_return_val_if_fail(interface != NULL, -EINVAL);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
impl = (struct impl *) handle;
treewide: replace strcmp() == 0 with spa_streq() This change is only done in source files for now, header files will be done separately.
2021-05-18 11:36:13 +10:00
if (spa_streq(type, SPA_TYPE_INTERFACE_Loop))
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
*interface = &impl->loop;
treewide: replace strcmp() == 0 with spa_streq() This change is only done in source files for now, header files will be done separately.
2021-05-18 11:36:13 +10:00
else if (spa_streq(type, SPA_TYPE_INTERFACE_LoopControl))
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
*interface = &impl->control;
treewide: replace strcmp() == 0 with spa_streq() This change is only done in source files for now, header files will be done separately.
2021-05-18 11:36:13 +10:00
else if (spa_streq(type, SPA_TYPE_INTERFACE_LoopUtils))
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
*interface = &impl->utils;
Make interface types a string This is more in line with wayland and it allows us to create new interfaces in modules without having to add anything to the type enum. It also removes some lookups to map type_id to readable name in debug.
2019-12-19 13:15:10 +01:00
else
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
return -ENOENT;
Make interface types a string This is more in line with wayland and it allows us to create new interfaces in modules without having to add anything to the type enum. It also removes some lookups to map type_id to readable name in debug.
2019-12-19 13:15:10 +01:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
return 0;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static int impl_clear(struct spa_handle *handle)
{
struct impl *impl;
use _list_consume to destroy objects
2018-09-25 13:05:13 +02:00
struct source_impl *source;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
uint32_t i;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
spa_return_val_if_fail(handle != NULL, -EINVAL);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
impl = (struct impl *) handle;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
spa_log_debug(impl->log, "%p: clear", impl);
loop: don't assert in cleanup Just issue a warning instead of asserting. Firefox does strange things to the fds that make things crash otherwise.
2022-11-08 15:45:55 +01:00
if (impl->enter_count != 0 || impl->polling)
spa_log_warn(impl->log, "%p: loop is entered %d times polling:%d",
impl, impl->enter_count, impl->polling);
spa: support: loop: assert loop is not polling when destroyed Assert that the loop is not polling when `impl_clear()` is called.
2022-03-05 22:00:18 +01:00
use _list_consume to destroy objects
2018-09-25 13:05:13 +02:00
spa_list_consume(source, &impl->source_list, link)
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
loop_destroy_source(impl, &source->source);
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
spa_list_consume(source, &impl->free_list, link) {
spa_list_remove(&source->link);
free(source);
}
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
for (i = 0; i < impl->n_queues; i++)
loop_queue_destroy(impl->queues[i]);
loop: free tss from the thread calling impl::clear The thread that calls the impl_clear method might be the main thread and is certainly not going to call the invoke function anymore so free the tss if there is any. Fixes a leak in the unit test.
2024-10-02 09:18:14 +02:00
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
spa_system_close(impl->system, impl->poll_fd);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_cond_destroy(&impl->cond);
pthread_mutex_destroy(&impl->lock);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
return 0;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
handle: make method to get size
2018-04-09 10:06:17 +02:00
static size_t
impl_get_size(const struct spa_handle_factory *factory,
const struct spa_dict *params)
{
return sizeof(struct impl);
}
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
#define CHECK(expression,label) \
do { \
if ((errno = (expression)) != 0) { \
res = -errno; \
spa_log_error(impl->log, #expression ": %s", strerror(errno)); \
goto label; \
} \
} while(false);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
static int
impl_init(const struct spa_handle_factory *factory,
struct spa_handle *handle,
const struct spa_dict *info,
const struct spa_support *support,
uint32_t n_support)
{
struct impl *impl;
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
const char *str;
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
pthread_mutexattr_t attr;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_condattr_t cattr;
improve error handling
2019-06-18 10:47:12 -04:00
int res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
spa_return_val_if_fail(factory != NULL, -EINVAL);
spa_return_val_if_fail(handle != NULL, -EINVAL);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
handle->get_interface = impl_get_interface;
handle->clear = impl_clear;
impl = (struct impl *) handle;
interface: add an interface struct The interface struct has the type,version and methods of the interface. Make spa interfaces extend from spa_interface and make a separate structure for the methods. Pass a generic void* as the first argument of methods, like we don in PipeWire. Bundle the methods + implementation in a versioned inteface and use that to invoke methods. This way we can do version checks on the methods. Make resource and proxy interfaces that we can can call. We can then make the core interfaces independent on proxy/resource and hide them in the lower layers. Add add_listener method to methods of core interfaces, just like SPA.
2019-05-20 16:11:23 +02:00
impl->loop.iface = SPA_INTERFACE_INIT(
SPA_TYPE_INTERFACE_Loop,
SPA_VERSION_LOOP,
&impl_loop, impl);
impl->control.iface = SPA_INTERFACE_INIT(
SPA_TYPE_INTERFACE_LoopControl,
SPA_VERSION_LOOP_CONTROL,
&impl_loop_control, impl);
impl->utils.iface = SPA_INTERFACE_INIT(
SPA_TYPE_INTERFACE_LoopUtils,
SPA_VERSION_LOOP_UTILS,
&impl_loop_utils, impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: handle queue overflow better When a queue overflows we place the queue back in the stack and try again. Because it's at the top of the stack we take exactly the same queue and keep on looping forever if the other thread is blocked for some reason. Instead, mark the queue as overflowed and only place it back in the stack when we have flushed it. This avoids a deadlock when the main-thread invokes on the data loop and blocks and when the data loop invokes on the main-thread and overflows the queue.
2024-11-14 15:58:09 +01:00
impl->rate_limit.interval = 2 * SPA_NSEC_PER_SEC;
impl->rate_limit.burst = 1;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
impl->retry_timeout = DEFAULT_RETRY;
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
if (info) {
if ((str = spa_dict_lookup(info, "loop.cancel")) != NULL &&
spa_atob(str))
impl->control.iface.cb.funcs = &impl_loop_control_cancel;
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
if ((str = spa_dict_lookup(info, "loop.retry-timeout")) != NULL)
impl->retry_timeout = atoi(str);
spa: some more invoke -> locked calls
2025-03-10 17:46:28 +01:00
if ((str = spa_dict_lookup(info, "loop.prio-inherit")) != NULL)
impl->prio_inherit = spa_atob(str);
loop: add optimized non-cancellable iterate Only use the more heavy cancellable loop when the loop.cancel property was set. Makes pipewire go from 5% to 3% in high frequency wakeups.
2023-05-05 17:41:37 +02:00
}
spa: some more invoke -> locked calls
2025-03-10 17:46:28 +01:00
CHECK(pthread_mutexattr_init(&attr), error_exit);
CHECK(pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE), error_exit_free_attr);
if (impl->prio_inherit)
CHECK(pthread_mutexattr_setprotocol(&attr, PTHREAD_PRIO_INHERIT),
error_exit_free_attr)
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
CHECK(pthread_mutex_init(&impl->lock, &attr), error_exit_free_attr);
pthread_mutexattr_destroy(&attr);
CHECK(pthread_condattr_init(&cattr), error_exit_free_mutex);
CHECK(pthread_condattr_setclock(&cattr, CLOCK_REALTIME), error_exit_free_mutex);
CHECK(pthread_cond_init(&impl->cond, &cattr), error_exit_free_mutex);
loop: move thread-loop to support loop Add more synchronization primitives to spa loop so that we can replace the thread-loop with it.
2025-03-10 13:31:41 +01:00
CHECK(pthread_cond_init(&impl->accept_cond, &cattr), error_exit_free_mutex);
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_condattr_destroy(&cattr);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
Make interface types a string This is more in line with wayland and it allows us to create new interfaces in modules without having to add anything to the type enum. It also removes some lookups to map type_id to readable name in debug.
2019-12-19 13:15:10 +01:00
impl->log = spa_support_find(support, n_support, SPA_TYPE_INTERFACE_Log);
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_topic_init(impl->log, &log_topic);
Make interface types a string This is more in line with wayland and it allows us to create new interfaces in modules without having to add anything to the type enum. It also removes some lookups to map type_id to readable name in debug.
2019-12-19 13:15:10 +01:00
impl->system = spa_support_find(support, n_support, SPA_TYPE_INTERFACE_System);
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
if (impl->system == NULL) {
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_error(impl->log, "%p: a System is needed", impl);
improve error handling
2019-06-18 10:47:12 -04:00
res = -EINVAL;
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
goto error_exit_free_cond;
System: More work on making system functions pluggable Move the epoll functions to the system functions and make the loop use those. Use simple mask for events instead of enum. Add the used system api in pw_loop. Add System API to spa_support and use it where possible. Pass the system API used in the realtime loops in spa_support as well and use this in the realtime paths. Improve bootstrapping, load only the log and cpu interfaces because those can/need to be shared between instances. Let the core load the other interfaces. Add keys to configure the System and Loop implementations used in pw_loop.
2019-06-06 15:21:40 +02:00
}
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
if ((res = spa_system_pollfd_create(impl->system, SPA_FD_CLOEXEC)) < 0) {
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_error(impl->log, "%p: can't create pollfd: %s",
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
impl, spa_strerror(res));
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
goto error_exit_free_cond;
improve error handling
2019-06-18 10:47:12 -04:00
}
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
impl->poll_fd = res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
spa_list_init(&impl->source_list);
Revert "loop: remove destroy list" This reverts commit c474846c42967c44db069a23b76a29da6f496f33. In addition, `s->loop` is also checked before dispatching a source. The destroy list is needed in the presence of threads. The issue is that a source may be destroyed between `epoll_wait()` returning and thread loop lock being acquired. If this source is active, then a use-after-free will be triggered when the thread loop acquires the lock and starts dispatching the sources. thread 1 thread 2 ---------- ---------- loop_iterate spa_loop_control_hook_before // release lock pw_thread_loop_lock spa_system_pollfd_wait // assume it returns with source A pw_loop_destroy_source(..., A) // frees storage of A pw_thread_loop_unlock spa_loop_control_hook_after // acquire the lock for (...) { struct spa_source *s = ep[i].data; s->rmask = ep[i].events; // use-after-free if `s` refers to // the previously freed `A` Fixes #2147
2022-02-18 18:36:36 +01:00
spa_list_init(&impl->destroy_list);
loop: keep a free_list of sources When a source is destroyed, move it to a free_list and reuse the memory when a new source is made. This way we can avoid doing a free() from the epoll thread.
2025-03-13 09:35:05 +01:00
spa_list_init(&impl->free_list);
Move listeners to spa and rename to hooks The listeners are generally useful in spa, move it there and rename to hooks. Implement loop hooks with it. Fix some cleanup issues in stream and remote
2017-08-08 16:56:29 +02:00
spa_hook_list_init(&impl->hooks_list);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
loop: cleanups
2019-05-23 15:11:49 +02:00
impl->wakeup = loop_add_event(impl, wakeup_func, impl);
improve error handling
2019-06-18 10:47:12 -04:00
if (impl->wakeup == NULL) {
res = -errno;
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_error(impl->log, "%p: can't create wakeup event: %m", impl);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
goto error_exit_free_poll;
improve error handling
2019-06-18 10:47:12 -04:00
}
loop: move invoke queue to separate object Make an internal queue object that implements the invoke queue. Because we can not do invokes concurrently from different threads, this is required to make per-thread invoke queues later.
2024-04-29 12:01:48 +02:00
support: make the loop queue handling lockfree Don't use TSS to store per-thread queues but keep a lockfree stack of queues. We can then pick off a queue and write to that one and place it back after use. We need to keep the queues indexed by id in the stack because otherwise we would need to compare-and-swap 128 bits (pointer + tag), which is more problematic. Because we keep the queues in an array and no queue is ever removed and the array can only grow, we can quite easily just iterate the array without a lock. Without the lock we also fix one of the potential problems with ardour where the queue_flush thread is canceled while flushing and the queue_mutex remains locked. Because we end up with all queues in the array now, we can overflow the fixed max amount of queues we can manage. When that happens, sleep for a while and try again. This is a case where more than QUEUES_MAX (128) threads are invoking at the same time and is rather unlikely. There is also the queue overflow case which we now also must handle with a retry. This potentially uses more eventfds but again this should be unlikely and cause no further problems. See #4356
2024-11-04 17:26:52 +01:00
impl->head.t.idx = IDX_INVALID;
loop: create a per-thread queue Keep a thread local queue. This makes it possible for multiple threads to write to the ringbuffer. There is a lock to protect the list of queues. It can only be contended when new queues are created in the threads but this can be done at thread startup. Fixes #3983
2024-04-29 15:17:45 +02:00
spa: sprinkle more log topics into spa
2021-09-27 15:31:35 +10:00
spa_log_debug(impl->log, "%p: initialized", impl);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
return 0;
improve error handling
2019-06-18 10:47:12 -04:00
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit_free_poll:
improve error handling
2019-06-18 10:47:12 -04:00
spa_system_close(impl->system, impl->poll_fd);
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
error_exit_free_cond:
pthread_cond_destroy(&impl->cond);
spa: serialize in_thread flushes with a mutex When we have no thread running the loop, we need to flush the queues from the invoking thread. Make sure that when multiple threads attempt this that we serialize the flushing because the flushing code is not thread safe.
2024-12-03 16:38:28 +01:00
error_exit_free_mutex:
spa: add locking to the loop We can add a PTHREAD_PRIO_INHERIT lock to the loop to protect the callbacks and then use this to update shared data in an RT-safe way. This can avoid some invoke calls that require a context switch but also due to the nature of epoll cause locking in the kernel with non-RT guarantees. Because we use PRIO_INHERIT, the code executed in the lock must not use any RT-unsafe functions.
2025-03-10 09:13:52 +01:00
pthread_mutex_destroy(&impl->lock);
error_exit_free_attr:
pthread_mutexattr_destroy(&attr);
system: make system functions return error on error Return -errno from system functions instead of -1 in errors. This makes it easier to pass along the result without having to go to errno etc..
2019-06-20 17:31:29 +02:00
error_exit:
improve error handling
2019-06-18 10:47:12 -04:00
return res;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
static const struct spa_interface_info impl_interfaces[] = {
More type cleanups SPA_ID_* -> SPA_TYPE_*. We use 'type' for all registered types. ID refers to either a registered type or an enum. Improve introspection of object ids. Make 0 an invalid enum and use it to look up the object id type. Move some type-info files together. Swap type and id of the object, we first specify the type and then the id because the meaning of the id depends on the object type.
2018-08-27 15:03:11 +02:00
{SPA_TYPE_INTERFACE_Loop,},
{SPA_TYPE_INTERFACE_LoopControl,},
{SPA_TYPE_INTERFACE_LoopUtils,},
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
static int
impl_enum_interface_info(const struct spa_handle_factory *factory,
const struct spa_interface_info **info,
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
uint32_t *index)
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
{
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
spa_return_val_if_fail(factory != NULL, -EINVAL);
spa_return_val_if_fail(info != NULL, -EINVAL);
spa_return_val_if_fail(index != NULL, -EINVAL);
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
if (*index >= SPA_N_ELEMENTS(impl_interfaces))
return 0;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
Use errno for result errors Make new enumeration for data transport status and use errno style error numbers for errors.
2017-11-13 09:41:41 +01:00
*info = &impl_interfaces[(*index)++];
return 1;
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
}
spa: don't use constructor attribute to register factories
2019-02-06 11:38:12 +01:00
const struct spa_handle_factory spa_support_loop_factory = {
Add versions to structures Use versions instead of size Remove user_data from callbacks, we can simply pass the callback struct.
2017-06-14 18:32:39 +02:00
SPA_VERSION_HANDLE_FACTORY,
names: add standard factory name definitions Define a set of standard factory names and document what they contain. This makes it possible to change the implementation by mapping the factory-name to a different shared library.
2019-06-21 13:31:34 +02:00
SPA_NAME_SUPPORT_LOOP,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
NULL,
handle: make method to get size
2018-04-09 10:06:17 +02:00
impl_get_size,
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
impl_init,
Add versions to structures Use versions instead of size Remove user_data from callbacks, we can simply pass the callback struct.
2017-06-14 18:32:39 +02:00
impl_enum_interface_info
loop: move to plugin Move the loop implementation to a plugin. Organize the hooks in a list so that we can add many.
2017-06-14 16:10:07 +02:00
};
Reference in a new issue Copy permalink