mirrors/pipewire
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/pipewire/pipewire.git synced 2025-10-29 05:40:27 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update docs a little

Browse source
This commit is contained in:
Wim Taymans 2017-06-21 13:17:43 +02:00
parent c1de5ba624
commit 68413d472f
1 changed files with 43 additions and 146 deletions
Download patch file Download diff file Expand all files Collapse all files

189
doc/design.txt
View file

@ -1,174 +1,71 @@
PipeWire PipeWire
-------- --------
The idea is to make a server where you can provide PipeWire is a media server that can run graphs of multimedia nodes.
and consume media to/from. Nodes can run inside the server or in separate processes.
Some of the requirements are: Some of the requirements are:
- must be efficient for raw video using fd passing - must be efficient for raw video using fd passing and audio with
shared ringbuffers
- must be able to provide/consume/process media from any process - must be able to provide/consume/process media from any process
- streaming media only (no seeking)
- policy to restrict access to devices and streams - policy to restrict access to devices and streams
- extensible
Although an initial goal, the design is not limited to raw video Although an initial goal, the design is not limited to raw video
only and should be able to handle compressed video and other only and should be able to handle compressed video and other
streamable media as well. media as well.
The design is in some part inspired by pulseaudio, hence its original PipeWire uses the SPA plugin API for the nodes in the graph. SPA is
name. Increasingly we also seem to add functionality of jack and a plugin API designed for low-latency and efficient processing of
GStreamer. any multimedia format.
Some of the application we intend to build
- v4l2 device provider. Provide controlled access to v4l2 devices
and share 1 device between multiple processes.
- gnome-shell video provider. Gnome-shell provides a node that
gives the contents of the frame buffer for screen sharing or
screen recording.
- audio server. Mix and playback multiple audio streams. The design
is more like CRAS (Chromium audio server) than pulseaudio and with
the added benefit that processing can be arranged in a graph.
- Pro audio graph processing like JACK.
- Media playback backend
Protocol Protocol
-------- --------
The protocol is similar to wayland but with custom The native protocol and object model is similar to wayland but with custom
serialization/deserialization of messages. This is because the datastructures serialization/deserialization of messages. This is because the datastructures
in the messages are more complicated. in the messages are more complicated and not easily expressable in xml format.
fd management Extensibility
------------- -------------
Clients receive fds with buffers and memory after a format was negotiated. The functionality of the server is implemented and extended with modules and
Updates to these buffers are notified by a message containing the id of the extensions. Modules are server side bits of logic that hook into various
buffer. places to provide extra features. This mostly means controlling the processing
graph in some way.
Wire Extensions are the client side version of the modules. Most extensions provide
---- both a client side and server side init function. New interfaces or new object
implementation can easily be added with modules/extensions.
The wire protocol for the node control channel is a serialization of Some of the extensions that can be written
structures.
- protocol extensions: a client/server side API (.h) together with protocol
extensions and server/client side logic to implement a new object or
interface.
+-----+ +----+ +----+ - a module to check security of method calls
| | | S | | |
| ----- ----- |
+-----+ +----+ +----+
|
|
|
|
+----+
| |
| C |
+----+
- a module to automatically create or link or relink nodes
Client Proxy - a module to suspend idle nodes
| INIT
node-update |
-------------------------------------->|
port-update |
-------------------------------------->|
state-change CONFIGURE | CONFIGURE
-------------------------------------->|
|<--- enum-ports
|<--- enum-formats
|<--- add-port
|<--- remove-port
set-property |<--- set-property
<--------------------------------------|
set-format |<--- set-format
<--------------------------------------|
|
port-update |
-------------------------------------->|
state-change READY | READY
-------------------------------------->|
|<--- port memory requirements
add-mem |<--- use-buffers
<--------------------------------------|
remove-mem |
<--------------------------------------|
add-buffer |
<--------------------------------------|
remove-buffer |
<--------------------------------------|
|
pause |<--- stop
<--------------------------------------|
state-change PAUSED | PAUSED
-------------------------------------->|
|
play |<--- start
<--------------------------------------|
state-change STREAMING | STREAMING
-------------------------------------->|
|
need-input |
<------------------------------------->|
have-output |
<------------------------------------->|
process-buffer |
<------------------------------------->|
reuse-buffer |
<------------------------------------->|
1) Update config C->S INIT
node-update
port-update
state change CONFIGURE
2) Set formats S->C CONFIGURE
set-property
enumerate ports
add-port
remove-port
enumerate formats
set-format
3) Buffer requirements update C->S
Update port status
state change READY if enough formats are set
4) Start S->C READY
read port memory requirements
add_mem
remove_mem
add_buffer
remove_buffer
command START/PAUSE
5) Pause S->C PAUSED
state change STREAMING
set-format to NULL -> state change to CONFIGURE
5) data transfer C->S STREAMING
need-input
have-output
process_buffer
reuse_buffer
state change PAUSED
6) data transfer S->C
process_buffer
reuse_buffer
7) format change C->S
port-update
state change CONFIGURE
8) format change S->C
Send set-format change on ports -> READY if new memory requirements
-> PAUSED/STREAMING if all ok
9) format-change to NULL
state change CONFIGURE
10) ERROR