mirrors/pipewire
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/pipewire/pipewire.git synced 2025-10-29 05:40:27 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Documentation Updates

Browse source
This commit is contained in:
jasker5183 2022-05-08 17:06:28 +00:00 • committed by Wim Taymans
parent 8afe5fe0f0
commit c71db353f1
27 changed files with 513 additions and 498 deletions
Download patch file Download diff file Expand all files Collapse all files

192
doc/pipewire-objects-design.dox
View file

@ -1,11 +1,12 @@
/** \page page_objects_design Objects Design
This document is a design reference on the various objects that exist
in the PipeWire media and session management graphs, explaining what these
objects are, how they are meant to be used and how they relate to other
in the PipeWire media and session management graphs. Explaining what these
objects are, how they are meant to be used, and how they relate to other
kinds of objects and concepts that exist in subsystems or other libraries.
## The media graph
# The Media Graph
The media graph represents and enables the media flow inside the PipeWire
daemon and between the daemon and its clients. It consists of nodes, ports
@ -21,7 +22,7 @@ and links.
+------------+ +------------+
```
### Node
## Node
A **node** is a media processing element. It consumes and/or produces buffers
that contain data, such as audio or video.
@ -33,26 +34,28 @@ client using the PipeWire protocol.
In an analogy to GStreamer, a _node_ is similar (but not equal) to a
GStreamer _element_.
### Port
## Port
A **port** is attached on a **node** and provides an interface for input
or output of media on the node. A node may have multiple ports.
A port always has a direction, input or output:
- Input: it allows media input into the node (in other terms, it is a _sink_)
- Output: it outputs media out of the node (in other terms, it is a _source_)
In an analogy to GStreamer, a _port_ is similar (but not equal) to a
GStreamer _pad_.
### Link
## Link
A **link** connects 2 ports of opposite direction, making media flow from
A **link** connects two ports of opposite direction, making media flow from
the output port to the input port.
## The session management graph
The session management graph is a virtual, higher-level representation of the
# The Session Management Graph
The session management graph is a virtual, higher level representation of the
media flow. It is created entirely by the session manager and it can affect
the routing on the media graph only through the session manager's actions.
@ -70,28 +73,31 @@ codebase.
+---------------------+ +----------------------+
```
### Endpoint
## Endpoint
An **endpoint** is a session management object that provides a representation
of user-conceivable places where media can be routed to/from.
of user conceivable places where media can be routed to/from.
Examples of endpoints associated with hardware on a desktop-like system:
- Laptop speakers
- USB webcam
- Bluetooth headset microphone
- Line out stereo jack port
- Laptop speakers.
- USB webcam.
- Bluetooth headset microphone.
- Line out stereo jack port.
Examples of endpoints associated with hardware in a car:
- Speakers amplifier
- Front right seat microphone array
- Rear left seat headphones
- Bluetooth phone voice gateway
- Hardware FM radio device
- Speakers amplifier.
- Front right seat microphone array.
- Rear left seat headphones.
- Bluetooth phone voice gateway.
- Hardware FM radio device.
Examples of endpoints associated with software:
- Desktop screen capture source
- Media player application
- Camera application
- Desktop screen capture source.
- Media player application.
- Camera application.
In most cases an endpoint maps to a node on the media graph, but this is not
always the case. An endpoint may be backed by several nodes or no nodes at all.
@ -107,7 +113,7 @@ be able to represent the *CD player endpoint* and the _endpoint link_ between
it and the amplifier, so that it can apply audio policy that takes into account
whether the CD player is playing or not.
#### Target
### Target
An **endpoint** may be grouping together targets that can be reached by
following the same route and they are mutually exclusive with each other.
@ -121,7 +127,7 @@ In this case, a session manager may choose to group these two targets into the
same endpoint, using a parameter on the _endpoint_ object to allow the user
to choose the target (if the hardware allows configuring this at all).
### Endpoint Stream
## Endpoint Stream
An **endpoint stream** is attached to an **endpoint** and represents a logical
path that can be taken to reach this endpoint, often associated with
@ -129,45 +135,48 @@ a _use case_.
For example, the "Speakers amplifier" endpoint in a car might have the
following streams:
- _Music_: a path to play music;
the implementation will output this to all speakers, using the volume
that has been configured for the "Music" use case
- _Voice_: a path to play a voice message, such as a navigation message or
feedback from a voice assistant; the implementation will output this
to the front speakers only, lowering the volume of the music (if any)
on these speakers at the same time
- _Emergency_: a path to play an emergency situation sound (a beep,
or equivalent); the implementation will output this on all speakers,
increasing the volume to a factory-defined value if necessary (to ensure
that it is audible) while muting audio from all other streams at the
same time
- _Music_: A path to play music;
the implementation will output this to all speakers, using the volume
that has been configured for the "Music" use case.
- _Voice_: A path to play a voice message; such as a navigation message or
feedback from a voice assistant, the implementation will output this
to the front speakers only. Lowering the volume of the music (if any)
on these speakers at the same time.
- _Emergency_: A path to play an emergency situation sound (a beep,
or equivalent); the implementation will output this on all speakers.
Increasing the volume to a factory defined value if necessary (to ensure
that it is audible) while muting audio from all other streams at the
same time.
In another example, a microphone that can be used for activating a voice
assistant might have the following streams:
- _Capture_: a path to capture directly from the microphone; this can be used
by an application that listens for the assistant's wake-word in order
to activate the full voice recognition engine
- _CaptureDelayed_: a path to capture with a constant delay (meaning that
starting capturing now will actually capture something that was spoken
a little earlier); this can be used by the full voice recognition engine,
allowing it to start after the wake-word has been spoken while capturing
audio that also includes the wake-word
- _Capture_: A path to capture directly from the microphone; this can be used
by an application that listens for the assistant's wake-word in order
to activate the full voice recognition engine.
- _CaptureDelayed_: A path to capture with a constant delay (meaning that
starting capturing now will actually capture something that was spoken
a little earlier); this can be used by the full voice recognition engine,
allowing it to start after the wake-word has been spoken while capturing
audio that also includes the wake-word.
Endpoint streams may be mutually exclusive or they may used simultaneously,
depending on the implementation.
Endpoint streams may be implemented in many ways:
- By plugging additional nodes in the media graph that link to the device node
(ex. a simple buffering node linked to an alsa source node could implement
the _CaptureDelayed_ stream in the above microphone example)
- By using a different device node (ex. different ALSA device on the same card)
that has a special meaning for the hardware
- By triggering switches on the hardware (ex. modify ALSA controls on the
same device)
### Endpoint Link
- By plugging additional nodes in the media graph that link to the device node
(ex. a simple buffering node linked to an alsa source node could implement
the _CaptureDelayed_ stream in the above microphone example).
- By using a different device node (ex. different ALSA device on the same card)
that has a special meaning for the hardware.
- By triggering switches on the hardware (ex. modify ALSA controls on the
same device).
An **endpoint link** connects 2 streams from 2 different endpoints, creating
## Endpoint Link
An **endpoint link** connects two streams from two different endpoints, creating
a logical representation of media flow between the endpoints.
An **endpoint link** may be implemented by creating one or more _links_ in the
@ -175,7 +184,7 @@ underlying media graph, or it may be implemented by configuring hardware
resources to enable media flow, in case the flow does not pass through the
media graph.
#### Constructing
### Constructing
Constructing an **endpoint link** is done by asking the _endpoint stream_
objects to prepare it. First, the source stream is asked to provide linking
@ -185,40 +194,43 @@ When this is done, the session manager is asked to create the link using the
provided information.
This mechanism allows stream implementations:
- to prepare for linking, adjusting hardware paths if necessary
- to check for stream linking compatibility; not all streams can be connected
to all others (ex. streams with media flow in the hardware cannot be linked
to streams that are backed by nodes in the media graph)
- to provide implementation-specific information for linking; in the standard
case this is going to be a list of _ports_ to be linked in the media graph,
but in a hardware-flow case it can be any kind of hardware-specific detail
## Other related objects
- To prepare for linking, adjusting hardware paths if necessary.
- To check for stream linking compatibility; not all streams can be connected
to all others (ex. streams with media flow in the hardware cannot be linked
to streams that are backed by nodes in the media graph).
- To provide implementation specific information for linking; in the standard
case this is going to be a list of _ports_ to be linked in the media graph,
but in a hardware-flow case it can be any kind of hardware-specific detail.
### Device
# Other Related Objects
## Device
A **device** represents a handle to an underlying API that is used to create
higher level objects, such as nodes, or other devices.
Well-known devices include:
| Device API | Description |
| :--- | :--- |
| alsa.pcm.device | A handle to an ALSA card (ex. `hw:0`, `hw:1`, etc) |
| alsa.seq.device | A handle to an ALSA Midi device |
| v4l2.device | A handle to a V4L2 device (`/dev/video0`, `/dev/video1`, etc..) |
| jack.device | A JACK client, allowing PipeWire to slave to JACK for audio input/output |
| alsa.pcm.device | A handle to an ALSA card (ex. `hw:0`, `hw:1`, etc). |
| alsa.seq.device | A handle to an ALSA Midi device. |
| v4l2.device | A handle to a V4L2 device (`/dev/video0`, `/dev/video1`, etc..). |
| jack.device | A JACK client, allowing PipeWire to slave to JACK for audio input/output. |
A device may have a _profile_, which allows the user to choose between
multiple configurations that the device may be capable of having, or to simply
turn the device _off_, which means that the handle is closed and not used
by PipeWire.
### Session
## Session
The **session** represents the session manager and can be used to expose
global properties or methods that affect the session management.
#### Default endpoints
### Default Endpoints
The session is responsible for book-keeping the default device endpoints (one
for each kind of device) that is to be used to link new clients when
@ -227,17 +239,18 @@ device preferences.
For example, a system may have both "Speakers" and "HDMI" endpoints on the
"Audio Output" category and the user may be offered to make a choice within
the UI to select which endpoint she wants to use by default for audio output.
the UI to select which endpoint they want to use by default for audio output.
This preference is meant to be stored in the session object.
#### Multiple sessions
### Multiple Sessions
It is not currently defined whether it is allowed to have multiple sessions
or not and how the system should behave if this happens.
## Mappings to underlying subsystem objects
### ALSA UCM
# Mappings To Underlying Subsystem Objects
## ALSA UCM
This is a ***proposal***
@ -252,15 +265,15 @@ This is a ***proposal***
In UCM mode, an ALSA card is represented as a PipeWire device, with the
available UCM verbs listed as profiles of the device.
Activating a profile (i.e. a verb) will create the necessary nodes for the
Activating a profile (ie. a verb) will create the necessary nodes for the
available PCM streams and at the same time it will also create one endpoint
for each UCM device. Optionally, conflicting UCM devices can be grouped in
for each UCM device. Optionally conflicting UCM devices can be grouped in
the same endpoint, listing the conflicting options as targets of the endpoint.
The available UCM modifiers for each UCM device will be added as streams, plus
one "default" stream for accessing the device with no modifiers.
### ALSA fallback
## ALSA Fallback
| ALSA | PipeWire |
| :--- | :--- |
@ -268,15 +281,15 @@ one "default" stream for accessing the device with no modifiers.
| PCM stream | node + endpoint |
In the case where UCM (or another similar mechanism) is not available,
ALSA cards are represented as PipeWire devices with only 2 profiles: On/Off
ALSA cards are represented as PipeWire devices with only two profiles on/off.
When the On profile is activated, a node and an associated endpoint are created
When the on profile is activated, a node and an associated endpoint are created
for every available PCM stream.
Endpoints in this case have only one "default" stream, unless they are extended
by the session manager to have software-backed streams.
### V4L2
## V4L2
***FIXME***
@ -284,11 +297,12 @@ by the session manager to have software-backed streams.
| :--- | :--- |
| device | device + node |
## Relationship to other APIs
### PulseAudio
# Relationship To Other API's
#### Mapping PipeWire objects for access by PulseAudio clients
## PulseAudio
### Mapping PipeWire Objects For Access By PulseAudio Clients
| PipeWire | PulseAudio |
| :--- | :--- |
@ -297,21 +311,21 @@ by the session manager to have software-backed streams.
| endpoint (associated with a device) | sink / source |
| endpoint (associated with a client) | sink-input / source-output |
| endpoint target | port |
| endpoint stream | N/A, pa clients will be limited to the default stream |
| endpoint stream | N/A, PA clients will be limited to the default stream |
#### Mapping PulseAudio clients to PipeWire
### Mapping PulseAudio Clients To PipeWire
| PulseAudio | PipeWire |
| :--- | :--- |
| stream | client + node + endpoint (no targets, 1 default stream) |
### Jack
## Jack
Note: This section is about JACK clients connecting to PipeWire through the
JACK compatibility library. The scenario where PipeWire connects to another
JACK server as a client is out of scope here.
#### Mapping PipeWire objects for access by JACK clients
### Mapping PipeWire Objects For Access By JACK Clients
| PipeWire | JACK |
| :--- | :--- |
@ -320,7 +334,7 @@ JACK server as a client is out of scope here.
| device | N/A |
| endpoint | N/A |
#### Mapping JACK clients to PipeWire
### Mapping JACK Clients To PipeWire
| JACK | PipeWire |
| :--- | :--- |
@ -328,6 +342,6 @@ JACK server as a client is out of scope here.
| port | port |
JACK clients do not create endpoints. A session manager should be JACK aware
in order to anticipate direct node linking
in order to anticipate direct node linking.
*/