PipeWire
--------

The idea is to make a server where you can provide
and consume media to/from.

Some of the requirements are:

 - must be efficient for raw video using fd passing
 - must be able to provide/consume/process media from any process
 - streaming media only (no seeking)
 - policy to restrict access to devices and streams

Although an initial goal, the design is not limited to raw video
only and should be able to handle compressed video and other
streamable media as well.

The design is in some part inspired by pulseaudio, hence its original
name. Increasinly we also seem to add functionality of jack and
GStreamer.


Protocol
--------

The protocol is similar to wayland but with custom
serialization/deserialization of messages. This is because the datastructures
in the messages are more complicated.


fd management
-------------

Clients receive fds with buffers and memory after a format was negotiated.
Updates to these buffers are notified by a message containing the id of the
buffer.

Wire
----

The wire protocol for the node control channel is a serialization of
structures.


               +-----+   +----+   +----+
               |     |   |  S |   |    |
               |     -----    -----    |
               +-----+   +----+   +----+
                           |
                           |
                           |
                           |
                         +----+
                         |    |
                         |  C |
                         +----+


    Client                                  Proxy
                                              |                         INIT
                     node-update              |
       -------------------------------------->|
                     port-update              |
       -------------------------------------->|
                 state-change CONFIGURE       |                         CONFIGURE
       -------------------------------------->|
                                              |<--- enum-ports
                                              |<--- enum-formats
                                              |<--- add-port
                                              |<--- remove-port
                       set-property           |<--- set-property
       <--------------------------------------|
                       set-format             |<--- set-format
       <--------------------------------------|
                                              |
                       port-update            |
       -------------------------------------->|
                   state-change READY         |                         READY
       -------------------------------------->|
                                              |<--- port memory requirements
                       add-mem                |<--- use-buffers
       <--------------------------------------|
                       remove-mem             |
       <--------------------------------------|
                       add-buffer             |
       <--------------------------------------|
                    remove-buffer             |
       <--------------------------------------|
                                              |
                       pause                  |<--- stop
       <--------------------------------------|
                   state-change PAUSED        |                         PAUSED
       -------------------------------------->|
                                              |
                       play                   |<--- start
       <--------------------------------------|
                   state-change STREAMING     |                         STREAMING
       -------------------------------------->|
                                              |
                   need-input                 |
       <------------------------------------->|
                   have-output                |
       <------------------------------------->|
                   process-buffer             |
       <------------------------------------->|
                   reuse-buffer               |
       <------------------------------------->|




1) Update config C->S                 INIT

   node-update
   port-update
   state change CONFIGURE

2) Set formats S->C                   CONFIGURE

   set-property
   enumerate ports
   add-port
   remove-port
   enumerate formats
   set-format

3) Buffer requirements update C->S

   Update port status
   state change READY if enough formats are set

4) Start S->C                         READY

   read port memory requirements
   add_mem
   remove_mem
   add_buffer
   remove_buffer
   command START/PAUSE

5) Pause S->C                         PAUSED

   state change STREAMING
   set-format to NULL -> state change to CONFIGURE

5) data transfer C->S                 STREAMING

   need-input
   have-output

   process_buffer
   reuse_buffer
   state change PAUSED

6) data transfer S->C

   process_buffer
   reuse_buffer

7) format change C->S

   port-update
   state change CONFIGURE

8) format change S->C

   Send set-format change on ports -> READY if new memory requirements
                                   -> PAUSED/STREAMING if all ok

9) format-change to NULL

   state change CONFIGURE

10) ERROR