#Designing a realtime audio/video conferencing service

## Features

Functionality to consider when considering tool/platform.

  * topology
    * mesh (P2P)
      all processing at end-points, participants limited by bandwidth
    * routing (SFU)
      light server processing, participants limited by bandwidth
    * mixing (MCU)
      heavy server processing
  * stream efficiency - recipient
    * Support suspending select streams at receiving end
      i.e. pause reception of stream (not just omit further processing it)
  * stream efficiency - forwarding
    * Support [simulcast](https://www.w3.org/TR/webrtc/#simulcast-functionality)
      i.e. encode multiple streams that an SFU can "hop" between
      * issue: **enabling** simulcast is done in [3 incompatible ways](https://www.meetecho.com/blog/simulcast-janus-ssrc/)
        * [Plan B]: SDP munging
          * test: SDP contains `a=ssrc-group:SIM` entry
        * [Unified Plan]: rid
          * test: SDP contains `a=rid` and `a=simulcast` entry mentioning "rid="
          * supported by Firefox [>= 46](https://www.meetecho.com/blog/simulcast-janus-ssrc/) [<< 72](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/72#removals_3)
        * [rfc8853][Simulcast in SDP]: rid
          * test: SDP contains `a=rid` and `a=simulcast` entry not mentioning "rid="
          * supported by Firefox [>= 68](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/72#removals_3)
          * supported by Chromium [>= M74](https://www.meetecho.com/blog/simulcast-janus-ssrc/)
      * issue: **using** simulcast requires support for [sendEncodings in RTCRtpTransceiverInit](https://bugzilla.mozilla.org/show_bug.cgi?id=1396918)
        * workaround: [adapter.js shim](https://github.com/webrtcHacks/adapter/pull/1004)
          * janus demos
    * Support spatial/temporal/quality [SVC](https://webrtcglossary.com/svc/)
      i.e. encode a stream that an SFU can efficiently "slice" without recoding
  * stream efficiency - source
    * Support suspending simulcast streams
      <https://webrtchacks.com/suspending-simulcast-streams/>
    * Support [SCReAM](https://github.com/EricssonResearch/scream)
      i.e. adapt encoding based on SCReAM feedback from stream remote end
  * stream efficiency - conference admin or shared room setting
    * Skip video streams beyond a threshold of participants
    * Skip video streams tied to quiet audio streams
    * Skip streams of explicitly tagged non-speaker participants
    * Mix audio streams (not stream each individually)
  * security
    * Support [PERC](https://webrtcglossary.com/perc/)
    * Support [ZRTP](https://en.wikipedia.org/wiki/ZRTP)
      i.e. end-to-end encryption (not only client-to-server)
  * meeting management
    * Personalized Meeting rooms
    * Scheduled/Meet-me Meetings
    * Instant/Direct Meetings
    * Presence Support
    * Recording
    * Text chat
    * Screen sharing
    * feedback on own audio level
    * feedback on encoding and streaming qualities
  * conference management
    * Conference Recording
    * force-mute participants
    * "Raise a hand" for muted participants
  * meeting room
    * Dual stream for dual screen
  * Dial in from telephone
  * Dial in from SIP audio-only
  * Dial in from SIP with video
  * Dial in from SIP with SIMPLE text chat

## See also

### SFU

[Janus Gateway](https://janus.conf.meetecho.com/) -
WebRTC SFU/bridge/broker
written in C,
shared under the GPL-3 license
on [Github](https://github.com/meetecho/janus-gateway)

[Mediasoup](https://mediasoup.org/) -
WebRTC SFU
written in C,
shared under the ISC license
on [Github](https://github.com/versatica/mediasoup).

Medooze SFU -
WebRTC SFU
written in Node.js,
shared under the Expat license
on [Github](https://github.com/medooze/sfu).

Spreed WebRTC -
WebRTC SFU
written in NodeJS and Go,
shared under the AGPL-3 license
on [Github](https://github.com/strukturag/spreed-webrtc).

Ion SFU -
WebRTC SFU
written in Go,
shared under the Expat license
on [Github](https://github.com/pion/ion-sfu).

[Jitsi Videobridge](https://jitsi.org/jitsi-videobridge/) -
XMPP SFU
written in Java,
shared under the Apache-2.0 license
on [Github](https://github.com/jitsi/jitsi-videobridge).
Seems to [require "plan B" SDP](https://github.com/jitsi/jitsi-meet/issues/4758).

[Open WebRTC Toolkit](https://01.org/open-webrtc-toolkit) -
WebRTC SFU and MCU
written in C++,
shared under the Apache-2.0 license
on [Github](https://github.com/open-webrtc-toolkit/owt-server).


### MCU

[Kurento Media Server](https://www.kurento.org/) -
WebRTC MCU
written in C++,
shared under the Apache-2.0 license
on [Github](https://github.com/Kurento/kurento-media-server).

[Licode](https://lynckia.com/licode/) -
WebRTC MCU
written in C++,
shared under the Expat license
on [Github](https://github.com/lynckia/licode).

[Medooze WebRTC Media Server](http://www.medooze.com/products/mcu.aspx) -
WebRTC/SIP MCU
written in C++,
shared under the GPL-2 license
on [Github](https://github.com/medooze/media-server).

[Red5 Server](https://www.red5pro.com/) -
WebRTC SFU
written in Java,
shared under the Apache-2.0 license
on [Github](https://github.com/Red5/red5-server).

[Ant Media Server](https://antmedia.io/) -
WebRTC SFU
written in Java,
shared under the Apache-2.0 license
on [Github](https://github.com/ant-media/Ant-Media-Server).
Fork of Red5 Server.


### Bridge/management

[SylkServer](https://sylkserver.com/) -
SIP/XMPP Application Server
with bridge to WebRTC
using Janus,
written in Python2,
shared under the GPL-3 license
on [Github](https://github.com/AGProjects/sylkserver).

Spreed standalone signaling server -
Nextcloud Talk bridge to WebRTC
using Janus and NATS,
written in Go,
shared under the AGPL-3 license
on [Github](https://github.com/strukturag/nextcloud-spreed-signaling).

Jigasi -
WebRTC bridge to Jitsi Videobridge
written in Java,
shared under the Apache-2.0 license
on [Github](https://github.com/jitsi/jigasi).

[Matrix Synapse](https://matrix.org/) -
Federated text chat
with WebRTC direct chat,
and with optional WebRTC conferencing
using Jigasi and Jitsi Videobridge,
written in Python3,
shared under the Apache-2.0 license
on [Github](https://github.com/matrix-org/synapse).

[BigBlueButton](https://bigbluebutton.org/) -
using Kurento Media Server
(only for [one-way streaming](https://docs.bigbluebutton.org/2.2/architecture.html#kurento-and-webrtc-sfu)
also using [internal SFU](https://github.com/bigbluebutton/bbb-webrtc-sfu)),
written in Java,
shared under the LGPL-3 license
on [Github](https://github.com/bigbluebutton/bigbluebutton).

[OpenMeetings](https://openmeetings.apache.org/) -
using Kurento Media Server and Red5 Server,
written in Java and JavaScript,
shared under the Apache-2.0 license
on [Github](https://github.com/apache/openmeetings).
WebRTC features ["might be not production ready"](https://openmeetings.apache.org/#News).
in release 5.0.0-M3 (newest as of 2020-04-10).

[openVidu](https://openvidu.io/) -
using Kurento Media Server,
written in Java and TypeScript,
shared under the Apache-2.0 license
on [Github](https://github.com/OpenVidu/openvidu).

Wire Audio Video Signaling -
[Wire](https://wire.com/) Application Server
using for [MLS](https://messaginglayersecurity.rocks/),
and using internal FIXME for WebRTC SFU,
written in Haskell, Rust, C,
FIXME.


### Web frontend

[multiparty-meeting](https://github.com/havfo/multiparty-meeting)
using Mediasoup
(and optionally drachtio and Kurento Media Server),
written in JavaScript
hosted at <https://letsmeet.no/>

[BitLink](https://github.com/oss-videochat/bitlink)
using Mediasoup,
written in JavaScript
hosted at <https://app.bitlink.live/>

[Jangouts](https://github.com/jangouts/jangouts)
using Janus,
written in CoffeeScript

[Roomler](https://github.com/gjovanov/roomler)
using Janus,
written in VueJS
hosted at <https://roomler.live/>

[tawk.space](https://github.com/invisible-college/tawk.space)
using Janus
written in CoffeeScript
hosted at <https://tawk.space/>

[PULT](https://gitlab.cvh-server.de/pgerwinski/pult)
using Janus, Mumble, and VNC,
written in JavaScript.

[open-museum](https://github.com/open-museum/web)
using Janus
written in Svelte
hosted at <https://open-museum.ch/>

[SIP2SIP](https://ag-projects.com/sip2sip/)
using SylkServer and Janus
hosted at <https://sip2sip.info/>
and <https://webrtc.sipthor.net/>

[Roll Call](https://github.com/mikeal/roll-call)
audio-only
hosted at <https://roll.call>

[Spreed.ME](https://www.spreed.me/)
using Spreed WebRTC

[Nextcloud Talk](https://nextcloud.com/talk/)
using Spreed standalone signaling server

[Galène](https://galene.org/)
using Ion SFU
written in Go.

[Jitsi Meet](https://jitsi.org/jitsi-meet/)
using Jigasi and Jitsi Videobridge
hosted at <https://meet.jit.si/>
and <https://beta.meet.jit.si/>.

[WorkAdventure](https://github.com/thecodingmachine/workadventure)
using Jigasi and Jitsi Videobridge
written in TypeScript
hosted at <https://workadventu.re/>

[mConf](http://mconf.org/)
using Kurento Media Server
written in Java and Ruby

[Veeting](https://veeting.com/)
cloud SFU service
using Janus
hosted at <https://rooms.veeting.com/>
(free trial at <https://rooms.veeting.com/home-office>)

[SkyWay Conference](https://github.com/skyway/skyway-conf)
cloud SFU service
written in TypeScript
hosted at <https://conf.webrtc.ecl.ntt.com/>

[Talky](https://about.talky.io/)
cloud SFU service
hosted at <https://talky.io/>

[Me](https://join.me/)
cloud SFU service

GoToMeeting
cloud SFU service

[Zoom Meetings](https://zoom.us/)
cloud SFU service
supporting "up to 50 participants at once"
(but client bandwidth and resource demands and stability of such session is unknown)

Hangouts Meet
cloud SFU service

Webex Meetings
cloud SFU service

Wowza Streaming Cloud
cloud streaming service

Skype
cloud SFU service
suporting "up to 25 participants at once"
(but client bandwidth and resource demands and stability of such session is unknown)

MoxieMeet
cloud SFU service
requiring Google account
supporting "up to 32 users all on video together"
(but client bandwidth and resource demands and stability of such session is unknown)

TeamViewer
cloud SFU service


Plan B: <https://datatracker.ietf.org/doc/html/draft-uberti-rtcweb-plan-00>
  "Plan B: draft spec expired 2013 on signaling multiple media sources in WebRTC by munging SDP"

Unified Plan: <https://tools.ietf.org/html/draft-roach-mmusic-unified-plan-00>
  "Unified Plan: draft spec expired 2014 on signaling multiple media sources in WebRTC by setting Restriction Identifier (RID)"

Simulcast in SDP: <https://datatracker.ietf.org/doc/html/rfc8853>
  "Simulcast in SDP: spec on signaling multiple media sources in WebRTC by setting Restriction Identifier (RID)"