aboutsummaryrefslogtreecommitdiff
path: root/DEVELOP.md
blob: 85b27c6e6f261fd4cde2b4be5c542b8eae3f900c (plain)

#Designing a realtime audio/video conferencing service

Features

Functionality to consider when considering tool/platform.

  • topology
    • mesh (P2P) all processing at end-points, participants limited by bandwidth
    • routing (SFU) light server processing, participants limited by bandwidth
    • mixing (MCU) heavy server processing
  • stream efficiency - recipient
    • suspending select streams at receiving end i.e. pause reception of stream (not just omit further processing it)
  • stream efficiency - forwarding
    • simulcast i.e. encode multiple streams that an SFU can "hop" between
    • spatial/temporal/quality SVC i.e. encode a stream that an SFU can efficiently "slice" without recoding
  • stream efficiency - source
  • stream efficiency - conference admin or shared room setting
    • Skip video streams beyond a threshold of participants
    • Skip video streams tied to quiet audio streams
    • Skip streams of explicitly tagged non-speaker participants
    • Mix audio streams (not stream each individually)
  • security
    • PERC
    • ZRTP i.e. end-to-end encryption (not only client-to-server)
  • meeting management
    • Personalized Meeting rooms
    • Scheduled/Meet-me Meetings
    • Instant/Direct Meetings
    • Presence
    • Recording
    • Text chat
    • Screen sharing
    • feedback on own audio level
    • feedback on encoding and streaming qualities
  • conference management
    • Conference Recording
    • force-mute participants
    • "Raise a hand" for muted participants
  • meeting room
    • Dual stream for dual screen
  • Dial in from telephone
  • Dial in from SIP audio-only
  • Dial in from SIP with video
  • Dial in from SIP with SIMPLE text chat

See also

SFU

Janus Gateway - WebRTC SFU/bridge/broker written in C, shared under the GPL-3 license on Github

Mediasoup - WebRTC SFU written in C, shared under the ISC license on Github.

Medooze SFU - WebRTC SFU written in Node.js, shared under the Expat license on Github.

Spreed WebRTC - WebRTC SFU written in NodeJS and Go, shared under the AGPL-3 license on Github.

Ion SFU - WebRTC SFU written in Go, shared under the Expat license on Github.

Jitsi Videobridge - XMPP SFU written in Java, shared under the Apache-2.0 license on Github. Seems to require "plan B" SDP.

Open WebRTC Toolkit - WebRTC SFU and MCU written in C++, shared under the Apache-2.0 license on Github.

MCU

Kurento Media Server - WebRTC MCU written in C++, shared under the Apache-2.0 license on Github.

Licode - WebRTC MCU written in C++, shared under the Expat license on Github.

Medooze WebRTC Media Server - WebRTC/SIP MCU written in C++, shared under the GPL-2 license on Github.

Red5 Server - WebRTC SFU written in Java, shared under the Apache-2.0 license on Github.

Ant Media Server - WebRTC SFU written in Java, shared under the Apache-2.0 license on Github. Fork of Red5 Server.

Bridge/management

SylkServer - SIP/XMPP Application Server with bridge to WebRTC using Janus, written in Python2, shared under the GPL-3 license on Github.

Spreed standalone signaling server - Nextcloud Talk bridge to WebRTC using Janus and NATS, written in Go, shared under the AGPL-3 license on Github.

Jigasi - WebRTC bridge to Jitsi Videobridge written in Java, shared under the Apache-2.0 license on Github.

Matrix Synapse - Federated text chat with WebRTC direct chat, and with optional WebRTC conferencing using Jigasi and Jitsi Videobridge, written in Python3, shared under the Apache-2.0 license on Github.

BigBlueButton - using Kurento Media Server (only for one-way streaming also using internal SFU), written in Java, shared under the LGPL-3 license on Github.

OpenMeetings - using Kurento Media Server and Red5 Server, written in Java and JavaScript, shared under the Apache-2.0 license on Github. WebRTC features "might be not production ready". in release 5.0.0-M3 (newest as of 2020-04-10).

openVidu - using Kurento Media Server, written in Java and TypeScript, shared under the Apache-2.0 license on Github.

Wire Audio Video Signaling - Wire Application Server using for MLS, and using internal FIXME for WebRTC SFU, written in Haskell, Rust, C, FIXME.

Web frontend

multiparty-meeting using Mediasoup (and optionally drachtio and Kurento Media Server), written in JavaScript hosted at https://letsmeet.no/

BitLink using Mediasoup, written in JavaScript hosted at https://app.bitlink.live/

Jangouts using Janus, written in CoffeeScript

Roomler using Janus, written in VueJS hosted at https://roomler.live/

tawk.space using Janus written in CoffeeScript hosted at https://tawk.space/

PULT using Janus, Mumble, and VNC, written in JavaScript.

open-museum using Janus written in Svelte hosted at https://open-museum.ch/

SIP2SIP using SylkServer and Janus hosted at https://sip2sip.info/ and https://webrtc.sipthor.net/

Roll Call audio-only hosted at https://roll.call

Spreed.ME using Spreed WebRTC

Nextcloud Talk using Spreed standalone signaling server

Galène using Ion SFU written in Go.

Jitsi Meet using Jigasi and Jitsi Videobridge hosted at https://meet.jit.si/ and https://beta.meet.jit.si/.

WorkAdventure using Jigasi and Jitsi Videobridge written in TypeScript hosted at https://workadventu.re/

mConf using Kurento Media Server written in Java and Ruby

Veeting cloud SFU service using Janus hosted at https://rooms.veeting.com/ (free trial at https://rooms.veeting.com/home-office)

SkyWay Conference cloud SFU service written in TypeScript hosted at https://conf.webrtc.ecl.ntt.com/

Talky cloud SFU service hosted at https://talky.io/

Me cloud SFU service

GoToMeeting cloud SFU service

Zoom Meetings cloud SFU service supporting "up to 50 participants at once" (but client bandwidth and resource demands and stability of such session is unknown)

Hangouts Meet cloud SFU service

Webex Meetings cloud SFU service

Wowza Streaming Cloud cloud streaming service

Skype cloud SFU service suporting "up to 25 participants at once" (but client bandwidth and resource demands and stability of such session is unknown)

MoxieMeet cloud SFU service requiring Google account supporting "up to 32 users all on video together" (but client bandwidth and resource demands and stability of such session is unknown)

TeamViewer cloud SFU service

Plan B: https://datatracker.ietf.org/doc/html/draft-uberti-rtcweb-plan-00 "Plan B: draft spec expired 2013 on signaling multiple media sources in WebRTC by munging SDP"

Unified Plan: https://tools.ietf.org/html/draft-roach-mmusic-unified-plan-00 "Unified Plan: draft spec expired 2014 on signaling multiple media sources in WebRTC by setting Restriction Identifier (RID)"

Simulcast in SDP: https://datatracker.ietf.org/doc/html/rfc8853 "Simulcast in SDP: spec on signaling multiple media sources in WebRTC by setting Restriction Identifier (RID)"