Blog

All articles

Technologies for Real-Time Multimedia

Technologies for Real-Time Multimedia | New Line Technologies

Updated:

Real-time multimedia is the delivery of audio and video from the source to the viewer with a delay ranging from a few hundred milliseconds for calls to 1–3 seconds for large live broadcasts. The delay depends on the protocol: according to Wowza, WebRTC delivers video in under 500 ms, Low-Latency HLS in 1–3 seconds, and standard HLS in 18–45 seconds or more.

Below, we look at the stages of delivery, what affects latency, how WebRTC, HLS, MPEG-DASH, RTMP, SRT, and RTSP differ, and how to choose a protocol for a specific product.

How Real-Time Delivery Works

Between the camera and the viewer’s screen, the stream passes through several stages, and each one adds to the delay.

Capture and Encoding

The camera and microphone record the signal, and the encoder compresses it with a codec such as H.264, HEVC, or AV1 for video and AAC or Opus for audio. Stronger compression means less data on the network, but encoding takes time and device resources.

Media Server and CDN

The server receives the stream, transcodes it into several quality levels if needed, and packages it in a delivery format. For a large audience, a CDN distributes the stream: copies of segments are stored on servers closer to viewers, which reduces the load but adds caching delay.

Playback

The player on a phone, computer, or TV receives the data, builds a small buffer, and shows the video. The buffer protects against network interruptions: the larger it is, the smoother the playback and the longer the delay.

What Affects Latency

For HTTP-based protocols, the main factor is segment length. In the Low-Latency HLS session at WWDC 2019, Apple showed that with standard 6-second segments, the delay in its example reaches about 12 seconds even before delivery through a CDN, and grows further with a CDN. Low-Latency HLS splits segments into smaller parts, and Apple designed it for a delay of 1–2 seconds. For comparison, cable and satellite TV run with a delay of 2–8 seconds.

UDP-based protocols such as WebRTC and SRT do not wait for a segment to finish and send packets right away. They achieve lower latency, but for thousands of viewers they need dedicated infrastructure: media servers (SFU) and a CDN that supports WebRTC.

Planning a streaming service?

Streaming Protocols

WebRTC

WebRTC is a W3C standard and a set of IETF protocols (RFC 8825) for sending audio, video, and data between browsers and apps without plugins. It is used in video calls, interactive webinars, online auctions, and camera control, where a delay under one second matters.

HLS and Low-Latency HLS

HTTP Live Streaming was developed by Apple. The stream is split into segments that the player downloads over regular HTTP, so HLS scales easily through a CDN and is supported on iOS, macOS, Android, and most web players. Adaptive bitrate switches quality depending on network speed. Low-Latency HLS keeps these advantages and cuts the delay to a few seconds.

MPEG-DASH

MPEG-DASH is an international, codec-agnostic standard for adaptive streaming. It works on the same segment principle as HLS and is widely used in browsers, on Android, and on smart TVs, while Apple devices rely primarily on HLS.

RTMP

Real-Time Messaging Protocol was created for Flash Player. Since Flash support ended in 2020, RTMP is rarely used for playback, but it remains common for sending a stream from encoders and OBS to servers and platforms such as YouTube Live. The secure version, RTMPS, encrypts the connection with TLS.

SRT

Secure Reliable Transport was developed by Haivision, which made its code open source in 2017. SRT runs over UDP, resends lost packets, encrypts the stream with AES, and keeps a 120 ms latency buffer by default. It is used to send a signal from an event venue to a studio or the cloud over the public internet.

RTSP

Real-Time Streaming Protocol controls a streaming session: playback, pause, and seeking. Most IP cameras support it, including ONVIF-compatible models, so RTSP is the main protocol in video surveillance. Browsers do not play RTSP directly, so for the web the stream is converted to WebRTC or HLS.

Protocol Comparison

ProtocolTypical latencyUsed forLimitations
WebRTCUnder 500 msVideo calls, interactive broadcasts, camera controlLarge audiences need media servers (SFU)
Low-Latency HLS1–3 sLive broadcasts to large audiences, sports, esportsRequires player and CDN support
HLS18–45 s or moreVideo on demand, linear OTTToo much delay for interaction
MPEG-DASHSimilar to HLS, depends on segmentsBrowsers, Android, smart TVsApple devices rely on HLS
RTMP / RTMPS1–5 s when tuned, usually 5–18 sSending a stream from an encoder to a server or platformNot played in modern browsers
SRT0.5–3 s depending on the scenario, 120 ms buffer by defaultSending a signal from an event venue over the internetNot designed for browser playback
RTSPJust over 1 sIP cameras, video surveillanceNot supported directly in browsers

Latency figures are approximate and based on Wowza data for typical scenarios, and for Low-Latency HLS also on Apple’s target of 1–2 seconds. Actual latency depends on encoder settings, segment length, the network, and the player.

How to Choose a Protocol

If users talk to each other or control an event, as in a video consultation or an online auction, choose WebRTC. If the product shows a live broadcast to thousands of viewers, it is more efficient to stream it through Low-Latency HLS or DASH with a CDN. For video on demand, standard HLS is enough.

The protocols for delivery to viewers and for sending to the server are often different. A typical setup: a camera or encoder sends the stream to the server over RTMP or SRT, the server transcodes it, and viewers receive HLS or WebRTC. If the product runs on several devices at once, plan the sync between them as well, which we covered in our article on the ecosystem of connected devices.

Need a team for your video platform?

How NLT Builds Real-Time Platforms

For more than 6 years, the New Line Technologies team has worked on video production and streaming products built on US-patented technology. They include Remote HD Recorder for Zoom for Cinamaker: apps for macOS, Windows, iOS, and Android and a cloud service record each call participant separately in 1080p, and AI detects the active speaker and assembles the final video. A team of 14 people built the product in 18 months.

In the same area, we are developing AI Sight Director, a module that automatically switches cameras and scenes during a live broadcast based on behavior and scene analysis. Our multi-camera apps connect up to 4 video devices and 12 audio devices at once, with a 40 ms signal delay between them. The stack of these projects includes AVFoundation, VideoToolbox, Metal, NDI, RTMP/S, HLS, and the Zoom SDK.

Learn more about our experience in real-time multimedia development, our software development services, and dedicated development teams.

Conclusions

There is no universal protocol for real time. Conversation and control need WebRTC, mass broadcasts need Low-Latency HLS or DASH, sending a signal to the server needs RTMP or SRT, and surveillance cameras use RTSP. Most products combine two or three protocols.

Before choosing a technology, define the acceptable latency for each scenario: conversation, viewing, recording. This number determines the architecture, the infrastructure cost, and the choice of CDN, so it is worth fixing before development starts.

Planning a streaming service, video calls, or a multi-camera broadcast? Talk to our team, and we will help you choose the protocols and architecture.

Frequently Asked Questions

What latency counts as “real time”?

Video calls and camera control need a delay under 500 ms, otherwise people talk over each other. For live broadcasts to a large audience, real time means 1–3 seconds, which Low-Latency HLS provides. Standard HLS, with a delay of 18–45 seconds, suits broadcasts where viewers do not interact with the event.

How is WebRTC different from HLS?

WebRTC sends audio and video directly between browsers or through a media server with a delay under 500 ms and suits two-way communication. HLS splits the stream into segments and delivers them over regular HTTP and CDNs, so it scales to millions of viewers easily, but its delay ranges from 1–3 seconds in the Low-Latency version to 18–45 seconds in the standard one.

Is RTMP still used in 2026?

Yes, but mostly for sending a stream from an encoder or OBS to a server or streaming platform. RTMP playback in browsers disappeared along with Flash, so viewers receive the stream through HLS, DASH, or WebRTC.

When should you choose SRT?

SRT fits when a signal has to travel from an event venue to a studio or the cloud over the public internet with packet loss. The protocol runs over UDP, resends lost packets, encrypts the stream with AES, and uses a 120 ms latency buffer by default.