Skip to content

Communication RTSP Client

Jump to bottom
pschatzmann edited this page Oct 22, 2026 · 1 revision

This page shows how to receive and play audio from an RTSP server - an IP camera, a media server like mediamtx or live555, or another microcontroller running the RTSP Server of this library.

Contents

Which Client?

Client Header Library Transport Notes
RTSPClient (RTSPClientWiFi, RTSPClientEthernet) AudioTools/Communication/RTSP.h part of this library UDP small, decodes via registered decoders, audio only
AudioClientRTSP AudioTools/Communication/RTSPClient555.h arduino-live555 TCP or UDP based on the proven live555 implementation, bigger

Start with RTSPClient. Use AudioClientRTSP if a server doesn't work with it, or if you need RTP over TCP.

RTSPClient

How it works

RTSPClient<TcpClient, UdpSocket> opens the RTSP session (OPTIONS, DESCRIBE, SETUP, PLAY), binds a local UDP port and receives the RTP packets. The SDP of the server tells it the format. The payload of each RTP packet is passed to an internal MultiDecoder, which selects the decoder that matches the format and writes the decoded PCM to your output - via a resampler, which can compensate small clock differences between server and client (see setResampleFactor()). During the session it sends an OPTIONS keep-alive.

The template parameters bind it to a network stack:

using RTSPClientWiFi = RTSPClient<WiFiClient, WiFiUDP>;          // RTSPClientWiFi.h, included by RTSP.h on ESP32
using RTSPClientEthernet = RTSPClient<EthernetClient, EthernetUDP>;  // RTSPClientEthernet.h

For Ethernet include AudioTools/Communication/RTSP/RTSPClientEthernet.h in addition to RTSP.h.

Example

From audio-client-audiokit.ino: plays MP3 or IMA ADPCM from an RTSP server on an AudioKit.

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecADPCM.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"  // https://github.com/pschatzmann/codec-helix
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/Communication/RTSP.h"  // brings RTSPClientWiFi

IPAddress srv(192, 168, 1, 39);  // RTSP server
const uint16_t rtspPort = 8554;
const char* rtspPath = "stream";  // path of the stream on the server

AudioBoardStream i2s(AudioKitEs8388V1);
RTSPClientWiFi client(i2s);
MP3DecoderHelix mp3;                                  // for audio/mpeg
ADPCMDecoder adpcm(AV_CODEC_ID_ADPCM_IMA_WAV, 512);  // for audio/adpcm

void setup() {
  Serial.begin(115200);
  // connect WiFi
  WiFi.begin("ssid", "password");
  while (WiFi.status() != WL_CONNECTED) delay(500);
  WiFi.setSleep(false);

  auto cfg = i2s.defaultConfig(TX_MODE);
  i2s.begin(cfg);

  client.addDecoder(mp3);
  client.addDecoder(adpcm);
  client.setHeaderTimeoutMs(8000);  // for slow servers
  if (!client.begin(srv, rtspPort, rtspPath)) {
    Serial.println("Failed to start RTSP client");
    stop();
  }
}

void loop() {
  client.copy();  // decodes the next RTP payload
}

begin(address, port, path) connects to rtsp://<address>:<port>/<path>/; leave the path out if the server provides its stream at the root. Call copy() as often as possible in loop().

Supported formats

The format is determined from the first a=rtpmap line of the server's SDP - from the static RTP payload type if there is one, else from the encoding name:

SDP MIME Decoder
L16 (payload type 10, 11) audio/L16 built in (network byte order to PCM)
L8 audio/L8 built in
PCMU (0), PCMA (8) audio/PCMU, audio/PCMA G.711 decoder, register with addDecoder()
GSM (3) audio/gsm GSM decoder
G722 (9) audio/g722 G.722 decoder
DVI4 (5, 6, 16, 17) audio/adpcm IMA ADPCM decoder
MPA (14) audio/mpeg MP3 decoder - see below
MPEG4-GENERIC audio/aac not supported out of the box: the AU headers of RFC 3640 are not removed, and the frames come without ADTS header
OPUS audio/opus Opus decoder

Register the decoders you need with addDecoder(decoder), which uses the decoder's own mime(), or with addDecoder("audio/...", decoder) if it doesn't match. See Encoding and Decoding of Audio for the decoders and the libraries they need. mime() and payloadType() tell you what the server announced, audioInfo() the format of raw PCM.

MP3 (payload type 14): RFC 2250 puts a 4 byte header in front of the MP3 data in each packet. Servers that send it (e.g. ffmpeg, and this library's server with setUseRfc2250Header(true)) need client.setPayloadOffset(4), so that the client skips these bytes; servers that send raw MP3 frames (the default of this library's server, which VLC expects) need no offset. If it plays with distortions, check this first.

API

Method Description
setOutput(out) defines where the decoded audio is written to (also possible in the constructor)
addDecoder(decoder), addDecoder(mime, decoder) registers a decoder for a format
begin(address, port, path) starts the session; returns false if the server can't be reached or rejects it
copy() passes the next received RTP payload to the decoder - call it in loop()
available() bytes received and not yet decoded
setActive(bool) pauses (RTSP PAUSE) and resumes (PLAY) the stream
end() ends the session (TEARDOWN)
mime(), payloadType(), audioInfo() what the server announced
setResampleFactor(factor) > 1.0 plays faster, < 1.0 slower: compensates clock differences that would otherwise let the buffers run over or empty
setPayloadOffset(bytes) bytes to skip after the RTP header (e.g. 4 for RFC 2250 MP3)
setHeaderTimeoutMs(ms) timeout for the server's responses (default 3000)
setConnectRetries(n), setConnectRetryDelayMs(ms) connection retries (default 2, 500 ms)
setIdleDelay(ms) delay in copy()/available() when there is no data, to avoid a busy loop

Limitations

  • UDP only: RTP is received over UDP. Lost packets are lost - over a weak WiFi connection you hear gaps. If that's a problem, use AudioClientRTSP with TCP.
  • Audio only: there are no video decoders in this pipeline, and an MPEG transport stream with video and audio (MP2T, like Streaming Video and Audio via RTSP produces) is not supported.
  • One track: the client uses the first a=rtpmap line of the SDP. A server that announces only a static payload type without an a=rtpmap line is not recognized (mime() returns nullptr).

AudioClientRTSP (live555)

AudioClientRTSP wraps the RTSP client of live555 - install arduino-live555. It writes the received payload to any Print, e.g. an EncodedAudioStream that decodes it, and can receive RTP over TCP or UDP. From audio-client-555-audiokit.ino:

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/Communication/RTSPClient555.h"  // https://github.com/pschatzmann/arduino-live555

AudioBoardStream i2s(AudioKitEs8388V1);
EncodedAudioStream out_mp3(&i2s, new MP3DecoderHelix());
AudioClientRTSP rtsp(1024);  // receive buffer size

void setup() {
  auto cfg = i2s.defaultConfig(TX_MODE);
  i2s.begin(cfg);
  out_mp3.begin();
  rtsp.setLogin("ssid", "password");  // optional: connects WiFi
  rtsp.begin("rtsp://192.168.1.38:8554/test.mp3", out_mp3);
}

void loop() {
  rtsp.loop();
}

The constructor takes the receive buffer size - make sure that one encoded frame fits -, whether to use TCP (setTCP()/setUDP()) and whether begin() should block (setBlocking()); in the non-blocking default call loop() in your loop(). The live555 library itself also provides an RTSPSimpleClient, used in audio-client-555-i2s.ino.

Testing

A desktop RTSP server is the easiest way to test a client. With mediamtx running, publish a file with ffmpeg:

ffmpeg -re -stream_loop -1 -i test.mp3 -c:a copy -f rtsp rtsp://localhost:8554/stream
ffmpeg -re -stream_loop -1 -i test.wav -c:a pcm_s16be -ar 16000 -ac 1 -f rtsp rtsp://localhost:8554/stream

and connect the client to <ip of the PC>, port 8554, path stream. Or use a second board with one of the RTSP server examples of this library. Check the stream with ffplay rtsp://<server>:8554/stream first: if ffplay can't play it, the client won't either.

Troubleshooting

  • begin() fails: check the address, port and path - many servers only accept their exact stream path. Increase setHeaderTimeoutMs() for slow servers.
  • No sound, but no error: no decoder matches the announced format - print client.mime() and register a decoder for it. If it is nullptr, the SDP has no a=rtpmap line the client understands.
  • MP3 sounds distorted: the RFC 2250 header - see setPayloadOffset() above.
  • Gaps in the sound: UDP packets got lost - check the WiFi connection, switch off WiFi power save (WiFi.setSleep(false)), or use AudioClientRTSP with TCP.
  • The playback slowly runs ahead or falls behind, with buffer over- or underflows: the clocks of server and client differ - compensate with setResampleFactor(), e.g. 1.001 or 0.999.

Clone this wiki locally