Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apple Music Lossless & AirPlay #1205

Closed
CarlBernard opened this issue Jun 11, 2021 · 21 comments
Closed

Apple Music Lossless & AirPlay #1205

CarlBernard opened this issue Jun 11, 2021 · 21 comments

Comments

@CarlBernard
Copy link

Hello,

First, thanks for this lovely little piece of software. Thanks thanks thanks.

Apple Music has started streaming in lossless format in both CD quality (16bit / 44.1k) and high resolution (24bit / 96k, 24bit / 196k).
Apple AirPlay 1 and 2 are capable of sending up to 16 bit / 44.1k in lossless ALAC format.
I use my Naim UnitiQute 1 amplifier via AirPlay (= shairport-sync) and everything works fine but I'm not sure what format I'm sending via my iPhone X (AAC lossy or ALAC lossless) to my Raspberry running shairport-sync.

I read on the Naim Audio forum (https://community.naimaudio.com/t/apple-music-hifi-tier-incoming/16445/632) that at the moment the only way to enjoy lossless is via cable as Apple Music via AirPlay decodes the file from ALAC to AAC before sending it to the receiver.
To do a further test, I also tried TIDAL in high resolution via AirPlay but my NAIM receiver informs me that it is always receiving 44.100Hz even when I am sending it 96k or 192k.
Do I have to configure shairport-sync in a particular way?
Does Shairport-sync always decode the file only in the 16bit / 44.1k format or send the file as received without any modification?

Is it possible to know which audio format (example: AAC, ALAC, FLAC), bit resolution and sample rate is received by shairport-sync before sending it to the audio system?
In ALSA, I remember, through a command from the terminal it was possible to verify this with

cat / proc / asound / card0 / pcm0p / sub0 / hw_params

but it was not possible to know what the codec was.

@CarlBernard CarlBernard changed the title Apple Lossless & AirPlay Apple Music Lossless & AirPlay Jun 11, 2021
@mikebrady
Copy link
Owner

Thanks for the post.

Shairport Sync uses the AirPlay 1 protocol. In my experience, the AirPlay 1 protocol uses a fixed format: 44,100 frames per second, 16 bit linear PCM, interleaved stereo, losslessly compressed in ALAC -- let's call it "CD quality ALAC" . It doesn't matter what it's being sent from or what it's being sent to, the format remains the same. If the source material is encoded in a different form, it is transcoded to this format before transmission. This accords with your experience. If the source material is actually CD quality, stored losslessly, then AFAIR it used to be the case that it was transmitted bit-perfectly over AirPlay 1 by iTunes. I don't know if that is still the case, but I presume it is.

It follows that there is no scope for, and no need for, adjustments to Shairport Sync to control how audio is encoded -- it's always sent in CD-quality ALAC.

At the receiving end, Shairport Sync is meant to modify the audio data as little as possible. For example -- assuming you are outputting to the alsa back end -- if you have a CD-quality DAC and use its hardware mixer for volume control, or if you choose to ignore the volume control information coming from the source, or if you set the volume control to max, the audio material is sent to the DAC unmodified except for an occasional sync event.

You can force Shairport Sync to choose a particular output depth, but if it's set to auto, it will select the greatest depth. For instance, if you have a CD-quality DAC and a hardware mixer but with 24 or 32 bit resolution, the highest resolution is chosen and the material is recoded from 16 to 24 or 32 bits but without doing anything to it -- 8 or 16 binary zeroes are simply added to each sample as appropriate. In other words, as before, the audio material is effectively unaltered except for occasional sync events.

If you use Shairport Sync's software mixer or if you select monophonic output (in other words, if Shairport Sync has to process the audio material), then the necessary calculations are done at 32-bit precision (giving 64-bit results for multiplications) and dithered to the appropriate bit depth -- 32 / 24 / 16 or 8 (!) bits for the output device.

As far as AirPlay 2 is concerned, what we know is that it uses two transmission modes -- Realtime and Buffered -- and two encoding schemes: CD-Quality ALAC for Realtime streams and AAC for Buffered streams. So it fits that the new lossless formats might be transcoded to AAC for onward transmission to AirPlay 2 players.

I hope this helps.

@susman
Copy link

susman commented Jun 15, 2021

but I'm not sure what format I'm sending via my iPhone X (AAC lossy or ALAC lossless) to my Raspberry running shairport-sync.

I just compared file captured from shairport-sync's stdout (streamed form iPhone 12 mini, Apple Music lossless) with file I ripped from a CD. They sound and look identical:
Screenshot 2021-06-15 at 22 15 16

For reference FFmpeg encoded AAC (bottom) vs CD version:
Screenshot 2021-06-15 at 22 24 58

@mikebrady
Copy link
Owner

Very interesting, thanks!

@susman
Copy link

susman commented Jun 16, 2021

Hi @mikebrady,
Could you please elaborate on:

the audio material is sent to the DAC unmodified except for an occasional sync event.

trying to understand how sync event is stored in payload.
Thanks!

@mikebrady
Copy link
Owner

@susman, thanks again for those diagrams. The syncing process (aka “stuffing”) is described here.

@CarlBernard
Copy link
Author

CarlBernard commented Jun 16, 2021

hello mikebrady and susman,

Thanks for your answers.

Susman, according to the analysis you did with "stdout", shairport-sync sends lossless ALAC format to the audio system as the diagrams are the same.
Question: Can you try to re-convert the lossy compressed AAC file to a lossless compressed ALAC file and compare it to the same lossless file you extracted from the CD? Are the diagrams always the same?

I ask you this question because (according to what I read on the various forums) Apple Music App if it has to send the stream to an AirPlay 2 device it converts the format received by Apple Music Server from ALAC (lossless compression) to AAC (lossy compression ) while if it has to send it to legacy devices that only accept ALAC (shairport-sync, AirConnect, LUMIN streamer etc etc) it does a double conversion from ALAC to AAC to ALAC.
If so we would have confirmation that all AirPlay devices are currently sending lossy compressed audio if the source is Apple Music.
The Naim Audio technician also confirms that if the source is QOBUZ or TIDAL, AirPlay sends the file in ALAC without loss to prove that the problem is neither AirPlay nor Apple Music Server but Apple Music App.

I also attach a small summary after reading the Naim and Roon forums (NAIM FORUM - https://community.naimaudio.com/t/apple-music-hifi-tier-incoming/16445/437 - ROON FORUM - https://community.roonlabs.com/t/apple-music-high-res/159869/1132)
Apple Music & AirPlay.pdf

@susman
Copy link

susman commented Jun 16, 2021

Hi @CarlBernard,
shairport-sync's stdout delivers raw PCM, as if it it was writing it into a sound device (file). AAC to PCM conversion is something that is happening every time you're playing a file, music players do that. DACs understand PCM (or DSD), not compressed audio.

I tested it once again today, here's the process:

  1. stop shairport-sync daemon
  2. run shairport-sync -u -o stdout > 'Awesome Band - Awesome Song.wav'
  3. start playing a song on your iOS/macOS device ('lossless', not 'High Resolution lossless')
  4. select shairport-sync host as output device in Apple Music app. No sound will be produced, we're writing to a file on disk. Wait for a song to finish
  5. load created file into Audacity as raw data: 'File' --> 'Import' --> 'Raw Data' --> select the file:
  • Encoding: Signed 16-bit PCM
  • Channels: 2
    Click 'Import'
  1. Load reference file into a second track in Audacity
  2. Align the tracks exactly at sample resolution (zoom all the way in) - this is important
  3. Select one of the tracks, in menu bar select 'Effect' --> 'Invert'. Now one of the tracks is out-of-phase relative to another
  4. Play both tracks. There should be no sound as they're supposed to cancel each other out.

This way we can confirm that tracks are identical. If you do the same process, but instead of raw PCM dumped from shairport-sync's output - AAC encoded reference file (decoded back to PCM, because that's what DAWs work with), you'll hear the artifacts introduced by lossy encoding instead of silence.

My tests results were as described. I can certainly tell shairport-sync delivers lossless CD quality (16/44.1k) to a DAC (backend).

Spectrogram test isn't very representative. Turns out FFmpeg's libfdk_aac does this low pass filtering around 18kHz, default aac encoder doesn't do that and compared spectrograms look very much alike, not exactly the same:
Screenshot 2021-06-16 at 20 34 11

Strictly speaking, streaming Apple Music Lossless over network to shairport-sync is not a lossless process. Let's imagine the chain:
Apple lossless files may be in 24 bit 48kHz format. Before sending it over network, Apple Music app has to dither it down to 16 bit and resample to 44.1kHz because that's the limit for AirPlay 1 ALAC streaming - that is a lossy process. In case of 16 bit/44.1kHz lossless files this process obviously doesn't have to happen. In case of 24/44.1kHz only dithering is required. Encoded again into ALAC it is streamed over network to shairport-sync, which employs "stuffing" process, adding or removing frames in order to keep in sync is a lossy process. But it's just dropping or adding frames, compared to AAC's MDCT it's nothing.

@CarlBernard
Copy link
Author

Hi susman,

Thanks for the reply.

I shared on another forum (https://audiophilestyle.com/ca/bits-and-bytes/apple-musics-lossless-and-hi-res-mess-r1022/page/6/#comments) your tests and I'm waiting for more HDCD testing from AudiophileStyle director.

As for the "stuffing" process, are there any possibilities to avoid adding or deleting frames for example via libsoxr?

Thanks again for all the explanations.

@mikebrady
Copy link
Owner

Stuffing isn’t done on outputs to a pipe or to stdout.

When stuffing is done using libsoxr, e.g. on the alsa backend, it resamples a 352-frame packet to be either 351 or 353 frames.

@susman
Copy link

susman commented Jun 18, 2021

Not sure I understand what HDCD has to do with any of this, but hey... whatever floats their boat.
Trust your ears, this is what matters in the end.

@easp
Copy link

easp commented Jun 23, 2021

@susman Comparing the spectrograms isn't a useful way to compare a lossless to a lossy file, at least not when the lossy file is 256kbps AAC. I started with an ALAC rip I made, then used Apple's AAC encoder at 256kbps via XLD to make an AAC. I loaded both the ALAC and the AAC into Audacity and made spectrograms of one of the stereo tracks to compare. The result is below. I'm hard pressed to find any differences. It's not any easier when zooming in.
losslessvslossy
At the same time, the audio residual test does produce audible output.

Your results with the residual comparison on the captured WAV are interesting. I'm not sure though that I understand the conditions of your test.

What is the origin of the lossless file/data being played from your iPhone to shairport-sync? If I understand you, the file you used as the reference file in your residual test was ripped from CD. Is this also the origin of the lossless file played from your phone, or are you playing a lossless stream from Apple Music's servers?

The distinction is important, because Apple Music (on MacOS, at least) behaves differently when playing an ALAC that was ripped from CD vs a track from Apple's servers with lossless playback enabled. In the latter case it will switch to the AAC version when an AirPort Express v1 is the playback destination. When an AirPlay 2 device is the playback destination it transcodes the lossless data received from Apple's servers and sends it as AAC in buffered mode.

@susman
Copy link

susman commented Jun 23, 2021

Hi @easp

Comparing the spectrograms isn't a useful way to compare a lossless to a lossy file

Yes, it's not very useful, as I said in my next comment. Although you can clearly see the difference in quiet part at the end of the track.

Is this also the origin of the lossless file played from your phone, or are you playing a lossless stream from Apple Music's servers?

My test is described in detail. I'm playing a song from Apple Music Lossless and comparing with the same song ripped from a CD I own.

In the latter case it will switch to the AAC version when an AirPort Express v1 is the playback destination. When an AirPlay 2 device is the playback destination it transcodes the lossless data received from Apple's servers and sends it as AAC in buffered mode.

Playback destination in case of shairport-sync is over AirPlay 1 protocol, no Airplay 2 involved at any step.

@easp
Copy link

easp commented Jun 23, 2021

Interesting @sussman. Thanks for the added clarification. I wanted to be sure because people sometimes conflate the two. It appears that AppleMusic on iOS behaves differently from AppleMusic on MacOS when playing over the original AirPlay protocol as with shairport-sync, or a 1st gen AirPort Express.

I will have dig out my 1st gen AirPort express again and figure out how to do port mirroring on my switch so I can sniff packets between my phone and my AirPorts Express so I can investigate further.

@sussman
Copy link

sussman commented Jun 24, 2021 via email

@CarlBernard
Copy link
Author

As I told you, further tests have been done by the AudiophileStyle Director.
The results confirm what Susman has already said.

https://audiophilestyle.com/ca/bits-and-bytes/apple-music-lossless-mess-part-2-airplay-r1026/

At this point, I'd like to better understand how "stuffing" works (as if it were to be explained to a child!) and if and how it could be improved perhaps with a more performing PC (but maybe it's better to open another post ??).

Small question about it: does the first generation AirPort Express also suffer from this problem?

@sonorejr
Copy link

sonorejr commented Jul 4, 2021

@mikebrady, you said, "Shairport Sync uses the AirPlay 1 protocol. In my experience, the AirPlay 1 protocol uses a fixed format: 44,100 frames per second, 16 bit linear PCM, interleaved stereo, losslessly compressed in ALAC -- let's call it "CD quality ALAC"

What about when streaming to an older Apple TV using AirPlay 1 (Apple TV with optical output) where it can send 16 bit or 24 bit (set in settings) at 48 kHz? Is it possible to emulate an older Apple TV to stream 16 bit or 24 bit at 48 kHz?

@mikebrady
Copy link
Owner

Which AppleTV model are you referring to, please? Is it the very original Intel-based AppleTV or one of the more “modern” Apple silicon based devices? Which settings do you mean?

@mikebrady
Copy link
Owner

As I told you, further tests have been done by the AudiophileStyle Director.
The results confirm what Susman has already said.

https://audiophilestyle.com/ca/bits-and-bytes/apple-music-lossless-mess-part-2-airplay-r1026/

At this point, I'd like to better understand how "stuffing" works (as if it were to be explained to a child!) and if and how it could be improved perhaps with a more performing PC (but maybe it's better to open another post ??).

Apologies for the delay. Please see an earlier reply.

Small question about it: does the first generation AirPort Express also suffer from this problem?

What problem do you mean?

@sonorejr
Copy link

sonorejr commented Jul 4, 2021

Which AppleTV model are you referring to, please? Is it the very original Intel-based AppleTV or one of the more “modern” Apple silicon based devices? Which settings do you mean?

I have an Apple TV (3rd generation) that I can toggle 16 bit and 24 bit in its settings (auto vs 16 bit output) It always sends 48 kHZ.

https://support.apple.com/kb/sp648?locale=en_US

@github-actions
Copy link

github-actions bot commented Sep 3, 2021

This issue has been inactive for 60 days so will be closed 7 days from now. To prevent this, please remove the "stale" label or post a comment.

@Biao7
Copy link

Biao7 commented Feb 22, 2024

Hi everyone,
I have a question:
I integrated sharport sync 4.3.2 in the Linux system, which supports airplay 2 functionality. I am using an iPhone to connect to a Linux device and can connect normally, but the audio encoding transmitted is default to using Alac. How can I have my phone send audio data encoded with AAC? Can we modify the code of sharport sync to force the use of aac encoding format for data processing during connection negotiation?

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants