some FLAC files are not loaded/parsed

[mixxxbot]

Reported by: sals3r0
Date: 2019-06-28T08:53:14Z
Status: Confirmed
Importance: High
Launchpad Issue: lp1834604

---

OS: Kubuntu 18.04
Kernel: 4.15.0-52-generic #⁠56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Architecture: x86_64
CPU: Intel(R) Core(TM) i5-4440 CPU @ 3.10GHz
Mixxx version: mixxx-2.3.0-alpha-pre-master-git6862-release-bionic-amd64 (nightly build)

Issue description:

Some FLAC files are not loaded/ analyzed by the software. Example FLAC file here: https://uloz.to/!dGxBLrXYOHS6/test-flac-zip

This file is loaded and parsed normally by Audacity and Virtual DJ.

From the application log:

Debug [Main]: BeatFactory::loadBeatsFromByteArray could not parse serialized beats.
Debug [Main]: SoundSourceProxy - SoundSourceProvider "Xiph.org libFLAC" created a SoundSource for file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac" of type "flac"
Debug [Main]: SoundSourceProxy - Skip importing of track metadata and embedded cover art from file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac"
Debug [Main]: BaseTrackCache(0x555fb6e94b50) updateIndexWithQuery took 0 ms
Debug [Main]: WCoverArt::slotCoverFound WCoverArt(0x555fbe84ed10)  "CoverInfoRelative(NONE,GUESSED,,0x0)" QSize(0, 0)
Debug [Main]: dropEventFiles() true (QUrl("file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac"))
Debug [Main]: mimeData.hasText() true "library"
Debug [Main]: dropEventFiles() true (QUrl("file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac"))
Debug [Main]: mimeData.hasText() true "library"
Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]"
Debug [CachingReaderWorker 1]: SoundSourceProxy - SoundSourceProvider "Xiph.org libFLAC" created a SoundSource for file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac" of type "flac"
Debug [CachingReaderWorker 1]: SoundSourceProxy - Opening file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac" with provider "Xiph.org libFLAC" using mode Strict
Warning [CachingReaderWorker 1]: AudioSource - No audio data available
Warning [CachingReaderWorker 1]: SoundSourceProxy - File is empty "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234  Descarga .flac"
Debug [Main]: BaseTrackPlayerImpl::slotTrackLoaded
Debug [Main]: TrackAnalysisScheduler - Resuming

[mixxxbot]

Commented by: uklotzde
Date: 2019-06-28T12:41:01Z

---

The FLAC stream metadata doesn't contain the total length (in sample frames) that Mixxx expects:

Format : FLAC
Format/Info : Free Lossless Audio Codec
File size : 15.5 MiB
Overall bit rate mode : Variable

Audio
Format : FLAC
Format/Info : Free Lossless Audio Codec
Bit rate mode : Variable
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 44.1 kHz
Bit depth : 16 bits
Compression mode : Lossless
Stream size : 15.5 MiB (100%)
Writing library : libFLAC 1.3.2 (UTC 2017-01-01)

I've never seen any file without length metadata before. We need to find out how to handle this without decoding the whole file upfront until EOF.

[mixxxbot]

Commented by: daschuer
Date: 2019-06-28T14:33:03Z

---

We can fix it during the analyzer and may estimate the length from the file size in the meantime

This bug can also be used as a prove of concept for possible steaming support.

A stream is a file without a fixed length.

How can we analyze that? A kind of on the fly analysis would be nice.
I think as a DJ a stream delay can be accepted if we can see what will come looking at the waveforms and beats in turn.

[mixxxbot]

Commented by: uklotzde
Date: 2019-06-28T15:04:07Z

---

The total length of the stream is used everywhere in Mixxx. This is the reason why files with undefined length are rejected. Supporting sources with undefined length is a major task!

[mixxxbot]

Commented by: daschuer
Date: 2019-06-28T15:13:44Z

---

Unfortunately yes ...

[mixxxbot]

Commented by: sals3r0
Date: 2019-06-28T15:42:02Z

---

In Virtual DJ the exact stream length is figured up by the analyzer (it has to decode the file anyway). Maybe it could work in a similar way in Mixxx?

This is not an infinite stream, it is a file with finite duration, but not known from metadata...handling this case should be easier than supporting streams with potentially infinite length (would be very nice though :)...

[mixxxbot]

Commented by: uklotzde
Date: 2019-06-30T22:37:11Z

---

We are not able to provide a solution other than improving the logging. The current architecture is not ready for streams with unknown or even unlimited length.

As a workaround I suggest re-encoding the FLAC files to generate the complete stream info metadata block for those files. Please check why your encoder didn't calculate and write the data and how to include it in the future.

This is the proposed solution even if Mixxx would support such files. There is no reason for excluding essential metadata from files that is known in advance.

[mixxxbot]

Commented by: daschuer
Date: 2019-06-30T23:19:11Z

---

Mixxx can already handle inaccurate length info in the file metadata.
Mayb I have missed something, but lenght = 0 is only a special case. So I think the fix is only a matter of making some lenght checks non fatal.

[mixxxbot]

Commented by: sals3r0
Date: 2019-07-01T08:58:28Z

---

Yeah I think for the time being I will just re-encode my FLAC collection with the official FLAC encoder. Thanks.

[mixxxbot]

Commented by: sals3r0
Date: 2019-07-01T15:26:03Z

---

FYI Someone had the same problem and posted a simple script to re-encode his entire FLAC collection:

https://forums.gentoo.org/viewtopic-t-831333-start-0.html

I am running a simplified version of this script now (skipping the meta-tags logic as I don't have any tags I need to preserve) and it works.