Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

headers for remote streams (including volatile) #367

Merged
merged 27 commits into from
Aug 5, 2020
Merged

headers for remote streams (including volatile) #367

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
eeb3f26
rebase
philippe44 Jun 22, 2020
6d6201c
typo + stop AAC parser if reached end of stsz table
philippe44 Jun 23, 2020
363cf0f
comments & tweaks
philippe44 Jun 24, 2020
fe3ba9f
move parsing to Audio::Scan
philippe44 Jun 25, 2020
44f9376
Linux requires seeking (and maybe a file handle copy, not sure why)
philippe44 Jun 25, 2020
1ff06f2
complete parsing move to Format::XXX
philippe44 Jun 26, 2020
5d79e33
integrate stash creation to audio_process
philippe44 Jun 26, 2020
e545b8b
proper demux process intialization
philippe44 Jun 26, 2020
6589f8f
removing unused bits
philippe44 Jun 27, 2020
72edf2d
add WIMP support and a bit of cleanup
philippe44 Jun 27, 2020
ffa016a
load classes in header parsing
philippe44 Jun 27, 2020
54b8323
really load class (this time) and add live header parsing
philippe44 Jun 27, 2020
54b69eb
tweak on-the-fly header + WIMP fix for direct streaming
philippe44 Jun 28, 2020
4eca380
forgot to create local mapping for _content_type
philippe44 Jun 28, 2020
08d0a45
addressing style comments
philippe44 Jun 28, 2020
401bb80
consolidate HTTP/HTTPS sysread + add canDirectStreamSong in HTTPS
philippe44 Jun 29, 2020
e26a444
handle recurse issue in HTTPS for ICY + avoid reference in InitialBlock
philippe44 Jun 29, 2020
b44a8c1
solve sysread issue + unlike temp files
philippe44 Jun 30, 2020
a74908e
Thank you Windows...
philippe44 Jun 30, 2020
311256a
comment + remove use os smartmatch
philippe44 Jul 2, 2020
2c6582b
avoid init invocation + prepare 'reliable'
philippe44 Jul 3, 2020
e26cb69
debug sentence fix (no auto invoke!)
philippe44 Jul 4, 2020
3a225bd
comment clarification
philippe44 Jul 4, 2020
5ffa970
improve comments
philippe44 Jul 5, 2020
474ec2b
don't stash range unless self is blessed (direct streaming)
philippe44 Jul 14, 2020
3bac645
streamUrl must be updated when scanUrl redirect/changes url
philippe44 Jul 15, 2020
7ca3490
Merge branch 'public/8.0' into remote-headers
philippe44 Aug 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 26 additions & 1 deletion Slim/Formats/AIFF.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,10 @@ sub getTag {
return $tags;
}

sub volatileInitialAudioBlock { 1 }

sub getInitialAudioBlock {
my ($class, $fh, $track) = @_;
my ($class, $fh, $track, $time) = @_;
my $length = $track->audio_offset() || return undef;

open(my $localFh, '<&=', $fh);
Expand All @@ -92,9 +94,32 @@ sub getInitialAudioBlock {
seek($localFh, 0, 0);
close($localFh);

# adjust header size according to seek position
my $trim = int($time * $track->bitrate / 8);
$trim -= $trim % ($track->block_alignment || 1);
substr($buffer, $length - 3*4, 4, pack('N', $track->audio_size - $trim));
substr($buffer, 4, 4, pack('N', unpack('N', substr($buffer, 4, 4) - $trim)));

return $buffer;
}

sub parseStream {
my ( $class, $dataref, $args ) = @_;

$args->{_scanbuf} .= $$dataref;
return -1 if length $args->{_scanbuf} < 32*1024;

my $fh = File::Temp->new();
$fh->write($args->{_scanbuf});
$fh->seek(0, 0);

my $info = Audio::Scan->scan_fh( aif => $fh )->{info};
$fh->truncate($info->{audio_offset});
$info->{fh} = $fh;

return $info;
}

*getCoverArt = \&Slim::Formats::MP3::getCoverArt;

sub canSeek { 1 }
Expand Down
209 changes: 207 additions & 2 deletions Slim/Formats/Movie.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,8 @@ sub _doTagMapping {
}
}

sub volatileInitialAudioBlock { 1 }

sub getInitialAudioBlock {
my ($class, $fh, $track, $time) = @_;

Expand All @@ -188,8 +190,12 @@ sub findFrameBoundaries {
if (!defined $fh || !defined $time) {
return 0;
}

my $info = Audio::Scan->find_frame_fh_return_info( mp4 => $fh, int($time * 1000) );

# I'm not sure why we need a localFh here ...
open(my $localFh, '<&=', $fh);
$localFh->seek(0, 0);
my $info = Audio::Scan->find_frame_fh_return_info( mp4 => $localFh, int($time * 1000) );
$localFh->close;

# Since getInitialAudioBlock will be called right away, stash the new seek header so
# we don't have to scan again
Expand All @@ -200,4 +206,203 @@ sub findFrameBoundaries {

sub canSeek { 1 }

sub parseStream {
my ( $class, $dataref, $args, $formats ) = @_;
return -1 unless defined $$dataref;

# stitch new data to existing buf and init parser if needed
$args->{_scanbuf} .= $$dataref;
$args->{_need} ||= 8;
$args->{_offset} ||= 0;

my $len = length($$dataref);
my $offset = $args->{_offset};
my $log = logger('player.streaming');

while (length($args->{_scanbuf}) > $args->{_offset} + $args->{_need} + 8) {
$args->{_atom} = substr($args->{_scanbuf}, $offset+4, 4);
$args->{_need} = unpack('N', substr($args->{_scanbuf}, $offset, 4));
$args->{_offset} = $args->{"_$args->{_atom}_"} = $offset;

# a bit of sanity check
if ($offset == 0 && $args->{_atom} ne 'ftyp') {
$log->warn("no header! this is supposed to be a mp4 track");
return 0;
}

$offset += $args->{_need};
main::DEBUGLOG && $log->is_debug && $log->debug("atom $args->{_atom} at $args->{_offset} of size $args->{_need}");

# mdat reached = audio offset & size acquired
if ($args->{_atom} eq 'mdat') {
$args->{_audio_size} = $args->{_need};
last;
}
}

return -1 unless $args->{_mdat_};

# now make sure we have acquired a full moov atom
if (!$args->{_moov_}) {
# no 'moov' found but EoF
if (!$len) {
$log->warn("no 'moov' found before EOF => track probably not playable");
return 0;
}

# already waiting for bottom 'moov', we need more
return -1 if $args->{_range};

# top 'moov' not found, need to seek beyond 'mdat'
$args->{_range} = $offset;
$args->{_scanbuf} = substr($args->{_scanbuf}, 0, $args->{_offset});
delete $args->{_need};
return $offset;
} elsif ($args->{_atom} eq 'moov' && $len) {
return -1;
}

# finally got it, add 'moov' size it if was last atom
$args->{_scanbuf} = substr($args->{_scanbuf}, 0, $args->{_offset} + ($args->{_atom} eq 'moov' ? $args->{_need} : 0));

# put at least 16 bytes after mdat or it confuses audio::scan (and header creation)
my $fh = File::Temp->new();
$fh->write($args->{_scanbuf} . pack('N', $args->{_audio_size}) . 'mdat' . ' ' x 16);
$fh->seek(0, 0);

my $info = Audio::Scan->scan_fh( mp4 => $fh )->{info};
$info->{fh} = $fh;
Comment on lines +269 to +274
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still believe that this file handle risks to never be released. The documentation for File::Temp says:

In a scalar context (where no filename is returned) the file is always deleted either (depending on the operating system) on exit or when it is closed

But when would this be closed? We don't close it actively. Would it be closed when the track object is discarded?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could use Slim::Player::Song::DESTROY to close the handle. Something like this:

diff --git a/Slim/Player/Song.pm b/Slim/Player/Song.pm
index 5db48cf7f..d965607f4 100644
--- a/Slim/Player/Song.pm
+++ b/Slim/Player/Song.pm
@@ -156,6 +156,12 @@ sub new {
 
 sub DESTROY {
        my $self = shift;
+
+       my $track = $self->_track;
+       if ( $track->can('initial_block_fh') && (my $fh = $track->initial_block_fh) ) {
+               $track->initial_block_fh(undef);
+               close $fh;
+       }
+
        $_liveCount--;
        if (main::DEBUGLOG && $log->is_debug)   {
                $log->debug(sprintf("DESTROY($self) live=$_liveCount: index=%d, url=%s", $self->index(), $self->_track()->url));

This effectively got rid of the temporary files. close $track->initial_block_fh did not work, for whatever reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think RemoteTrack::delete could be used instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also tried to add a DESTROY method to RemoteTrack but it does not seem to be called. I have to check the Song as well but I think I remember in a previous modification, when I was trying to understand how all this works, I tested this liveCount and I never saw it decreased

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RemoteTrack::delete() is more than that. I actually was wondering whether that information/file handle should belong to the song object rather than the track, as it's only relevant when the track is being played. The track object can be around for much longer, eg. when it's queued up, or after it had been played, it would still be part of the current playlist. But the file handle is not required at that point.
RemoteTrack's DESTROY probably isn't called for a long time, as these objects are kept around for a while for aforementioned reasons.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I think this fh is really an attribute of the track that must 'survive' between $song instances as the url might not be rescanned. So if I delete it, I won't be able to re-create it. And when I populate the track information, I might not have a song object. For example, scanUrl does not always have one when called, as it can scan multiple tracks, keep them ready and then create song when getNextTrack/Song is called. It's like for File.pm, the audio_offset and audio_size (e.g.) are attributes that are created independently of a song object. But I agree that we don't want to keep them around forever. I'm trying to find under which circumstances the $track object is destroyed (other than the call to delete which obviously happen when the track changes). At least, when LMS stops, things are cleaned, I verified :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to see whether RemoteTrack's DESTROY would be called or not you could reduce the size of %Cache in that file. It's initially set to 500 items, but in init()'s $maxPlaylistLengthCB you could set it to any small size to make sure it soon would need to run some cleanup.

And yes, I've seen LMS clean up behind, too. But some people listen to hundreds of tracks before they restart LMS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will try that. Maybe I could as well have a global that tracks the total size of these headers and kicks in some DESTROY when a max is reached?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I did not know it is possible to register a callback on a prefs change ... there are so many well thought features in LMS. I've discovered a couple of inconsistencies in codec management between pcm/wav/aif and mp4/aac but these are very minor compared to extremely well done global architecture, IMHO

$info->{audio_offset} = $args->{_mdat_} + 8;

# MPEG-4 audio = 64, MPEG-4 ADTS main = 102, MPEG-4 ADTS Low Complexity = 103
# MPEG-4 ADTS Scalable Sampling Rate = 104
if ($info->{tracks}->[0] && $info->{tracks}->[0]->{audio_type} == 64 && (!$formats || grep(/aac/i, @{$formats}))) {
$info->{audio_initiate} = \&setADTSProcess;
$info->{audio_format} = 'aac';
}

return $info;
}

sub setADTSProcess {
my ($bufref) = @_;
my $pos;
my $codec;
my %atoms = (
stsd => 16,
mp4a => 36,
meta => 12,
moov => 8,
trak => 8,
mdia => 8,
minf => 8,
stbl => 8,
utda => 8,
ilst => 8,
);

while ($pos < length $$bufref) {
my $len = unpack("N", substr($$bufref, $pos, 4));
my $type = substr($$bufref, $pos + 4, 4);
$pos += 8;

last if $type eq 'mdat';

if ($type eq 'esds') {
my $offset = 4;
last unless unpack("C", substr($$bufref, $pos + $offset++, 1)) == 0x03;
my $data = unpack("C", substr($$bufref, $pos + $offset, 1));
$offset += 3 if $data == 0x80 || $data == 0x81 || $data == 0xfe;
$offset += 4;
last unless unpack("C", substr($$bufref, $pos + $offset++, 1)) == 0x04;
$data = unpack("C", substr($$bufref, $pos + $offset, 1));
$offset += 3 if $data == 0x80 || $data == 0x81 || $data == 0xfe;
$offset += 14;
last unless unpack("C", substr($$bufref, $pos + $offset++, 1)) == 0x05;
$data = unpack("C", substr($$bufref, $pos + $offset, 1));
$offset += 3 if $data == 0x80 || $data == 0x81 || $data == 0xfe;
$offset++;
$data = unpack("N", substr($$bufref, $pos + $offset, 4));
$codec->{freq_index} = ($data >> 23) & 0x0f;
$codec->{object_type} = $data >> 27;
# Fix because Touch and Radio cannot handle ADTS header of AAC Main.
$codec->{object_type} = 2 if $codec->{object_type} == 5;
$codec->{channel_config} = ($data >> 19) & 0x0f;
$pos += $len - 8;
} elsif ($type eq 'stsz') {
my $offset = 4;
$codec->{frame_size} = unpack("N", substr($$bufref, $pos + $offset, 4));
if (!$codec->{frame_size}) {
$offset += 4;
$codec->{frames}= [];
$codec->{entries} = unpack("N", substr($$bufref, $pos + $offset, 4));
$offset += 4;
$codec->{frames} = [ unpack("N[$codec->{entries}]", substr($$bufref, $pos + $offset)) ];
if ($codec->{entries} != scalar @{$codec->{frames}}) {
logger('player.source')->warn("inconsistent stsz entries $codec->{entries} vs ", scalar @{$codec->{frames}});
$codec->{entries} = scalar @{$codec->{frames}};
}
}
$pos += $len - 8;
} else {
$pos += ($atoms{$type} || $len) - 8;
}

last if $codec->{frame_size} || $codec->{entries} && $codec->{channel_config};
}

# don't want to send a header when doing AAC demuxs
$$bufref = '';
return (\&extractADTS, $codec);
}

sub extractADTS {
my ($codec, undef, $chunk_size, $offset) = @_;
my $consumed = 0;
my @ADTSHeader = (0xFF,0xF1,0,0,0,0,0xFC);

$codec->{inbuf} .= substr($_[1], $offset);
$_[1] = substr($_[1], 0, $offset);

while ($codec->{frame_size} || $codec->{frame_index} < $codec->{entries}) {
my $frame_size = $codec->{frame_size} || $codec->{frames}->[$codec->{frame_index}];
last if $frame_size + $consumed > length($codec->{inbuf}) || length($_[1]) + $frame_size + 7 > $chunk_size;

$ADTSHeader[2] = (((($codec->{object_type} & 0x3) - 1) << 6) + ($codec->{freq_index} << 2) + ($codec->{channel_config} >> 2));
$ADTSHeader[3] = ((($codec->{channel_config} & 0x3) << 6) + (($frame_size + 7) >> 11));
$ADTSHeader[4] = ( (($frame_size + 7) & 0x7ff) >> 3);
$ADTSHeader[5] = (((($frame_size + 7) & 7) << 5) + 0x1f) ;

$_[1] .= pack("CCCCCCC", @ADTSHeader) . substr($codec->{inbuf}, $consumed, $frame_size);

$codec->{frame_index}++;
$consumed += $frame_size;
}

$codec->{inbuf} = substr($codec->{inbuf}, $consumed);
return length $codec->{inbuf};
}

# AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP
#
# Header consists of 7 bytes without CRC.
#
# Letter Length (bits) Description
# A 12 syncword 0xFFF, all bits must be 1
# B 1 MPEG Version: 0 for MPEG-4, 1 for MPEG-2
# C 2 Layer: always 0
# D 1 set to 1 as there is no CRC
# E 2 profile, the MPEG-4 Audio Object Type minus 1
# F 4 MPEG-4 Sampling Frequency Index (15 is forbidden)
# G 1 private bit, guaranteed never to be used by MPEG, set to 0 when encoding, ignore when decoding
# H 3 MPEG-4 Channel Configuration (in the case of 0, the channel configuration is sent via an inband PCE)
# I 1 originality, set to 0 when encoding, ignore when decoding
# J 1 home, set to 0 when encoding, ignore when decoding
# K 1 copyrighted id bit, the next bit of a centrally registered copyright identifier, set to 0 when encoding, ignore when decoding
# L 1 copyright id start, signals that this frame's copyright id bit is the first bit of the copyright id, set to 0 when encoding, ignore when decoding
# M 13 frame length, this value must include 7 bytes of header
# O 11 Buffer fullness
# P 2 Number of AAC frames (RDBs) in ADTS frame minus 1, for maximum compatibility always use 1 AAC frame per ADTS frame


1;
26 changes: 25 additions & 1 deletion Slim/Formats/Wav.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ sub getTag {
return $tags;
}

sub volatileInitialAudioBlock { 1 }

sub getInitialAudioBlock {
my ($class, $fh, $track) = @_;
my ($class, $fh, $track, $time) = @_;
my $length = $track->audio_offset() || return undef;

open(my $localFh, '<&=', $fh);
Expand All @@ -69,6 +71,11 @@ sub getInitialAudioBlock {
read ($localFh, my $buffer, $length);
seek($localFh, 0, 0);
close($localFh);

# adjust header size according to seek position
my $trim = int($time * $track->bitrate / 8);
$trim -= $trim % ($track->block_alignment || 1);
substr($buffer, $length - 4, 4, pack('V', $track->audio_size - $trim));

return $buffer;
}
Expand All @@ -88,6 +95,23 @@ sub doTagMapping {
}
}

sub parseStream {
my ( $class, $dataref, $args ) = @_;

$args->{_scanbuf} .= $$dataref;
return -1 if length $args->{_scanbuf} < 128;

my $fh = File::Temp->new();
$fh->write($args->{_scanbuf});
$fh->seek(0, 0);

my $info = Audio::Scan->scan_fh( wav => $fh )->{info};
$fh->truncate($info->{audio_offset});
$info->{fh} = $fh;

return $info;
}

*getCoverArt = \&Slim::Formats::MP3::getCoverArt;

sub canSeek { 1 }
Expand Down
14 changes: 6 additions & 8 deletions Slim/Player/Protocols/File.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ sub open {
$streamLength = $song->streamLength();
$seekoffset = $seekdata->{restartOffset};
} elsif ($seekdata->{sourceStreamOffset}) { # used for seeking
$seekoffset = $seekdata->{sourceStreamOffset} + $seekdata->{sourceHeaderOffset};
$seekoffset = $seekdata->{sourceStreamOffset};
$streamLength -= $seekdata->{sourceStreamOffset} - $offset;
} else {
$seekoffset = $offset; # normal case
Expand All @@ -151,7 +151,7 @@ sub open {
${*$sock}{'streamFormat'} = $args->{'transcoder'}->{'streamformat'};

if ( $seekoffset

&& !$song->stripHeader
# We do not need to worry about an initialAudioBlock when we are restarting
# as getSeekDataByPosition() will not have allowed a restart within the
# initialAudioBlock.
Expand All @@ -172,18 +172,16 @@ sub open {
main::DEBUGLOG && $log->debug("Got initial audio block of size $length");
if ($seekoffset <= $length) {
# Might as well just play from the start normally
$offset = $seekoffset = 0;
$streamLength = $size + $offset;
$offset = $seekoffset = 0;
} else {
${*$sock}{'initialAudioBlockRemaining'} = $length;
${*$sock}{'initialAudioBlockRef'} = \($song->initialAudioBlock());
$streamLength += $length;
}

# For MP4 files, we can't cache the audio block because it's different each time
if ($streamClass eq 'Slim::Formats::Movie') {
$song->initialAudioBlock(undef);
}
# For some files, we can't cache the audio block because it's different each time
$song->initialAudioBlock(undef) if $streamClass->can('volatileInitialAudioBlock') && $streamClass->volatileInitialAudioBlock($track);
}
}

Expand All @@ -192,7 +190,7 @@ sub open {
if (!defined(sysseek($sock, $seekoffset, 0))) {
logError("could not seek to $seekoffset for $filepath: $!");
} else {
$client->songBytes($seekoffset - ${*$sock}{'initialAudioBlockRemaining'});
$client->songBytes($seekoffset - ($song->stripHeader ? $song->offset : ${*$sock}{'initialAudioBlockRemaining'}));
${*$sock}{'position'} = $seekoffset;
if ($seekoffset > $offset && $seekdata && $seekdata->{'timeOffset'}) {
$song->startOffset($seekdata->{'timeOffset'});
Expand Down