Repository navigation
Send partial content is not possible with BinaryFileResponse with memory_limit less than filesize #48692
Description
Symfony version(s) affected
6.1.7
Description
Hello. I got OutOfMemroy in BinaryFileResponse in stream_copy_to_stream function.
I try to send partial content with Accept-Range bytes but it doesn't work.
I have memory_limit 256M in php.ini and i have file with size >= 512Mb. When i try to send response
return new BinaryFileResponse('path_to_mp4_video_fil_greater_memory_limit');I got
Fatal error: Allowed memory size of X bytes exhausted (tried to allocate X bytes) in /var/www/html/vendor/symfony/http-foundation/BinaryFileResponse.php on line 323
What i expect?
I expect that response will use Range headers and will read file by chunkSize. And i see it in source code
// symfony/vendor/symfony/http-foundation/BinaryFileResponse.php
elseif ($request->headers->has('Range') && $request->isMethod('GET')) {
// Process the range headers.
if (!$request->headers->has('If-Range') || $this->hasValidIfRangeHeader($request->headers->get('If-Range'))) {
$range = $request->headers->get('Range');
if (str_starts_with($range, 'bytes=')) {....I tried to manually set Content-Type to video/mp4 and manually set Accept-Range like this
$r = new BinaryFileResponse($file->filePath);
$r->headers->set('content-type', 'video/mp4');
$r->headers->set('accept-ranges', 'bytes');
// i expected that BinaryFileReponse has own logic to create Content-Range headers and Accept-Ranges based on current memory_limit or something else, but it doesn't
return $r;but not works for me.
I did't know there is bug or not, but look's like buggy/unexpected behavior. It could use check for memory limit to switch "Range mode" if filesize >= memory_limit. And i think that HTTP-header response to Range header in BinaryFileResponse now incorrect, because it doesn't return Content-Range to "bytes=0-"
curl -v -H "Range: bytes=0-"
no Content-Range returned
How to reproduce
First.
bin/console make:controller
Second create/get mp4 file with size > php memory_limit and put it to any not public directory.
Then in controller
return new BinaryFileResponse('path_to_mp4_video_fil_greater_memory_limit');Possible Solution
I create own custom class for this case
class StreamedFileResponse extends Response
{
private ?Request $request;
private ?int $start;
private ?int $end;
private ?int $chunkSize;
public function prepare(Request $request): static
{
$this->request = $request;
parent::prepare($request);
$this->headers->set('Accept-Ranges', 'bytes');
$this->headers->set('Content-Type', 'video/mp4'); //<!-- FIX TODO:
$this->setStatusCode(206);
$fileSize = filesize($this->content);
$this->headers->set('Content-Length', $fileSize);
$start = 0;
$end = $fileSize - 1;
if($this->request->headers->has('Range'))
{
$range = $this->request->headers->get('Range');
if (str_starts_with($range, 'bytes='))
{
$rangeExp = array_filter(explode('bytes=', $range));
[$start, $end] = explode('-', implode('', $rangeExp));
$start = (int) $start;
$end = (int) $end;
if($start > $end || empty($end))
{
$end = $fileSize - 1;
}
}
$this->headers->set('Content-Range', "bytes {$start}-{$end}/{$fileSize}");
}
$this->start = $start;
$this->end = $end;
$this->chunkSize = $this->start - $this->end <= 0 ? 1024 : $this->start - $this->end;
return $this;
}
/**
* Sends content for the current web response.
*
* @return $this
*/
public function sendContent(): static
{
$fd = fopen($this->content, 'r');
// move to begining
fseek($fd, $this->start);
while(!feof($fd))
{
$content = fread($fd, $this->chunkSize);
echo $content;
flush();
}
fclose($fd);
return $this;
}
}Additional Context
No response
Activity
xabbuh commentedon Dec 17, 2022
Maybe it would be better if we didn't use
stream_copy_to_stream()inBinaryFileResponseas it seems to load the full file into memory when copying.mw3e21a commentedon Dec 19, 2022
I couldn't reproduce your problem. I didn't get
memory Allowed memory size of X bytes exhausted.BinaryFileResponse supports the Range header out of the box. When you send a request with a Range header, the response will include the status code 206, and the body of the response will contain the requested range of bytes from the file.
In your curl snippet:
The value Range: bytes=0- specifies that the client wants to retrieve all bytes from the start of the resource to the end. That's why no Content-Range header is returned
For example try with:
The value Range: bytes=1- specifies that the client wants to retrieve all bytes from the second byte of the resource to the end. In this case you will get Content-Range: bytes 1-Content-Length/Content-Length header
As default when you do not provide Range related headers
maxlenproperty inBinaryFileResponseis equal to -1, so it means that the entire file will be read and returned.ivanbogomoloff commentedon Dec 19, 2022
I'm demonstrating it in screencast https://www.youtube.com/watch?v=OR1bHGQUPVU
Only you need just mp4 file greater than
php.memory_limitin my screecast memory_limit is 128mb. Also i shown that google-chrome browser sendsRange: bytes=0-and it doesn't matter, because response down instream_copy_to_streamline.So, i decided that
BinaryFileResponseworks incorrect.Yes, and in my opinoin
BinaryFileResponsein this case must use cheking formemory_limitand replace empty value tominimum chunkand send it to client. But now it makesOutOfMemoryI try like a browser -> google-chrome sends
bytes=0-=)Fix BinaryFileResponse for huge files
Fix memory limit problems in BinaryFileResponse
glady commentedon Jan 13, 2023
We expected this issue too, the $this->maxlen=1 handling is not compatible, as chunked reading of the file will not be applied because of the $read is determined based on $length > $this->chunkSize. If this is -1 its just ignored and reads / copies the whole file at once without chunks.
I added pull request for 5.4 branch.
glady commentedon Jan 18, 2023
As commented in #48972 - the memory_limit problem occurs only, if there is any output_buffering active. (see #48972 (comment)) We had two levels and just closed one before we called sendContent() and it was not visible for us, that this is sent to memory. Maybe BinaryFileResponse close them all or just throw an error if any output buffering is active?
EDIT: it looks like its dependent on the buffer&chunk size of the output buffer, our second one (ob_get_status(true);) has chunk size 0
bug #48972 [HttpFoundation] Fix memory limit problems in BinaryFileRe…