Stream File Download with Octane/Swoole gives blank file #260

dbelyaeff · 2021-05-03T18:57:35Z

Octane Version: v0.4.0
Laravel Version: 8.40.0
PHP Version: 8.0.3
Laravel Sail: 1.5
Server: Swoole
Database Driver & Version:

Description:

Running file downloading from remote server gives blank file (0kb) when octane is enabled.

public function download(Video $video)
    {
        return response()->streamDownload(function () use ($video) {
            echo file_get_contents($video->mp3);
        }, basename($video->mp3));
    }

Switching back to sail artisan server works fine – full length file is downloaded.

Steps To Reproduce:

Install Laravel.
Install Sail.
Install Octane for Sail via Octane github docs.
Create simple route with callback like the given one upwards.
Try to stream any remote file (img, whatever).
Blank file is downloaded (0 bytes).
Switch back from octane to standard Sail/Laravel artisan server.
Now remote file is downloaded via stream correctly, w/full size.

nunomaduro · 2021-05-04T16:57:24Z

I don't have Sail currently installed - yet, without sail seems to work as expected when simply using octane:start with both RoadRunner and Swoole. Is someone else able to reproduce this issue?

sy-records · 2021-05-05T02:27:56Z

What swoole configuration items have you set

dbelyaeff · 2021-05-05T08:35:22Z

I've basically followed the instructions provided via Octanes github README.md.
So here's a part of supervisord.conf:

command=/usr/bin/php -d variables_order=EGPCS /var/www/html/artisan octane:start --watch --server=swoole --host=0.0.0.0 --port=80

nunomaduro · 2021-05-07T16:24:24Z

I am sorry - but I was unable to reproduce your issue, even after following the "Steps To Reproduce" in your issue description.

If other developers are facing the same issue please let me know, meanwhile I believe this must be something related to your environment/setup.

fridzema · 2021-05-19T13:17:04Z

@nunomaduro I am currently facing the same issue altough a little bit different.

laravel/framework v8.42.0 
laravel/octane v1.0.1
php 8.0.5
Ubuntu 20.04.2 (forge provisioned)

php --ri swoole

Swoole => enabled
Author => Swoole Team <team@swoole.com>
Version => 4.6.5
Built => Apr 13 2021 11:47:21
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
pcre => enabled
zlib => 1.2.11
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

I am streaming basic pdf files that come from a third party api. Assets around ~80mb gives me a 0kb "corrupt" file, where the smaller files download just as expected.

    return response()->streamDownload(function () use ($file_data) {
      echo base64_decode($file_data['data']);
    }, $file_data['file_name']);

First things i tried myself before i found this issue, increasing php_memory, disabled opcache etc. with no succes.
When i was looking in the logs i stumbled upon the following interesting values package_max_length and socket_buffer_size :

{
    "masterProcessId": 1654,
    "managerProcessId": 1733,
    "state": {
        "appName": "DEV",
        "host": "127.0.0.1",
        "port": "8000",
        "workers": 2,
        "taskWorkers": 2,
        "maxRequests": "500",
        "publicPath": "\/home\/forge\/DOMAIN\/public",
        "storagePath": "\/home\/forge\/DOMAIN\/storage",
        "defaultServerOptions": {
            "buffer_output_size": 10485760,
            "enable_coroutine": false,
            "daemonize": false,
            "log_file": "\/home\/forge\/DOMAIN\/storage\/logs\/swoole_http.log",
            "log_level": 5,
            "max_request": "500",
            "package_max_length": 10485760,
            "reactor_num": 2,
            "send_yield": true,
            "socket_buffer_size": 10485760,
            "task_max_request": "500",
            "task_worker_num": 2,
            "worker_num": 2
        },
        "octaneConfig": {
            "server": "swoole",
            "https": true,
            "listeners": {
                "Laravel\\Octane\\Events\\WorkerStarting": [
                    "Laravel\\Octane\\Listeners\\EnsureUploadedFilesAreValid"
                ],
                "Laravel\\Octane\\Events\\RequestReceived": [
                    "Laravel\\Octane\\Listeners\\CreateConfigurationSandbox",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToAuthorizationGate",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToBroadcastManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToDatabaseManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToHttpKernel",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToMailManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToNotificationChannelManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToPipelineHub",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToQueueManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToRouter",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToValidationFactory",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToViewFactory",
                    "Laravel\\Octane\\Listeners\\FlushDatabaseRecordModificationState",
                    "Laravel\\Octane\\Listeners\\FlushArrayCache",
                    "Laravel\\Octane\\Listeners\\PrepareInertiaForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareScoutForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareSocialiteForNextOperation",
                    "Laravel\\Octane\\Listeners\\FlushLocaleState",
                    "Laravel\\Octane\\Listeners\\FlushQueuedCookies",
                    "Laravel\\Octane\\Listeners\\FlushSessionState",
                    "Laravel\\Octane\\Listeners\\FlushAuthenticationState",
                    "Laravel\\Octane\\Listeners\\EnforceRequestScheme",
                    "Laravel\\Octane\\Listeners\\EnsureRequestServerPortMatchesScheme",
                    "Laravel\\Octane\\Listeners\\GiveNewRequestInstanceToApplication"
                ],
                "Laravel\\Octane\\Events\\RequestHandled": [],
                "Laravel\\Octane\\Events\\RequestTerminated": [],
                "Laravel\\Octane\\Events\\TaskReceived": [
                    "Laravel\\Octane\\Listeners\\CreateConfigurationSandbox",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToAuthorizationGate",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToBroadcastManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToDatabaseManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToHttpKernel",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToMailManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToNotificationChannelManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToPipelineHub",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToQueueManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToRouter",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToValidationFactory",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToViewFactory",
                    "Laravel\\Octane\\Listeners\\FlushDatabaseRecordModificationState",
                    "Laravel\\Octane\\Listeners\\FlushArrayCache",
                    "Laravel\\Octane\\Listeners\\PrepareInertiaForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareScoutForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareSocialiteForNextOperation"
                ],
                "Laravel\\Octane\\Events\\TickReceived": [
                    "Laravel\\Octane\\Listeners\\CreateConfigurationSandbox",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToAuthorizationGate",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToBroadcastManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToDatabaseManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToHttpKernel",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToMailManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToNotificationChannelManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToPipelineHub",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToQueueManager",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToRouter",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToValidationFactory",
                    "Laravel\\Octane\\Listeners\\GiveNewApplicationInstanceToViewFactory",
                    "Laravel\\Octane\\Listeners\\FlushDatabaseRecordModificationState",
                    "Laravel\\Octane\\Listeners\\FlushArrayCache",
                    "Laravel\\Octane\\Listeners\\PrepareInertiaForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareScoutForNextOperation",
                    "Laravel\\Octane\\Listeners\\PrepareSocialiteForNextOperation"
                ],
                "Laravel\\Octane\\Contracts\\OperationTerminated": [
                    "Laravel\\Octane\\Listeners\\FlushTemporaryContainerInstances",
                    "Laravel\\Octane\\Listeners\\DisconnectFromDatabases",
                    "Laravel\\Octane\\Listeners\\CollectGarbage"
                ],
                "Laravel\\Octane\\Events\\WorkerErrorOccurred": [
                    "Laravel\\Octane\\Listeners\\ReportException",
                    "Laravel\\Octane\\Listeners\\StopWorkerIfNecessary"
                ],
                "Laravel\\Octane\\Events\\WorkerStopping": []
            },
            "warm": [
                "auth",
                "cache",
                "cache.store",
                "config",
                "cookie",
                "db",
                "db.factory",
                "encrypter",
                "files",
                "hash",
                "log",
                "router",
                "routes",
                "session",
                "session.store",
                "translator",
                "url",
                "view"
            ],
            "flush": [],
            "cache": {
                "rows": 1000,
                "bytes": 10000
            },
            "tables": {
                "example:1000": {
                    "name": "string:1000",
                    "votes": "int"
                }
            },
            "watch": [
                "app",
                "bootstrap",
                "config",
                "database",
                "public\/**\/*.php",
                "resources\/**\/*.php",
                "routes",
                "composer.lock",
                ".env"
            ],
            "garbage": 50,
            "max_execution_time": 10
        }
    }

Is there anybody who know how to increase the package_max_length and socket_buffer_size as i could only find it "hardcoded" in StartSwooleCommand.php or am i looking in the wrong direction?

sy-records · 2021-05-20T02:16:29Z

@fridzema

config/octane.php

    'swoole' => [
        'options' => [
        ],
    ],

fridzema · 2021-05-20T13:50:01Z

@sy-records Looks like it is a step in the right direction!

Had to increase the values to a massive ammount:

  'swoole' => [
    'options' => [
      'package_max_length' => 100000000,
      'socket_buffer_size' => 100000000,
      'buffer_output_size' => 100000000,
    ],
  ],

The weird thing i am facing right now is that sometimes it works, and sometimes i get after around ~30mb suddenly a network error. Very strange...

Currently trying to reproduce it an seperate repo to share here.

fridzema · 2021-05-20T15:03:43Z

i have setup a repo that demonstrates the problem:
https://github.com/fridzema/stream-download-octane-sample

I also made a short clip to demonstrate:
https://user-images.githubusercontent.com/8180660/119002043-ce24a480-b98c-11eb-8689-56de50f0fa11.mp4

The sample files i used for testing:
https://github.com/fridzema/stream-download-octane-sample/releases/download/0.0.1/sample-files.zip

sy-records · 2021-05-21T03:36:45Z

@fridzema The problem was reproduced and this option was removed to fix it.

#286 After the merge, you can upgrade Swoole to v4.6.7 without setting 'buffer_output_size' => 100000000,

Fix #260

nunomaduro added the needs more info label May 4, 2021

nunomaduro closed this as completed May 7, 2021

sy-records mentioned this issue May 21, 2021

Remove buffer_output_size from Swoole's default options #286

Merged

taylorotwell pushed a commit that referenced this issue May 21, 2021

Remove buffer_output_size from Swoole's default options (#286)

9683607

Fix #260

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stream File Download with Octane/Swoole gives blank file #260

Stream File Download with Octane/Swoole gives blank file #260

dbelyaeff commented May 3, 2021

nunomaduro commented May 4, 2021

sy-records commented May 5, 2021

dbelyaeff commented May 5, 2021 •

edited

Loading

nunomaduro commented May 7, 2021

fridzema commented May 19, 2021

sy-records commented May 20, 2021

fridzema commented May 20, 2021

fridzema commented May 20, 2021

sy-records commented May 21, 2021

Stream File Download with Octane/Swoole gives blank file #260

Stream File Download with Octane/Swoole gives blank file #260

Comments

dbelyaeff commented May 3, 2021

Description:

Steps To Reproduce:

nunomaduro commented May 4, 2021

sy-records commented May 5, 2021

dbelyaeff commented May 5, 2021 • edited Loading

nunomaduro commented May 7, 2021

fridzema commented May 19, 2021

sy-records commented May 20, 2021

fridzema commented May 20, 2021

fridzema commented May 20, 2021

sy-records commented May 21, 2021

dbelyaeff commented May 5, 2021 •

edited

Loading