Scheduler : Large (longer) jobs, multiple tenants, stops.

[BartMommens]

Hi,

I'm not 100% sure if it's a bug or just something i'm doing wrong.
I have an issue with a cronjob running the Laravel scheduler.

Info:
The Cronjob runs every minute like described in the laravel docs :
app/production/current && php artisan tenants:artisan schedule:run >> /tmp/export-job-prod.log 2>&1

This will trigger the run schedule of laravel and it will start executing the jobs:

namespace App\Console;

use App\Jobs\CloseExpiredWorkSessionsJob;
use App\Jobs\ExportAllCSVDataToGoogleDriveJob;
use App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob;
use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
use App\Models\Tenant\Tenant;

class Kernel extends ConsoleKernel
{
    /**
     * The Artisan commands provided by your application.
     *
     * @var array
     */
    protected $commands = [
        //
    ];

    /**
     * Define the application's command schedule.
     *
     * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
     * @return void
     */
    protected function schedule(Schedule $schedule)
    {
        if(Tenant::current()){
            //SCHEDULES CAN ONLY BE ADJUST ON STAGING
            $schedule->job(ExportAllCSVDataToGoogleDriveJob::class)->dailyAt('2:00');
            $schedule->job(ExportLmraOverviewCSVDataToGoogleDriveJob::class)->hourly();
            $schedule->job(CloseExpiredWorkSessionsJob::class)->hourly();
        }
    }

    /**
     * Register the commands for the application.
     *
     * @return void
     */
    protected function commands()
    {
        $this->load(__DIR__.'/Commands');

        require base_path('routes/console.php');
    }
}

Let's say it's now 2:00 and the cronjob is triggered:

When it starts it loops over all 7 tenants one by one and executes all 3 jobs per tenant.
The largest job is the ExportAllCSVDataToGoogleDriveJob::class and depending on the client it take up somewhere from 10 upto 40 seconds. The first 3 or 4 tenant jobs all are being executed , the 4th tenant is a rather big one and sometimes it takes upto 40seconds.
So when it hits the 5th to 7th client, nothing gets executed anymore because by that time it is already 2:01 and it just doesn't execute the "hourly" jobs anymore.

Log below:

Running command for tenant (id: 1)...
---------------------------------------------------------

  2023-09-22 2:00:01 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  14,647ms DONE


Running command for tenant (id: 2)...
---------------------------------------------------------

  2023-09-22 2:00:16 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  17,467ms DONE


Running command for tenant (id: 3)...
---------------------------------------------------------

  2023-09-22 2:00:33 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  17,963ms DONE


Running command for tenant (id: 4)...
---------------------------------------------------------

  2023-09-22 2:00:51 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  20,772ms DONE


Running command for tenant (id: 5)...
---------------------------------------------------------



Running command for tenant  (id: 6)...
---------------------------------------------------------



Running command for tenant (id: 7)...
---------------------------------------------------------

When it runs at other times and does not execute the large job per tenant the other two jobs are successfully executed due to the fact that it runs under a minute.

When i change all jobs to run ->everyMinute() and only execute the cron job every hour everything works fine since it always hits the jobs and the "condition" every minute still stands.

Running command for tenant (id: 1)...
---------------------------------------------------------

  2023-09-22 08:10:01 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  14,761ms DONE
  2023-09-22 08:10:16 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,671ms DONE
  2023-09-22 08:10:18 Running [App\Jobs\CloseExpiredWorkSessionsJob]  4ms DONE


Running command for tenant (id: 2)...
---------------------------------------------------------

  2023-09-22 08:10:18 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  16,942ms DONE
  2023-09-22 08:10:34 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,666ms DONE
  2023-09-22 08:10:36 Running [App\Jobs\CloseExpiredWorkSessionsJob]  3ms DONE


Running command for tenant (id: 3)...
---------------------------------------------------------

  2023-09-22 08:10:36 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  16,857ms DONE
  2023-09-22 08:10:53 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,987ms DONE
  2023-09-22 08:10:55 Running [App\Jobs\CloseExpiredWorkSessionsJob]  3ms DONE


Running command for tenant (id: 4)...
---------------------------------------------------------

  2023-09-22 08:10:55 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  20,195ms DONE
  2023-09-22 08:11:15 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,698ms DONE
  2023-09-22 08:11:17 Running [App\Jobs\CloseExpiredWorkSessionsJob]  3ms DONE


Running command for tenant (id: 5)...
---------------------------------------------------------

  2023-09-22 08:11:17 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  16,180ms DONE
  2023-09-22 08:11:33 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  2,120ms DONE
  2023-09-22 08:11:35 Running [App\Jobs\CloseExpiredWorkSessionsJob]  4ms DONE


Running command for tenant (id: 6)...
---------------------------------------------------------

  2023-09-22 08:11:35 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  35,580ms DONE
  2023-09-22 08:12:11 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,709ms DONE
  2023-09-22 08:12:13 Running [App\Jobs\CloseExpiredWorkSessionsJob]  4ms DONE


Running command for tenant (id: 7)...
---------------------------------------------------------

  2023-09-22 08:12:13 Running [App\Jobs\ExportAllCSVDataToGoogleDriveJob]  22,672ms DONE
  2023-09-22 08:12:35 Running [App\Jobs\ExportLmraOverviewCSVDataToGoogleDriveJob]  1,831ms DONE
  2023-09-22 08:12:37 Running [App\Jobs\CloseExpiredWorkSessionsJob]  4ms DONE

So i'm not sure it's a bug or i'm just using the wrong approach here. Is there a way to run the cron task every minute but use the laravel ->hourly() functionality without it stopping on tenants that get executed on hour:01?

Thanks in advance.

Best regards,
Bart M.