Not Enumerating Apps via Docker and Swag

[Teflon98]

I've looked at #300 and #290.

I just installed Heimdall (in 2021) and I'm getting the same issue. I've tried what was suggested but to no avail.

When I used Docker Compose, I also noticed the last App\Jobs\ProcessApps fail as well.

Is this still a bug, or is there a set fix for this? The suggested solutions seem to vary and I can't seem to find the 'one' solution that fixed it for everyone.

Thank you!

[Martin-Schnaible]

I'm not sure if it's the same problem, but I can't install Heimdall since the update of the app "Cloud9". The files for Cloud9 are downloaded but it searches for the "Cloud" app.

[TheGamerFace]

Same issue here...

[Deathnerd]

I've tracked down this issue and it's related to the app names returned from https://apps.heimdall.site/list containing invalid PHP class name characters, specifically having digits. Once I modified ProcessApps.php to filter out the apps below, the job completes without issue

• Cloud9
• Tar1090
• n8n

A couple of simple fixes would be to wrap the contents of the loop inside ProcessApps.php with a try/catch or filter out any apps that have a name with a digit in it (which I tried to do with a preg_match but my PHP must be rusty because it matched everything). I think the real fix though is to enforce apps that are submitted to the repo to follow a schema where the name doesn't contain numbers.

[Deathnerd]

If anyone is curious and would like to apply my fix for their own installation, open the app/Jobs/ProcessApps.php and add the following code where appropriate

// Logging is optional
use Illuminate\Support\Facades\Log;
// ...snip...
  $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
  foreach($list->apps as $app) {
    if(in_array($app->name, $AppsToReject)) {
      Log::info('Rejected app '.$app->name); // Will write this to storage/logs/laravel.log
      continue;
    }
// ...snip...

And here's the whole file for reference:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Log;
use App\Application;
use App\SupportedApps;

class ProcessApps implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $localapps = Application::all();
        $list = json_decode(SupportedApps::getList()->getBody());
        $validapps = [];
        $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
        foreach($list->apps as $app) {
          Log::info('Considering app '.$app->name . ' type '.gettype($app->name));
          if(in_array($app->name, $AppsToReject)) {
            Log::info('Rejected app '.$app->name);
            continue;
          }
            $validapps[] = $app->appid;
            $localapp = $localapps->where('appid', $app->appid)->first();

            $application = ($localapp) ? $localapp : new Application;

            if(!file_exists(app_path('SupportedApps/'.className($app->name)))) {
                SupportedApps::getFiles($app);
                SupportedApps::saveApp($app, $application);
            } else {
                // check if there has been an update for this app
                $localapp = $localapps->where('appid', $app->appid)->first();
                if($localapp) {
                    if($localapp->sha !== $app->sha) {
                        SupportedApps::getFiles($app);
                        SupportedApps::saveApp($app, $application);
                    }
                }  else {
                    SupportedApps::getFiles($app);
                    SupportedApps::saveApp($app, $application);

                }
            }
        }
        //$delete = Application::whereNotIn('appid', $validapps)->delete(); // delete any apps not in list
        // removed the delete so local apps can be added

    }
}

If running in Docker, make sure you docker restart your container after saving the changes to the file back to the container. Then, run the app list update command again. docker logs -f heimdall while doing so will let you know if it succeeds. If it says the ProcessApps.php job failed, search for lines in the storage/logs/laravel.log file containing laravel.INFO (cat laravel.logs | grep 'laravel.INFO') and check which ones weren't rejected (should be the last line) and add it to the $AppsToReject variable, restart heimdall, and run again.

[Teflon98]

Thanks for the effort: I have edited the PHP file; restarted the container; tried the logs. All to no avail.

Still getting the Failed Job. Looking in laravel.log, I couldn't find and laravel.INFO lines in it.

Is there something I can post here that may shed some light?

Thank you again for your help.

Here's my PHP file.

I tried to insert what you posted above, but I don't follow why the code from the first part differs from the code that appears to have been inserted into the who PHP file. Also, I added a } after the continue as I was getting a PHP error. I may have totally put that in the wrong place.

`<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Log;
use App\Application;
use App\SupportedApps;

class ProcessApps implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

/**
 * Create a new job instance.
 *
 * @return void
 */
public function __construct()
{
    //
}

/**
 * Execute the job.
 *
 * @return void
 */
public function handle()
{
    $localapps = Application::all();
    $list = json_decode(SupportedApps::getList()->getBody());
    $validapps = [];
	$AppsToReject = ["Cloud9", "Tar1090", "n8n"];
	foreach($list->apps as $app) {
		if(in_array($app->name, $AppsToReject)) {
			Log::info('Rejected app '.$app->name); // Will write this to storage/logs/laravel.log
			continue;
			}
		}
	
    
    foreach($list->apps as $app) {
        $validapps[] = $app->appid;
        $localapp = $localapps->where('appid', $app->appid)->first();

        $application = ($localapp) ? $localapp : new Application;

        if(!file_exists(app_path('SupportedApps/'.className($app->name)))) {
            SupportedApps::getFiles($app);
            SupportedApps::saveApp($app, $application);
        } else {
            // check if there has been an update for this app
            $localapp = $localapps->where('appid', $app->appid)->first();
            if($localapp) {
                if($localapp->sha !== $app->sha) {
                    SupportedApps::getFiles($app);
                    SupportedApps::saveApp($app, $application);
                }
            }  else {
                SupportedApps::getFiles($app);
                SupportedApps::saveApp($app, $application);
  
            }
        }
    }
    //$delete = Application::whereNotIn('appid', $validapps)->delete(); // delete any apps not in list
    // removed the delete so local apps can be added

}

}
`

[igazka]

#596 (comment)

@Deathnerd THANKYOU!!!!!!!!! I have been looking for a solution for days now, this solved for me.

[alexbutoi]

@Deathnerd big thanks from me also, but any chance on getting back even more apps ?

root@880a7a5aaa7f:/var/www/localhost/heimdall/storage/logs# cat laravel.log | grep 'laravel.INFO' | grep -i rutorrent
[2021-06-17 20:47:54] laravel.INFO: Considering app ruTorrent type string
[2021-06-17 20:48:00] laravel.INFO: Considering app ruTorrent type string
[2021-06-17 20:48:16] laravel.INFO: Considering app ruTorrent type string
[2021-06-17 20:48:22] laravel.INFO: Considering app ruTorrent type string
[2021-06-17 20:48:26] laravel.INFO: Considering app ruTorrent type string
[2021-06-17 20:48:37] laravel.INFO: Considering app ruTorrent type string
root@880a7a5aaa7f:/var/www/localhost/heimdall/storage/logs#

while on the Application Type I don't see rutorrent listed

[alexbutoi]

my bad, sorry ...

Actually the list was growing, I do see now flood and rutorrent

@Deathnerd thank you very much

[mgoeppl]

Thanks for the fix @Deathnerd!
However I think that this kind of fixed the symptoms and not the problem, and would probably reset each time the container is recreated.
@devs is there an official fix coming?

[Deathnerd]

@Teflon98 have you tried copy/pasting the whole file in the second part of my reply?

Either way, you don't need two loops. The check needs to happen inside the for loop that already exists. Check out the full file in my post

[Deathnerd]

I wish I had more time to find a true solution for this, but I really only had a couple of hours to do anything with it today. Hopefully the devs can use this information to come up with a true fix.

Devs, if y'all wanna chime in with any information, I'll try to carve out some time to get a patch together based on your input for a possible fix. I don't know enough about the apps repository to make any kind of judgement call on a path of action on my own.

[SumitBajoria]

If anyone is curious and would like to apply my fix for their own installation, open the app/Jobs/ProcessApps.php and add the following code where appropriate

// Logging is optional
use Illuminate\Support\Facades\Log;
// ...snip...
  $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
  foreach($list->apps as $app) {
    if(in_array($app->name, $AppsToReject)) {
      Log::info('Rejected app '.$app->name); // Will write this to storage/logs/laravel.log
      continue;
    }
// ...snip...

And here's the whole file for reference:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Log;
use App\Application;
use App\SupportedApps;

class ProcessApps implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $localapps = Application::all();
        $list = json_decode(SupportedApps::getList()->getBody());
        $validapps = [];
        $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
        foreach($list->apps as $app) {
          Log::info('Considering app '.$app->name . ' type '.gettype($app->name));
          if(in_array($app->name, $AppsToReject)) {
            Log::info('Rejected app '.$app->name);
            continue;
          }
            $validapps[] = $app->appid;
            $localapp = $localapps->where('appid', $app->appid)->first();

            $application = ($localapp) ? $localapp : new Application;

            if(!file_exists(app_path('SupportedApps/'.className($app->name)))) {
                SupportedApps::getFiles($app);
                SupportedApps::saveApp($app, $application);
            } else {
                // check if there has been an update for this app
                $localapp = $localapps->where('appid', $app->appid)->first();
                if($localapp) {
                    if($localapp->sha !== $app->sha) {
                        SupportedApps::getFiles($app);
                        SupportedApps::saveApp($app, $application);
                    }
                }  else {
                    SupportedApps::getFiles($app);
                    SupportedApps::saveApp($app, $application);

                }
            }
        }
        //$delete = Application::whereNotIn('appid', $validapps)->delete(); // delete any apps not in list
        // removed the delete so local apps can be added

    }
}

If running in Docker, make sure you docker restart your container after saving the changes to the file back to the container. Then, run the app list update command again. docker logs -f heimdall while doing so will let you know if it succeeds. If it says the ProcessApps.php job failed, search for lines in the storage/logs/laravel.log file containing laravel.INFO (cat laravel.logs | grep 'laravel.INFO') and check which ones weren't rejected (should be the last line) and add it to the $AppsToReject variable, restart heimdall, and run again.

Thank you so, so, so much. Been trying to get this to work for the past week. You fixed it. Now, hopefully the changes persist and the developers fix this for good.

Thank you once again.

[Teflon98]

@Deathnerd ok... So I guess my copy/paste fu was weak.

I know (or at least I swore) I tried to replace the entire file before and it didn't work.

I tried again today and it worked!!

Thank you very much!

[sanmagal]

If anyone is curious and would like to apply my fix for their own installation, open the app/Jobs/ProcessApps.php and add the following code where appropriate

// Logging is optional
use Illuminate\Support\Facades\Log;
// ...snip...
  $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
  foreach($list->apps as $app) {
    if(in_array($app->name, $AppsToReject)) {
      Log::info('Rejected app '.$app->name); // Will write this to storage/logs/laravel.log
      continue;
    }
// ...snip...

And here's the whole file for reference:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Log;
use App\Application;
use App\SupportedApps;

class ProcessApps implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $localapps = Application::all();
        $list = json_decode(SupportedApps::getList()->getBody());
        $validapps = [];
        $AppsToReject = ["Cloud9", "Tar1090", "n8n"];
        foreach($list->apps as $app) {
          Log::info('Considering app '.$app->name . ' type '.gettype($app->name));
          if(in_array($app->name, $AppsToReject)) {
            Log::info('Rejected app '.$app->name);
            continue;
          }
            $validapps[] = $app->appid;
            $localapp = $localapps->where('appid', $app->appid)->first();

            $application = ($localapp) ? $localapp : new Application;

            if(!file_exists(app_path('SupportedApps/'.className($app->name)))) {
                SupportedApps::getFiles($app);
                SupportedApps::saveApp($app, $application);
            } else {
                // check if there has been an update for this app
                $localapp = $localapps->where('appid', $app->appid)->first();
                if($localapp) {
                    if($localapp->sha !== $app->sha) {
                        SupportedApps::getFiles($app);
                        SupportedApps::saveApp($app, $application);
                    }
                }  else {
                    SupportedApps::getFiles($app);
                    SupportedApps::saveApp($app, $application);

                }
            }
        }
        //$delete = Application::whereNotIn('appid', $validapps)->delete(); // delete any apps not in list
        // removed the delete so local apps can be added

    }
}

If running in Docker, make sure you docker restart your container after saving the changes to the file back to the container. Then, run the app list update command again. docker logs -f heimdall while doing so will let you know if it succeeds. If it says the ProcessApps.php job failed, search for lines in the storage/logs/laravel.log file containing laravel.INFO (cat laravel.logs | grep 'laravel.INFO') and check which ones weren't rejected (should be the last line) and add it to the $AppsToReject variable, restart heimdall, and run again.

i've searched the config folder and can't find the app/Jobs/ProcessApps.php file.
could anyone point me to the right location?

[KodeStar]

I think this was already fixed a while ago, but if not 2.3.0 has code that should allow numbers in app names so shouldn't be an issue anyway