Error in file permissions

[kkz6]

[2] An error occurred in file /var/www/html/app/vendor/duzun/hquery/src/hQuery.php on line 629: filemtime(): stat failed

I am using custom framework with blade templating engine. The cache files for blade is getting stored without any issues. But getting this error in your plugin. I have set the default permission for the cache folder as rwx for all users. But still getting error

[duzun]

hQuery does nothing special with cache, just creates, writes and reads files in a folder, with locks. Make sure the user or group under which PHP is run has write access to the cache folder.

I suppose you have something like:

hQuery::$cache_path = "/path/to/cache/";

Try to change it to

hQuery::$cache_path = "/path/to/cache/hquery/"

then chmod 777 "/path/to/cache/hquery/" (rwx for everyone).

This way you guaranty a writeable cache folder for hQuery, no matter under which *nix user it is run.

If this works, check the owner (and group) of the cache files created by hQuery (uid/gid). I hope this would give you an insight into what the permissions should be for your cache folder.

When you know the right user (uid) and group (gid),
change the owner of cache folder: chown uid:gid "/path/to/cache/hquery/",
and set permissions to rwx for user/group only: chmod 775 "/path/to/cache/hquery/".

[kkz6]

Did the same as you said.. I am getting two error

[2] An error occurred in file /var/www/testing/app/vendor/duzun/hquery/src/hQuery.php on line 629: filemtime(): stat failed for /var/www/testing/app/core/bootstrap/scrape/hquery/bd3febb5710fbaea6f04389d52e039e8b2e0cfce.gz

[2] An error occurred in file /var/www/testing/app/vendor/duzun/hquery/src/hQuery.php on line 684: mkdir(): File exists

[duzun]

Do you know the uid and/or gid pf the PHP process hQuery is run under?
Or maybe you run hQuery from the command line (CLI)?
Did you set the owner of /var/www/testing/app/core/bootstrap/scrape/hquery/ folder to that of the PHP process? And also set the owner of files in this folder (chown -R ...) or just remove those files.
The second error is not that important. That line just tries to make sure the cache folder exists. I'm thinking of changing this line.

There might be other causes too, depending on the platform.
What OS are you using?
What filesystem is /var/www/testing/app/core/bootstrap/scrape/hquery/ on? (ext4/NTFS/FAT/...)
What software do you use to run PHP? (Apache, PHP-FPM, PHP-CLI etc)

[kkz6]

the gid and uid is www-data.
The owner of the /var/www/testing/app/core/bootstrap/scrape/hquery/ folder is www-data

The second error is not that important.

this thing is raising errors. So I am getting email like there is some error and the program just stops there.

I am using Ubuntu Server. Running PHP on apache2. So now you also know what is the filesystem there.

[duzun]

Well, there might be one of many other causes to the error.
The command filemtime('/var/www/testing/app/core/bootstrap/scrape/hquery/bd3febb5710fbaea6f04389d52e039e8b2e0cfce.gz') should not fail when everything is set up correctly. And it is not specific to hQuery.

[kkz6]

I felt the same. But I found one thing. Creating the files by hQuery has only read and write permissions. I checked umask and the default file creation permission. But everything is perfect there. So you have any idea on updating the file permission to also execute then will it work.
Or my answer is totally irrelevant please let me know.

Also, remove another error that is being generated by the program.

[duzun]

hQuery creates cache files with fopen($fn, 'c+'), thus the file permissions are set by PHP (I guess the default is rw only), but that should be enough. Why x on a data file?

Also, make sure hQuery::$cache_path does not contain symlinks, use something like hQuery::$cache_path = realpath('/path/to/cache/').

See v2.2.3 for second error fix.

[kkz6]

Ok. As I said previously I was not sure about this x on data file. But its so clear the files are created with file permissions as mentioned by you.
I get this error when I use to sent the error reports in email.

I use to the following code to trigger this thing
`//set custom error handler
set_error_handler([new \App\Classes\ErrorHandler(), 'handleErrors'], E_ALL | E_STRICT);

//Fatal custom error handler
register_shutdown_function([new \App\Classes\ErrorHandler(), 'handleFatal']);`

and the class that sends the email is
`<?php

namespace App\Classes;

class ErrorHandler
{

public function handleFatal()
{

    $last_error = error_get_last();
    if ($last_error['type'] === E_ERROR) {
        // fatal error
        $this->handleErrors(E_ERROR, $last_error['message'], $last_error['file'], $last_error['line']);
    }

}

public function handleErrors($error_number, $error_message, $error_file, $error_line)
{
    $error = "[{$error_number}] An error occurred in file
                {$error_file} on line $error_line: $error_message";

    $environment = APP_ENV;

    if ($environment === 'development') {
        $whoops = new \Whoops\Run;
        $whoops->pushHandler(new \Whoops\Handler\PrettyPageHandler);
        $whoops->register();
    } else {
        ini_set("display_errors", "off");
        $data = [
            'to' => DEV_EMAIL,
            'subject' => 'System Error - AMZ One Step',
            'view' => 'errors',
            'name' => 'Admin',
            'body' => $error
        ];
        ErrorHandler::emailAdmin($data)->outputFriendlyError();
    }
}

public function outputFriendlyError()
{
    ob_end_clean();
    view('errors/generic');
    exit;
}

public static function emailAdmin($data)
{
    $mail = new Mail();
    $mail->send($data);
    return new static;
}

}`

with this I am getting error in APP_ENV = "production" only.
But the program actually runs.

since I have added exit() after outputFriendlyError(); this makes the program to stop working. So when I remove it. I get the error emails. but the program works..

Thats why I am not sure how this is happening here.

anywhy I will try the realpath() thing and update you.

[duzun]

I think I know exactly what is wrong now :-)
I use to hide notice errors in production:

error_reporting(E_ALL & ~E_NOTICE);

Judging from your code, I can tell that you catch E_NOTICE and exit; where I try to hide it.
Often times I use something like this:

$fm = @filemtime($fn);
if ( $fm ) { // file exists
	if ( $fm + $expires > time() ) { // but expired
		// ... 
	}
	else { // and not expired
		// ...
	}
} 

Did you notice the @ sign before filemtime? I use this pattern instead of $fm = file_exists($fn) ? filemtime($fn) : false;, this way I avoid one extra call to filesystem. filemtime returns false when the file doesn't exist and triggers an E_NOTICE, which should be quieted by @, but your custom error handler catches it.

I'm not saying your choice to catch E_NOTICE is bad or good, but I would not stop program execution because of an E_NOTICE. I suppose many other libraries out there would trigger an E_NOTICE for one reason or another.

[kkz6]

Ok then your library is great so for sure I am going to use your library. And yea I found this @ before some functions in your library. I actually had no idea about it. So I just ignored it. If I have searched on google then I might have found some answers previously. So to use your library I have to update the code by removing E_NOTICE from the snippet. That's the solutions for it right?

[duzun]

So to use your library I have to update the code by removing E_NOTICE from the snippet. That's the solutions for it right?

Not exactly... not that it’s my intention.
I’ve written this code long time ago, when I was still a student, thus it lack some rigors in terms of code style. Actually performance was my only focus, thus some tricks and shortcuts. I wish there were no E_NOTICE at all.

[kkz6]

Hy Awesome man. You have updated the code too and yea I am not getting any errors emails too. Great job and yours is a really good scraper.
One small advice instead of the dom crawler that you are currently using you can use symphony DOM crawler. That is much better. I am closing the issue