Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

avoid rebuilding cache during upload #1367

Open
plegall opened this issue Mar 16, 2021 · 5 comments
Open

avoid rebuilding cache during upload #1367

plegall opened this issue Mar 16, 2021 · 5 comments
Assignees
Milestone

Comments

@plegall
Copy link
Member

plegall commented Mar 16, 2021

As discussed on https://piwigo.org/forum/viewtopic.php?id=28338 we have a problem on high volume galleries when an upload is running and each photo is uploaded very quickly. The problem comes from the cache regeneration.

In include/user.inc.php we already have:

$user = build_user( $user['id'],
          ( defined('IN_ADMIN') and IN_ADMIN ) ? false : true // use cache ?
         );

Meaning we avoided to regenerate cache if we were in the administration. That was perfectly valid when we didn't use the API to upload photos. Now that we use pwg.images.upload (and other methods) we skip this IN_ADMIN test and the cache is regenerated each time we call ws.php. We should replace it with something like:

$use_cache = true;
if (defined('IN_ADMIN') and IN_ADMIN)
{
  $use_cache = false;
}

if (isset($_REQUEST['method']) and 'pwg.images.upload' == $_REQUEST['method'])
{
  $use_cache = false;
}

$user = build_user( $user['id'], $use_cache);
@plegall
Copy link
Member Author

plegall commented Mar 16, 2021

I'm currently testing it on a few Piwigo.com servers :-)

@plegall plegall self-assigned this Mar 16, 2021
@plegall plegall added this to the 12.0.0beta1 milestone Mar 16, 2021
@plegall
Copy link
Member Author

plegall commented Mar 18, 2021

The test shows a problem. During Community upload, the first photo is correctly uploaded, but on next photo:

Warning:  [mysql error 1064] You have an error in your SQL syntax;
    AND level > 8
    AND level <= 8
  GROUP BY category_id' at line 5

SELECT
    category_id
  FROM image_category
    JOIN images ON image_id = id
  WHERE category_id IN ()
    AND level > 8
    AND level <= 8
  GROUP BY category_id
; in include/dblayer/functions_mysqli.inc.php on line 856

@plegall plegall closed this as completed Mar 18, 2021
@plegall plegall reopened this Mar 18, 2021
@extorn
Copy link

extorn commented Mar 18, 2021

I don't know how easy it is to do, but, if the website were to use the ws api under the hood, this would have the effect of both having a single code base to test and maintain, and keep the ws api fully featured.

@plegall
Copy link
Member Author

plegall commented May 21, 2021

I think we need to go deeper on this experiment and try to change Community to keep the idea of not rebuilding cache each time we do a pure admin task like an upload.

@plegall
Copy link
Member Author

plegall commented Oct 7, 2023

With the lounge we already have a nice optimization.

For now and the immediate future, I don't see any easy way to reduce user cache resets. I postpone to 15 the "in depth" brainstorm to make things better.

@plegall plegall modified the milestones: 14.0.0beta4, 15.0.0beta1 Oct 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants