Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit media files count in media manager #2758

Open
mxbossard opened this issue Apr 20, 2019 · 9 comments

Comments

@mxbossard
Copy link

commented Apr 20, 2019

Hi,

We currently have a lot of images in the default namespace. We choose to work like this for multiple reasons, but the loading is now really slow.

The media manager lack pagination, so I checked the source code. Every files in the namespace directory are scanned and media meta informations are gathered in a PHP array and then sent to the client.

I didn't think much of how to paginate results of the search function (

function search(&$data,$base,$func,$opts,$dir='',$lvl=1,$sort='natural'){
). But I think this function would at least deserve a size limit guard which build and return a sized limited array.

By default this size limit guard would be infinite (-1) to not break current implementation. And I could definitely use a limit in media manager plugin to display a max number of media file. The guard could be set to a reasonable value like 1000, because I cannot imagine à scenario where 1000 files need to be searched, but I may be wrong.

Dokuwiki would benefit of this guard by consuming less resources when every files are not necessary, but only first ones.

I can submit a PR to limit the result count of the search function. And another to add a media size result limit configuration in media manager.

@phy25

This comment has been minimized.

Copy link
Collaborator

commented Apr 20, 2019

Releted to #2619

@mxbossard

This comment has been minimized.

Copy link
Author

commented Apr 21, 2019

The implementation of the #2619 does not modify the search function. So for each browsed page, the search will be called and process every files in the directory.
Imo, the pagination should be implemented at search level.

@b01xy

This comment has been minimized.

Copy link

commented Apr 30, 2019

+1001 to modify the media manager behavior :)
don't know if I understand everything correctly but an ideal behavior could be :

  • to have the ability to search on all images
  • to only display a limited amount of images, with different possibility to sort this short list (last uploaded, by name, ...) to speed-up the loading of the media manager
@nderambure

This comment has been minimized.

Copy link

commented Apr 30, 2019

Really need a fix for this media manager, but pagination seems a bit overloaded. We need AJAX loading of small groups of images, and thus cut down the limit of first search return.

@phy25

This comment has been minimized.

Copy link
Collaborator

commented Apr 30, 2019

Due to how search() works now, it will be a lot of work to add pagination to it. PR welcome.

@nderambure I believe "AJAX loading of small groups of images" is the same as "pagination" here.

@nderambure

This comment has been minimized.

Copy link

commented Apr 30, 2019

I think pagination as mxbossard describes is a PHP pagination with reloading of media manager each time a click is made. But I may be wrong.

Anyway, I hope a fix will be found as this is a real major problem when a wiki has more than 100 images in root namespace (and it still is too much too load).

@Klap-in

This comment has been minimized.

Copy link
Collaborator

commented May 1, 2019

The clicking in namespace tree of the media managers uses already ajax.php for retrieving the images of a namespace. So I understand that you like instead of this entire list, to retrieve the list of media in smaller segments by ajax.

However, I'm wondering: what is causing the slow user experience? Is it due showing the list in the browser? Or server side, the assembling of the html response? or is it the search action (by search() )?.

@mxbossard

This comment has been minimized.

Copy link
Author

commented May 1, 2019

I think the problem is the slow build of the page which delay js loading of media manager a lot.

I wrote a patch to limit the size of the data array manipulated in the search function. If it works I'll submit the patch. For now, only the media manager will use this limit. The purpose is to always display X first media in a namespace or after a search.

@nderambure

This comment has been minimized.

Copy link

commented May 2, 2019

@Klap-in : there is 2 distincts problems that could be solved independantly or together.

As @mxbossard said, the initial PHP build of the images array is a huge work for the server.
FOr example, in our server, we have 1700+ images in root level, resulting on 2,5 seconds of loading, just for the PHP (the server is a Xeon octocore with 16Go Ram from 2014).
Then, the loading of the hundreds of thumbnails in the browser takes 65 seconds and 23Mo each time we need to insert an image in wysiwyg (see below).

We could retrieve small segments of images as you said, and in top of this, lazy-loading images would be great for bandwith.

waterfall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.