-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Full Text Search Finds No Results For External Storage #546
Comments
The user interface doesn't returns search results from local external storage on NC 16 either. It is frustrating, because I can see the results when I use curl, but I can't get them into the user interface. (My users aren't going to be running curl by hand!) |
Same problem here.. Indexing with
Anyone any suggestions? Thanks in advance. |
Me either but with shared folders... Indexing with Since I use FTS in a productive environment, I am very interested in a solution for this matter. Thanks in advance. |
Ok.. I debug a while and I think there is a problem with the owner rights in the search request. private function generateSearchQueryAccess(IDocumentAccess $access): array {
$query = [];
$query[] = ['term' => ['owner' => $access->getViewerId()]];
$query[] = ['term' => ['users' => $access->getViewerId()]];
$query[] = ['term' => ['users' => '__all']];
foreach ($access->getGroups() as $group) {
$query[] = ['term' => ['groups' => $group]];
}
foreach ($access->getCircles() as $circle) {
$query[] = ['term' => ['circles' => $circle]];
}
return $query;
} build a search request for the logged in user (owner). Files on external storages didn't have a owner (owner is an empty string). I can provide PR, but I'm unshure about security risks. For everybody who is interessted in a solution, can fix this by adding a simple line of code: private function generateSearchQueryAccess(IDocumentAccess $access): array {
$query = [];
$query[] = ['term' => ['owner' => $access->getViewerId()]];
// add this:
$query[] = ['term' => ['owner' => '']]; // Files on external storages didn't have owner information
$query[] = ['term' => ['users' => $access->getViewerId()]];
$query[] = ['term' => ['users' => '__all']];
foreach ($access->getGroups() as $group) {
$query[] = ['term' => ['groups' => $group]];
}
foreach ($access->getCircles() as $circle) {
$query[] = ['term' => ['circles' => $circle]];
}
return $query;
} In my case the solution works and I can search for files on external storages 👍. |
Thank you for your investigation. Glad to read that fixing the owner's rights fixes the problem for you. Unfortunately, it doesn't help in my case. Do you have some advice how I can track down the root cause? |
I think it's an access problem, too.. |
I adopted your search query to
It seems you are right. But your fix didn't help. |
Maybe there is an other problem with group folders... In my case the groups and metatags are:
Your user is member of |
I can confirm this problem with nextcloud 16.0.7 and fulltextsearch 1.3.6. But the search in nextcloud shows only the number of results but no results. But I think my current problem has to do with the core of nextcloud. The releases for 1.3.6 fulltextsearch are all from july or august 2019 and the search for my user account worked till December 2019. I think it stopped working for me with the 16.0.7 update. |
I've installed today fulltextsearch 1.3.7 and the search results returned to my user. |
I've installed fulltextsearch 1.3.8 today, while using files_fulltextsearch 1.3.6, fulltextsearch_elasticsearch 1.3.6. and nextcloud 16.0.7 |
I'm on full text version 1.4.1 and elastic platform 1.5 with NC 18 and i'm also encountering the error that external files (a samba share in my case) are not found. Like @wiphi stated there is a problem when checking the In my opinion the code is correct but lacks a sufficient check of external files. So there should be a check if
If all conditions match, the document should be taken into account for the current search. By the way it's definately not enough to check whether the |
Hi! curl "http://localhost:9200/my_index/_search?q=test&pretty" I have installed NC 17.0.3 (stable) and fulltextsearch v1.3.8 |
@jtitov for me that sounds like a different kind of problem. In my case the request is properly sent to the elastic server but with "incomplete" parameters. The situation you mentioned is different because it seems that no request is sent from the NC php-backend to the elasticsearch-server. So maybe it would be worth opening a different issue? |
@R0Wi , the description of the problem says that the problem is with both local and external storage. I think that is the same application problem. Can you show tcpdump? |
@jtitov for me it sounds like a connection problem between your webserver (NC instance) and elasticsearch server. The |
@R0Wi, yes I inspect all log, and don't show errors or other warnings. On this I began to research TCP traffic and don't see any packets. |
Seems to be a duplicate of #301 |
Hi, any news on this issue? I have the same symptoms. Files which directly belong to the user works fine.
output of fulltextsearch:check:
a search via
but searching via elasticsearch directly returns a list of files (I shortened the result here):
permissions for the external storage Seems good for me:
and the External Storage Folder is included as Musik in nextcloud. What am I doing wrong? |
...
... fulltextsearch:live is for updates of the index, see here |
@daita And may be it would be nice to hear about any plan how nextcloud intends to handle this issue that is existing now for more than 2 years in the code. Thanks a lot. |
@lhurt @ravermeister i implemented a fix for this in nextcloud/fulltextsearch_elasticsearch#100. Currently via the perspective of a modular software approach there are a few things mixed up and we need to have a plan on how to implement this properly but it works for the moment. |
@R0Wi thanks, but I didn't get your Version to run with current nextcloud:
|
@ravermeister sorry for that, this is because the PR is a bit older now and NC did a major upgrad in the meantime. For a quick fix you could just change |
@R0Wi thanks, applying the patch to the 1.5.2 branch/tag worked. but
still no results. do I have to re-index even if Update: searching inside nextcloud works now. I don't understand but Thanks 👍 |
hmm, but something is still weird, I have following scenario:
why is musik2 not scanned? is it because a part of the foldername is the name of an already scanned folder? Update: adding a new file (tested with .md file) seems to work (I ran the The Re-indexing did it. 👍 Thanks in advance |
For those still facing this issue: |
Interesting workaround :-) So if the group setting is explicitly set for the external files provider (i guess the smb provider in that case) this setting is reflected in Elasticsearch group property so therefore taken into account while indexing? Will test that, too but from my point of view it would be nicer to check permissions for external files in the moment when creating a search request (see #546 (comment)). Otherwise after any changes to permissions you would have to run a full reindex. |
Seems to be fixed in version 24. Please see nextcloud/fulltextsearch_elasticsearch#100 (comment) and comment if the issue still exists. |
Full Text Search is matching no results from my local external storage, using Nextcloud 17.0 on Ubuntu 18.04.3 with Apache 2.4.29 and PHP 7.2 It does find files in local storage.
Configuration:
I ran occ fulltextsearch:index and waited for it to complete.
Search finds no documents.
The document ARE indexed:
Full Text Search does find documents in internal storage; just not external storage:
The output of your Nextcloud log in Admin > Logging:
config.php is:
The text was updated successfully, but these errors were encountered: