Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

occ recognize:classify : "Could not find file" using S3 as main storage #987

Open
Aeris1One opened this issue Oct 5, 2023 · 5 comments
Open
Labels
bug Something isn't working
Projects

Comments

@Aeris1One
Copy link

Aeris1One commented Oct 5, 2023

Which version of recognize are you using?

4.3.2

Enabled Modes

Face recognition

TensorFlow mode

WASM mode

Downstream App

other, please specify below

Which Nextcloud version do you have installed?

27.0.2

Which Operating system do you have installed?

Debian 11

Which database are you running Nextcloud on?

MariaDB

Which Docker container are you using to run Nextcloud? (if applicable)

N.A.

How much RAM does your server have?

4GiB

What processor Architecture does your CPU have?

ARM64

Describe the Bug

It seems that when using S3 as main storage, recognize can't access the files.
When using occ recognize:classify just after enabling face recognition, it spams Could not find file and removing 123456 from faces queue because it couldn't be found and only succeed to run on images hosted on my SFTP external storage, not on files hosted on S3 main storage.

I've ~600 photos.

It seems to work when using regular background job though, only occ recognize:classify seems to have this problem.

Expected Behavior

It should detect faces also on S3 main storage.

To Reproduce

  • Setup a Nextcloud with S3 main storage
    • Note : I use Hetzner CAX11 (ARM64) and Scaleway Object Storage.
  • Add some photos
  • (Add a SFTP external storage)
  • Install Recognize
  • Activate Face Recognition
  • Run occ recognize:classify
  • See that it succeeds only for images hosted on SFTP, not on S3

Debug log

https://paste.evolix.org/?e24017eeb4db4b87#9N4X5b5Sm1YM6n7t6gQmJDC1pe1Sku6K3etSQG51N1vR

@Aeris1One Aeris1One added the bug Something isn't working label Oct 5, 2023
@github-actions github-actions bot added this to Backlog in Recognize Oct 5, 2023
@Aeris1One Aeris1One changed the title "Could not find file" using S3 as main storage occ recognize:classify : "Could not find file" using S3 as main storage Oct 5, 2023
@Aeris1One
Copy link
Author

Aeris1One commented Oct 5, 2023

It also outputs a bunch of warning in Admin > Logging, though it seems to work at least for some files when using the regular background job
image

@marcelklehr
Copy link
Member

S3 main storage

This means you've mounted your S3 storage as a local file system right? So, for Nextcloud the S3 storage looks like a normal fs?

@marcelklehr
Copy link
Member

That exception means a local file cannot be generated, likely because it isn't available. I cannot tell you much more than that.

@marcelklehr marcelklehr moved this from Backlog to Triaging in Recognize Nov 5, 2023
@Aeris1One
Copy link
Author

No, S3 isn't mounted as a local filesystem. I use the native objectstore configuration in Nextcloud that allow to use S3 as primary storage.
Here's the doc about that feature : https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html

@github-actions github-actions bot added the stale label Nov 20, 2023
@nextcloud nextcloud deleted a comment from github-actions bot Nov 20, 2023
@marcelklehr marcelklehr removed the stale label Nov 20, 2023
@Aeris1One
Copy link
Author

You left the "waiting for more information" tag, which information do you need ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Recognize
Triaging
Development

No branches or pull requests

2 participants