Access token expiry causes failed uploads on OCIS test server

[prohtex]

I've been scratching my head over token expiry errors that cause failed uploads on OCIS documented on owncloud/ocis#8368 and owncloud/ocis#8372. Today I reproduced these errors on OCIS continuous deployment test server.

I obviously don't have access to those logs, but screenshots are below.

Steps to reproduce:

1. Log into OCIS test server: https://ocis.ocis-wopi.latest.owncloud.works admin/admin
2. Create a new space
3. Upload a directory containing large flies to the new space
4. Upload fails with unknown error. Observe console errors caused by token expiry

[prohtex]

owncloud/ocis#5066 really seems related. After downgrading to 4.0.6 and observing the same behavior, I am now trying setting these values:

IDP_ACCESS_TOKEN_EXPIRATION=2592000
IDP_ID_TOKEN_EXPIRATION=2592000

Update: Doesn't resolve issue

[ScharfViktor]

I tried to reproduce it on https://ocis.ocis-wopi.latest.owncloud.works/ but instance seems broken. no possible to create project space (500 error) so I tested it using our examples see here https://github.com/owncloud/ocis/blob/master/deployments/examples/

localy works fine to me:

• up ocis with s3 https://github.com/owncloud/ocis/blob/master/deployments/examples/ocis_s3/docker-compose.yml
• add envs to ocis env section: IDP_ID_TOKEN_EXPIRATION: 30, IDP_ACCESS_TOKEN_EXPIRATION: 30
• login as admin and create project space
• set unrestricted quota
• upload a big file to space

Actual:
Uploading a large file works fine. The token update happens between upload chunks correctly. No interference, no errors

token_refreshing_while_uploading.mov

[prohtex]

I tried to reproduce it on https://ocis.ocis-wopi.latest.owncloud.works/ but instance seems broken. no possible to create project space (500 error) so I tested it using our examples see here https://github.com/owncloud/ocis/blob/master/deployments/examples/

localy works fine to me:

• up ocis with s3 https://github.com/owncloud/ocis/blob/master/deployments/examples/ocis_s3/docker-compose.yml
• add envs to ocis env section: IDP_ID_TOKEN_EXPIRATION: 30, IDP_ACCESS_TOKEN_EXPIRATION: 30
• login as admin and create project space
• set unrestricted quota
• upload a big file to space

Actual: Uploading a large file works fine. The token update happens between upload chunks correctly. No interference, no errors

token_refreshing_while_uploading.mov

Hi @ScharfViktor thank you very much for taking the time to help troubleshoot this.

I followed your steps exactly, except I am using the WOPI deployment. I noticed that setting IDP_ID_TOKEN_EXPIRATION in the .env file did not work, so I set it in docker-compose.yml.

When I set the expiration to 30, I can observe the token being refreshed every 30 seconds during file upload.

However, after a little while the uploads fail.

This happens to me every single time, in every test environment I can try. I've tried different servers, versions, hosts, files, networks, everything. The only common denominator I can possibly think of is that I am using Mac Safari. The other difference I see in your test is that I am not dragging a single file, but a directory full of large files, although I cannot imagine that would matter.

Here are the system logs:

ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:28Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:28Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:29Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:29Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:29Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"Get \"http://tika:9998/version\": dial tcp 172.27.0.10:9998: connect: connection refused","time":"2024-02-12T17:48:29Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/server/grpc/server.go:43","message":"Error initializing search service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"userlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","time":"2024-02-12T17:48:36Z","line":"github.com/owncloud/ocis/v2/services/userlog/pkg/service/http.go:57","message":"cant get service account"}
ocis_wopi-ocis-1                         | {"level":"error","service":"thumbnails","time":"2024-02-12T17:48:37Z","line":"github.com/owncloud/ocis/v2/services/thumbnails/pkg/service/grpc/v0/service.go:289","message":"resource info is missing checksum"}
ocis_wopi-ocis-appprovider-collabora-1   | {"level":"error","service":"app-provider-collabora","error":"unable to register services: rgrpc: grpc service appprovider could not be started,: Get \"https://collabora.server.com/hosting/discovery\": tls: failed to verify certificate: x509: certificate is valid for 4b76ae2c4756019ce748b86384d4cb95.f120f9e28f0701c44561454aa554e6e9.traefik.default, not collabora.server.com","time":"2024-02-12T17:48:28Z","message":"error starting the grpc server"}
ocis_wopi-ocis-1                         | {"level":"error","service":"userlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","time":"2024-02-12T17:48:55Z","line":"github.com/owncloud/ocis/v2/services/userlog/pkg/service/http.go:57","message":"cant get service account"}
ocis_wopi-ocis-1                         | {"level":"error","service":"thumbnails","time":"2024-02-12T17:48:56Z","line":"github.com/owncloud/ocis/v2/services/thumbnails/pkg/service/grpc/v0/service.go:289","message":"resource info is missing checksum"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T17:49:14Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"b60fae1cd1d9af3b021f79b1ef18b12e","request-id":"5cfc52c9-2784-4e7a-afa7-320aaeabf887","error":"Patch \"http://localhost:9158/data/tus/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e\": context canceled","time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"b60fae1cd1d9af3b021f79b1ef18b12e","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iYzFmNGE5OC03MWFhLTRiYzktYmM4My1mOTFkOGMxNTU3MGUifQ.-SGwGWceseINP5-NwUmFFE40Egr5LlRHdzAIIh4pxTg","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iYzFmNGE5OC03MWFhLTRiYzktYmM4My1mOTFkOGMxNTU3MGUifQ.-SGwGWceseINP5-NwUmFFE40Egr5LlRHdzAIIh4pxTg","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6979758923,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"dd590e15520ce7cf1e16653a92fd7a4e","request-id":"0e238ea7-93f6-4f96-b24b-ee65f59e28d9","error":"Patch \"http://localhost:9158/data/tus/bcefde43-7968-47bb-8de2-048f9e3c56a9\": context canceled","time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"dd590e15520ce7cf1e16653a92fd7a4e","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iY2VmZGU0My03OTY4LTQ3YmItOGRlMi0wNDhmOWUzYzU2YTkifQ.1Gs2NCSedQcNjdNIosZ9s0sE4ORQrt796FxXjSUdevY","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iY2VmZGU0My03OTY4LTQ3YmItOGRlMi0wNDhmOWUzYzU2YTkifQ.1Gs2NCSedQcNjdNIosZ9s0sE4ORQrt796FxXjSUdevY","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:50:55 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":16836909403,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"5a7325aee22b25c371ac61630c378ec0","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/bcefde43-7968-47bb-8de2-048f9e3c56a9","url":"/bcefde43-7968-47bb-8de2-048f9e3c56a9","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:50:55 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":16836613797,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"c205f72bb8af42294758c157c56c852b","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e","url":"/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6979626694,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"fdaf5662f984648aa48ac58f26e1476a","request-id":"15fa1780-d237-42c6-bbc8-dd75532d520e","error":"Patch \"http://localhost:9158/data/tus/5bf5b875-aa81-4cef-b671-0be3ab177e02\": context canceled","time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"066c6d629aa88baf52ef6c593cd03bea","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/796a6d25-4feb-40f7-8a27-8e864de31712","url":"/796a6d25-4feb-40f7-8a27-8e864de31712","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:50:58 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":13653007397,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"fdaf5662f984648aa48ac58f26e1476a","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy81YmY1Yjg3NS1hYTgxLTRjZWYtYjY3MS0wYmUzYWIxNzdlMDIifQ.kMzpjv4oAZB36qN9SxIEwOgsFqd569AVELrACkA2vdc","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy81YmY1Yjg3NS1hYTgxLTRjZWYtYjY3MS0wYmUzYWIxNzdlMDIifQ.kMzpjv4oAZB36qN9SxIEwOgsFqd569AVELrACkA2vdc","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6972339183,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"49e3e76aa4f50b782aac8bda84866de4","request-id":"ac98fab4-13cd-41d1-abca-94bca0d87cc3","error":"Patch \"http://localhost:9158/data/tus/c74f5fc1-895f-4895-bdca-92ebe6455f0e\": context canceled","time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"49e3e76aa4f50b782aac8bda84866de4","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jNzRmNWZjMS04OTVmLTQ4OTUtYmRjYS05MmViZTY0NTVmMGUifQ.Z2eBpG8sAwGM-dAMbA-dhNXlCvOPoYnzylQEj5c_cBw","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jNzRmNWZjMS04OTVmLTQ4OTUtYmRjYS05MmViZTY0NTVmMGUifQ.Z2eBpG8sAwGM-dAMbA-dhNXlCvOPoYnzylQEj5c_cBw","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6980121753,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"553609acb5575d298b27ff481dc9ec60","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/c74f5fc1-895f-4895-bdca-92ebe6455f0e","url":"/c74f5fc1-895f-4895-bdca-92ebe6455f0e","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6979671615,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"5fd65fe97877c40948701593d8e50762","request-id":"4cb6b6e1-e20d-4eed-b990-d3a86574b09e","error":"Patch \"http://localhost:9158/data/tus/796a6d25-4feb-40f7-8a27-8e864de31712\": context canceled","time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"5fd65fe97877c40948701593d8e50762","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83OTZhNmQyNS00ZmViLTQwZjctOGEyNy04ZTg2NGRlMzE3MTIifQ.0l2rYSk5quSfSnuVsCGPhTZzGY4rzjfb5jUELIiQY7A","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83OTZhNmQyNS00ZmViLTQwZjctOGEyNy04ZTg2NGRlMzE3MTIifQ.0l2rYSk5quSfSnuVsCGPhTZzGY4rzjfb5jUELIiQY7A","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:50:58 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":13653489356,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"5f8bea0b6680902eac478f964fd43ea7","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/5bf5b875-aa81-4cef-b671-0be3ab177e02","url":"/5bf5b875-aa81-4cef-b671-0be3ab177e02","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:51:05 +0000","end":"12/Feb/2024:17:51:12 +0000","time_ns":6972041352,"time":"2024-02-12T17:51:12Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"ec5fbfb31d77d6386a47d3cd79070422","request-id":"74dc5cd8-fc6f-4d95-83ee-7bd8f1adebfa","error":"Patch \"http://localhost:9158/data/tus/796a6d25-4feb-40f7-8a27-8e864de31712\": context canceled","time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"ec5fbfb31d77d6386a47d3cd79070422","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83OTZhNmQyNS00ZmViLTQwZjctOGEyNy04ZTg2NGRlMzE3MTIifQ.0l2rYSk5quSfSnuVsCGPhTZzGY4rzjfb5jUELIiQY7A","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83OTZhNmQyNS00ZmViLTQwZjctOGEyNy04ZTg2NGRlMzE3MTIifQ.0l2rYSk5quSfSnuVsCGPhTZzGY4rzjfb5jUELIiQY7A","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:54:52 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":16520841165,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"808ef987c6dcd41dbdcabe118a21aea1","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/796a6d25-4feb-40f7-8a27-8e864de31712","url":"/796a6d25-4feb-40f7-8a27-8e864de31712","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:54:52 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":16520204221,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"2263d3c77f618012e3532653d0353b0d","request-id":"dd54edee-785e-4a9e-8bd3-d749aaed3c4f","error":"Patch \"http://localhost:9158/data/tus/bcefde43-7968-47bb-8de2-048f9e3c56a9\": context canceled","time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"2263d3c77f618012e3532653d0353b0d","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iY2VmZGU0My03OTY4LTQ3YmItOGRlMi0wNDhmOWUzYzU2YTkifQ.1Gs2NCSedQcNjdNIosZ9s0sE4ORQrt796FxXjSUdevY","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iY2VmZGU0My03OTY4LTQ3YmItOGRlMi0wNDhmOWUzYzU2YTkifQ.1Gs2NCSedQcNjdNIosZ9s0sE4ORQrt796FxXjSUdevY","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:54:58 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":9823776219,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"e1888a11ef3c4409fbfda97090fde55e","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/bcefde43-7968-47bb-8de2-048f9e3c56a9","url":"/bcefde43-7968-47bb-8de2-048f9e3c56a9","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:54:58 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":9823531712,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"d7f9ba89c3aed933ca79051b6be30115","request-id":"6b5f9578-7f89-4d1d-bf07-5c752fba03cc","error":"Patch \"http://localhost:9158/data/tus/c74f5fc1-895f-4895-bdca-92ebe6455f0e\": context canceled","time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"d7f9ba89c3aed933ca79051b6be30115","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jNzRmNWZjMS04OTVmLTQ4OTUtYmRjYS05MmViZTY0NTVmMGUifQ.Z2eBpG8sAwGM-dAMbA-dhNXlCvOPoYnzylQEj5c_cBw","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jNzRmNWZjMS04OTVmLTQ4OTUtYmRjYS05MmViZTY0NTVmMGUifQ.Z2eBpG8sAwGM-dAMbA-dhNXlCvOPoYnzylQEj5c_cBw","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:54:59 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":9468322422,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"7a77f80df50cb84720b1c9befcec3604","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/5bf5b875-aa81-4cef-b671-0be3ab177e02","url":"/5bf5b875-aa81-4cef-b671-0be3ab177e02","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:54:54 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":13808641189,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"4c595c87f76f566e682cdc9700a49238","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/c74f5fc1-895f-4895-bdca-92ebe6455f0e","url":"/c74f5fc1-895f-4895-bdca-92ebe6455f0e","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:54:59 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":9468039842,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"010d70727ba9af2b0a4ccabfa1cb31d8","request-id":"c8e68053-3ffb-4328-adae-f642c18f4792","error":"Patch \"http://localhost:9158/data/tus/5bf5b875-aa81-4cef-b671-0be3ab177e02\": context canceled","time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"010d70727ba9af2b0a4ccabfa1cb31d8","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy81YmY1Yjg3NS1hYTgxLTRjZWYtYjY3MS0wYmUzYWIxNzdlMDIifQ.kMzpjv4oAZB36qN9SxIEwOgsFqd569AVELrACkA2vdc","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy81YmY1Yjg3NS1hYTgxLTRjZWYtYjY3MS0wYmUzYWIxNzdlMDIifQ.kMzpjv4oAZB36qN9SxIEwOgsFqd569AVELrACkA2vdc","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:54:54 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":13809142501,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"2485c6c070133c915ada4dc430166537","request-id":"63e58822-0bce-4d0b-bbd9-1916f1d0ea0f","error":"Patch \"http://localhost:9158/data/tus/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e\": context canceled","time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/services/datagateway/datagateway.go:390","message":"error doing PATCH request to data service"}
ocis_wopi-ocis-1                         | {"level":"error","service":"storage-users","pkg":"rhttp","traceid":"4c4a55fe6e1b30bce6956822056def5d","host":"127.0.0.1","method":"PATCH","uri":"/data/tus/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e","url":"/bc1f4a98-71aa-4bc9-bc83-f91d8c15570e","proto":"HTTP/1.1","status":500,"size":15,"start":"12/Feb/2024:17:54:50 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":17936842303,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"frontend","pkg":"rhttp","traceid":"2485c6c070133c915ada4dc430166537","host":"127.0.0.1","method":"PATCH","uri":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iYzFmNGE5OC03MWFhLTRiYzktYmM4My1mOTFkOGMxNTU3MGUifQ.-SGwGWceseINP5-NwUmFFE40Egr5LlRHdzAIIh4pxTg","url":"/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzA3ODQ2NTU0LCJpYXQiOjE3MDc3NjAxNTQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9iYzFmNGE5OC03MWFhLTRiYzktYmM4My1mOTFkOGMxNTU3MGUifQ.-SGwGWceseINP5-NwUmFFE40Egr5LlRHdzAIIh4pxTg","proto":"HTTP/1.1","status":500,"size":0,"start":"12/Feb/2024:17:54:50 +0000","end":"12/Feb/2024:17:55:08 +0000","time_ns":17937286936,"time":"2024-02-12T17:55:08Z","line":"github.com/cs3org/reva/v2@v2.18.1-0.20240208105019-d5e420d294be/internal/http/interceptors/log/log.go:112","message":"http"}
ocis_wopi-ocis-1                         | {"level":"error","service":"userlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","time":"2024-02-12T17:55:10Z","line":"github.com/owncloud/ocis/v2/services/userlog/pkg/service/http.go:57","message":"cant get service account"}
ocis_wopi-ocis-1                         | {"level":"error","service":"userlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","time":"2024-02-12T17:55:18Z","line":"github.com/owncloud/ocis/v2/services/userlog/pkg/service/http.go:57","message":"cant get service account"}
ocis_wopi-ocis-1                         | {"level":"error","service":"thumbnails","time":"2024-02-12T17:55:26Z","line":"github.com/owncloud/ocis/v2/services/thumbnails/pkg/service/grpc/v0/service.go:289","message":"resource info is missing checksum"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"entity not found","Id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"ed7ee40b-d9ff-43da-8f3c-e3cd4ad2e554","space_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T17:55:33Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/service.go:452","message":"failed to remove item from index"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T17:55:34Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T17:55:57Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"thumbnails","time":"2024-02-12T17:55:58Z","line":"github.com/owncloud/ocis/v2/services/thumbnails/pkg/service/grpc/v0/service.go:289","message":"resource info is missing checksum"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T17:56:10Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"clientlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","event":{"Type":"events.UploadReady","ID":"5e3d7fe9-6123-45b8-bcbc-9e7701baa30a","TraceParent":"00-98a5b5d587843dd87ee97f4b65cd271b-156daa504520b31b-00","Event":{"UploadID":"c6f52bab-250c-49d9-a539-061f95c4903f","Filename":"DSCF0292.psd","SpaceOwner":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"ExecutingUser":{"id":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"username":"admin"},"FileRef":{"resource_id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"aece9e81-d86a-4c38-b645-441af5b8e68c","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"path":"./Test/DSCF0292.psd"},"Failed":false,"Timestamp":{"seconds":1707761486,"nanos":401430349}}},"time":"2024-02-12T18:11:26Z","line":"github.com/owncloud/ocis/v2/services/clientlog/pkg/service/service.go:87","message":"error authenticating service user"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:11:27Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"entity not found","Id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"654f5481-4311-4f9a-9c53-e633abd50a01","space_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:15:38Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/service.go:452","message":"failed to remove item from index"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T18:15:39Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"entity not found","Id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"d218b2b0-094f-472b-907d-ae8f3e898bae","space_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:15:46Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/service.go:452","message":"failed to remove item from index"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T18:15:47Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"entity not found","Id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"3398dbe9-72c7-4efa-a352-b9409fa0ac86","space_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:15:51Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/service.go:452","message":"failed to remove item from index"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$75f2f92b-9fd3-4ccd-b161-31232df3f349"},"userID":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349","type":1},"time":"2024-02-12T18:15:52Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"clientlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","event":{"Type":"events.UploadReady","ID":"2dde96e8-7e32-4f81-b5a4-d82ce03238cc","TraceParent":"00-98a5b5d587843dd87ee97f4b65cd271b-156daa504520b31b-00","Event":{"UploadID":"72eecb85-2da8-4a21-9605-3ea860f6586b","Filename":"DSCF0292 copy.psd","SpaceOwner":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"ExecutingUser":{"id":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"username":"admin"},"FileRef":{"resource_id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"aece9e81-d86a-4c38-b645-441af5b8e68c","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"path":"./Test/DSCF0292 copy.psd"},"Failed":false,"Timestamp":{"seconds":1707762380,"nanos":137506726}}},"time":"2024-02-12T18:26:20Z","line":"github.com/owncloud/ocis/v2/services/clientlog/pkg/service/service.go:87","message":"error authenticating service user"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 10m32.382249178s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:26:20Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:26:21Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 10m55.205300986s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:26:43Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m16.290653992s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:27:04Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m36.673976295s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:27:24Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m55.106970228s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:27:43Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m14.641554557s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:28:02Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m30.708796202s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:28:18Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m50.466031144s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:28:38Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m10.190948568s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:28:58Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m33.351693475s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:29:21Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m56.190269643s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:29:44Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 14m15.682274596s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:30:03Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 14m33.840235475s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:30:21Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 14m49.774160325s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:30:37Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 15m7.203914225s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:30:55Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 15m24.70163714s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:31:12Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 15m37.225811732s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:31:25Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 15m59.145899338s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:31:47Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 16m22.981000565s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:32:10Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 16m43.316509171s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:32:31Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 17m0.26155035s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:32:48Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 17m14.240283836s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:33:02Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 17m30.945218797s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:33:18Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 17m49.973397436s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:33:37Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 18m4.153917171s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:33:52Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 18m22.993991619s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:34:10Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 18m35.412734534s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:34:23Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 18m47.932306549s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:34:35Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 19m1.745981039s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:34:49Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 19m18.744345171s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:35:06Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 19m39.873524541s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:35:27Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 19m58.54615136s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T18:35:46Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"clientlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","event":{"Type":"events.UploadReady","ID":"46ee437d-02bc-40f5-acf3-783589bfb8f6","TraceParent":"00-98a5b5d587843dd87ee97f4b65cd271b-156daa504520b31b-00","Event":{"UploadID":"32ded349-e277-48e9-a819-491551ffdffe","Filename":"marc-brick-deliverable-flat-layers copy.psb","SpaceOwner":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"ExecutingUser":{"id":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"username":"admin"},"FileRef":{"resource_id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"aece9e81-d86a-4c38-b645-441af5b8e68c","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"path":"./Test/marc-brick-deliverable-flat-layers copy.psb"},"Failed":false,"Timestamp":{"seconds":1707763021,"nanos":277449242}}},"time":"2024-02-12T18:37:01Z","line":"github.com/owncloud/ocis/v2/services/clientlog/pkg/service/service.go:87","message":"error authenticating service user"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:37:02Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"clientlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","event":{"Type":"events.UploadReady","ID":"be04bfb6-6d8f-415a-8a0b-2e7d1312b19d","TraceParent":"00-98a5b5d587843dd87ee97f4b65cd271b-156daa504520b31b-00","Event":{"UploadID":"e06a7300-a272-4960-92b9-2ad95033a2df","Filename":"nola2-edit copy.psb","SpaceOwner":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"ExecutingUser":{"id":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"username":"admin"},"FileRef":{"resource_id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"aece9e81-d86a-4c38-b645-441af5b8e68c","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"path":"./Test/nola2-edit copy.psb"},"Failed":false,"Timestamp":{"seconds":1707763063,"nanos":697409937}}},"time":"2024-02-12T18:37:43Z","line":"github.com/owncloud/ocis/v2/services/clientlog/pkg/service/service.go:87","message":"error authenticating service user"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T18:37:44Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m15.778187988s","authenticator":"oidc","path":"/ocs/v1.php/cloud/user","time":"2024-02-12T18:54:56Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"userlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","time":"2024-02-12T19:02:01Z","line":"github.com/owncloud/ocis/v2/services/userlog/pkg/service/http.go:57","message":"cant get service account"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"entity not found","Id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"8e060e52-b4e7-48cc-9fb1-40ec78c9cdb3","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"time":"2024-02-12T19:02:23Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/service.go:452","message":"failed to remove item from index"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T19:02:24Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T19:02:27Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"clientlog","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","event":{"Type":"events.UploadReady","ID":"3de43cb0-e3cb-4397-82ab-3d6f3957ed1f","TraceParent":"00-98a5b5d587843dd87ee97f4b65cd271b-156daa504520b31b-00","Event":{"UploadID":"9237480e-a504-4b0d-be75-1cd00487d332","Filename":"DSCF0292.psd","SpaceOwner":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"ExecutingUser":{"id":{"idp":"https://ocis.server.com","opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"username":"admin"},"FileRef":{"resource_id":{"storage_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a","opaque_id":"aece9e81-d86a-4c38-b645-441af5b8e68c","space_id":"aece9e81-d86a-4c38-b645-441af5b8e68c"},"path":"./Test/DSCF0292.psd"},"Failed":false,"Timestamp":{"seconds":1707765269,"nanos":725001651}}},"time":"2024-02-12T19:14:29Z","line":"github.com/owncloud/ocis/v2/services/clientlog/pkg/service/service.go:87","message":"error authenticating service user"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 10m29.933994417s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:14:29Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"search","error":"unexpected status code while authenticating service user: CODE_UNAUTHENTICATED","spaceID":{"opaque_id":"a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c"},"userID":{"opaque_id":"75f2f92b-9fd3-4ccd-b161-31232df3f349"},"time":"2024-02-12T19:14:30Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/search/events.go:69","message":"error while indexing a space"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 10m41.764343392s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:14:41Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 10m53.985016317s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:14:53Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m7.247513027s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:15:07Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m20.947918708s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:15:20Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m35.268173705s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:15:35Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 11m48.814082091s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:15:48Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m4.487141407s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:16:04Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m18.323599316s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:16:18Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m33.611486473s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:16:33Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 12m50.795579723s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:16:50Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m10.561757856s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:17:10Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m34.895997227s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:17:34Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 13m55.546131949s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:17:55Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}
ocis_wopi-ocis-1                         | {"level":"error","service":"proxy","error":"failed to verify access token: token is expired by 14m20.265665121s","authenticator":"oidc","path":"/remote.php/dav/spaces/a22d6de7-a4e0-4f09-8483-bb2452162a3a$aece9e81-d86a-4c38-b645-441af5b8e68c/Test","time":"2024-02-12T19:18:20Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/oidc_auth.go:185","message":"failed to authenticate the request"}

[kulmann]

Hey @prohtex thank you so much for looking into this issue with such dedication and attention to detail!

The issue you are describing seems to be related to browsers throttling inactive tabs...

I have this as a reproducer:

1. Start a long running upload (e.g. like in your reproducer)
2. Ban the browser tab into the background
3. Open the network tab in the dev tools, so that network requests get recorded
4. Grab a coffee or do something else without coming back to the ownCloud browser tab for at least the access token lifetime (5 minutes)
5. Come back to the ownCloud browser tab

Could you please try the following?

1. Start a long running upload (e.g. like in your reproducer)
2. Open the network tab in the dev tools, so that network requests get recorded
3. Grab a coffee or click around in the ownCloud browser tab. Make sure to not put the browser or the tab into hibernation.

Throttling in modern browsers exists so that you can keep 500 browser tabs "open" at the same time, not drain your device battery, keep the memory footprint and CPU usage low (haha, laughing in Chromium). However, if you start an upload and want it to finish, this is kind of annoying...

More info: https://aboutfrontend.blog/tab-throttling-in-browsers/

There seems to be a way to disable inactive tab throttling in Safari, but that's really only for debugging purposes, not for your everyday browser usage. See https://apple.stackexchange.com/questions/280679/preventing-safari-background-tabs-disconnecting-from-server

The only solution I currently see in our code base would be to implement long running actions via Web Workers, i.e. in separate threads. Looks like those are still not throttled by modern browsers... we already thought about that nearly 2 years ago in a different context, see #7039
This issue here is a very strong "+1" for Web Workers because in an ownCloud you typically want your long running actions (upload, batch delete, etc) to complete in a timely manner. Pausing those when in the background is not desirable at all. However, please understand that that's not a low hanging fruit. I don't see an easier mitigation for the throttling.

For the token expiration you could try to add the offline_access to the scopes that the web ui requests. You can do so by setting the environment variable WEB_OIDC_SCOPE to openid profile email offline_access (assuming that you are currently on the default value openid profile email). With that ownCloud Web would be able to recreate a valid access token when your browser tab is inactive for longer than the access token lifetime.

cc @tbsbdr this needs PM attention.

[rhafer]

@kulmann I've played with this a bit as well. It is also pretty easily reproducible when just throttling the network bandwith (using the developer tools) to e.g. DSL or 4G/LTE (Wifi might also work). In that case the token refresh request seems to just take too long when an upload is happening in parallel and the token expires.

[rhafer]

... transferring to web

[kulmann]

We're requesting a new access token 10 seconds before the current one expires. Maybe we could do that earlier, that's an easy fix. Maybe give it 30 seconds if the access token lifetime is longer than e.g. 1 minute (1 minute is already a magic number in the oidc-client-ts lib that we use).

[tbsbdr]

All I can say is that uploads must continue in "background" tabs; assessing as p2 as uploading is essential for us. as mentioned by you, @kulmann solving this issue via web worker might be heavy lifting, so a effort estimation / reasearch beforehand should happen before we go in.

[prohtex]

Could you please try the following?

1. Start a long running upload (e.g. like in your reproducer)
2. Open the network tab in the dev tools, so that network requests get recorded
3. Grab a coffee or click around in the ownCloud browser tab. Make sure to not put the browser or the tab into hibernation.

Hi @kulmann, I am glad the issue was found! I can confirm that taking further actions keeps the tab active and the upload going. This also explains why the point at which the upload failed was rather inconsistent (as I was taking various actions with the browser, and not always keeping it in the background). I can also confirm that setting IDP_ID_TOKEN_EXPIRATION mitigates the issue; I had mistakenly tried to set the value in the .env file previously and it was not taking effect.

Our use case is for photo retouching, and so we are frequently sending massive files over the web client. A big part of why we wanted to migrate to OCIS was to improve reliability sending large files this way; we often encounter timeouts and broken uploads with OC10. I am assuming sites like DropBox and Wetransfer use Web Workers for their uploads.

For the meantime, we will likely deploy more than one instance of OCIS so we can continue to test, and to keep one with a very long token timeout for these purposes. It doesn't appear possible to get the WOPI deployment working without Docker so we will continue to use the Ubuntu VM. I can always test anything in a fresh Ubuntu VM or on macOS.

Thanks again to the team of dedicated individuals working to make OCIS an incredible product. I'm a PHP developer so can't contribute much in the way of code, but am happy to do whatever I can to help.

[prohtex]

After further testing, I am noticing some quirks: #10492

[appiekap653]

I am receiving the same error, but then when trying to upload using the android client, rclone and web.
I don't think this has something to do with browsers throttling tabs, rclone and android client doesn't have tabs...

I have also set the access tokens to expire after 24 hours in my external idp. using authentik.
I doubt that the access token expired error message is referring to the token from the idp, I think it is referring to some internal used access token.

Errors I'm getting are:

Rclone:

DEBUG : webdav root '': Bearer token expired: 401 Unauthorized
ERROR : rc: "operations/uploadfile": error: Bad Gateway: 502 Bad Gateway

[JammingBen]

I am receiving the same error, but then when trying to upload using the android client, rclone and web.
I don't think this has something to do with browsers throttling tabs, rclone and android client doesn't have tabs...

It sounds like a different error then. We have a similar issue open in the server repo, which makes more sense if you're experiencing these problems with other clients as well. Maybe related? owncloud/ocis#8372

Either way, there are definitely issues with tab throttling. #10955 addresses those.