Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce metadata gateway #3602

Merged
merged 15 commits into from
May 2, 2022
Merged

introduce metadata gateway #3602

merged 15 commits into from
May 2, 2022

Conversation

butonic
Copy link
Member

@butonic butonic commented Apr 27, 2022

We wrapped the metadata storage in a minimal reva instance with a dedicated gateway, including static storage registry, static auth registry, in memory userprovider and machine authprovider. This allows us to preconfigure the service user for the ocis settings service, share and public share providers.

requires cs3org/reva#2781

@update-docs
Copy link

update-docs bot commented Apr 27, 2022

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@butonic butonic marked this pull request as draft April 27, 2022 13:15
@ownclouders
Copy link
Contributor

ownclouders commented Apr 27, 2022

💥 Acceptance test localApiTests-apiAccountsHashDifficulty-ocis failed. Further test are cancelled...

@butonic butonic marked this pull request as ready for review April 27, 2022 15:54
@butonic butonic requested review from C0rby and rhafer April 27, 2022 15:55
@micbar micbar mentioned this pull request Apr 29, 2022
45 tasks
@rhafer rhafer force-pushed the metadata-gateway branch 2 times, most recently from 6962b9b to 23f8a42 Compare April 29, 2022 15:30
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@rhafer
Copy link
Contributor

rhafer commented Apr 29, 2022

After increasing the log-level the debug log shows this for the failing test:

"level":"info","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","time":"2022-04-29T17:11:47Z","message":"user idp:\"internal\" opaque_id:\"95cb8724-03b2-11eb-a0a6-c33ef8ef53ad\" type:USER_TYPE_PRIMARY  authenticated"}
{"level":"debug","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","user-agent":"grpc-go/1.46.0","from":"tcp://127.0.0.1:48166","uri":"/cs3.auth.provider.v1beta1.ProviderAPI/Authenticate","start":"29/Apr/2022:17:11:47 +0000","end":"29/Apr/2022:17:11:47 +0000","time_ns":1328258,"code":"OK","time":"2022-04-29T17:11:47Z","message":"unary"}
{"level":"debug","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","user-agent":"grpc-go/1.46.0","from":"tcp://127.0.0.1:48162","uri":"/cs3.gateway.v1beta1.GatewayAPI/Authenticate","start":"29/Apr/2022:17:11:47 +0000","end":"29/Apr/2022:17:11:47 +0000","time_ns":2817598,"code":"OK","time":"2022-04-29T17:11:47Z","message":"unary"}
{"level":"debug","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","user-agent":"grpc-go/1.46.0","from":"tcp://127.0.0.1:48044","uri":"/cs3.permissions.v1beta1.PermissionsAPI/CheckPermission","start":"29/Apr/2022:17:11:47 +0000","end":"29/Apr/2022:17:11:47 +0000","time_ns":91179,"code":"OK","time":"2022-04-29T17:11:47Z","message":"unary"}
{"level":"debug","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","ref":{"resource_id":{"storage_id":"f1bdd61a-da7c-49fc-8203-0558109d1b4f","opaque_id":"f1bdd61a-da7c-49fc-8203-0558109d1b4f"}},"grant":{"grantee":{"type":1,"Id":{"UserId":{"idp":"internal","opaque_id":"95cb8724-03b2-11eb-a0a6-c33ef8ef53ad","type":1}}},"permissions":{"add_grant":true,"create_container":true,"delete":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_grants":true,"list_container":true,"list_file_versions":true,"list_recycle":true,"move":true,"remove_grant":true,"purge_recycle":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true}},"time":"2022-04-29T17:11:47Z","message":"AddGrant()"}
{"level":"debug","service":"ocis","pkg":"rgrpc","traceid":"00000000000000000000000000000000","user-agent":"grpc-go/1.46.0","from":"tcp://127.0.0.1:48160","uri":"/cs3.storage.provider.v1beta1.ProviderAPI/CreateStorageSpace","start":"29/Apr/2022:17:11:47 +0000","end":"29/Apr/2022:17:11:47 +0000","time_ns":1358746,"code":"OK","time":"2022-04-29T17:11:47Z","message":"unary"}
{"level":"error","service":"ocis","error":"error: permission denied: permission denied","time":"2022-04-29T17:11:47Z","message":"error initializing metadata client"}

So the authentication of the service seems to succeed with the new internal user and auth providers. It seems to fail after that in CreateStorageSpace

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic requested a review from wkloucek May 2, 2022 10:34
Copy link
Collaborator

@kobergj kobergj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor issue, not critical

extensions/storage-metadata/pkg/command/command.go Outdated Show resolved Hide resolved
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
.vscode/launch.json Outdated Show resolved Hide resolved
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@sonarcloud
Copy link

sonarcloud bot commented May 2, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

4.2% 4.2% Coverage
4.5% 4.5% Duplication

@butonic butonic merged commit 398caee into master May 2, 2022
@delete-merged-branch delete-merged-branch bot deleted the metadata-gateway branch May 2, 2022 13:16
ownclouders pushed a commit that referenced this pull request May 2, 2022
Merge: 0bb8300 3074d70
Author: Jörn Friedrich Dreyer <jfd@owncloud.com>
Date:   Mon May 2 15:16:27 2022 +0200

    Merge pull request #3602 from owncloud/metadata-gateway

    introduce metadata gateway
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants