Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata and tags to mc find #4531

Merged
merged 8 commits into from
Apr 8, 2023

Conversation

klauspost
Copy link
Contributor

@klauspost klauspost commented Mar 30, 2023

Description

Add metadata and tag search.

In both cases a MinIO sever must be on the backend. Tags and --versions requires the latest release.

This is required for fixing unmarshal: minio/minio-go#1795

Motivation and Context

Examples

λ mc find -json -tags="key=v*" myminio/testbucket/
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-05T10:26:13.12+01:00",
 "size": 712001,
 "key": "myminio/testbucket/out.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 },
 "tags": {
  "key": "value",
  "key2": "",
  "key3": "else"
 }
}

Metadata:

λ go build&&mc find -json -metadata="content-type=text/csv" myminio/testbucket/
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-19T17:38:43.803+01:00",
 "size": 342934,
 "key": "myminio/testbucket/go2/src/cmd/go/internal/modfetch/zip_sum_test/testdata/zip_sums.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 }
}
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-19T17:41:03.011+01:00",
 "size": 342934,
 "key": "myminio/testbucket/go3/src/cmd/go/internal/modfetch/zip_sum_test/testdata/zip_sums.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 }
}

If a value is empty -metadata="X-Amz-Server-Side-Encryption=" it must either not exist or have an empty value.

Multiple tags or metadata fields can be queried, eg -metadata="content-type=text/csv" -metadata="X-Amz-Server-Side-Encryption=.". In that case all of them must exist.

Regex is Go RE2 syntax.

How to test this PR?

See examples above.

Types of changes

  • New feature (non-breaking change which adds functionality)

λ mc find -json -tags="key=v*" myminio/testbucket/
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-05T10:26:13.12+01:00",
 "size": 712001,
 "key": "myminio/testbucket/out.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 },
 "tags": {
  "key": "value",
  "key2": "",
  "key3": "else"
 }
}
```

Metadata:

```
λ go build&&mc find -json -metadata="content-type=text/csv" myminio/testbucket/
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-19T17:38:43.803+01:00",
 "size": 342934,
 "key": "myminio/testbucket/go2/src/cmd/go/internal/modfetch/zip_sum_test/testdata/zip_sums.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 }
}
{
 "status": "success",
 "type": "",
 "lastModified": "2023-01-19T17:41:03.011+01:00",
 "size": 342934,
 "key": "myminio/testbucket/go3/src/cmd/go/internal/modfetch/zip_sum_test/testdata/zip_sums.csv",
 "etag": "",
 "metadata": {
  "X-Amz-Server-Side-Encryption": "aws:kms",
  "content-type": "text/csv"
 }
}
```
klauspost added a commit to klauspost/minio that referenced this pull request Mar 30, 2023
Everything is implemented similar to `ListObjectsV2M`, except this is for versions as well.

Related PRs: minio/mc#4531 and minio/minio-go#1795
Copy link
Member

@krisis krisis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff! PR looks good. Need to update go.mod to use latest minio-go.

@klauspost
Copy link
Contributor Author

Merged...

@harshavardhana harshavardhana merged commit 36aa65b into minio:master Apr 8, 2023
go.mod Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants