make sure lifecycle rule ID is present

[findmyname666]

Description

PR #10053 (#10027) introduced check that lifecycle rule ID must be unique. Based on that we should also check that ID is provided for new rules.

Motivation and Context

AWS S3 enforces presence of rule ID as well.

How to test this PR?

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Documentation needed
• Unit tests needed

[findmyname666]

hmm not sure whether it is really required - per DOC

An S3 Lifecycle configuration can have up to 1,000 rules. The <ID> element uniquely identifies a rule. ID length is limited to 255 characters. 

But rule cannot be created without ID via website ....

On other hand this DOC mentions:

Each rule consists of the following:

    Rule metadata that include a rule ID, and status indicating whether the rule is enabled or disabled. If a rule is disabled, Amazon S3 doesn't perform any actions specified in the rule.

ID enforcing just decrease confusion i guess ...

Furthermore i'm not sure what impact it will have on already set-up rules with duplicated IDs ....

Any opinion on that ?

[vadmeste]

@findmyname666 AWS does not require ID field.. but it generates a random one when it is not set. This could break some deployments. Maybe you can also do the same.. setting a random ID when parsing the lifecycle configuration when an S3 client sends it.

[harshavardhana]

@findmyname666 AWS does not require ID field.. but it generates a random one when it is not set. This could break some deployments. Maybe you can also do the same.. setting a random ID when parsing the lifecycle configuration when an S3 client sends it.

@findmyname666 can you do this? please use mustGetUUID() to generate this and save as a unique ID.

[findmyname666]

please use mustGetUUID() to generate this and save as a unique ID.

The function mustGetUUID is defined in package cmd

$ grep -r 'func mustGetUUID' cmd
cmd/object-api-utils.go:func mustGetUUID() string {

There are two approaches in my mind:

1. define the similar functions in lifecycle package which returns error instead e.g. getNewUUID
2. i see the similar function in pkg/event/target/common.go

func getNewUUID() (string, error) {
        u, err := uuid.NewRandom()
        if err != nil {
                return "", err
        }

        return u.String(), nil
}

maybe we can create package pkg/uuid ? @vadmeste , @harshavardhana ?

[harshavardhana]

please use mustGetUUID() to generate this and save as a unique ID.

The function mustGetUUID is defined in package cmd

$ grep -r 'func mustGetUUID' cmd
cmd/object-api-utils.go:func mustGetUUID() string {

There are two approaches in my mind:

1. define the similar functions in lifecycle package which returns error instead e.g. getNewUUID
2. i see the similar function in pkg/event/target/common.go

func getNewUUID() (string, error) {
        u, err := uuid.NewRandom()
        if err != nil {
                return "", err
        }

        return u.String(), nil
}

maybe we can create package pkg/uuid ? @vadmeste , @harshavardhana ?

just write the function like in pkg/event no new package is needed @findmyname666

[findmyname666]

just write the function like in pkg/event no new package is needed @findmyname666

@harshavardhana please take a look ;) THX

[minio-trusted]

Mint Automation

Test	Result
mint-large-bucket.sh	✔️
mint-fs.sh	✔️
mint-dist-xl.sh	✔️
mint-gateway-s3.sh	✔️
mint-gateway-azure.sh	✔️
mint-zoned.sh	✔️
mint-gateway-nas.sh	✔️
mint-xl.sh	❌ more...

10084-7cc3288/mint-xl.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev3.minio.io:30234
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 1fcab14380b4:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 0 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 42 seconds
(4/15) Running aws-sdk-ruby tests ... done in 2 seconds
(5/15) Running awscli tests ... done in 1 minutes and 12 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 0 seconds
(8/15) Running minio-dotnet tests ... done in 29 seconds
(9/15) Running minio-go tests ... done in 46 seconds
(10/15) Running minio-java tests ... done in 50 seconds
(11/15) Running minio-js tests ... FAILED in 19 seconds
{
  "name": "minio-js",
  "function": "fPutObject(bucketName, objectName, filePath, metaData)",
  "args": "bucketName:minio-js-test-428530af-0ec0-4dbb-bd3c-4602da6a8cff, objectName:datafile-65-MB, filePath:/tmp/datafile-65-MB",
  "duration": 687,
  "status": "FAIL",
  "error": "Error: read ECONNRESET at TCP.onStreamRead (internal/stream_base_commons.js:205:27)"
}
(11/15) Running minio-py tests ... done in 1 minutes and 47 seconds
(12/15) Running s3cmd tests ... done in 18 seconds
(13/15) Running s3select tests ... done in 3 seconds
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally