audit: Add field to know who triggered the operation

[vadmeste]

Description

This is for now needed to know if an external S3 request deleted a file
or it was the scanner.

Signed-off-by: Anis Elleuch anis@min.io

Motivation and Context

Add indication what triggered an operation in the audit

How to test this PR?

1. Setup an audit endpoint
2. Create a bucket, upload a file, setup ILM with one day expiry, forward the clock and see the audit json when the file is removed.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Optimization (provides speedup with no functional changes)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Documentation updated
• Unit tests added/updated

[minio-trusted]

Mint Automation

Test	Result
mint-large-bucket.sh	✔️
mint-fs.sh	✔️
mint-gateway-s3.sh	✔️
mint-erasure.sh	✔️
mint-dist-erasure.sh	✔️
mint-gateway-nas.sh	✔️
mint-zoned.sh	❌ more...
mint-compress-encrypt-dist-erasure.sh	❌ more...

12129-3abed99/mint-zoned.sh.log:

Running with
SERVER_ENDPOINT:      minio-c3.minio.io:32634
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp b97ea48ec38a:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 1 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 42 seconds
(4/15) Running aws-sdk-ruby tests ... done in 4 seconds
(5/15) Running awscli tests ... done in 1 minutes and 24 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 57 seconds
(8/15) Running minio-dotnet tests ... done in 42 seconds
(9/15) Running minio-go tests ... done in 2 minutes and 30 seconds
(10/15) Running minio-java tests ... done in 1 minutes and 17 seconds
(11/15) Running minio-js tests ... FAILED in 18 seconds
{
  "name": "minio-js",
  "function": "\"after all\" hook in \"functional tests\"",
  "duration": 6,
  "status": "FAIL",
  "error": "S3Error: The bucket you tried to delete is not empty at Object.parseError (node_modules/minio/dist/main/xml-parsers.js:79:11) at /mint/run/core/minio-js/node_modules/minio/dist/main/transformers.js:156:22 at DestroyableTransform._flush (node_modules/minio/dist/main/transformers.js:80:10) at DestroyableTransform.prefinish (node_modules/readable-stream/lib/_stream_transform.js:129:10) at prefinish (node_modules/readable-stream/lib/_stream_writable.js:611:14) at finishMaybe (node_modules/readable-stream/lib/_stream_writable.js:620:5) at endWritable (node_modules/readable-stream/lib/_stream_writable.js:643:3) at DestroyableTransform.Writable.end (node_modules/readable-stream/lib/_stream_writable.js:571:22) at IncomingMessage.onend (internal/streams/readable.js:684:10) at endReadableNT (internal/streams/readable.js:1327:12) at processTicksAndRejections (internal/process/task_queues.js:80:21)"
}
(11/15) Running minio-py tests ... FAILED in 2 minutes and 15 seconds
{
  "name": "minio-py:test_thread_safe",
  "status": "FAIL",
  "args": {
    "bucket_name": "minio-py-test-fd8d5964-a5ba-4ca1-99c6-300b236cb01d",
    "object_name": "5d89b6f2-5f91-44e3-9d65-f7912fcf2ec7"
  },
  "message": "S3 operation failed; code: BucketNotEmpty, message: The bucket you tried to delete is not empty, resource: /minio-py-test-fd8d5964-a5ba-4ca1-99c6-300b236cb01d, request_id: 16787EA1B555644B, host_id: 8d988d57-0cb6-440b-8f56-cadb404bbeca, bucket_name: minio-py-test-fd8d5964-a5ba-4ca1-99c6-300b236cb01d",
  "error": "Traceback (most recent call last):\n  File \"/mint/run/core/minio-py/tests.py\", line 126, in _call_test\n    func(log_entry, *args, **kwargs)\n  File \"/mint/run/core/minio-py/tests.py\", line 1640, in test_thread_safe\n    _CLIENT.remove_bucket(bucket_name)\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 676, in remove_bucket\n    self._execute(\"DELETE\", bucket_name)\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 409, in _execute\n    preload_content=preload_content,\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 385, in _url_open\n    raise response_error\nminio.error.S3Error: S3 operation failed; code: BucketNotEmpty, message: The bucket you tried to delete is not empty, resource: /minio-py-test-fd8d5964-a5ba-4ca1-99c6-300b236cb01d, request_id: 16787EA1B555644B, host_id: 8d988d57-0cb6-440b-8f56-cadb404bbeca, bucket_name: minio-py-test-fd8d5964-a5ba-4ca1-99c6-300b236cb01d\n",
  "duration": 6497
}
(11/15) Running s3cmd tests ... done in 21 seconds
(12/15) Running s3select tests ... done in 5 seconds
(13/15) Running security tests ... done in 0 seconds

Executed 13 out of 15 tests successfully.

12129-3abed99/mint-compress-encrypt-dist-erasure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev4.minio.io:30315
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp e3747d54058e:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 1 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 42 seconds
(4/15) Running aws-sdk-ruby tests ... done in 4 seconds
(5/15) Running awscli tests ... FAILED in 31 seconds
{
  "name": "awscli",
  "duration": 2584,
  "function": "aws --endpoint-url http://minio-dev4.minio.io:30315 s3api copy-object --bucket awscli-mint-test-bucket-7218 --key datafile-1-kB-copy --copy-source awscli-mint-test-bucket-7218/datafile-1-kB\n",
  "status": "FAIL",
  "error": "Hash mismatch expected 084e1383b70fb0c51acc680fef370023, got ac57de7156d7fc25ac1a65f81fa3989b"
}
(5/15) Running healthcheck tests ... done in 0 seconds
(6/15) Running mc tests ... done in 48 seconds
(7/15) Running minio-dotnet tests ... done in 45 seconds
(8/15) Running minio-go tests ... FAILED in 2 minutes and 27 seconds
{
  "args": {},
  "duration": 410,
  "error": "At least one of the pre-conditions you specified did not hold",
  "function": "CopyObjectPart(destination, source)",
  "message": "CopyObjectPart call failed",
  "name": "minio-go: testUnencryptedToSSES3CopyObjectPart",
  "status": "FAIL"
}
(8/15) Running minio-java tests ... FAILED in 1 minutes and 21 seconds
{
  "name": "minio-java",
  "function": "copyObject()",
  "args": "[match etag]",
  "duration": 173,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code = PreconditionFailed, message = At least one of the pre-conditions you specified did not hold, bucketName = minio-java-test-3su8eov, objectName = minio-java-test-8tlpef-copy, resource = /minio-java-test-3su8eov/minio-java-test-8tlpef-copy, requestId = 16787E709874C7BC, hostId = 906dde20-4b2c-414d-a2e7-3b624f522176)\nrequest={method=PUT, url=http://minio-dev4.minio.io:30315/minio-java-test-3su8eov/minio-java-test-8tlpef-copy, headers=x-amz-copy-source-if-match: 71cff0a060f852067e443ad1e24ae26c-1\nx-amz-copy-source: /minio-java-test-246m1eb/minio-java-test-8tlpef\nHost: minio-dev4.minio.io:30315\nAccept-Encoding: identity\nUser-Agent: MinIO (Linux; amd64) minio-java/8.0.3\nContent-MD5: 1B2M2Y8AsgTpgAmY7PhCfg==\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20210423T130448Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20210423/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-copy-source;x-amz-copy-source-if-match;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=412, headers=Accept-Ranges: bytes\nContent-Length: 416\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nETag: \"71cff0a060f852067e443ad1e24ae26c\"\nLast-Modified: Fri, 23 Apr 2021 13:04:48 GMT\nServer: MinIO\nVary: Origin\nX-Amz-Request-Id: 16787E709874C7BC\nX-Xss-Protection: 1; mode=block\nDate: Fri, 23 Apr 2021 13:04:48 GMT\n}\n >>> [io.minio.MinioClient.execute(MinioClient.java:775), io.minio.MinioClient.execute(MinioClient.java:563), io.minio.MinioClient.executePut(MinioClient.java:904), io.minio.MinioClient.copyObject(MinioClient.java:1232), FunctionalTest.testCopyObjectMatchETag(FunctionalTest.java:1850), FunctionalTest.copyObject(FunctionalTest.java:2016), FunctionalTest.runObjectTests(FunctionalTest.java:3757), FunctionalTest.runTests(FunctionalTest.java:3783), FunctionalTest.main(FunctionalTest.java:3927)]"
}
(8/15) Running minio-js tests ... done in 52 seconds
(9/15) Running minio-py tests ... done in 2 minutes and 44 seconds
(10/15) Running s3cmd tests ... FAILED in 6 seconds
{
  "name": "s3cmd",
  "duration": "3600",
  "function": "test_put_object_multipart",
  "status": "FAIL",
  "error": "WARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Too many failures. Giving up on '/mint/data/datafile-65-MB'\nERROR: \nUpload of '/mint/data/datafile-65-MB' part 1 failed. Use\n  /usr/local/bin/s3cmd abortmp s3://s3cmd-test-bucket-24293/s3cmd-test-object-1875 0caa5eae-f938-4039-87af-100c1e33b5d8\nto abort the upload, or\n  /usr/local/bin/s3cmd --upload-id 0caa5eae-f938-4039-87af-100c1e33b5d8 put ...\nto continue the upload.\nERROR: Upload of '/mint/data/datafile-65-MB' failed too many times (Last reason: )"
}
(10/15) Running s3select tests ... done in 6 seconds
(11/15) Running security tests ... done in 0 seconds

Executed 11 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally