Add json output for mint mc test cases #121

nitisht · 2017-08-23T16:39:09Z

Fixes: #114

Here is the new output for mint mc tests:

{
  "name": "mc",
  "duration": "14",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-19857",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "74",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-8268",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "971",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-3041",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "68",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-27023",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "99",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-12823",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "1088",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-6179",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "48",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-18387",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "5",
  "function": "./mc mb target/Mc-mint-test-bucket-1023",
  "status": "PASS"
}
{
  "name": "mc",
  "duration": "37",
  "function": "./mc rm --force --recursive target/mc-mint-test-bucket-5913",
  "status": "PASS"
}

balamurugana · 2017-08-23T17:26:44Z

run/core/mc/test.sh

+log_pass() {
+    jq -n --arg name "${1}" --arg function "${2}" --arg args "${3}" \
+    --arg duration "${4}" --arg status "pass" \
+    '{"name": $name, "function": $function, "args": $args, "duration": $duration, "status": $status }'


Unable to understand jq usage for this json structure. You could use echo or printf command

As this function is for PASS case, you would need to have hard-coded status in the json

balamurugana · 2017-08-23T17:29:53Z

run/core/mc/test.sh

+log_fail() {
+    jq -n --arg name "${1}" --arg function "${2}" --arg args "${3}" \
+    --arg duration "${4}" --arg status "fail" --arg message "${5}" --arg error "${6}" \
+    '{"name": $name, "function": $function, "args": $args, "duration": $duration, "status": $status, "message": $message, "error": $error }'


same as above

balamurugana · 2017-08-23T17:31:30Z

run/core/mc/test.sh

+    '{"name": $name, "function": $function, "args": $args, "duration": $duration, "status": $status, "message": $message, "error": $error }'
+}
+
+log_fail_with_alert() {


you could name it as log_alert()

balamurugana · 2017-08-23T17:33:33Z

run/core/mc/test.sh

 # Create a bucket and check if it exists on server
-makeBucket(){
+make_bucket(){


I would recommend to follow function <name> { style which is more readable.

You could need to fix formatting by given proper spacing for {

balamurugana · 2017-08-23T17:34:10Z

run/core/mc/test.sh

 # Create a bucket and check if it exists on server
-makeBucket(){
+make_bucket(){
+    name="mc"


You would need to keep this globally for all functions

balamurugana · 2017-08-23T17:47:38Z

run/core/mc/test.sh

-    if ! ./mc mb "target/${bucketName}" > /dev/null
-    then 
-        >&2 echo "Make Bucket Test 1 Failure"
+    function="./mc mb target/${bucketName}"


camelCase style is not a convention. You would need to use bucket_name

balamurugana · 2017-08-23T17:51:10Z

run/core/mc/test.sh

+        # log end time in ms
+        end=$(($(date +%s%N)/1000000))
+        # find the duration
+        duration=$(( end - start ))


You would need to calculate duration before this if block.

You would need to round to ms at duration computing than getting start and end

start and end need to be named as start_time and end_time

balamurugana · 2017-08-23T17:54:05Z

run/core/mc/test.sh

+        # find the duration
+        duration=$(( end - start ))
+        # update log file
+        log_fail_with_alert "${name}" "${function}" "${args}" "${duration}" "Blocker" "mc mb failed" "$error" 
        return 1
    fi

    # remove bucket and cleanup
    remove_bucket "${bucketName}"


Any reason why you skip checking remove_bucket?

nitisht · 2017-08-25T04:58:10Z

@balamurugana updated the PR.

balamurugana · 2017-08-25T18:58:02Z

run/core/mc/test.sh


-remove_bucket() {
+function remove_bucket() {


You can fix this function too.

balamurugana · 2017-08-25T19:00:49Z

run/core/mc/test.sh

+    printf '{"name": "mc", "duration": "%d", "function": "%s", "status": "FAIL", "error": "%d"}\n' "$1" "$2" "$3"
+}
+
+function log_failure_and_alert() {


You can name this function as log_alert() or log_alert_failure()

balamurugana · 2017-08-25T19:01:35Z

run/core/mc/test.sh

+}
+
+function log_failure() {
+    printf '{"name": "mc", "duration": "%d", "function": "%s", "status": "FAIL", "error": "%d"}\n' "$1" "$2" "$3"


"error" is "%s"

balamurugana · 2017-08-25T19:03:00Z

run/core/mc/test.sh

+}
+
+function log_failure_and_alert() {
+    printf '{"name": "mc", "duration": "%d", "function": "%s", "status": "FAIL", "alert": "%d", error": "%d"}\n' "$1" "$2" "$3" "$4"


This function is not used anywhere. You could remove it.

balamurugana · 2017-08-25T19:06:22Z

run/core/mc/test.sh

+    if [ $rv -eq 0 ]; then
+        log_success "$(get_duration "$start_time")" "${function}"
+    else
+        log_failure "$(get_duration "$start_time")" "${function}" "${out}"


You could leave unquoted get_duration and start_time

Shellcheck (the linter we use) recommends adding quotes here.

balamurugana · 2017-08-25T19:30:20Z

run/core/mc/test.sh


    # Make bucket
-    ./mc mb "target/${bucketName}" > /dev/null
+    bucket_name="mc-mint-test-bucket-$RANDOM"
+    function="${MC_CMD} mb ${SERVER_ALIAS}/${bucket_name}"


You could do have setup and teardown style than creating/removing bucket every time.

balamurugana · 2017-08-25T19:37:23Z

run/core/mc/test.sh

-        return 1
+    # if download succeeds, verify downloaded file
+    if [ $rv -eq 0 ]; then 
+        if [ "$hash1" == "$hash2" ] && [ "$(basename "$(echo "$out" | jq -r .target)")" == "datafile-1-MB" ]; then


Why do we need to check target with the name?

This is to check the downloaded file name same as the uploaded file name.

balamurugana · 2017-08-25T21:47:38Z

run/core/mc/test.sh

+
+    # Make bucket - invalid bucket name
+    bucket_name="Mc-mint-test-bucket-$RANDOM"
+    function="${MC_CMD} mb ${SERVER_ALIAS}/${bucket_name}"


Unknown to understand this test and the behavior. This function always fails.

This is a negative test to make sure a bucket with invalid name is not created on the server.

balamurugana · 2017-08-25T21:52:16Z

run/core/mc/test.sh

-    local bucketName
-    bucketName="Abcd"
+# Upload an object, with invalid object name
+function put_object_error() {


This function would need a better than conveying about invalid object name.

balamurugana · 2017-08-25T21:53:46Z

run/core/mc/test.sh

-    printf "\n %s End of tests %s \n\n" "${blue}" "${normal}" 1
+    # Negative tests
+    make_bucket_error
+    put_object_error


I would recommend to have test function name prefixing test_

balamurugana · 2017-08-25T21:58:07Z

run/core/mc/test.sh

-    bucketName=$(create_random_string)
+# Tests for the cat object success by comparing the STDOUT of "cat run.sh",
+# to "mc cat run.sh" by uploading run.sh, and comparing the two outputs.
+function cat_objects() {


why do we need this test for server? It doesn't make any different than cp in server point.

This was not introduced in this commit. I assume the test is to check if cat streams the content properly.

I think it doesn't hurt to make sure the mc specific commands are working as expected with Minio server.

As mint is meant for server testing, adding mc's cat command test is not a good idea.

For server, there is no difference whether returned http response body is saved, streamed or printed on the screen.

same as above

balamurugana · 2017-08-25T21:58:36Z

run/core/mc/test.sh

-    bucketName=$(create_random_string)
+# Tests for list object success, by mirroring an fs store to minio, 
+# and then comparing the ls results.
+function mirror_list_objects() {


Its not good idea to upload data dir for list objects. You could create objects using smallest file with different names. Mirror is not the right choice.

This was not introduced in this commit. I assume the test was to check if mirror works and uploads all the files and then it is verified via ls.

I will update the comments in the test case accordingly.

As mint is for server testing, adding mc's mirror test is not applicable. It is mc specific feature syncing directory.

As this test is for list objects, using mirror is not a good idea.

As mint is for server testing, adding mc's mirror test is not applicable. It is mc specific feature syncing directory.

The point of Mint is to test functionally if various libraries/SDKs/tools are working as expected with Minio server and of course if the server is working as expected.

If you say mirror is not applicable as mint does server testing, and mirror is just one feature of mc that does cp in the background. It implies we don't even need multiple SDKs, we can just test one SDK and be done -- since all the SDKs will call the same server APIs.

We use all SDKs to check if there is something broken in a specific scenario which may otherwise get overlooked. As mint runs all the SDKs / tools with their specific way to interact with server, we have the confidence that overall system (various clients + server) are all working fine.

So, I think mirror is needed and should not be removed. I can for sure add a new test case that does what you proposed - upload multiple files via cp and then perform a ls.

nitisht · 2017-09-01T04:30:05Z

Updated with make_bucket delete_bucket @balamurugana . Please take a look

Fixes: minio#114

balamurugana

I am taking this immediately for minio PR verification test

harshavardhana

LGTM

nitisht requested review from harshavardhana and balamurugana August 23, 2017 16:40

nitisht added this to the Current milestone Aug 23, 2017

nitisht force-pushed the mc-json-logs branch from 20cecf6 to 2f6f5a3 Compare August 23, 2017 16:45

balamurugana requested changes Aug 23, 2017

View reviewed changes

nitisht force-pushed the mc-json-logs branch 2 times, most recently from 1f812bf to f4c9740 Compare August 25, 2017 04:57

nitisht force-pushed the mc-json-logs branch from f4c9740 to f2cc570 Compare August 25, 2017 05:16

balamurugana requested changes Aug 25, 2017

View reviewed changes

nitisht force-pushed the mc-json-logs branch 2 times, most recently from d6a9283 to a6504af Compare September 1, 2017 04:28

Add json output for mint mc test cases

0d01566

Fixes: minio#114

nitisht force-pushed the mc-json-logs branch from a6504af to 0d01566 Compare September 1, 2017 08:34

balamurugana approved these changes Sep 1, 2017

View reviewed changes

balamurugana merged commit 62d3383 into minio:master Sep 1, 2017

harshavardhana reviewed Sep 1, 2017

View reviewed changes

nitisht deleted the mc-json-logs branch September 14, 2017 09:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add json output for mint mc test cases #121

Add json output for mint mc test cases #121

nitisht commented Aug 23, 2017 •

edited

Loading

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

balamurugana Aug 23, 2017

nitisht commented Aug 25, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

nitisht Aug 26, 2017

balamurugana Aug 28, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

nitisht Aug 26, 2017

balamurugana Aug 25, 2017

nitisht Aug 26, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

balamurugana Aug 25, 2017

nitisht Aug 26, 2017

balamurugana Aug 28, 2017

nitisht Aug 31, 2017

balamurugana Aug 25, 2017

nitisht Aug 26, 2017

balamurugana Aug 28, 2017

nitisht Aug 31, 2017

nitisht commented Sep 1, 2017

balamurugana left a comment

harshavardhana left a comment

Add json output for mint mc test cases #121

Add json output for mint mc test cases #121

Conversation

nitisht commented Aug 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nitisht commented Aug 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nitisht commented Sep 1, 2017

balamurugana left a comment

Choose a reason for hiding this comment

harshavardhana left a comment

Choose a reason for hiding this comment

nitisht commented Aug 23, 2017 •

edited

Loading