Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved logging and tests for mc #105

Merged
merged 1 commit into from
Aug 17, 2017
Merged

Improved logging and tests for mc #105

merged 1 commit into from
Aug 17, 2017

Conversation

CarterMcClellan
Copy link
Contributor

@CarterMcClellan CarterMcClellan commented Jul 24, 2017

Prior to the changes mc printed every step to output.log, and logged the tests
which were intended to throw errors under error.log.

mc output.log will now provide colorized print of all tests which are executed,
all progress bars and extraneous prints have been removed to make output.log
more readable. mc error.log will only hold errors if an unintentional error is
thrown, the convention followed for logging is that the test which failed will
be printed to both output and error.log, inside of error.log a line number which
the test failed on will also be provided. Because these tests are being done
on mc error logging can be improved with the --debug flag. For mc share
upload, and mc share download (the more complicated commands), --debug
stack traces are logged under error.log if an error is found.

In accordance with the doc tests for watch (experimental), cat, and ls have
all been added to mc.

Fixes #86

@nitisht
Copy link
Contributor

nitisht commented Jul 25, 2017

@CarterMcClellan looks like shellcheck is failing, please check

@CarterMcClellan
Copy link
Contributor Author

CarterMcClellan commented Jul 25, 2017

Also, here a document of all tests which I am working towards implementing in mint.

Mint MC Tests.docx

Doc has been updated to reflect updated tests, and removal of event related tests from the TODO agenda

@nitisht
Copy link
Contributor

nitisht commented Jul 28, 2017

@CarterMcClellan I think it will be better to add the doc as .md in /run/core/mc directory

@@ -43,9 +43,10 @@ main() {
setupMCTarget >>"$logfile" 2>&1 || { echo 'mc setup failed' ; exit 1; }

run 2>>"$errfile" 1>>"$logfile" || { echo 'mc run failed.'; rc=1; }
grep -e '<ERROR>' "$logfile" >> "$errfile"
# grep -e '<ERROR>' "$logfile" >> "$errfile"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be better to remove the old echo statements instead of commenting them out, for all the occurences..

if [ "$(basename "$(./mc cp --json "target/${bucketName}/datafile-65-MB" /tmp | jq -r .target)")" != "datafile-65-MB" ]; then
>&2 echo "Put Object Multipart Test 1 Failure"
>&2 echo "Line Number: 105"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are these static line numbers for? If it is to indicate failure point, I think it is better to remove these as line numbers will change in due course and it will be difficult to maintain.

then
printf "\tList Objects Test 1 Failure\n"
>&2 echo "List Objects Test 1 Failure"
>&2 echo "Line Number 263"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like spacing issue here..

printf "\tCat Objects Test 1 Failure\n"
>&2 echo "Cat Objects Test 1 Failure"
>&2 echo "Line 291"
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spacing issue..

watchObjects(){
local bucketName
bucketName=$(create_random_string)
./mc mb play/"$bucketName" >/dev/null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of tabbing issues..

remove_bucket "${bucketName}"
printf "\tTest Success\n"
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comments for each tests.. ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@nitisht
Copy link
Contributor

nitisht commented Aug 2, 2017

@CarterMcClellan I tried running these changes locally

$ cd mc
$ cat error.log 
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
curl: (6) Could not resolve host: null
Presigned Upload Test 1 Failure
Error on line 164
jq: error: Could not open file myVar: No such file or directory
Watch Objects Test 1 Error
rm: cannot remove 'myVar': No such file or directory

$ cat output.log 

  Make Bucket Tests  

	Entering Make Bucket Test 1
	Test Success

  Put Object Tests  

	Entering Put Object Test 1
	Entering Put Object Test 2
	Test Success

  Put Object Multipart Tests  

	Entering Put Object Multipart Test 1
	Entering Put Object Multipart Test 2
	Test Success

  Mirror Object Tests  

	Entering Mirror Object Test 1
	Test Success

  Presigned Upload Object Tests  

	Entering Share Upload Test 1
	Share Upload Test 1 Failure

  Presigned Download Object Tests  

	Entering Presigned Download Object Test 1
	Test Success

  List Object Tests  

	Test Success

  Cat Object Tests  

	Test Success

  Watch  Object Tests  

0a1
> ObjectCreated ObjectRemoved
\ No newline at end of file
	Watch Objects Test 1 Error
	Test Success

  Put Object Error Tests  

	Entering Put Object Error Test 1
	Test Success

  Make Bucket Object Error Tests  

	Entering Make Bucket Error Test 1
	Test Success

  End of tests  

 1 End of tests  

Looks like the logs are not properly reported (see Watch Object Tests section), also there is text in error.log can you please check.

@harshavardhana harshavardhana changed the title Improved Logging/ Tests for MC Improved logging and tests for mc Aug 15, 2017
Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use a shell mode in your editor and fix these tabbing problems?

local bucketName
bucketName=$(create_random_string)

./mc mb "target/$bucketName" >/dev/null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is some tabbing problem still in these scripts.

@harshavardhana
Copy link
Member

@CarterMcClellan i added a comment please take a look.

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@nitisht nitisht merged commit f9c9d05 into minio:master Aug 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants