Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display emojis with test results #533

Merged
merged 6 commits into from
Oct 8, 2022

Conversation

joelostblom
Copy link
Contributor

  • This PR adds emojis for individual test that pass or fail, as well as if all tests pass. I think the emojis for individual tests makes it easier to quickly spot what is right and what is wrong, and it gives a certain satisfaction to see many green checkmarks (personal opinion of course).
  • Although the emojis are green and red, I try to choose sufficiently different shapes to not be an issue for people with color vision deficiencies.
  • The addition of the failure message will also facilitate implementing Hide traceback and verbose otter output to only show assertion message instead #419 where the traceback is optionally removed.
  • The random positive emoji when all tests passes is similar to what test_that in R does and is more of a fun touch.

Individual tests:

image

All tests:

image

image

Copy link
Member

@chrispyles chrispyles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also look at adding the emojis to the new exception-based test files? I think editing message in the run() method should be sufficient: https://github.com/ucbds-infra/otter-grader/blob/master/otter/test_files/exception_test.py#L156

otter/test_files/ok_test.py Outdated Show resolved Hide resolved
@chrispyles
Copy link
Member

@joelostblom can you also add an entry to the changelog?

@joelostblom
Copy link
Contributor Author

Can you also look at adding the emojis to the new exception-based test files?

@chrispyles I had a go at doing this the same way as for the regular ok-tests, but there seem to be not test files in otter CI to check exception based tests so not sure I have gotten it right. I am not familiar with this new format of tests and I am not clear on the differences versus the old format after reading the docs, so I might need some help if there are significant other changes needed.

@codecov
Copy link

codecov bot commented Oct 2, 2022

Codecov Report

Base: 72.53% // Head: 72.17% // Decreases project coverage by -0.36% ⚠️

Coverage data is based on head (cc66129) compared to base (7c00b46).
Patch coverage: 37.50% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #533      +/-   ##
==========================================
- Coverage   72.53%   72.17%   -0.37%     
==========================================
  Files          64       62       -2     
  Lines        3921     3748     -173     
==========================================
- Hits         2844     2705     -139     
+ Misses       1077     1043      -34     
Impacted Files Coverage Δ
otter/test_files/exception_test.py 33.75% <0.00%> (ø)
otter/test_files/abstract_test.py 75.36% <25.00%> (-0.38%) ⬇️
otter/test_files/ok_test.py 100.00% <100.00%> (ø)
otter/utils.py
otter/__init__.py

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@chrispyles
Copy link
Member

but there seem to be not test files in otter CI to check exception based tests so not sure I have gotten it right

+1. I need to add tests for this behavior.

@chrispyles chrispyles added this to the v4.1.0 milestone Oct 8, 2022
@chrispyles chrispyles added the enhancement New feature or request label Oct 8, 2022
@chrispyles chrispyles added this to In Progress in Otter-Grader Issues and Tasks via automation Oct 8, 2022
@chrispyles chrispyles merged commit e281f10 into ucbds-infra:master Oct 8, 2022
Otter-Grader Issues and Tasks automation moved this from In Progress to Done Oct 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

Successfully merging this pull request may close these issues.

None yet

2 participants