Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add a more descriptive test failure message to NoticeFieldsTest. #1369

Merged
merged 2 commits into from
Apr 4, 2023

Conversation

bdferris-v2
Copy link
Collaborator

Summary:

To make it easier to understand what this test failure means, I've added a more descriptive failure message. The failure now looks like:

Screenshot 2023-03-31 at 11 54 16 AM

  • Run the unit tests with gradle test to make sure you didn't break anything
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@github-actions
Copy link
Contributor

✅ Rule acceptance tests passed.
New Errors: 0 out of 1423 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1423 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1423 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1423 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1423 sources (~0 %) are corrupted.
Commit: b483921
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 4, 2023

✅ Rule acceptance tests passed.
New Errors: 0 out of 1424 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Errors: 0 out of 1424 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
New Warnings: 0 out of 1424 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
Dropped Warnings: 0 out of 1424 datasets (~0%) are invalid due to code change, which is less than the provided threshold of 1%.
0 out of 1424 sources (~0 %) are corrupted.
Commit: 3662854
Download the full acceptance test report here (report will disappear after 90 days).
✅ Rule acceptance tests passed.

@bdferris-v2 bdferris-v2 merged commit ead50b2 into master Apr 4, 2023
@bdferris-v2 bdferris-v2 deleted the issue/1324/rules_documentation_test branch April 4, 2023 17:15
KClough pushed a commit to JarvusInnovations/gtfs-validator that referenced this pull request Apr 7, 2023
KClough added a commit that referenced this pull request Apr 13, 2023
* chore: fix deprecated dependency

* chore: auto-format

* feat: handle missing RULES.md

* feat: refactor upload process and handle more errors

* Add a more descriptive test failure message to NoticeFieldsTest. (#1369)

* Docs: Update notice migration file (#1357)

* refactor migration file

* update documentation with new file

* fix wording

* Make easier to read

* remove colors

---------

Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>

* feat: Fail acceptance tests on significant change in warnings (#1377)

* Refactor logic out of of outputcomparator/cli/Main and into a new ValidationReportComparator class for easier testing.  In addition, update ValidationReportComparator to trigger a comparison error if changed warnings are above configured thresholds.

* Additional unit tests for ValidationReportComparator.

---------

Co-authored-by: Ryon Coleman <ryon55@gmail.com>
Co-authored-by: Brian Ferris <bdferris+v2@google.com>
Co-authored-by: isabelle-dr <isabelle@mobilitydata.org>
Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>
Co-authored-by: David Gamez <1192523+davidgamez@users.noreply.github.com>
KClough added a commit that referenced this pull request Apr 18, 2023
…1384)

* chore: fix deprecated dependency

* chore: auto-format

* feat: handle missing RULES.md

* feat: refactor upload process and handle more errors

* Add a more descriptive test failure message to NoticeFieldsTest. (#1369)

* Docs: Update notice migration file (#1357)

* refactor migration file

* update documentation with new file

* fix wording

* Make easier to read

* remove colors

---------

Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>

* feat: Fail acceptance tests on significant change in warnings (#1377)

* Refactor logic out of of outputcomparator/cli/Main and into a new ValidationReportComparator class for easier testing.  In addition, update ValidationReportComparator to trigger a comparison error if changed warnings are above configured thresholds.

* Additional unit tests for ValidationReportComparator.

* Add sentry script to static html header

* Add sentry script to web service

* Remove sentry properties from source control

---------

Co-authored-by: Ryon Coleman <ryon55@gmail.com>
Co-authored-by: Brian Ferris <bdferris+v2@google.com>
Co-authored-by: isabelle-dr <isabelle@mobilitydata.org>
Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>
bradyhunsaker pushed a commit to bradyhunsaker/gtfs-validator that referenced this pull request Apr 25, 2023
bradyhunsaker pushed a commit to bradyhunsaker/gtfs-validator that referenced this pull request Apr 25, 2023
* chore: fix deprecated dependency

* chore: auto-format

* feat: handle missing RULES.md

* feat: refactor upload process and handle more errors

* Add a more descriptive test failure message to NoticeFieldsTest. (MobilityData#1369)

* Docs: Update notice migration file (MobilityData#1357)

* refactor migration file

* update documentation with new file

* fix wording

* Make easier to read

* remove colors

---------

Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>

* feat: Fail acceptance tests on significant change in warnings (MobilityData#1377)

* Refactor logic out of of outputcomparator/cli/Main and into a new ValidationReportComparator class for easier testing.  In addition, update ValidationReportComparator to trigger a comparison error if changed warnings are above configured thresholds.

* Additional unit tests for ValidationReportComparator.

---------

Co-authored-by: Ryon Coleman <ryon55@gmail.com>
Co-authored-by: Brian Ferris <bdferris+v2@google.com>
Co-authored-by: isabelle-dr <isabelle@mobilitydata.org>
Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>
Co-authored-by: David Gamez <1192523+davidgamez@users.noreply.github.com>
bradyhunsaker pushed a commit to bradyhunsaker/gtfs-validator that referenced this pull request Apr 25, 2023
…obilityData#1384)

* chore: fix deprecated dependency

* chore: auto-format

* feat: handle missing RULES.md

* feat: refactor upload process and handle more errors

* Add a more descriptive test failure message to NoticeFieldsTest. (MobilityData#1369)

* Docs: Update notice migration file (MobilityData#1357)

* refactor migration file

* update documentation with new file

* fix wording

* Make easier to read

* remove colors

---------

Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>

* feat: Fail acceptance tests on significant change in warnings (MobilityData#1377)

* Refactor logic out of of outputcomparator/cli/Main and into a new ValidationReportComparator class for easier testing.  In addition, update ValidationReportComparator to trigger a comparison error if changed warnings are above configured thresholds.

* Additional unit tests for ValidationReportComparator.

* Add sentry script to static html header

* Add sentry script to web service

* Remove sentry properties from source control

---------

Co-authored-by: Ryon Coleman <ryon55@gmail.com>
Co-authored-by: Brian Ferris <bdferris+v2@google.com>
Co-authored-by: isabelle-dr <isabelle@mobilitydata.org>
Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>
isabelle-dr added a commit that referenced this pull request Apr 25, 2023
* chore: fix deprecated dependency

* chore: auto-format

* feat: handle missing RULES.md

* feat: refactor upload process and handle more errors

* Add a more descriptive test failure message to NoticeFieldsTest. (#1369)

* Docs: Update notice migration file (#1357)

* refactor migration file

* update documentation with new file

* fix wording

* Make easier to read

* remove colors

---------

Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>

* feat: Fail acceptance tests on significant change in warnings (#1377)

* Refactor logic out of of outputcomparator/cli/Main and into a new ValidationReportComparator class for easier testing.  In addition, update ValidationReportComparator to trigger a comparison error if changed warnings are above configured thresholds.

* Additional unit tests for ValidationReportComparator.

* Add sentry script to static html header

* Add sentry script to web service

* Remove sentry properties from source control

* feat: add newrelic agent to web service

---------

Co-authored-by: Ryon Coleman <ryon55@gmail.com>
Co-authored-by: Brian Ferris <bdferris+v2@google.com>
Co-authored-by: isabelle-dr <isabelle@mobilitydata.org>
Co-authored-by: Emma Blue <90632321+emmambd@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants