Skip to content

Create Separate Integration Test Workflow Run & Nightly Report for Firestore #830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Feb 2, 2022
Merged

Create Separate Integration Test Workflow Run & Nightly Report for Firestore #830

merged 9 commits into from
Feb 2, 2022

Conversation

sunmou99
Copy link
Contributor

@sunmou99 sunmou99 commented Jan 27, 2022 •
edited
Loading

Description

Separate nightly workflow run & report for firestore.

Testing

Nightly Report for firestore: #832
Nightly Report for the rest: #833

Type of Change

Place an x the applicable box:

  • Bug fix. Add the issue # below if applicable.
  • New feature. A non-breaking change which adds functionality.
  • Other, such as a build process or documentation change.

Notes

  • Bug fixes and feature changes require an update to the Release Notes section of release_build_files/readme.md.
  • Read the contribution guidelines CONTRIBUTING.md.
  • Changes to the public API require an internal API review. If you'd like to help us make Firebase APIs better, please propose your change in a feature request so that we can discuss it together.

@sunmou99 sunmou99 added tests-requested: quick Trigger a quick set of integration tests. skip-release-notes Skip release notes check labels Jan 27, 2022
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests-requested: quick Trigger a quick set of integration tests. labels Jan 27, 2022
@github-actions
Copy link

github-actions bot commented Jan 27, 2022 •
edited
Loading

Integration test with FLAKINESS (succeeded after retry)

Requested by @sunmou99 on commit e14cd1a
Last updated: Thu Jan 27 18:26 PST 2022
View integration test log & download artifacts

Failures Configs
firestore [TEST] [FLAKINESS] [Android] [1/3 os: ubuntu] [1/2 android_device: android_target]
(1 failed tests)  CRASH/TIMEOUT

Add flaky tests to go/fpl-cpp-flake-tracker

@github-actions github-actions bot added the tests: failed This PR's integration tests failed. label Jan 27, 2022
@sunmou99 sunmou99 added the tests-requested: quick Trigger a quick set of integration tests. label Jan 27, 2022
@github-actions github-actions bot removed tests-requested: quick Trigger a quick set of integration tests. tests: failed This PR's integration tests failed. labels Jan 27, 2022
@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Jan 28, 2022
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 28, 2022
jonsimantov
jonsimantov reviewed Feb 1, 2022
View reviewed changes
.github/workflows/integration_tests.yml
@@ -169,7 +171,13 @@ jobs:

# To feed input into the job matrix, we first need to convert to a JSON
# list. Then we can use fromJson to define the field in the matrix for the tests job.
apis=$( python scripts/gha/print_matrix_configuration.py -c -w integration_tests -k apis -o "${{github.event.inputs.apis}}" ${TEST_MATRIX_PARAM} )
if [[ "${{ github.event.schedule }}" == "0 9 * * *" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I HATE doing it this way. Hate it hate it hate it.

That said, can you think of any other way to do this? :-/

Copy link
Contributor Author

@sunmou99 sunmou99 Feb 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For scheduled jobs, There is no Webhook payload can be attached to Github Context. This is why I cannot use the {{ github.events.inputs.(field) }}. And the current code is based on the example below:
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule

However, I'm aware of this as it cause confusion. And there are few things I could do to improve:

  1. add more comments.
  2. put this part of logic into print_matrix_configuration.py.
  3. Having a separate cron CI, which leverages it_workflow.py to trigger the nightly integration tests. Or reuse the same CI, and leverages it_workflow.py to trigger the nightly integration tests.

Please let me know what do you think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for now let's add more comments, since refactoring our workflows much more than that seems out of the scope.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments added. Should I have a ticket to refactor this part later?

@sunmou99 sunmou99 requested a review from jonsimantov February 1, 2022 18:29
@sunmou99 sunmou99 merged commit 4366051 into main Feb 2, 2022
@sunmou99 sunmou99 deleted the feature/firestore-nightly-report branch February 2, 2022 00:14
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests: succeeded This PR's integration tests succeeded. labels Feb 2, 2022
@github-actions
Copy link

github-actions bot commented Feb 2, 2022 •
edited
Loading

Integration test with FLAKINESS (succeeded after retry)

Requested by @sunmou99 on commit 4366051
Last updated: Tue Feb 1 18:50 PST 2022
View integration test log & download artifacts

Failures Configs
dynamic_links [TEST] [FLAKINESS] [iOS] [macos] [1/2 ios_device: ios_target]
(1 failed tests)  CRASH/TIMEOUT

Add flaky tests to go/fpl-cpp-flake-tracker

@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Feb 2, 2022
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2022
@sunmou99 sunmou99 changed the title Create Integration Test Nightly Report for firestore Create Separate Integration Test Workflow Run & Nightly Report for Firestore Feb 25, 2022
@firebase firebase locked and limited conversation to collaborators Mar 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@dconeybe dconeybe Awaiting requested review from dconeybe

1 more reviewer

@jonsimantov jonsimantov jonsimantov approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
skip-release-notes Skip release notes check tests: succeeded This PR's integration tests succeeded.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sunmou99 @jonsimantov