Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test result ingestion in the API #218

Merged
merged 30 commits into from
Jan 24, 2024
Merged

Conversation

joseph-sentry
Copy link
Contributor

@joseph-sentry joseph-sentry commented Oct 31, 2023

Purpose/Motivation

This is being done to support failed test ingestion in the API.

Links to relevant tickets

codecov/engineering-team#664

Depends on: codecov/worker#197

What does this PR do?

  • Create Test and TestInstance models
  • Create test_results endpoint

Copy link

codecov bot commented Oct 31, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (0e58149) 95.73% compared to head (b7b532c) 95.74%.

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

Files Patch % Lines
upload/views/test_results.py 98.24% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #218     +/-   ##
=======================================
+ Coverage   95.73   95.74   +0.01     
=======================================
  Files        743     746      +3     
  Lines      16823   16914     +91     
=======================================
+ Hits       16104   16194     +90     
- Misses       719     720      +1     
Flag Coverage Δ
unit 96.07% <98.91%> (+0.01%) ⬆️
unit-latest-uploader 96.07% <98.91%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-staging
Copy link

codecov-staging bot commented Oct 31, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Files Patch % Lines
upload/views/test_results.py 98.24% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codecov-qa
Copy link

codecov-qa bot commented Oct 31, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (0e58149) 96.05% compared to head (b7b532c) 96.07%.

Files Patch % Lines
upload/views/test_results.py 98.24% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #218      +/-   ##
==========================================
+ Coverage   96.05%   96.07%   +0.01%     
==========================================
  Files         628      631       +3     
  Lines       16307    16398      +91     
==========================================
+ Hits        15664    15754      +90     
- Misses        643      644       +1     
Flag Coverage Δ
unit 96.07% <98.91%> (+0.01%) ⬆️
unit-latest-uploader 96.07% <98.91%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

codecov-public-qa bot commented Oct 31, 2023

Codecov Report

Merging #218 (b7b532c) into main (0e58149) will increase coverage by 0.01%.
The diff coverage is 98.91%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #218      +/-   ##
==========================================
+ Coverage   96.05%   96.07%   +0.01%     
==========================================
  Files         628      631       +3     
  Lines       16307    16398      +91     
==========================================
+ Hits        15664    15754      +90     
- Misses        643      644       +1     
Flag Coverage Δ
unit 96.07% <98.91%> (+0.01%) ⬆️
unit-latest-uploader 96.07% <98.91%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
reports/migrations/0013_test_testinstance.py 100.00% <100.00%> (ø)
reports/models.py 97.01% <100.00%> (+0.52%) ⬆️
services/archive.py 80.00% <100.00%> (+0.21%) ⬆️
upload/helpers.py 94.02% <100.00%> (ø)
upload/serializers.py 100.00% <100.00%> (ø)
upload/urls.py 100.00% <100.00%> (ø)
utils/rollouts.py 100.00% <100.00%> (ø)
upload/views/test_results.py 98.24% <98.24%> (ø)

Impacted file tree graph

reports/models.py Outdated Show resolved Hide resolved
@joseph-sentry joseph-sentry force-pushed the joseph/test-ingestion branch 3 times, most recently from 38b93a7 to 09f44e9 Compare December 8, 2023 20:47
@joseph-sentry joseph-sentry force-pushed the joseph/test-ingestion branch 2 times, most recently from 304aee1 to 0a5aa44 Compare December 13, 2023 19:32
@joseph-sentry joseph-sentry changed the title add Test, TestRun, TestSession models and accept upload_file_type in upload endpoint Add test result ingestion in the API Dec 15, 2023
@joseph-sentry joseph-sentry force-pushed the joseph/test-ingestion branch 2 times, most recently from badf3f9 to fbcbfb9 Compare December 21, 2023 15:29
@joseph-sentry joseph-sentry force-pushed the joseph/test-ingestion branch 3 times, most recently from 42977e0 to e155dee Compare January 3, 2024 16:52
@joseph-sentry joseph-sentry marked this pull request as ready for review January 3, 2024 17:58
Copy link

@barnett-yuxiang barnett-yuxiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RFC

@joseph-sentry joseph-sentry force-pushed the joseph/test-ingestion branch 3 times, most recently from c82b8b9 to 2e7899f Compare January 12, 2024 19:09
)
duration_seconds = models.FloatField()
outcome = models.IntegerField()
report = models.ForeignKey(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: rename this to upload

on_delete=models.CASCADE,
)
failure_message = models.TextField(null=True)
timestamp = models.TextField()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Food for thought: do we need a timezone? Would we want to save a timestamp or a datetime? Does it matter?

)
name = models.TextField()
testsuite = models.TextField()
env = models.TextField(default="")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
This field will be used to keep track of which TestInstances we should
be using to compute the PR comment.

Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
The reason for this change is to support concurrent writing of Test and
TestInstance objects, by making the foreign key of the Test computable
using the information, allowing us to create Test objects in a
"fire-and-forget" way
This commit adds this field so we can order TestInstances according to
their timestamp so we can show the latest TestInstance for each Test
for a given commit in the PR comment.

Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
the active field on the TestInstance model is no longer needed because
we can use the timestamp field to determine which TestInstance should be
active.

Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
The timestamp field is now unnecessary as we'll be using the upload
created_at time to determine the ordering of TestInstance objects.

Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
Signed-off-by: joseph-sentry <joseph.sawaya@sentry.io>
@joseph-sentry joseph-sentry merged commit 0b82da1 into main Jan 24, 2024
21 of 22 checks passed
@joseph-sentry joseph-sentry deleted the joseph/test-ingestion branch January 24, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants