Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow upload requests originating from Shelter to set storage path #172

Merged
merged 4 commits into from
Oct 27, 2023

Conversation

scott-codecov
Copy link
Contributor

Purpose/Motivation

Shelter will be buffering uploads on a queue and then calling the API async. Since Shelter won't know about the report and upload's external ID we need to construct our own storage path inside Shelter.

Links to relevant tickets

TODO: link to Shelter PR

What does this PR do?

This PR introduces a new config option which is a secret that will be shared between API and Shelter. Shelter will add this to some API requests in a custom header: X-Shelter-Token

When this token is present in the upload handler then we'll allow the storage path to be set explicitly instead of generating that path in the API.

@codecov
Copy link

codecov bot commented Oct 3, 2023

Codecov Report

Merging #172 (c74edf0) into main (5ddb063) will not change coverage.
The diff coverage is 100.00%.

❗ Current head c74edf0 differs from pull request most recent head 282afa6. Consider uploading reports for the commit 282afa6 to get more accurate results

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

@@          Coverage Diff          @@
##            main    #172   +/-   ##
=====================================
  Coverage   95.51   95.51           
=====================================
  Files        716     715    -1     
  Lines      15673   15658   -15     
=====================================
- Hits       14970   14955   -15     
  Misses       703     703           
Flag Coverage Δ
unit 95.60% <100.00%> (-0.01%) ⬇️
unit-latest-uploader 95.60% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
codecov/settings_base.py 88.13% <100.00%> (+0.06%) ⬆️
upload/helpers.py Critical 93.42% <ø> (-0.31%) ⬇️
upload/serializers.py 100.00% <100.00%> (ø)
upload/views/base.py 100.00% <100.00%> (ø)
upload/views/legacy.py 96.35% <100.00%> (+0.05%) ⬆️
upload/views/uploads.py 98.93% <100.00%> (+0.02%) ⬆️

... and 4 files with indirect coverage changes

@codecov-staging
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!.

@codecov-qa
Copy link

codecov-qa bot commented Oct 3, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5ddb063) 95.61% compared to head (282afa6) 95.61%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #172   +/-   ##
=======================================
  Coverage   95.61%   95.61%           
=======================================
  Files         602      602           
  Lines       15267    15279   +12     
=======================================
+ Hits        14597    14609   +12     
  Misses        670      670           
Flag Coverage Δ
unit 95.61% <100.00%> (+<0.01%) ⬆️
unit-latest-uploader 95.61% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-public-qa
Copy link

codecov-public-qa bot commented Oct 3, 2023

Codecov Report

Merging #172 (282afa6) into main (5ddb063) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #172   +/-   ##
=======================================
  Coverage   95.61%   95.61%           
=======================================
  Files         602      602           
  Lines       15267    15279   +12     
=======================================
+ Hits        14597    14609   +12     
  Misses        670      670           
Flag Coverage Δ
unit 95.61% <100.00%> (+<0.01%) ⬆️
unit-latest-uploader 95.61% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
codecov/settings_base.py 85.31% <100.00%> (+0.08%) ⬆️
upload/helpers.py 93.72% <ø> (ø)
upload/serializers.py 100.00% <100.00%> (ø)
upload/views/base.py 100.00% <100.00%> (ø)
upload/views/legacy.py 96.35% <100.00%> (+0.05%) ⬆️
upload/views/uploads.py 98.93% <100.00%> (+0.02%) ⬆️

Impacted file tree graph

uploads cannot access.
"""
shelter_token = self.request.META.get("HTTP_X_SHELTER_TOKEN")
return shelter_token and shelter_token == settings.SHELTER_SHARED_SECRET
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Is this a go-to approach when trying to identify specific requests? (for my own learning heh)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think shared secrets are a good approach when you're making internal requests between systems (i.e. similar to the Sentry supertoken)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I recall doing some identification through headers, cool!

upload = ReportSession.objects.filter(
report_id=commit_report.id, upload_extras={"format_version": "v1"}
).first()
assert response.status_code == 201
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused: the other code I see is being gated by not self.is_shelter_request(), but this endpoint works. Is this intended or did I miss something? As far as I understand this, the existing logic should only work if it is not a shelter request

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic within not self.is_shelter_request() is the default behavior of generating the storage path in the API. Only requests from Shelter are allowed to pass in their own storage path.

@scott-codecov scott-codecov merged commit f74c61d into main Oct 27, 2023
17 of 18 checks passed
@scott-codecov scott-codecov deleted the scott/upload-path branch October 27, 2023 14:00
scott-codecov added a commit that referenced this pull request Nov 6, 2023
* main:
  Test ats run (#229)
  Release 23.11.2 (#228)
  feat: Add comparison endpoint for impacted files (#212)
  Update shared version (#226)
  feat: add a filter for public/private repo filter for viewableRepositories (#224)
  fix: speedup flag filtering in GraphQL API (#223)
  fix: Disable early notify comment (#219)
  Add 'value' key to the plan representation resolver  (#216)
  713 adjust available plans in plan service (#214)
  feat: Allow upload requests originating from Shelter to set storage path (#172)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants