bug 1746940: debug_id lookup by code_file/code_id (part 2) #2791
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This captures some additional data for sym files at the time of upload so we can later use it to do code id to debug id lookups and sym file debugging.
When the Socorro processor stackwalker requests sym files with bogus debug id 000000000000000000000000000000000 Tecken would return a 404 because it had no idea what sym file was being requested. This changes the download API so that if there are valid code_file and code_id querystring params, it'll look up the FileUpload record and return the correct debug_id. Then it'll issue an HTTP 302 with the correct download API url. This keeps track of how often it does a successful code_file/code_id lookup with: * `tecken.download_symbol_code_id_lookup`
This adds the data to the upload_file API and adds the data in a Symbol File Data section to the File page in the ui.
willkg
commented
Sep 7, 2023
|
|
||
| class DownloadForm(forms.Form): | ||
| code_file = forms.CharField(required=False) | ||
| code_id = forms.CharField(required=False) |
There was a problem hiding this comment.
We should have removed this form and its usage when we removed the missing symbols bookkeeping a while back. It's vestigial and doesn't do anything now.
willkg
commented
Sep 7, 2023
| @@ -38,9 +42,6 @@ def _ignore_symbol(symbol, debugid, filename): | |||
| if filename == "file.ptr": | |||
| return True | |||
|
|
|||
| if debugid == "000000000000000000000000000000000": | |||
| return True | |||
There was a problem hiding this comment.
We no longer want to ignore this case.
willkg
commented
Sep 7, 2023
| file_upload = ( | ||
| FileUpload.objects.filter(code_file=code_file, code_id=code_id) | ||
| .order_by("created_at") | ||
| .last() |
There was a problem hiding this comment.
.last() returns either an object or None.
willkg
commented
Sep 7, 2023
| :returns: None (no record with that combination) or the debug_id | ||
|
|
||
| """ | ||
| logging.info("looking up by code_file %s code_id %s", code_file, code_id) |
There was a problem hiding this comment.
Oops--this shouldn't be a .info.
willkg
commented
Sep 7, 2023
| settings.MIME_OVERRIDES = {"html": "text/html"} | ||
| assert get_key_content_type("foo.bar") is None | ||
| assert get_key_content_type("foo.html") == "text/html" | ||
| assert get_key_content_type("foo.HTML") == "text/html" |
There was a problem hiding this comment.
I moved these tests to a separate file for upload utils functions.
willkg
commented
Sep 7, 2023
| key_existing.invalidate(client, "publicbucket", "somefile.txt") | ||
| size, metadata = key_existing(client, "publicbucket", "somefile.txt") | ||
| assert size == 0 | ||
| assert metadata is None |
There was a problem hiding this comment.
I moved these tests to a separate file with upload utils functions.
willkg
commented
Sep 7, 2023
| INFO CODE_ID 5DDC1E9E8B000 basic-opt64.dll | ||
| INFO GENERATOR mozilla/dump_syms XYZ | ||
| FILE 0 C:\\Users\\Calixte\\dump_syms\\test_data\\basic.cpp | ||
| FILE 1 d:\\agent\\_work\\2\\s\\src\\vctools\\crt\\vcstartup\\src\\heap\\delete_scalar_size.cpp |
There was a problem hiding this comment.
This header comes from the dump_syms tests. It seemed convenient to test with their test data.
willkg
commented
Sep 7, 2023
| MODULE Linux x86_64 20AD60B0B4C68177552708AA192E77390 basic.full | ||
| INFO CODE_ID B060AD20C6B47781552708AA192E7739FAC7C84A | ||
| FILE 0 /home/calixte/dev/mozilla/dump_syms.calixteman/test_data/linux/basic.cpp | ||
| PUBLIC 1000 0 _init |
There was a problem hiding this comment.
This header comes from the dump_syms tests. It seemed convenient to test with their test data.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We pushed the migration and migrated both stage and prod. This is the followup PR with all the code changes to populate those fields in the upload API and use them for looking up debug_id given a code_file/code_id in the download API.