Skip to content

Fix automatic contributor refresh#3755

Merged
sgoggins merged 6 commits intomainfrom
bug/contributor_refresh
Mar 10, 2026
Merged

Fix automatic contributor refresh#3755
sgoggins merged 6 commits intomainfrom
bug/contributor_refresh

Conversation

@MoralCode
Copy link
Contributor

Description
in #3748 (comment) @shlokgilda pointed out that A) a bug was preventing the periodic hourly contributor profile refresh task from running, and b) this task was not pulling any useful updated info from the users profile

This PR fixes #3748

Notes for Reviewers
this PR by itself may be broken if #3754 is not also merged - they should be independent as far as merge order though

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
@MoralCode MoralCode added high priority Blocking multiple other things, causing data loss, or other incredibly urgent things deployed version Live problems with deployed versions and removed high priority Blocking multiple other things, causing data loss, or other incredibly urgent things labels Mar 4, 2026
shlokgilda
shlokgilda previously approved these changes Mar 4, 2026
Copy link
Collaborator

@shlokgilda shlokgilda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@MoralCode
Copy link
Contributor Author

tested and now working

2026-03-04 22:11:59 d0c4fff081f1 process_contributors[283] INFO Contributor 177 of 477
2026-03-04 22:11:59 d0c4fff081f1 process_contributors[283] INFO Contributor 178 of 477
2026-03-04 22:11:59 d0c4fff081f1 process_contributors[283] INFO Contributor 179 of 477
2026-03-04 22:11:59 d0c4fff081f1 process_contributors[283] INFO Contributor 180 of 477
2026-03-04 22:12:00 d0c4fff081f1 process_contributors[283] INFO Contributor 181 of 477
2026-03-04 22:12:00 d0c4fff081f1 process_contributors[283] INFO Contributor 182 of 477
2026-03-04 22:12:00 d0c4fff081f1 process_contributors[283] INFO Contributor 183 of 477

Copy link
Member

@sgoggins sgoggins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems to depend on the other one I just reviewed, since its importing it. Lets make sure we merge them both or merge neither of them.

@MoralCode
Copy link
Contributor Author

MoralCode commented Mar 5, 2026

theres a dependency in that sense but the two PRs are independent/can be merged in any order since the divide between them is inside vs outside that data access function

they dont have to both be merged, but thats what i would strongly recommend doing. the testing i did above was with all 4 prs

@MoralCode MoralCode added this to the v0.93.0 milestone Mar 6, 2026
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
to get a better sense of which users cause this hourly task to fail

Signed-off-by: Adrian Edwards <adredwar@redhat.com>
@MoralCode
Copy link
Contributor Author

Pushed additional commits to partially address this failure we are seeing in prod

Stack Trace
Traceback (most recent call last):
  File "/augur/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/contributors.py", line 52, in process_contributors
    data = retrieve_dict_data(url, key_auth, logger)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/contributors.py", line 84, in retrieve_dict_data
    page_data = response.json()
                ^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/httpx/_models.py", line 743, in json
    return jsonlib.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

TL;DR it seems like one contributor is returning bad data in a way that raises an exception and isnt caught/skipped by existing logic.

I have manually checked a few users to see what the issue could be (my own, the GH ghost user, and a user i know has a private profile) and none returned non-JSON data.

The commits i added allow the process to skip and log contributors that demonstrate this error so the task can proceed.

Waiting for an oppotunity to redeploy this to our prod and test

Signed-off-by: Adrian Edwards <17362949+MoralCode@users.noreply.github.com>
@sgoggins sgoggins merged commit 301b2cd into main Mar 10, 2026
20 of 21 checks passed
@MoralCode MoralCode deleted the bug/contributor_refresh branch March 10, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deployed version Live problems with deployed versions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Periodic checks to contributor GH profile for new information

4 participants