PR: Use GITHUB_TOKEN to make authorized user requests against GitHub (CI)

[mrclary]

Some unit tests intermittently fail due to 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/spyder-ide/spyder/releases.

Rate Limit Failure

=================================== FAILURES ===================================
__________________________ test_updates_appenv[1.0.0] __________________________

qtbot = <pytestqt.qtbot.QtBot object at 0x7efba47508e0>
mocker = <pytest_mock.plugin.MockerFixture object at 0x7efba4750940>
version = '1.0.0'

    @pytest.mark.parametrize("version", ["1.0.0", "1000.0.0"])
    def test_updates_appenv(qtbot, mocker, version):
        """
        Test whether or not we offer updates for our installers according to the
        current Spyder version.
    
        Uses UpdateManagerWidget in order to also test QThread.
        """
        mocker.patch.object(update, "__version__", new=version)
        # Do not execute start_update after check_update completes.
        mocker.patch.object(
            UpdateManagerWidget, "start_update", new=lambda x: None
        )
        mocker.patch.object(workers, "__version__", new=version)
        mocker.patch.object(workers, "is_anaconda", return_value=True)
        mocker.patch.object(workers, "is_conda_based_app", return_value=True)
        mocker.patch.object(
            workers, "get_spyder_conda_channel",
            return_value=("conda-forge", "https://conda.anaconda.org/conda-forge")
        )
    
        um = UpdateManagerWidget(None)
        um.start_check_update()
        qtbot.waitUntil(um.update_thread.isFinished)
    
        update_available = um.update_worker.update_available
        if version.split('.')[0] == '1':
>           assert update_available
E           assert False

spyder/plugins/updatemanager/tests/test_update_manager.py:60: AssertionError
------------------------------ Captured log call -------------------------------
DEBUG    spyder.plugins.updatemanager.widgets.update:update.py:166 Checking for updates. startup = False.
INFO     spyder.plugins.updatemanager.workers:workers.py:153 Checking for updates from https://api.github.com/repos/spyder-ide/spyder/releases
WARNING  spyder.plugins.updatemanager.workers:workers.py:182 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/spyder-ide/spyder/releases
Traceback (most recent call last):
  File "/home/runner/work/spyder/spyder/spyder/plugins/updatemanager/workers.py", line 156, in start
    page.raise_for_status()
  File "/home/runner/micromamba/envs/test/lib/python3.8/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/spyder-ide/spyder/releases
________________________ test_updates_appenv[1000.0.0] _________________________

qtbot = <pytestqt.qtbot.QtBot object at 0x7efba4750b50>
mocker = <pytest_mock.plugin.MockerFixture object at 0x7efba4750d30>
version = '1000.0.0'

    @pytest.mark.parametrize("version", ["1.0.0", "1000.0.0"])
    def test_updates_appenv(qtbot, mocker, version):
        """
        Test whether or not we offer updates for our installers according to the
        current Spyder version.
    
        Uses UpdateManagerWidget in order to also test QThread.
        """
        mocker.patch.object(update, "__version__", new=version)
        # Do not execute start_update after check_update completes.
        mocker.patch.object(
            UpdateManagerWidget, "start_update", new=lambda x: None
        )
        mocker.patch.object(workers, "__version__", new=version)
        mocker.patch.object(workers, "is_anaconda", return_value=True)
        mocker.patch.object(workers, "is_conda_based_app", return_value=True)
        mocker.patch.object(
            workers, "get_spyder_conda_channel",
            return_value=("conda-forge", "https://conda.anaconda.org/conda-forge")
        )
    
        um = UpdateManagerWidget(None)
        um.start_check_update()
        qtbot.waitUntil(um.update_thread.isFinished)
    
        update_available = um.update_worker.update_available
        if version.split('.')[0] == '1':
            assert update_available
        else:
            assert not update_available
>       assert len(um.update_worker.releases) > 1
E       TypeError: object of type 'NoneType' has no len()

spyder/plugins/updatemanager/tests/test_update_manager.py:63: TypeError
------------------------------ Captured log call -------------------------------
DEBUG    spyder.plugins.updatemanager.widgets.update:update.py:166 Checking for updates. startup = False.
INFO     spyder.plugins.updatemanager.workers:workers.py:153 Checking for updates from https://api.github.com/repos/spyder-ide/spyder/releases
WARNING  spyder.plugins.updatemanager.workers:workers.py:182 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/spyder-ide/spyder/releases
Traceback (most recent call last):
  File "/home/runner/work/spyder/spyder/spyder/plugins/updatemanager/workers.py", line 156, in start
    page.raise_for_status()
  File "/home/runner/micromamba/envs/test/lib/python3.8/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/spyder-ide/spyder/releases

What is the root cause of this and how do we resolve it?

[mrclary]

I modified the test workflows for macOS, Linux, and Windows to report the rate limits. Following are excerpts from a failed Linux workflow:
16_Rate Limit Before Tests.txt
17_Run tests.txt
18_Rate Limit After Tests.txt

These show that before and after the tests are run, rate limits have not been exceeded. Nevertheless, the test fails due to an exceeded rate limit.

I conclude from this that the rate limit exceeded in the test is a "secondary" rate limit which is not reported in the rate limit queries.

[mrclary]

Reference: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28

[mrclary]

So I've discovered a few things

• There are no requests to github in our tests that are being counted against a rate limit except for those in test_update_manager.py. These amount to 12 requests: 2 for each test times 6 tests (1 macOS, 2 Windows, and 3 Linux).
• The GITHUB_TOKEN was not passed to requests.get in the update manager worker, so the rate limit was counting against unauthenticated requests (limited to 60).
• Passing the GITHUB_TOKEN to requests.get results in the expected 5000 limit and we see the counts incurred by the tests (see https://github.com/spyder-ide/spyder/actions/runs/9215667723/job/25354472458)
• Unfortunately, both logging and warnings that report the rate limits from within the update manager call are not displayed in the CI logs. This is because these are displayed at the end of the test run which cut short when segfaults occur. Subsequent automated rerun of the test will not run the update manager test because it already succeeded.

But I think that the changes in this PR will fix the problem.

One thing I'm curious about is whether unauthenticated requests are counted against individual IP addresses or against the repository. If the repository, then we could have an issue if more Spyder instances in the wild are sending unauthenticated requests.

[mrclary]

Oh yep, "Unauthenticated requests are associated with the originating IP address". Okay, I think we're good.

[mrclary]

I think we were just hitting the 60 count limit by having multiple (~5) PRs running tests in close proximity.

[ccordoba12]

Thanks @mrclary for your help to solve this!

[ccordoba12]

Is this really necessary for production code? Or only for testing?

[mrclary]

The motivation was for investigating the issue in our test suite. However, when our installers become mainstream, it may be prudent to have a debug log message in production in case users encounter a rate limit error. In production, headers={}, then requests.get(url) and requests.get(url, headers={}) are equivalent.

[mrclary]

Clarification: currently, only macOS and Windows 5.x standalone installers check Github for updates. When 6.0 is released, we could (hopefully) see much more widespread adoption of our installers, resulting in increased unauthorized requests to Github.

[ccordoba12]

Looks good to me now, thanks @mrclary!