Does python-gitlab support http based caching?

[notsatyarth]

Hi there 👋🏽 .
I was wondering if the python-gitlab sdk supports http layer caching? I've noticed that the amount of time taken to execute a request for historical data results in a consistent time of execution. whereas I would expect that subsequent requests for the same dataset would be a lot faster.

[nejch]

Hi @notsatyarth, the library itself does not, but it's extremely easy to add using https://pypi.org/project/requests-cache/.

It's been asked often enough that we should at least document it or maybe add it as an optional feature. I'm unfortunately busy now, but will come back here to give you an example if you don't find a solution yourself 🙇

Please note, it's not always good to cache by default, as you might get outdated data when fresh data might matter.

[notsatyarth]

Thanks @nejch !

I agree with the point you raise, however I would expect gitlab's etag header to be requested for this? So that should also help in invalidating the cache for those requests.
There may still be other requests which we may want to explicitly not cache, and I agree that we should handle those carefully.

[nejch]

@notsatyarth true, at least requests-cache handles all of that already (https://requests-cache.readthedocs.io/en/stable/user_guide/headers.html). So maybe less of an issue but depends on how paranoid about data accuracy you need to be 😅

Let me know how you go with this. Just found an old example of mine, which was as simple as this (obviously no granular caching here):

import os

import gitlab
import requests_cache

CACHE_DIR = Path.cwd() / ".cache"
CACHE_SECONDS = 60 * 60 * 24
GITLAB_URL = "https://gitlab.example.com"

requests_cache.install_cache(CACHE_DIR, expire_after=CACHE_SECONDS)
gl = gitlab.Gitlab(
    GITLAB_URL,
    private_token=os.getenv("GITLAB_TOKEN"),
    per_page=100,
    retry_transient_errors=True,
)

[notsatyarth]

Thank you again, the docs for helped for this too. Looks like gitlab apis do not have much caching in place either :)
The most common header I see for listing apis are cache-control: max-age=0, private, must-revalidate which means I would need to look into customizing the caching logic here. Thanks for the swift reply!