Failsafe on API errors

[vic1707]

Hi,

TL/DR: Might be great if gickup throws an error (and/or a push notif) when encountering an API 4XX error.

I was experimenting with Gickup and trying to find a way to backup whole organisations without being a member of it, or how to backup specific repositories.

For orgs I still can't figure it out.
For specific repos using the any source and simply putting the github url does the trick but I wanted to try other ways.

But upon testing

see current try:

# cron: 0 22 * * *

log:
    timeformat: 2006-01-02 15:04:05
    file-logging:
        dir: data
        file: gickup.log
        maxage: 7

destination:
    local:
        -
            path: /data
            structured: true
            zip: true
            keep: 5
            bare: true
            lfs: false

source:
    github:
        -
            token_file: /config/token # fine grained token for my specific user
            user: vic1707
            excludeorgs:
                - adeo
                - Zenika
                - Zenika-Training
            wiki: true
            filter:
                excludeforks: true

        -
            user: hsutter
            include:
                - cppfront

        -
            user: Zelda-Reverse-Engineering-Team


    # works as is for one repo but I'm trying different things
    # any:
    #     -
    #         user: https://github.com/hsutter/cppfront

I got lots of errors from github-api (due ton invalid config so kinda expected) and gickup started spamming their API at astonishing speed reaching rate limits

2024-07-27 22:34:40 ERR GET https://api.github.com/users/hsutter/repos?page=1&per_page=50: 403 API rate limit of 60 still exceeded until 2024-07-27 23:16:23 +0200 CEST, not making remote request. [rate reset in 41m42s] stage=github url=https://github.com
2024-07-27 22:34:40 ERR GET https://api.github.com/users/hsutter/repos?page=1&per_page=50: 403 API rate limit of 60 still exceeded until 2024-07-27 23:16:23 +0200 CEST, not making remote request. [rate reset in 41m42s] stage=github url=https://github.com
2024-07-27 22:34:40 ERR GET https://api.github.com/users/hsutter/repos?page=1&per_page=50: 403 API rate limit of 60 still exceeded until 2024-07-27 23:16:23 +0200 CEST, not making remote request. [rate reset in 41m42s] stage=github url=https://github.com
// hundreds if not thousands of the same lines because go is fast x)

I think it would be nice to have a way for gickup to not freak out when things like these happen and eventually to share that news via push notif or something.

---

on a side note, if you can hint me on how (if possible) I can backup specific orgs even when I'm not a member as I don't want to star everything manually. And if you can tell me what I messed up with the hsutter config

[cooperspencer]

Hi,
Yeah, the hsutter part is easy, you also need to add a token to this part of the configuration. As for the failsafe, I will need to think a bit about that. It makes total sense, but I need to think on where to add that.

[vic1707]

Oh ok, easy fix indeed. Thought for another user + public repo meant I could omit the token 👍
As for my question about organization (I believed it got lost in my message) is it possible to backup specific orgs even when I'm not a member of said org? I naively tried this but didn't work as orgs are not users.

        - user: Zelda-Reverse-Engineering-Team
          token_file: /config/token # fine grained token for vic1707

2024-07-28 21:05:59 ERR GET https://api.github.com/users/Zelda-Reverse-Engineering-Team/repos?page=1&per_page=50: 404 Not Found [] stage=github url=https://github.com

---

It makes total sense, but I need to think on where to add that.

I'm not a go developer but, maybe you can add a middleware or wrapper to every kind of API calls and check for the result ?
Return either

• the data
• the error (if 5XX or else)
• a special variable that tells gickup to not try again
  At least that's probably how I'd do it in JS/TS/RUST 🤔

[cooperspencer]

Sorry, I forgot to answer the org question. Users and Organizations are treated in a similar way, so you can just use the organization as user and it should work. For your case with the Zelda-Reverse-Engineering-Team, are you sure they are not just called zeldaret?
https://github.com/zeldaret

I need to investigate why it prompts the error message a gazillion times when something doesn't exist. Smells like a bug.
But otherwise, most of the modules I use to grab repositories also return the error code. That would be a great place to start.

[vic1707]

display name != username 🤦, crap, I feel stupid now x) thank you that was it