"Live" support for git submodules

[JustArchi]

I've set up dependabot today for several of my open-source projects and I'm loving it so far, thank you a lot! ❤️

I'm wondering if it could be possible to ask about potential enhancement to submodules feature by giving it "live" updates if dependabot can receive proper webhooks.

Let me explain my setup, I have a core repo C that includes several submodules:

• reference to wiki of repo C (on GitHub)
• a subproject that is a part of core project C
• a tool that is being used during building of project C

Right now dependabot is limited to fetching all of submodules and checking them in periods, but it's totally possible to use GitHub's webhooks for other projects, as long as:

• it's a wiki of the project (that we already have access to)
• or it's other repo under same account/organization that we can access, even if it's not configured by dependabot right now

This way dependabot could configure all found submodules via webhooks and receive live submodules triggers for all the projects that it has access to. It'd basically be a trigger for "Bump now", appropriately optimized to handle only the submodule that the webhook has referenced. Moreover, in the future it could be used even in cross-account scenario where user could get this output even while referencing somebody's else repo, as long as that other person is using dependabot for his own repo too (as you're able to receive webhooks from it in this case, and use them for first user's advantage).

Is there any interest for making something like this happen? I'm mostly interested in the wiki part personally, but I believe that this feature can shine in a lot of different aspects, especially considering that dependabot already has access to everything that is needed, it only requires a bit more logic to add.

Thank you in advance for considering this suggestion. I hope that we can see it happen 🤞

[greysteil]

I'm not sure I understand what you mean by "it's a wiki of the project" - any chance you can link me to an example?

For the case where Dependabot has been given GitHub access to a repo it shouldn't be that hard for us to listen for webhooks. For public repos that don't have Dependabot installed it will be a bit trickier...

[JustArchi]

Sure I can, take a look at my ASF repo: https://github.com/JustArchiNET/ArchiSteamFarm

I'm using GitHub wiki of the project to the maximum degree, so I've also included it as a git submodule. I have it fully localized in several languages, so right now I have a pretty neat setup of dependabot doing wiki submodule bump once wiki is edited, then merging the PR, then my CI building the commit and as part of the process uploading updated source strings for translations on Crowdin (localization) platform. All of that now done automatically thanks to your awesome bot, this is why I'd be interested in cutting daily updates to instant ones, but only as a potential improvement of already working mechanism.

GitHub offers "wiki update" event in the webhook, dependabot could detect that submodule is a GitHub wiki and enable that option for the parent repo in this case.

For public repos that don't have Dependabot installed it will be a bit trickier...

I don't expect miracles, polling each minute for repo update could be possible, but it's very inefficient and would put a lot of pressure on the internal infrastructure if too many people used this at once. However, if you stick only to the repos you have access to (like in my case, where I have all the repos including parent repo of the wiki set up), then you can do it very efficiently and without any issues by just making use of GitHub webhooks mechanism (or any similar API available for bots), and you'd not waste a single request in the process since that webhook would happen only if repo was truly edited. This is why I consider it a good idea for submodules that you can set up in a way to give you live notifications, while those that aren't possible to do that (e.g. public, non-configured, non-supported infrastructures) would still use daily check as a balance between pressure and freshness. Of course you could always improve on this idea if you have some better solution than the one I offer here, but fact is that doing it for already-set repos can be done cost-free and very efficiently for the backend infrastructure.

In any case thank you a lot for considering this, I hope I made it a bit more clear 🙂

[greysteil]

I had no idea wikis worked like that. That's pretty awesome.

I'm 👍 on your suggestion of listening for webhooks on repos (and wikis) we have access to. I'll try to get this done tomorrow, but it might slip a few days. Pester me if I haven't shipped it by the end of the week.

Thanks for the feedback!

[anuejn]

Any progress on this?

[rebelagentm]

Hi! Not yet, unfortunately. The team is still pretty swamped integrating Dependabot into GitHub, so we haven't yet been able to work on new features like this.

[DuBistKomisch]

This would make submodules a lot less painful, have to do this manually at the moment.

[AraHaan]

This can be done easily using simple github actions now, take a look at the workflows in https://github.com/Elskom/Sdk/

I think it could be changed however to always check for live updates though, currently it updates on mine after the 1st 20 minutes of every hour instead of every 20 minutes like I intended to avoid spamming github actions every second on that.

[jeffwidman]

👋 Hey there! Sorry for the slow response... we've been a bit buried.

The "live" feature was not built out when Dependabot preview was migrated to native GitHub, so this isn't currently supported on any ecosystem, not just git submodules. It's unclear when/if we will support this in the future in the native GitHub service. For details/updates, subscribe to this issue:

• Support for schedule.interval: live in GitHub Native Dependabot #3488

However, as a workaround, you can always run Dependabot yourself in a custom GitHub action. I haven't thought through the details, but it's probably not that difficult to wire up the bits necessary to simulate "live" updates via polling... You can also wire up pub/sub if you're willing to run your own service that listens for events. But the gist of it is dependabot-core is just a library... you can run it whatever way you wish.

[jtbandes]

you can always run Dependabot yourself in a custom GitHub action

Are there any examples of what it would take to do this? It would really help to see a minimal working example that could be modified to individual needs. 🙌

[jglick]

If you are running the tool yourself in GHA anyway, you can also consider the likes of https://www.updatecli.io/.