[Open Discussion] - Create a core cog repo under Cog-Creators

[Drapersniper]

Summary

I've made a suggestion internally, but it would be nice to get the opinion from the community and org members alike.

Points for discussion

• What do you think of the idea?
• Would you use this if it was available?
• Would you want to test out new features in the Experimental branch (Would you use that branch)?
• Can you think of other Pros and Cons?
• Do you think it would be less pressure contributing to the core cogs separately from the main Red repo?
• Do you have any alternative suggestions?

What was proposed

• Create an official repo containing all Core cogs under Cog-Creators (Except Downloader)
• Have 3 branches in the repo, prod, experimental and dev
  • Prod branch would contain all the user safe changes
  • Experimental would have early features which we would be wanting feedback on before these go to the Prod branch
  • Dev where the active development is happening
• Keep the core cogs bundled with Red
  • At every release of Red we pin the cogs currently on Prod and update the Bundled cogs with any changes that gone into the Prod branch

Pros

• Allow users to choose to update their core cogs when they want to
• Allow us to more easily fix issues with a core cog and push out the fix without waiting on a hotfix release from Red (For example:)
  • Fix config conversion in image cog Fix config conversion in image cog #3617
  • Feedback to try to help fix Streamalerts Feedback to try to help fix Streamalerts #3489
  • [Audio] Config schema 2 migration incorrectly recorded as upgraded to _SCHEMA_VERSION = 3 [Audio] Config schema 2 migration incorrectly recorded as upgraded to _SCHEMA_VERSION = 3 #3552
  • [Audio] Local tracks with special characters in path wont return a value in all cases
• Try early features from the Experimental branch and give us early feedback before they are implemented into the prod branch (For example:)
  • [p]streamalert commands should accept links [p]streamalert commands should accept links #3480
  • [Audio] API tutorials should be made to work in DMs [Audio] API tutorials should be made to work in DMs #3471
• Allow users to use the bundled Downloader cog, to core cogs to a specific release using [p]cog pin
• When Downloader allows versioning, update to a specific version of the core cog rather than the latest
• If you don't want to use it you will still be able to use the bundled cogs that will get updated with every release
• Be able to contribute to cog issues without directly PR'ing to the main Red repo
• Allows the org to separate the Red issues and the core cog issues into 2 repos allowing for better prioratization

Cons

• It relies on Downloader to user this repo
• We could just do implement the aforementioned fixes in Red hotfix releases
• It's a radical change
• Would be a little extra work to update the bundled cogs on a release if we want to update the bundled cogs

---

Disclaimer: I'm strongly in favor of going this route, I believe it would be beneficial to Red users and such view may be clouding potential Cons.

[Kowlin]

As you've mentioned, this is a very radical change and I'm gonna say a no with a full stop on it. Downloader is not designed for this kind of load or behavior, and we've discussed the potential issues and bottlenecks regarding this subject within Discord. However this requires a fundamental rewrite on how Downloader works and on how we distribute Red as a software.

As example we have 56 open PRs at the time of writing this post. >90% is directed towards a cog. Asking people to close their PRs and move over to another repository, with another workflow is simply, in my opinion, unacceptable from an open source community.

Other challenges arise too such as CI Workflows, contributing guidelines, repository setup and management. The overhead is simply too big to be managed sanely.

More pressing concerns have also been raised in Discord with updating a individual cog. As of right now when you install Red, you can be guaranteed that it works. (3rd party cogs excluded, hence why their 3rd party in the first place, each person has their own implementation on an issue after all) Some cogs within Red require other cogs to function correctly, think Modlog that relies on Mod, and Economy on Bank.

Hotfixes and minor releases are the answer here, and as I've already said in our Discord channels. Having a clear policy on when and where to do them is the correct way forward here. We already have the infrastructure in effect for them and they require no radical rework on how we work as an open source community.

This is gonna be a hard no from me, it requires too much maintenance to keep working and stable and only adds to our already large workload for an open source piece of software.
I will leave this issue open, however this doesn't mean that this is anywhere near green-lit.

[Dav-Git]

I agree with what Kowlin wrote.

I'd like to also point out that a bunch of issues will inevitably be opened on the wrong repo if this happens. It could turn out to be much harder to communicate the difference to the end user.

Something I would like however is the possibility to choose versions of the core cogs. Without the seperation that's not going to be possible though and the arguments in favor of keeping it as is seem more important overall to me.

[Drapersniper]

Unsure what downloader rewrite would be needed here.

But the idea is that the bundled cogs would be updated on releases as they are now, this repo would just have fixes/updates before they make into the bundled cogs, the user could choose to use in case they need fixes that haven't made into Red yet.

but since cogs won't be delivered via "carrier pigeon" closing this as any attempt to discuss this seems pointless.