Sustaining the Aspire Community Toolkit as Aspire evolves #1338
Replies: 1 comment 6 replies
-
|
The CI stability thing seems to be getting pressing. Just slung up a PR today (powershell) but five other integrations/hosting implementations failed their CI :/ |
Beta Was this translation helpful? Give feedback.
-
|
You need to avoid running tests for integraitons that have not changed. This is the single biggest win for the CT because integrations do not depend on each other. They are usually islands. @radical is building out tech on the aspire repo do explore this for our integrations. We'd basically skip running a large number of tests and reducing the chances of flaking out because of a networking / docker / filesystem / gha blip. We also auto-re-run some actions if they fail in specific ways. CI reliability is something the core aspire repo struggles with as well but making it reliable while having so many external dependencies is a non trivial problem. |
Beta Was this translation helpful? Give feedback.
-
|
I've just merged #1355 in which will hopefully reduce the amount of tests that are run on a PR so there's a tighter feedback loop. I'd be keen to have the auto-re-run stuff you mention @davidfowl - can you point me to that in the aspire repo? |
Beta Was this translation helpful? Give feedback.
-
|
I am not sure what is the recommendation but you can take a look at https://kritner-blogs.github.io/FlakyTest.XUnit/ |
Beta Was this translation helpful? Give feedback.
-
|
It's not the tests that are flaky, it's the runner itself |
Beta Was this translation helpful? Give feedback.
-
|
Ah, I see. Do you know if this is specific to the xUnit runner or just the MTP? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
You may have noticed that activity in the Aspire Community Toolkit has slowed over the last few months. I want to talk about that, what happened during the 13.2 / 13.3 release cycle, and how we make the Toolkit more sustainable going forward.
But before we get into that, I want to celebrate that we shipped support for Aspire 13.3, including experimental support for the TypeScript AppHost. Thanks to all those who contributed to the release.
I also want to acknowledge that we still have quite a few open issues and PR's, including ones that are introducing new integrations. There's still work to be done.
Where did 13.2 go?
If you look at our release history, we shipped 13.1.1 but then the next jump was to 13.3.0, skipping 13.2.x entirely, and not just in our release numbering, we also never shipped a release that supported Aspire 13.2, so that begs the question - where did it go?
The short version is that too much of the release-readiness work still depends on one person having enough time and context to drive it. That does not scale well for a project that supports a fast-moving ecosystem. The Toolkit has always sat adjacent to my core role rather than being the focus of it, so it relied on available time that has been harder to find over the last six months as other projects have demanded more attention.
We have a dedicated group of contributors to the Toolkit, and they help keep the integrations they know best maintained. But the cross-cutting work, such as release readiness, CI health, and coordinating broad compatibility updates, still tends to fall to me because I have the most context across the whole repo.
But there is another critical factor I want to mention.
Keeping up with Aspire's velocity
Aspire is moving quickly, and that is a good thing for the ecosystem. But it also means the Toolkit needs a better way to stay aligned with changes before they land in a stable release.
Aspire 13.2 was an inflection point for us, especially around supporting the TypeScript AppHost. We were not close enough to those changes early enough to have the right testing infrastructure or a clear release plan in place. By the time the release was out, the Toolkit needed more cross-cutting work than a simple version bump.
That is the pattern we need to improve: not slowing Aspire down, but making sure the Toolkit has earlier signals, better validation, and more people able to help when platform changes affect integrations.
Stretching our CI to breaking point
The Toolkit has always put a lot of emphasis on confidence: integrations should have tests, sample apps, and validation across Windows and Linux. That has served us well, but the repo has grown to the point where each PR now runs 120+ GitHub checks across the integration test matrix.
Running those jobs in parallel keeps total runtime manageable, but it also means every PR spins up a large number of runners. In practice, some jobs fail intermittently for reasons unrelated to the change being tested, and rerunning the failed jobs often turns the build green.
That makes CI less of a clear signal and more of something maintainers have to babysit. Improving that reliability is now part of making the Toolkit sustainable.
Making the Toolkit more sustainable
Between a perfect storm of a lack of time and an increased complexity requirement of said time, we find ourselves where we are at today, and that begs the question, what now?
I have one main commitment here: the Aspire Community Toolkit should be seen as the best source of integrations for Aspire outside of the core repo, with the level of maintenance and confidence the community needs to trust those integrations are worth using.
Over the last couple of weeks, I've been discussing with the Aspire team how we can continue with this mission, and one of the goals we have is to improve the lines of communication. Hopefully we can avoid being slow on the adoption of large compatibility-impacting changes like we were with 13.2 and polyglot support. Even still, as complex features land, we are looking to the Aspire team to help uplift the Toolkit with them, bringing subject-matter experts to contribute rather than leaving us to figure it out from PRs and source code.
The other request I have is for the community. I'm looking for passionate members of the Aspire community to help with:
Through this, we remove the single bottleneck on the project that I can become. Because of how the Community Toolkit org operates, there as certain jobs that will always have to fall to me (such as the initial release of a new integration), other tasks to ensure a sustainable OSS project can be democratised.
If that's something that interests you, please do reach out, my email is on my profile, or you can hit me up on Discord (or there's plenty of other ways someone can figure out how to get in contact with me).
I'm really proud of what we've built so far, so let's make sure we can keep moving it forward.
Beta Was this translation helpful? Give feedback.
All reactions