Request to maintain xk6 extensions as a part of monorepo

[rverma-dev]

Currently, there are many different xk6 repos for each extension and thus making the extensions easy to develop. However, it seems that the extension's dependency is getting out of sync a lot and many versions of the same artifact are available in different extensions, including k6 itself (0.28...0...0.40.0). In fact go versions are also ranging out from 1.15 -> 1.18 within extensions.

Being said that it's very common for us to use different extensions together, like load test SQL and send metrics to Prometheus.

I sincerely request that k6 maintainers consider providing a monorepo for all the k6 extensions. This also lowers the barrier to submitting new k6 extensions to a common pool, auto-generate docs, enforcing better code reviews etc...

We also created an xk6 extension for Cassandra/scylladb https://github.com/rverma-nsl/xk6-cassandra which we would love to contribute.

Note: We can merge these repos without loosing commits from initial contributors using the steps https://stackoverflow.com/questions/1425892/how-do-you-merge-two-git-repositories.

If grafana creates an empty repo with xk6-extensions with a readme, I can contribute with merged repo for the extensions we are using.

[imiric]

Hi there, thanks for opening this issue.

A monorepo is not something we've considered for k6 extensions, but it sounds interesting. I think it would be a challenge to keep the same k6 dependency and Go versions pinned for all extensions, simply because there may be incompatibilities that would be overwhelmingly difficult to fix across all extensions. Separate repositories per extension makes more sense in this case, since authors are free to upgrade at their discretion.

Another reason to keep extensions separate is because code ownership and licensing becomes an issue. Will users expect the Grafana organization to take over maintenance of all extensions in the monorepo? This would be unmanageable for our k6 teams. Some extension authors might want to use a different license, and we don't want to enforce any one in particular.

Tooling is another problem. Monorepos usually require specific build systems, CI configuration, VCS workflows, etc. to be manageable. It's a lot of overhead that we don't have to deal with in separate repos.

Monorepos are a good fit in a single organization, where a single development model can be followed, common libraries can be shared, etc. Making them work in a distributed way where projects are only loosely connected would introduce more issues than it would solve IMO.

So I don't think we'll adopt this anytime soon, but I'm curious to read other opinions about it. I'll share this with the rest of the team.

[javaducky]

I'm in agreement with @imiric here. While it would be awesome to keep all the repositories up-to-date, I don't think this is feasible given the volatility of some of the extension codebases in general.

Another issue I can think of as well is that we introduced our own breaking changes to the extensions API within k6 itself--a necessary evil that we planned for. In a monorepo, of course, we'd have to upgrade N number of extensions which would have been a major undertaking---again, one that needs to be done, but as priorities allow.

On another note with regard to developer experience building with xk6, we're hoping to provide a facility to make this easier for developers and testers alike to simply select desired extensions (as with https://k6.io/docs/extensions/getting-started/bundle/) and download the resulting binary. Again priorities.

Thank you so much for your feedback though! We really appreciate any suggestions/ideas as we cannot improve within a vacuum; any insights from the community really make a difference.

[rverma-dev]

Thanks for sharing feedback. I just want to share that we started the concept of monorepo for

1. We want to use multiple extensions together which becomes incompatible because of the k6 api version mismatch. Which means that over the time if extensions grow they could easily become unmaintained too.
2. If you look at adoption lifecycle even helm started with a monorepo for charts, one of too 5 fastest growing community maintained project. Xk6 i personally beleive is more of a package manager for k6 too rather then a contract bind microservice architecture. Polyrepo packge managers like npm are also centralized behind the scene, while brew apt etc are already centralized.
3. With GitHub actions the automation for multiple k6 extensions is not a challenge, as I mentioned i can help with that. Infact it will encourage to write automation for 1 extension which can be used by others. Write a test automation for 1 and use n times. The whole principle of DRY.
4. Licensing is tricky depending on root licence grafana want to opt for. I personally saw that community is very active for Apache and MIT licence in term of contribution or donation. We personally offered our Apache license Cassandra/scylladb extension too.

[rverma-dev]

Here is an example of the approach https://github.com/nslhb/xk6-extensions.
Note the few advantages

1. Build time with multiple extensions is very much optimised (atleast on my m1 Mac). No benchmark but I can see that for me when I am building with just 4 extensions it used to be ~2min which is no <20sec
2. Dependency management is easy. We can certainly explore go work or bazel to make it bit lean.
3. Reduction of boilerplate and automations
4. We are not at all sacrificing the ability to pick particular module, in fact we are enforcing that at a given point of time all the modules will work with each other.
5. We can further use dependabot to keep all the modules deps updated.
6. Barrier to write new modules should replace, if require we can introduce a bootstrap script which can generate register.go and basic module extension + test etc... to reduce the barrier further.

[mstoykov]

Hi @rverma-nsl, thank you for opening this issue 🙇

We have discussed this internally within the k6 core team and have unanimously agreed to not do this at this time. Maybe at some future point we might decide to do something like the proposed thing for some grafana owned extensions, but that is to be decided in the future. And here future is at least a year, probably more from now.

Let me explain our reasoning, which will echo some of the above posts, and the original reason we did make xk6 and the extensions possible.

Some history

Extensions support in k6 was added close to two years ago.¹

The main reason why the core k6 team wanted support for that was actually to enable users to write extensions that the core team will not need to review, support and maintain. This is still the primary use for extensions - functionality for k6 that the core team doesn't need to take care off.

This issue in practice asks this to not be the case and the core team to start maintaining all/most extensions.

We do have a mono repo in practice ;)

We do have a mono repo of extensions actually - that is k6 (from here on I will refer to it as "k6 core"). Not only are extensions mostly just go code that looks exactly the same as any output and/or js exposed module that lives in k6, we even now have experimental modules that are extensions. You can see the last few commits on this PR on what we needed to do to not have a broken k6 or a broken extension for any period of time.

Doing this for a lot of extensions is definitely something that will impact how fast we can do any kind of change.

That being said we(I specifically have done it on multiple occasions) go and make PRs to some extensions fixing the incompatibilities. That has shown us that, both not every extension is maintained, even as little as merging PRs, but also that some extensions do "strange" stuff that we can't fix or even ... don't really work ;)

Extensions are experiments(for the most part)

Both the extensions API and the majority(I would argue all) of the extensions are experiments. We intentionally have never given any guarantees that your extension will compile with the next k6 or that any extension will be maintained in any respect from the k6 core team.

Yes there are extensions made by k6 core team members and are continuously worked on. There are even some that we try to put in the core - the experimental ones from above. But even the majority of the extensions under the grafana organization are experiments somebody did. They might've needed to test a k6 feature, load test something the core doesn't have support for and made an extension for it, had great idea that don't have time for. All of those were valuable, and might still be, but are not maintained by k6 core team.

We can take even less responsibility for anything that anybody else has made. In some of the cases we won't even have the legal right to do so due to licensing.

But even if we could, and we moved them under the grafana organization - that then means that we say "this works, somewhat". Maintaining it will be a burden on the team, we will need to actually see if it is not malicious, fix bugs with it, add functionality and so on.

Extensions are/should be mostly community driven

IMO (and not only) if somebody did make an extension and then abandoned them and nobody else from the community wants to fork and work on it - that extension isn't that useful² and should not be maintained. If at some point this changes, somebody will fork and work on it.

This is what OSS is arguably great at - you can fork and continue working on code and use it even if the original author doesn't want to work on it anymore.

So anybody (you included) can fork and work on maintaining the extension they want to use. One of the great things about this is that you don't need to care about what my opinion is on what is:

1. important
2. good UX
3. enough functionality
4. w/e else

You can do w/e and people have done so - they have made extensions the core team has clearly said they will never merge in core (like writing arbitrary files to the filesystems thought the test).

Obviously this is all within respecting the actual licensing of the original extension.

Technical problems

I am not going to even try to argue on the problems of this approach technically as ... well we are not going to do it.

But you are free to do - no idea whether some licensing issue will be a problem, I am not a lawyer.

You have even already done it - now you just need to maintain it with the extension that you find valuable and others can help you and/or just use it 🎉.

Automation ideas:

I am okay with more template repos for extensions and examples on how to automate them. I would prefer if those start outside of official k6 repos, and then we can bring them in once people have used them for a while. We do have a template for output extensions, but never did one for js module ones.

You can even make this monorepo for extension as a template so that people can use that to make their own.

I personally have made just a new package (within one of the extensions) with a main.go file that mimics what xk6 does and use that when I have had problems with reproducing something and needed to recompile a lot.

But the actual reason why I needed it was to get go tool pprof to work, which also is useful.

With that I think I will close this issue. If you want to open issue for a concrete improvement, or maybe a template you want to contribute - go for it.

Thanks again for the issue and hope this explains to you why the k6 core team doesn't want and won't be doing a mono repo of all extensions soon.

Footnotes

1. There is a lot of usage of the word plugin in some of the early discussion, but due to the way the final system works we are calling them extensions ↩

2. maybe it was super useful at the time and is no longer useful due to changes to k6. Or was just a bad idea. Or the user moved on to use a different tool, career, something else. Or some better idea came up. No matter the case, if it is useful somebody will maintain it. ↩