Batch Changes library MVP

[malomarrec]

Problem to solve

To use Batch Changes, users needs to write a spec that are made of several batch change steps. Those can be a simple as a script that uses a tool like sed, call an existing call rewrite tool like fastmod, or be a completely custom script. It can take some time to get to a usable batch spec, so we've started building a library of examples for users to get started faster here.

We want to make it easy for experience code rewrite tool users to get started, building on a template. We also want to make it possible for users that have no knowledge of code rewrite tools to find and run a spec that does what they want, and hence make Batch Changes approachable to many users. This is explained in details in PD 32 REVIEW: Making it easier for users to create or reuse batch changes steps.

Ultimately, we think that there should be a library of batch specs, maintained by Sourcegraph and the community.

Also see RFC 649 WIP: A unified Sourcegraph templates library

Measure of success

For the MVP stage, success means hearing customer feedback in customer interviews that customers were able to create a batch change significantly faster/more easily by using a template.

This should affect MAUs and # changesets merged in the long run.

Customers we're iterating with

• https://github.com/sourcegraph/accounts/issues/580
• https://github.com/sourcegraph/accounts/issues/537
• https://github.com/sourcegraph/accounts/issues/547

Workflow

Batch spec creator perspective

• As an enterprise user of Batch Changes, I want to save specs that I've built other privately with my team so that they can easily reuse it, or maybe with the broader world (to be validated)
• The Batch Changes team wants to curate useful specs, so that users can discover and run them plug and play
• (Needs validation, future use case) As a maintainer of a popular library, I want to ship a batch change along library updates that contain breaking changes to make it easy for downstream users to upgrade.

Batch spec user perspective

As a batch spec user, when I have to automate something with a batch change, the first question I ask myself is "what tool should I use for this?". I want to:

• find ideas of how to automate something
• ideally find a ready-made template, either built inside my company or by the community

Proposal

I think the long-term vision should be to have a library.sourcegraph.com, where users can contribute specs and Sourcegraph also contributes standard specs.

This is loaded onto the sourcegraph instance at runtime, if the instance has internet access. The latest stable library version is also vendored in each release so that if the instance does not have internet access, this can still be available.

Lastly, there's a private library that is stored alongside the instance for batch changes users to privately share their specs.

The library can start by pulling from a single repository, but maybe in the future it should start pulling from multiple repositories (like https://registry.terraform.io), some of which can be community maintained.

Our roadmap will likely look like:

• Step 1: make current examples easily available in-product
• Step 2: make more examples
• Step 3: validate the assumption that batch changes users contribute batch specs if they can do so internally in a private library, to share with their teams
• Step 4: validate the assumption that there would be contributions to a public library (can be done in parallel in a hacky way by encouraging contributions to batch changes examples)

### MVP

MVP will be defined during the offsite

I think what would have value as a first step would be:

• clean up a subset batch changes examples to be usable as is, a few dozen examples.
• make them available from the batch changes instance, and easy to search and navigate

User flows:

• Find the template that I want. The navigation here needs to be though through, but it can be "upgrade java dependencies" or "change docker image" or "change TravisCI config". Should we have a meta section in the spec that has tags, such as ["java","comby","batch spec templating"]
• Select the template to customize then run

Metrics

• number of batch changes created from a template (that is, anything else than the minimal template)
• breakdown of template usage by group / topic

Current status

• Collecting feedback on useful specs to provide
• Defining what the MVP looks like
• Understanding what proportion of users would allow downloading specs from the internet slack
• @malomarrec take batch changes examples, and reorganize it so that there's a contrib folder with all specs, and standard with the specs that work reliably and ard high quality enough to be shipped to customers

[malomarrec]

I think a next step here would be to have a small design workshop thinking about
(1) the library user persona, trying to find a spec to get started with
(2) a user that may want to contribute a spec, either privately or to a public repo of specs

[chrispine]

I love this, thanks! I hadn't really thought much about the origin of a recipe, but now that you call it out, that makes a lot of sense: Is this recipe one we built internally, or one that Sourcegraph made/curated, or a public contribution? (I might trust that last category the least, unless I could tell that lots of people had used it and/or it had a high rating.)

[malomarrec]

Discussing how this could become a Sourcegraph-wide unified library in RFC 649 WIP: A unified Sourcegraph templates library.
This RFC will cover how we link objects (insights, monitors, batch changes) together

[malomarrec]

Discussed with @danielmarquespt, defining the MVP of this will be the goal of the batch changes offsite design sprint.

[malomarrec]

There are great ideas for this in: #36109