Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/batch_release_pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: "Creates Batch Release for go_router package"

on:
repository_dispatch:
types: [batch_release_pr]

jobs:
create_release_pr:
runs-on: ubuntu-latest
env:
BRANCH_NAME: ${{ github.event.client_payload.package }}-${{ github.run_id }}-${{ github.run_attempt }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the cron job is constantly creating new branches? What cleans up branches that don't actually end up published?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that can be a problem. then I think the right thing to do will be re-use the same branch name and do a clean up before creating a new release

steps:
- name: checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
- name: Set up tools
run: dart pub get
working-directory: ${{ github.workspace }}/script/tool
# Give some time for LUCI checks to start becoming populated.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed in the context of this action?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is copied from release.yaml. Do you want me to factor the set up tools github action to a separate one? so that we don't duplicate code in a bunch of place

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's copied from release.yaml, I'm asking why. The next step in release.yaml is to wait for all the checks to complete, which is why the comment on it says:

Because of latency in Github Webhooks, we need to wait for a while
before being able to look at checks scheduled by LUCI.

I don't see anything in the rest of this file that looks at LUCI-created check runs.

Do you want me to factor the set up tools github action to a separate one?

Waiting for LUCI to schedule tasks is unrelated to repo tool setup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I see, I misunderstood the reason for adding the delay. yeah I will remove it

# Because of latency in Github Webhooks, we need to wait for a while
# before being able to look at checks scheduled by LUCI.
- name: Give webhooks a minute
run: sleep 60s
shell: bash
- name: create batch release PR
run: |
git config --global user.name ${{ secrets.USER_NAME }}
git config --global user.email ${{ secrets.USER_EMAIL }}
dart ./script/tool/lib/src/main.dart branch-for-batch-release --packages=${{ github.event.client_payload.package }} --branch=${{ env.BRANCH_NAME }}"
- name: Create Pull Request
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "Batch release PR for ${{ github.event.client_payload.package }} package"
title: "Batch Release PR for ${{ github.event.client_payload.package }} package"
body: "This PR was created automatically to batch release the ${{ github.event.client_payload.package }} package."
branch: ${{ env.BRANCH_NAME }}
base: release


16 changes: 16 additions & 0 deletions .github/workflows/go_router_batch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: "Creates Batch Release for go_router package"

on:
schedule:
- cron: "0 0 * * *"

jobs:
dispatch_release_pr:
runs-on: ubuntu-latest
steps:
- name: Repository Dispatch
uses: peter-evans/repository-dispatch@v4
with:
token: "${{ secrets.GITHUB_TOKEN }}"
event-type: batch_release_pr
client-payload: '{"package": "go_router"}'
6 changes: 6 additions & 0 deletions packages/go_router/pending_changelogs/template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Use this file as template to draft a unreleased changelog file.
# Make a copy of this file in the same directory, rename it, and fill in the details.
changelog: |
- Can include a list of changes.
- with markdown supported.
version: <major|minor|patch|skip>
6 changes: 6 additions & 0 deletions packages/go_router/pending_changelogs/test_only_1.yaml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these files are going to be created as part of the PR process, then the tooling should be updated to understand that, and the override labels shouldn't be needed on this PR. If the PR isn't landable without overrides, then the normal commit path isn't working correctly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that part of the work will be in flutter/flutter#176433.

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# This file is for test purposes only.
# TODO(chuntai): remove this file before publishing.
changelog: |
- Added 'batch' option to CI config for go_router package.
- Updated GitHub Actions workflow for batch releases of go_router.
version: major
6 changes: 6 additions & 0 deletions packages/go_router/pending_changelogs/test_only_2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# This file is for test purposes only.
# TODO(chuntai): remove this file before publishing.
changelog: |
- Added 'batch' option to CI config for go_router package.
- Updated GitHub Actions workflow for batch releases of go_router.
version: major
311 changes: 311 additions & 0 deletions script/tool/lib/src/branch_for_batch_release_command.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,311 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:io' as io;
import 'dart:math' as math;

import 'package:file/file.dart';
import 'package:git/git.dart';
import 'package:pub_semver/pub_semver.dart';
import 'package:yaml/yaml.dart';
import 'package:yaml_edit/yaml_edit.dart';

import 'common/core.dart';
import 'common/output_utils.dart';
import 'common/package_command.dart';
import 'common/repository_package.dart';

const int _kExitPackageMalformed = 2;
const int _kGitFailedToPush = 3;

const String _kTemplateFileName = 'template.yaml';

/// A command to create a remote branch with release changes for a single package.
class BranchForBatchReleaseCommand extends PackageCommand {
/// Creates a new `branch-for-batch-release` command.
BranchForBatchReleaseCommand(
super.packagesDir, {
super.processRunner,
super.platform,
super.gitDir,
}) {
argParser.addOption(
'branch',
mandatory: true,
abbr: 'b',
help: 'The branch to push the release PR to.',
);
}

@override
final String name = 'branch-for-batch-release';

@override
final String description = 'Creates a release PR for a single package.';

@override
Future<void> run() async {
final String branchName = argResults!['branch'] as String;

final List<RepositoryPackage> packages = await getTargetPackages()
.map((PackageEnumerationEntry e) => e.package)
.toList();
if (packages.length != 1) {
printError('Exactly one package must be specified.');
throw ToolExit(2);
}
final RepositoryPackage package = packages.single;

final GitDir repository = await gitDir;

print('Parsing package "${package.displayName}"...');
final PendingChangelogs pendingChangelogs =
await _getPendingChangelogs(package);
if (pendingChangelogs.entries.isEmpty) {
print('No pending changelogs found for ${package.displayName}.');
return;
}

final Pubspec pubspec =
Pubspec.parse(package.pubspecFile.readAsStringSync());
final ReleaseInfo releaseInfo =
_getReleaseInfo(pendingChangelogs.entries, pubspec.version!);

if (releaseInfo.newVersion == null) {
print('No version change specified in pending changelogs for '
'${package.displayName}.');
return;
}

print('Creating and pushing release branch...');
await _pushBranch(
repository: repository,
package: package,
branchName: branchName,
pendingChangelogFiles: pendingChangelogs.files,
releaseInfo: releaseInfo,
);
}

Future<PendingChangelogs> _getPendingChangelogs(
RepositoryPackage package) async {
final Directory pendingChangelogsDir =
package.directory.childDirectory('pending_changelogs');
if (!pendingChangelogsDir.existsSync()) {
printError(
'No pending_changelogs folder found for ${package.displayName}.');
throw ToolExit(_kExitPackageMalformed);
}
final List<File> pendingChangelogFiles = pendingChangelogsDir
.listSync()
.whereType<File>()
.where((File f) =>
f.basename.endsWith('.yaml') && f.basename != _kTemplateFileName)
.toList();
try {
final List<PendingChangelogEntry> entries = pendingChangelogFiles
.map<PendingChangelogEntry>(
(File f) => PendingChangelogEntry.parse(f.readAsStringSync()))
.toList();
return PendingChangelogs(entries, pendingChangelogFiles);
} on FormatException catch (e) {
printError('Malformed pending changelog file: $e');
throw ToolExit(_kExitPackageMalformed);
}
}

ReleaseInfo _getReleaseInfo(
List<PendingChangelogEntry> pendingChangelogEntries, Version oldVersion) {
final List<String> changelogs = <String>[];
int versionIndex = VersionChange.skip.index;
for (final PendingChangelogEntry entry in pendingChangelogEntries) {
changelogs.add(entry.changelog);
versionIndex = math.min(versionIndex, entry.version.index);
}
final VersionChange effectiveVersionChange =
VersionChange.values[versionIndex];

Version? newVersion;
switch (effectiveVersionChange) {
case VersionChange.skip:
break;
case VersionChange.major:
newVersion = Version(
oldVersion.major + 1,
0,
0,
);
case VersionChange.minor:
newVersion = Version(
oldVersion.major,
oldVersion.minor + 1,
0,
);
case VersionChange.patch:
newVersion = Version(
oldVersion.major,
oldVersion.minor,
oldVersion.patch + 1,
);
}
return ReleaseInfo(newVersion, changelogs);
}

Future<void> _pushBranch({
required GitDir repository,
required RepositoryPackage package,
required String branchName,
required List<File> pendingChangelogFiles,
required ReleaseInfo releaseInfo,
}) async {
print(' Creating new branch "$branchName"...');
final io.ProcessResult checkoutResult =
await repository.runCommand(<String>['checkout', '-b', branchName]);
if (checkoutResult.exitCode != 0) {
printError(
'Failed to create branch $branchName: ${checkoutResult.stderr}');
throw ToolExit(_kGitFailedToPush);
}

print(' Updating pubspec.yaml to version ${releaseInfo.newVersion}...');
// Update pubspec.yaml.
final YamlEditor editablePubspec =
YamlEditor(package.pubspecFile.readAsStringSync());
editablePubspec
.update(<String>['version'], releaseInfo.newVersion.toString());
package.pubspecFile.writeAsStringSync(editablePubspec.toString());

print(' Updating CHANGELOG.md...');
// Update CHANGELOG.md.
final String newHeader = '## ${releaseInfo.newVersion}';
final List<String> newEntries = releaseInfo.changelogs;

final String oldChangelogContent = package.changelogFile.readAsStringSync();
final StringBuffer newChangelog = StringBuffer();

newChangelog.writeln(newHeader);
newChangelog.writeln();
newChangelog.writeln(newEntries.join('\n'));
newChangelog.writeln();
newChangelog.write(oldChangelogContent);

package.changelogFile.writeAsStringSync(newChangelog.toString());

print(' Removing pending changelog files...');
for (final File file in pendingChangelogFiles) {
final io.ProcessResult rmResult =
await repository.runCommand(<String>['rm', file.path]);
if (rmResult.exitCode != 0) {
printError('Failed to rm ${file.path}: ${rmResult.stderr}');
throw ToolExit(_kGitFailedToPush);
}
}

print(' Staging changes...');
final io.ProcessResult addResult = await repository.runCommand(
<String>['add', package.pubspecFile.path, package.changelogFile.path]);
if (addResult.exitCode != 0) {
printError('Failed to git add: ${addResult.stderr}');
throw ToolExit(_kGitFailedToPush);
}

print(' Committing changes...');
final io.ProcessResult commitResult = await repository.runCommand(<String>[
'commit',
'-m',
'${package.displayName}: Prepare for release'
]);
if (commitResult.exitCode != 0) {
printError('Failed to commit: ${commitResult.stderr}');
throw ToolExit(_kGitFailedToPush);
}

print(' Pushing to remote...');
final io.ProcessResult pushResult = await repository
.runCommand(<String>['push', 'origin', branchName, '--force']);
if (pushResult.exitCode != 0) {
printError('Failed to push to $branchName: ${pushResult.stderr}');
throw ToolExit(_kGitFailedToPush);
}
}
}

/// A data class for pending changelogs.
class PendingChangelogs {
/// Creates a new instance.
PendingChangelogs(this.entries, this.files);

/// The parsed pending changelog entries.
final List<PendingChangelogEntry> entries;

/// The files that the pending changelog entries were parsed from.
final List<File> files;
}

/// A data class for processed release information.
class ReleaseInfo {
/// Creates a new instance.
ReleaseInfo(this.newVersion, this.changelogs);

/// The new version for the release, or null if there is no version change.
final Version? newVersion;

/// The combined changelog entries.
final List<String> changelogs;
}

/// The type of version change for a release.
enum VersionChange {
/// A major version change (e.g., 1.2.3 -> 2.0.0).
major,

/// A minor version change (e.g., 1.2.3 -> 1.3.0).
minor,

/// A patch version change (e.g., 1.2.3 -> 1.2.4).
patch,

/// No version change.
skip,
}

/// Represents a single entry in the pending changelog.
class PendingChangelogEntry {
/// Creates a new pending changelog entry.
PendingChangelogEntry({required this.changelog, required this.version});

/// Creates a PendingChangelogEntry from a YAML string.
factory PendingChangelogEntry.parse(String yamlContent) {
final dynamic yaml = loadYaml(yamlContent);
if (yaml is! YamlMap) {
throw FormatException(
'Expected a YAML map, but found ${yaml.runtimeType}.');
}

final dynamic changelogYaml = yaml['changelog'];
if (changelogYaml is! String) {
throw FormatException(
'Expected "changelog" to be a string, but found ${changelogYaml.runtimeType}.');
}
final String changelog = changelogYaml.trim();

final String? versionString = yaml['version'] as String?;
if (versionString == null) {
throw const FormatException('Missing "version" key.');
}
final VersionChange version = VersionChange.values.firstWhere(
(VersionChange e) => e.name == versionString,
orElse: () =>
throw FormatException('Invalid version type: $versionString'),
);

return PendingChangelogEntry(changelog: changelog, version: version);
}

/// The changelog messages for this entry.
final String changelog;

/// The type of version change for this entry.
final VersionChange version;
}
Loading