Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starts a command line tool for assisting engine dev workflows #50642

Merged
merged 1 commit into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .clang-tidy
Original file line number Diff line number Diff line change
Expand Up @@ -61,4 +61,4 @@ CheckOptions:
# updated.
#
# tl;dr: I'm sorry.
HeaderFilterRegex: "[..\/]+\/flutter\/(assets|benchmarking|build|ci|common|display_list|docs|examples|flow|flutter_frontend_server|flutter_vma|fml|impeller|lib|runtime|shell|skia|sky|testing|tools|vulkan|wasm|web_sdk)\/.*"
HeaderFilterRegex: "[..\/]+\/flutter\/(assets|benchmarking|bin|build|ci|common|display_list|docs|examples|flow|flutter_frontend_server|flutter_vma|fml|impeller|lib|runtime|shell|skia|sky|testing|tools|vulkan|wasm|web_sdk)\/.*"
68 changes: 68 additions & 0 deletions bin/et
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/bin/bash
#
# Copyright 2013 The Flutter Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

set -e

# Needed because if it is set, cd may print the path it changed to.
unset CDPATH

# On Mac OS, readlink -f doesn't work, so follow_links traverses the path one
# link at a time, and then cds into the link destination and find out where it
# ends up.
#
# The function is enclosed in a subshell to avoid changing the working directory
# of the caller.
function follow_links() (
cd -P "$(dirname -- "$1")"
file="$PWD/$(basename -- "$1")"
while [[ -h "$file" ]]; do
cd -P "$(dirname -- "$file")"
file="$(readlink -- "$file")"
cd -P "$(dirname -- "$file")"
file="$PWD/$(basename -- "$file")"
done
echo "$file"
)

SCRIPT_DIR=$(follow_links "$(dirname -- "${BASH_SOURCE[0]}")")
ENGINE_DIR="$(cd "$SCRIPT_DIR/.."; pwd -P)"

case "$(uname -s)" in
Linux)
OS="linux"
;;
Darwin)
OS="macos"
;;
*)
echo "The host platform is not supported by this tool"
exit 1
esac

case "$(uname -m)" in
arm64)
CPU="arm64"
;;
x86_64)
CPU="x64"
;;
*)
echo "The host platform is not supported by this tool"
exit 1
esac

PLATFORM="${OS}-${CPU}"
DART_SDK_DIR="${ENGINE_DIR}/prebuilts/${PLATFORM}/dart-sdk"
DART="${DART_SDK_DIR}/bin/dart"

cd "${ENGINE_DIR}/tools/engine_tool"

if [ ! -d ".dart_tool" ]; then
echo "You must run 'gclient sync -D' before using this tool."
exit 1
fi

"${DART}" --disable-dart-dev bin/et.dart "$@"
61 changes: 61 additions & 0 deletions tools/engine_tool/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# The Engine Tool

This is a command line Dart program that automates workflows in the
`flutter/engine` repository.

### Prerequisites

The tool requires an initial `gclient sync -D` as described in the [repo setup
steps](https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment#getting-the-source)
before it will work.

## Status

The tool has the following commands.

* `help` - Prints helpful information about commands and usage.
* `query builds` - Lists the CI builds described under `ci/builders` that the
host platform is capable of executing.

### Missing features

There are currently many missing features. Some overall goals are listed in the
GitHub issue [here](https://github.com/flutter/flutter/issues/132807). Some
desirable new features would do the following:

* Support Windows hosts.
* Add a `doctor` command.
* Update the engine checkout so that engine developers no longer have to remeber
zanderso marked this conversation as resolved.
Show resolved Hide resolved
to run `gclient sync -D`.
* Build and test the engine using CI configurations locally, with the
possibility to override or add new build options and targets.
* Build engines using options coming only from the command line.
* List tests and run them locally, automatically building their dependencies
first. Automatically start emulators or simulators if they're needed to run a
test.
* Spawn individual builders remotely using `led` from `depot_tools`.
* Encapsulate all code formatters, checkers, linters, etc. for all languages.
* Find a compatible version of the flutter/flutter repo, check it out, and spawn
tests from that repo with a locally built engine to run on an emulator,
simulator or device.
* Use a real logging package for prettier terminal output.
* Wire the tool up to a package providing autocomplete like
[cli_completion](https://pub.dev/packages/cli_completion.).

The way the current tooling in the engine repo works may need to be rewritten,
especially tests spawned by `run_tests.py`, in order to provide this interface.

## Contributing

* Follow the [Flutter style guide](https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo)
for Dart code that are relevant outside of the framework repo. It contains
conventions that go beyond code formatting, which
we'll follow even if using `dart format` in the future.
* Do not call directly into `dart:io` except from `main.dart`. Instead access
the system only through the `Enviroment` object.
* All commands must have unit tests. If some functionality needs a fake
implementation, then write a fake implementation.
* When adding or changing functionality, update this README.md file.
* *Begin with the end in mind* - Start working from what the interface provided
by this tool *should* be, then modify underlying scripts and tools to provide
APIs to support that.
7 changes: 7 additions & 0 deletions tools/engine_tool/bin/et.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:engine_tool/main.dart' as m;

void main(List<String> args) => m.main(args);
66 changes: 66 additions & 0 deletions tools/engine_tool/lib/main.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:ffi' as ffi show Abi;
import 'dart:io' as io show Directory, exitCode, stderr, stdout;

import 'package:engine_build_configs/engine_build_configs.dart';
import 'package:engine_repo_tools/engine_repo_tools.dart';
import 'package:path/path.dart' as p;
import 'package:platform/platform.dart';
import 'package:process_runner/process_runner.dart';

import 'src/commands/command_runner.dart';
import 'src/environment.dart';

void main(List<String> args) async {
// Find the engine repo.
final Engine engine;
try {
engine = Engine.findWithin();
} catch (e) {
io.stderr.writeln(e);
io.exitCode = 1;
return;
}

// Find and parse the engine build configs.
final io.Directory buildConfigsDir = io.Directory(p.join(
engine.flutterDir.path, 'ci', 'builders',
));
final BuildConfigLoader loader = BuildConfigLoader(
buildConfigsDir: buildConfigsDir,
);

// Treat it as an error if no build configs were found. The caller likely
// expected to find some.
final Map<String, BuildConfig> configs = loader.configs;
if (configs.isEmpty) {
io.stderr.writeln(
'Error: No build configs found under ${buildConfigsDir.path}',
);
io.exitCode = 1;
return;
}
if (loader.errors.isNotEmpty) {
loader.errors.forEach(io.stderr.writeln);
io.exitCode = 1;
}

// Use the Engine and BuildConfig collection to build the CommandRunner.
final ToolCommandRunner runner = ToolCommandRunner(
environment: Environment(
abi: ffi.Abi.current(),
engine: engine,
platform: const LocalPlatform(),
processRunner: ProcessRunner(),
stderr: io.stderr,
stdout: io.stdout,
),
configs: configs,
);

io.exitCode = await runner.run(args);
return;
}
18 changes: 18 additions & 0 deletions tools/engine_tool/lib/src/commands/command.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:args/command_runner.dart';

import '../environment.dart';

/// The base class that all commands and subcommands should inherit from.
abstract base class CommandBase extends Command<int> {
/// Constructs the base command.
CommandBase({
required this.environment
});

/// The host system environment.
final Environment environment;
}
53 changes: 53 additions & 0 deletions tools/engine_tool/lib/src/commands/command_runner.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:args/command_runner.dart';

import 'package:engine_build_configs/engine_build_configs.dart';

import '../environment.dart';
import 'query_command.dart';

/// The root command runner.
final class ToolCommandRunner extends CommandRunner<int> {
/// Constructs the runner and populates commands, subcommands, and global
/// options and flags.
ToolCommandRunner({
required this.environment,
required this.configs,
}) : super(toolName, toolDescription) {
addCommand(QueryCommand(
environment: environment,
configs: configs,
));
}

/// The name of the tool as reported in the tool's usage and help
/// messages.
static const String toolName = 'et';

/// The description of the tool reported in the tool's usage and help
/// messages.
static const String toolDescription = 'A command line tool for working on '
'the Flutter Engine.';

/// The host system environment.
final Environment environment;

/// Build configurations loaded from the engine from under ci/builders.
final Map<String, BuildConfig> configs;

@override
Future<int> run(Iterable<String> args) async {
try{
return await runCommand(parse(args)) ?? 1;
} on FormatException catch (e) {
environment.stderr.writeln(e);
return 1;
} on UsageException catch (e) {
environment.stderr.writeln(e);
return 1;
}
}
}
Loading