From 908e231539b6d9567953f340f1a2b738b4d152a2 Mon Sep 17 00:00:00 2001 From: Eric Seidel Date: Thu, 23 May 2024 11:17:25 -0700 Subject: [PATCH] docs: add thoughts on future design (#157) * docs: add thoughts on future design * chore: fix bsdiff to bidiff * chore: run formatter --- README.md | 15 ++++++++++----- patch/README.md | 42 ++++++++++++++++++++++++++++++++---------- 2 files changed, 42 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 1f0a080f..f62ad485 100644 --- a/README.md +++ b/README.md @@ -16,11 +16,16 @@ into the Flutter engine during build time. ## Parts -- `dart_cli`: Test ffi wrapping of updater library. -- `library`: The rust library that does the actual update work. -- `dart_bindings`: The Dart bindings for the updater library. - -All of the interesting code is in the `library` directory. There is also +- `library`: Runtime library linked into Flutter Engine to unpack and apply + updates. Build into libupdater.a and linked into libflutter.so. +- `patch`: Developer tooling to package Shorebird updates ("patches"). Built + into `patch.exe` and downloaded and run by `shorebird` command line tool: + https://github.com/shorebirdtech/shorebird/tree/main/packages/shorebird_cli. +- `shorebird_code_push`: The Dart bindings for communicating with the updater + library from within a Flutter app. Published to pub.dev, usage is optional by + developers. + +Most interesting code is in the `library` directory. There is also a [README.md](library/README.md) in that directory explaining the design. ## Developing diff --git a/patch/README.md b/patch/README.md index 38c85c29..4fc9c2aa 100644 --- a/patch/README.md +++ b/patch/README.md @@ -1,22 +1,43 @@ # patch command line tool -This is the tool used by the `shorebird` command line to compute the patch -file for uploading to the server. - -This currently uses the rust `bidiff` crate to compute the patch file. -and could just use the `bic` command line tool included in that crate. However -we're explicitly writing our own command line to allow us to change the -underlying compression without affecting the `shorebird` command line callers. +This is the tool used by the `shorebird` command line to package up a patch +file for uploading to Shorebird's servers. ## Usage patch +## Context, design and future thoughts. + +Originally `patch` was written on top of the `bidiff` tool, but really no longer +needs to be. For expediency we used the `comde` crate to have a generic api +across multiple compression algorithms. However, we've since decided to use the +`zstd` crate for compression and decompression and could remove our `comde` and +`bidiff` dependencies eventually. + +Because `bidiff` does not check what it's patching, it's possible to apply a +patch to the wrong file. To avoid this we currently have a separate hash +which we save in Shorebird's database and then `library` validates that the +patched file matches what we expected after inflation. This is the wrong design +and we intend to move to a system whereby `patch` is responsible for including +its own hash in the patch file and validating during application. + +In that world we might end up with two hashes. One who's purpose is to validate +the patch file itself and another to validate the resulting inflated file or +the original file (both are equivalent). Both of these hashes could be stored +inside the patch container (.vmcode) and validated by the `library` code. + +We should also probably rename `patch` to `packager` or similar since it should +do more than just bidiff. Also some of the apply/inflate code in library might +want to move into this directory to be more of the same place. + +We also probably eventually want to move to something like sigstore.dev rather +than rolling our own packaging/signing system. ## Generating test expectations -The string_patch target can be used to generate test expectations for testing -the updater. It takes two strings as arguments and prints the necessary +The `string_patch` target can be used to generate test expectations for testing +the updater. It takes two strings as arguments and prints the necessary variables you will need in your test to stdout. ``` @@ -28,9 +49,10 @@ Hash (new): fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9 ``` Which will translate into: + ```rust let base = "foo"; let new = "bar"; let patch: Vec = vec![40, 181, 47, 253, 0, 128, 113, 0, 0, 223, 177, 0, 0, 0, 16, 0, 0, 0, 3, 98, 97, 114, 0]; let hash = "fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9"; -``` \ No newline at end of file +```