Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing tower-grpc w/ Tonic #230

Merged
merged 357 commits into from Nov 11, 2019

Conversation

@tjtelan
Copy link
Collaborator

tjtelan commented Nov 9, 2019

I rebased my master branch to the L11 master, because I am getting some really deeply confusing branch conflicts. I hope this will be the last 100+ commit diff, since my branch was only about 5 commits...

Closes #227
Fixes #229

tjtelan added 30 commits May 20, 2019
…how to structure the structs and enums for the various optional flags
Preparing to split subcommand handlers into their own files.
Still refining the pattern for subcommands for consistency
Still needs a fine-grained pass to add all the needed documentation.
Good enough for integration
Changes to vagrant env for local dev builds
Leaving launch.json settings for VSCode debugging of werker.
broadcast-address option was placed on wrong service
Parsing git remote
Adding WIP stub of orb cli
…d commenting out existing swagger openapi generation
tjtelan and others added 21 commits Jun 25, 2019
orbital_api is a crate that encompasses generated Rust bindings
for our protobuf 3 protos representing client to server interactions
for orbital.
Cleaning up build warnings
Adding client utilities to cut down on repetition
Adding slack handler
Adding artifact handler
Adding apple handler
…d commenting out existing swagger openapi generation
…vendored grpc library installation target in GNUMakefile.
* Organizing modules into crates and adding to the workspace. Compile times are improved significantly.

* Creating orb cli module structure. Adding orb dev command to test some libraries for while I'm bootstrapping the server side

* Passing global options down into subcommand handlers. Renaming command to subcommand to be more in-line with structopt.

* Laying down boilerplate for inner subcommands, including second-level subcommands. They all return Result types too.

* Adding shell autocomplete. A little bit off pattern.

* Updating git_info to return Result types, and use more of that sweet sweet question mark operator

* Replacing unwraps with expects

* rustfmt, plus minor comments, tweaks to build.rs

* Adding in dev commands using docker apis. Pulling images, and a stub for creating containers.

* Plumbing for starting server. Rustfmt.

* Got a server running using the tower-grpc examples.

* Example client call working for orb build, based on tokio-grpc examples

* Removing the panics on both client request and server response. Its working!

* Cleaning up build warnings

* Starting to implement build functionality by doing it locally first under the dev commands

* Some refactoring of crates to root level of the workspace. Renaming the crates with more meaningful names
* Organizing modules into crates and adding to the workspace. Compile times are improved significantly.

* Creating orb cli module structure. Adding orb dev command to test some libraries for while I'm bootstrapping the server side

* Passing global options down into subcommand handlers. Renaming command to subcommand to be more in-line with structopt.

* Laying down boilerplate for inner subcommands, including second-level subcommands. They all return Result types too.

* rustfmt, plus minor comments, tweaks to build.rs

* Some refactoring of crates to root level of the workspace. Renaming the crates with more meaningful names

* First pass for the orbital config parser using serde. Connected to orb dev validate

* Cleaning up config parser, and the cli driver. Running rustfmt.

* Fixing minor syntax. Missing closing paren

* Adding conversion trait to SubcommandError, so we can questionmark with SubcommandError

* Starting the docker container builder library.

* Starting the release pipeline automation. Including an example orb.yml to test out local builds

* Making fields in OrbitalConfig pub, and cleaning up debug output

* Feating image from config and pulling the container
Rust fmt ran, and cleaning up warnings

* Adding env_logger and swapping out many println

* Creating docker containers. Wired to cli 'dev docker create' to test library and 'dev build' to test the server-side workflow

* Checkpoint commit. Starting, stopping, exec commands in container. Not quite working with local builds but it is all commented out while trying to figure it out.

* Adding Vagrantfile for dev environments

* Provisioning w/ unprivileged user to all the rust stuff gets set up properly for dev purposes

* Possibly got exec working with logs visible from docker logs

* Updated shiplift to the git repo version so we can set the working dir of a container. Volume mounts working as well using some hardcoded paths. First docker build successful

* Changing hardcoded path to the vagrant path.

* Bumping cpu on vagrantfile up to speed builds. Plumbing for env vars and stub for vol mounts.

* Able to pass in env vars and paths to mount from local builds. Cleaning up build warnings.

* Wiring docker build crate into dev orb docker.
Moving reusable code and refactoring.

* Fixing inconsistencies with volume mounting code
Fixing Cargo.toml files to only reference version major.minor

* Organizing modules into crates and adding to the workspace. Compile times are improved significantly.

* Creating orb cli module structure. Adding orb dev command to test some libraries for while I'm bootstrapping the server side

* Passing global options down into subcommand handlers. Renaming command to subcommand to be more in-line with structopt.

* Laying down boilerplate for inner subcommands, including second-level subcommands. They all return Result types too.

* Adding shell autocomplete. A little bit off pattern.

* Adding in dev commands using docker apis. Pulling images, and a stub for creating containers.

* Got a server running using the tower-grpc examples.

* Example client call working for orb build, based on tokio-grpc examples

* Some refactoring of crates to root level of the workspace. Renaming the crates with more meaningful names

* First pass for the orbital config parser using serde. Connected to orb dev validate

* Starting the docker container builder library.

* Adding env_logger and swapping out many println

* Updated shiplift to the git repo version so we can set the working dir of a container. Volume mounts working as well using some hardcoded paths. First docker build successful
Minor refactoring with packages, updating static url for ipv4 + Vagrant
Updating proto style to calm build warnings
Workaround for codegen issues #229
@tjtelan tjtelan marked this pull request as ready for review Nov 9, 2019
tjtelan added 2 commits Nov 9, 2019
Deleting oldmodels
Restoring ansible file from legacy
Uncommenting a module in orb cli
Restoring docstring that was lost during rebase
We don't need the protobuf libraries, so removed from Makefile
Adding Makefile docs target
@tjtelan

This comment has been minimized.

Copy link
Collaborator Author

tjtelan commented Nov 9, 2019

I just cleaned up this PR. The files changed now accurately reflect the files I've changed.

In summary:

  • Replace tower-grpc with tonic
  • Clean up unused dependencies in all Cargo.toml
  • Removed unnecessary clone of protobuf definitions for build.rs (tonic-build)
  • Touch all of the protos to update the naming conventions (Including a workaround for #229)
  • Update all of the grpc server/client stubs with their new names and async types, and added .await? in the BuildService subcommand_handler client/server stack
  • Add .gitattributes to have Github ignore the legacy directory, so the project language becomes Rust
Copy link
Collaborator

eccramer left a comment

LGTM, this looks great.

.map_err(|e| panic!("connect error: {:?}", e))
.and_then(move |conn| {
use orbital_headers::build_metadata::client::BuildService;
let response = client.start_build(request).await?;

This comment has been minimized.

Copy link
@eccramer

eccramer Nov 11, 2019

Collaborator

This is amazing. What a massive reduction in LOC and a huge increase in Readability. Tonic FTW!

)
.unwrap_or_else(|e| panic!("protobuf compilation failed: {}", e))
fn main() -> Result<(), Box<dyn std::error::Error>> {
tonic_build::compile_protos("../protos/build_metadata.proto")?;

This comment has been minimized.

Copy link
@eccramer

eccramer Nov 11, 2019

Collaborator

More benefit from tonic. This is wonderful.

}

/// Generated Rust from protobufs for high-level units, Organizations
pub mod organization {
include!(concat!(env!("OUT_DIR"), "/organization.rs"));
tonic::include_proto!("organization");

This comment has been minimized.

Copy link
@eccramer

eccramer Nov 11, 2019

Collaborator

Yeah, tonic is just better in every way.

MANUAL = 2;
SUBSCRIBE_TRIGGER = 3;
COMMIT_MSG_TRIGGER = 4;
enum JobTrigger {

This comment has been minimized.

Copy link
@eccramer

eccramer Nov 11, 2019

Collaborator

Thank you for converting these to best practice naming conventions. I was unaware of them, and matched the enums to the existing names in the Ocelot DB schema.

@tjtelan tjtelan merged commit d01270c into level11consulting:master Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.