Skip to content

Commit

Permalink
Rename Turbo Cache to Native Link (#402)
Browse files Browse the repository at this point in the history
This project has expanded well beyond build caching.
  • Loading branch information
MarcusSorealheis committed Nov 18, 2023
1 parent 8de3014 commit 7d9deaf
Show file tree
Hide file tree
Showing 146 changed files with 1,368 additions and 394 deletions.
4 changes: 2 additions & 2 deletions .bazelignore
Expand Up @@ -5,7 +5,7 @@ bazel-out
bazel-remote-bin
bazel-remote-out
bazel-remote-testlogs
bazel-remote-turbo-cache
bazel-remote-native-link
bazel-root
bazel-testlogs
bazel-turbo-cache
bazel-native-link
2 changes: 1 addition & 1 deletion .bazelrc
@@ -1,4 +1,4 @@
# Copyright 2022 The Turbo Cache Authors. All rights reserved.
# Copyright 2022 The Native Link Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion .dockerignore
@@ -1,5 +1,5 @@
target/
*/*/.terraform-turbo-cache-builder
*/*/.terraform-native-link-builder
*/*/.update_scheduler_ips.zip
*/*/terraform.tfstate
*/*/.terraform
Expand Down
24 changes: 12 additions & 12 deletions .github/workflows/main.yml
Expand Up @@ -39,29 +39,29 @@ jobs:
build-args: |
OS_VERSION=${{ matrix.os_version }}
load: true # This brings the build into `docker images` from buildx.
tags: allada/turbo-cache:dependencies
tags: trace_machina/native-link:dependencies
target: dependencies

- name: Create container for cargo
run: |
docker run --name=turbo-cache-cargo allada/turbo-cache:dependencies bash -c ' \
docker run --name=native-link-cargo trace_machina/native-link:dependencies bash -c ' \
DEBIAN_FRONTEND=noninteractive apt-get install -y curl libssl-dev gcc pkg-config python3 && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain=1.70.0 \
' && \
docker commit turbo-cache-cargo allada/turbo-cache:cargo
docker commit native-link-cargo trace_machina/native-link:cargo
- name: Check Cargo.toml is up to date
run: |
docker run --rm -w /root/turbo-cache -v $PWD:/root/turbo-cache allada/turbo-cache:cargo python3 ./tools/build_cargo_manifest.py && \
docker run --rm -w /root/native-link -v $PWD:/root/native-link trace_machina/native-link:cargo python3 ./tools/build_cargo_manifest.py && \
git diff --exit-code || \
(echo "Cargo.toml is out of date. Please run: python ./tools/build_cargo_manifest.py" && exit 1)
- name: Compile & test with cargo
run: |
docker run --rm -w /root/turbo-cache -v $PWD:/root/turbo-cache allada/turbo-cache:cargo bash -c ' \
docker run --rm -w /root/native-link -v $PWD:/root/native-link trace_machina/native-link:cargo bash -c ' \
. /root/.cargo/env && \
cargo build --all && \
cargo test --all \
'
docker-compose-compiles-turbo-cache:
docker-compose-compiles-native-link:
# The type of runner that the job will run on.
runs-on: ubuntu-22.04
strategy:
Expand All @@ -83,7 +83,7 @@ jobs:
OS_VERSION=${{ matrix.os_version }}
ADDITIONAL_SETUP_WORKER_CMD=DEBIAN_FRONTEND=noninteractive apt-get install -y gcc g++ lld pkg-config python3
load: true # This brings the build into `docker images` from buildx.
tags: allada/turbo-cache:latest
tags: trace_machina/native-link:latest
- uses: docker/build-push-action@v4
with:
context: .
Expand All @@ -92,24 +92,24 @@ jobs:
OPT_LEVEL=opt
OS_VERSION=${{ matrix.os_version }}
load: true # This brings the build into `docker images` from buildx.
tags: allada/turbo-cache:builder
tags: trace_machina/native-link:builder
target: builder

- name: Compile turbo cache with turbo cache
- name: Compile Native Link with Native Link
run: |
mkdir -p ~/.cache && \
cd deployment-examples/docker-compose && \
docker-compose up -d && \
cd ../../ && \
docker run --rm --net=host -w /root/turbo-cache -v $PWD:/root/turbo-cache allada/turbo-cache:builder sh -c ' \
docker run --rm --net=host -w /root/native-link -v $PWD:/root/native-link trace_machina/native-link:builder sh -c ' \
bazel clean && \
bazel test //... \
--remote_instance_name=main \
--remote_cache=grpc://127.0.0.1:50051 \
--remote_executor=grpc://127.0.0.1:50052 \
--remote_default_exec_properties=cpu_count=1 \
' && \
docker run --rm --net=host -w /root/turbo-cache -v $PWD:/root/turbo-cache allada/turbo-cache:builder sh -c ' \
docker run --rm --net=host -w /root/native-link -v $PWD:/root/native-link trace_machina/native-link:builder sh -c ' \
bazel clean && \
bazel test //... \
--remote_instance_name=main \
Expand Down Expand Up @@ -138,7 +138,7 @@ jobs:
OPT_LEVEL=fastbuild
OS_VERSION=${{ matrix.os_version }}
load: true # This brings the build into `docker images` from buildx.
tags: allada/turbo-cache:latest
tags: trace_machina/native-link:latest

- name: Run tests
run: ./run_integration_tests.sh
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
@@ -1,11 +1,11 @@
# Contributing to Turbo Cache
# Contributing to Native Link

Turbo Cache welcomes contribution from everyone. Here are the guidelines if you
Native Link welcomes contribution from everyone. Here are the guidelines if you
are thinking of helping us:

## Contributions

Contributions to Turbo Cache or its dependencies should be made in the form of
Contributions to Native Link or its dependencies should be made in the form of
GitHub pull requests. Each pull request will be reviewed by a core contributor
(someone with permission to land patches) and either landed in the main tree or
given feedback for changes that would be required. All contributions should
Expand Down Expand Up @@ -58,5 +58,5 @@ various Cloud CLIs for you:

## Conduct

Turbo Cache Code of Conduct is available in the
Native Link Code of Conduct is available in the
[CODE_OF_CONDUCT](CODE_OF_CONDUCT.md) file.
36 changes: 36 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 8 additions & 8 deletions README.md
@@ -1,6 +1,6 @@
# Turbo Cache
# Native Link

[![CI](https://github.com/allada/turbo-cache/workflows/CI/badge.svg)](https://github.com/allada/turbo-cache/actions/workflows/main.yml)
[![CI](https://github.com/tracemachina/native-link/workflows/CI/badge.svg)](https://github.com/tracemachina/native-link/actions/workflows/main.yml)

An extremely fast and efficient bazel cache service (CAS) written in rust.

Expand All @@ -12,7 +12,7 @@ The goals of this project are:

## Overview

Turbo Cache is a project that implements the Bazel's [Remote Execution protocol](https://github.com/bazelbuild/remote-apis) (both CAS/Cache and remote execution portion).
Native Link is a project that implements the Bazel's [Remote Execution protocol](https://github.com/bazelbuild/remote-apis) (both CAS/Cache and remote execution portion).

When properly configured this project will provide extremely fast and efficient build cache for any systems that communicate using the [RBE protocol](https://github.com/bazelbuild/remote-apis/blob/main/build/bazel/remote/execution/v2/remote_execution.proto) and/or extremely fast, efficient and low foot-print remote execution capability.

Expand All @@ -26,7 +26,7 @@ If you have not updated Rust or Cargo recently, run:

To compile and run the server:
```sh
# Install dependencies needed to compile Turbo Cache with bazel on
# Install dependencies needed to compile Native Link with bazel on
# worker machine (which is this machine).
apt install -y gcc g++ lld libssl-dev pkg-config python3

Expand All @@ -45,13 +45,13 @@ bazel test //... \
--remote_executor=grpc://127.0.0.1:50051 \
--remote_default_exec_properties=cpu_count=1
```
This will cause bazel to run the commands through an all-in-one `CAS`, `scheduler` and `worker`. See [here](https://github.com/allada/turbo-cache/tree/master/config) for configuration documentation and [here](https://github.com/allada/turbo-cache/tree/main/deployment-examples/terraform) for an example of multi-node cloud deployment example.
This will cause bazel to run the commands through an all-in-one `CAS`, `scheduler` and `worker`. See [here](https://github.com/tracemachina/native-link/tree/master/config) for configuration documentation and [here](https://github.com/tracemachina/native-link/tree/main/deployment-examples/terraform) for an example of multi-node cloud deployment example.

## Example Deployments
We currently have a few example deployments in [deployment-examples directory](https://github.com/allada/turbo-cache/tree/master/deployment-examples).
We currently have a few example deployments in [deployment-examples directory](https://github.com/tracemachina/native-link/tree/master/deployment-examples).

### Terraform
The [terraform deployment](https://github.com/allada/turbo-cache/tree/master/deployment-examples/terraform) is the currently preferred method as it leverages a lot of cloud resources to make everything much more robust.
The [terraform deployment](https://github.com/tracemachina/native-link/tree/master/deployment-examples/terraform) is the currently preferred method as it leverages a lot of cloud resources to make everything much more robust.

The terraform deployment is very easy to setup and configure. This deployment will show off remote execution capabilities and cache capabilities.

Expand Down Expand Up @@ -111,7 +111,7 @@ cargo build --release
### Configure

Configuration is done via a JSON file that is passed in as the first parameter to the `cas` program. See [here](https://github.com/allada/turbo-cache/tree/master/config) for more details and examples.
Configuration is done via a JSON file that is passed in as the first parameter to the `cas` program. See [here](https://github.com/tracemachina/native-link/tree/master/config) for more details and examples.

## How to update internal or external rust deps

Expand Down
10 changes: 5 additions & 5 deletions SECURITY.md
@@ -1,25 +1,25 @@
# Security policy

GitHub's bots crawl `turbo-cache` to detect security vulnerabilities wherever
GitHub's bots crawl `native-link` to detect security vulnerabilities wherever
possible.

TraceMachina and the `turbo-cache` authors place a high emphasis on fixing any
TraceMachina and the `native-link` authors place a high emphasis on fixing any
vulnerabilities. Please send a report if something doesn't look right.

## Supported versions

At the moment no version of `turbo-cache` is officially supported. Consider
At the moment no version of `native-link` is officially supported. Consider
using the latest commit on the `main` branch until official production binaries
are released.

## Reporting a vulnerability

Prefer reporting vulnerabilities via [GitHub](https://github.com/TraceMachina/turbo-cache/security).
Prefer reporting vulnerabilities via [GitHub](https://github.com/TraceMachina/native-link/security).

If you'd rather communicate via email please contact <blaise@tracemachina.com>,
<marcus@tracemachina.com>, <blake@tracemachina.com> or <aaron@tracemachina.com>.

## Vulnerability disclosure and advisories

See [Advisories](https://github.com/TraceMachina/turbo-cache/security/advisories)
See [Advisories](https://github.com/TraceMachina/native-link/security/advisories)
for publicly disclosed vulnerabilities.
2 changes: 1 addition & 1 deletion WORKSPACE
@@ -1,4 +1,4 @@
workspace(name = "turbo-cache")
workspace(name = "native-link")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

Expand Down
12 changes: 6 additions & 6 deletions cas/cas_main.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -58,12 +58,12 @@ use worker_api_server::WorkerApiServer;
const DEFAULT_PROMETHEUS_METRICS_PATH: &str = "/metrics";

/// Name of environment variable to disable metrics.
const METRICS_DISABLE_ENV: &str = "TURBO_CACHE_DISABLE_METRICS";
const METRICS_DISABLE_ENV: &str = "NATIVE_LINK_DISABLE_METRICS";

/// Backend for bazel remote execution / cache API.
#[derive(Parser, Debug)]
#[clap(
author = "Trace Machina, Inc. <turbo-cache@tracemachina.com>",
author = "Trace Machina, Inc. <native-link@tracemachina.com>",
version = "0.0.1",
about,
long_about = None
Expand All @@ -75,7 +75,7 @@ struct Args {
}

async fn inner_main(cfg: CasConfig, server_start_timestamp: u64) -> Result<(), Box<dyn std::error::Error>> {
let mut root_metrics_registry = <Registry>::with_prefix("turbo_cache");
let mut root_metrics_registry = <Registry>::with_prefix("native_link");

let store_manager = Arc::new(StoreManager::new());
{
Expand Down Expand Up @@ -367,8 +367,8 @@ async fn inner_main(cfg: CasConfig, server_start_timestamp: u64) -> Result<(), B
.map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))
.map(|_| {
// This is a hack to get around this bug: https://github.com/prometheus/client_rust/issues/155
buf = buf.replace("turbo_cache_turbo_cache_stores_", "");
buf = buf.replace("turbo_cache_turbo_cache_workers_", "");
buf = buf.replace("native_link_native_link_stores_", "");
buf = buf.replace("native_link_native_link_workers_", "");
let body = Body::from(buf);
Response::builder()
.header(
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/ac_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/bytestream_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/capabilities_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/cas_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/execution_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/tests/ac_server_test.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/tests/bytestream_server_test.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/grpc_service/tests/cas_server_test.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
4 changes: 2 additions & 2 deletions cas/grpc_service/tests/worker_api_server_test.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -28,7 +28,7 @@ use proto::build::bazel::remote::execution::v2::{
ActionResult as ProtoActionResult, ExecuteResponse, ExecutedActionMetadata, LogFile, OutputDirectory, OutputFile,
OutputSymlink,
};
use proto::com::github::trace_machina::turbo_cache::remote_execution::{
use proto::com::github::trace_machina::native_link::remote_execution::{
execute_result, update_for_worker, worker_api_server::WorkerApi, ExecuteResult, KeepAliveRequest,
SupportedProperties,
};
Expand Down
5 changes: 3 additions & 2 deletions cas/grpc_service/worker_api_server.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -29,7 +29,8 @@ use common::DigestInfo;
use config::cas_server::WorkerApiConfig;
use error::{make_err, Code, Error, ResultExt};
use platform_property_manager::PlatformProperties;
use proto::com::github::trace_machina::turbo_cache::remote_execution::{

use proto::com::github::trace_machina::native_link::remote_execution::{
execute_result, worker_api_server::WorkerApi, worker_api_server::WorkerApiServer as Server, ExecuteResult,
GoingAwayRequest, KeepAliveRequest, SupportedProperties, UpdateForWorker,
};
Expand Down
2 changes: 1 addition & 1 deletion cas/scheduler/action_messages.rs
@@ -1,4 +1,4 @@
// Copyright 2022 The Turbo Cache Authors. All rights reserved.
// Copyright 2022 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cas/scheduler/cache_lookup_scheduler.rs
@@ -1,4 +1,4 @@
// Copyright 2023 The Turbo Cache Authors. All rights reserved.
// Copyright 2023 The Native Link Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down

0 comments on commit 7d9deaf

Please sign in to comment.