Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust-port): Split up shim and cli #2970

Merged
merged 144 commits into from
Dec 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
d9f80e3
Created two separate chains for executing commands: One that uses the…
NicholasLYang Oct 18, 2022
06f7c10
Adjusted json serialization. Now debugging
NicholasLYang Oct 19, 2022
fb5489c
Hooked up new CLI to commands
NicholasLYang Oct 19, 2022
4409106
Fixing compile error
NicholasLYang Oct 27, 2022
41a5cb8
fixing lint error
NicholasLYang Oct 27, 2022
0543e72
Fixing more Go lint errors
NicholasLYang Oct 27, 2022
8f78e8b
Fixing lint errors
NicholasLYang Oct 28, 2022
0f2995f
fixing up tests
NicholasLYang Oct 31, 2022
c20470d
Merge branch 'main' into nicholasyang/turbo-362-port-turborepo-cli-to…
NicholasLYang Oct 31, 2022
59fd409
Only using turbo state with link, login, and logout commands
NicholasLYang Oct 31, 2022
4210fbd
Merge branch 'main' into nicholasyang/turbo-362-port-turborepo-cli-to…
NicholasLYang Nov 1, 2022
cd16510
Reverting changes
NicholasLYang Nov 1, 2022
b6db539
Removing more changes
NicholasLYang Nov 1, 2022
303c2ce
Deleting code and fixing lints
NicholasLYang Nov 1, 2022
9be15a7
Removed unlink command
NicholasLYang Nov 1, 2022
826a3aa
Fixing bugs
NicholasLYang Nov 1, 2022
b70cf95
Reverting more changes
NicholasLYang Nov 1, 2022
8a51b2f
fixed all tests
NicholasLYang Nov 1, 2022
d3f3ed5
Refactored away duplicates using CLIConfigProvider interface
NicholasLYang Nov 1, 2022
6a88e54
Lint fixes
NicholasLYang Nov 1, 2022
d4b1eb3
Minor order tweak
NicholasLYang Nov 1, 2022
b695a08
Merge branch 'main' into nicholasyang/turbo-362-port-turborepo-cli-to…
NicholasLYang Nov 1, 2022
7a13456
Added error if command is not handled
NicholasLYang Nov 1, 2022
6d77ad8
PR feedback
NicholasLYang Nov 2, 2022
df7be75
Minor fixes. Adding --__test-run flag
NicholasLYang Nov 3, 2022
a22614d
Added workaround for subcommand help
NicholasLYang Nov 4, 2022
0b10007
Moved logic around to accomodate login/logout/link/unlink
NicholasLYang Nov 4, 2022
6aee7a2
Added some documentation, removed some printf
NicholasLYang Nov 7, 2022
d52e687
Adding more tests.
NicholasLYang Nov 7, 2022
bc51eee
Hooked up link/login/unlink to --__test-run flag. Also added prysk te…
NicholasLYang Nov 7, 2022
0cadfc0
Split up derive into multiple lines
NicholasLYang Nov 7, 2022
e064ab6
Merge branch 'main' into nicholasyang/turbo-362-port-turborepo-cli-to…
NicholasLYang Nov 7, 2022
ec06fa5
Added comment explaining clap v3 in Cargo.toml
NicholasLYang Nov 7, 2022
98e888e
Ported daemon
NicholasLYang Nov 7, 2022
2afa43e
Cleaning up code. Removed imports
NicholasLYang Nov 8, 2022
c28fc20
Fix lints
NicholasLYang Nov 8, 2022
061e416
Fixing lints
NicholasLYang Nov 8, 2022
6f4e549
Ported prune to Rust CLI
NicholasLYang Nov 8, 2022
6931732
Going all the way. Porting run to Rust CLI and upgrading back to clap v4
NicholasLYang Nov 8, 2022
fa57848
Always gotta be testing
NicholasLYang Nov 9, 2022
05c6bf2
Always be testing. Always. Be. Testing.
NicholasLYang Nov 9, 2022
1ebda69
Added tests for each singular flag. Now for combination tests
NicholasLYang Nov 10, 2022
30e6c23
WIP: Hooking up Rust CLI to Go run logic
NicholasLYang Nov 10, 2022
647665b
Disable color
NicholasLYang Nov 10, 2022
8ec278c
Removed root inference from Go side
NicholasLYang Nov 10, 2022
3802381
Fixing compile errors
NicholasLYang Nov 10, 2022
6a2d250
Update cli/cmd/turbo/main.go
NicholasLYang Nov 11, 2022
4765e28
Patching up code for run
NicholasLYang Nov 14, 2022
4160db5
PR changes
NicholasLYang Nov 14, 2022
02e7205
Restored labeler
NicholasLYang Nov 14, 2022
2767e91
Restore og.tsx
NicholasLYang Nov 14, 2022
8e4d795
Reverted run changes that somehow made it in here
NicholasLYang Nov 14, 2022
7e7140e
Merge branch 'main' into nicholasyang/turbo-362-port-turborepo-cli-to…
NicholasLYang Nov 14, 2022
633c1e1
Fix run
NicholasLYang Nov 14, 2022
be68a8f
Update cli/internal/turbostate/turbostate.go
NicholasLYang Nov 14, 2022
edcbeba
feat: polyfill `global` with `globalThis` (#2666)
ecklf Nov 14, 2022
c383365
delete IsYarn util (#2668)
chris-olszewski Nov 14, 2022
064c174
feat(docs): new authors & consistent filesnames (#2680)
tknickman Nov 14, 2022
cc864ab
Added caveat to environment variables docs (#2703)
mattpocock Nov 14, 2022
c7a3f08
Fix publish workflow (#2681)
mehulkar Nov 14, 2022
e59e032
change from yarn to pnpm (#2686)
nus3 Nov 14, 2022
c9e0741
Merge branch 'nicholasyang/turbo-362-port-turborepo-cli-to-rust' into…
NicholasLYang Nov 14, 2022
0656abd
Reverted workflow changes
NicholasLYang Nov 14, 2022
1c4ba7a
Fixing tests. Added run args to base
NicholasLYang Nov 14, 2022
06b5f8b
Trying to fix tests
NicholasLYang Nov 15, 2022
4321bcf
Rewrote some logic around run args handling
NicholasLYang Nov 15, 2022
df56c34
Merge branch 'main' into nicholasyang/port-rest-of-cli
NicholasLYang Nov 16, 2022
98d9ca7
Fixing up tests by adding flags and fixing parsing bugs
NicholasLYang Nov 16, 2022
106c2e7
Hooked up RepoState to Go code
NicholasLYang Nov 16, 2022
eab7010
Merge branch 'main' into nicholasyang/port-rest-of-cli
NicholasLYang Nov 17, 2022
7fa9bab
Fixing e2e tests
NicholasLYang Nov 29, 2022
b493a99
Merge branch 'main' into nicholasyang/port-rest-of-cli
NicholasLYang Nov 29, 2022
1735b2e
Fixing e2e
NicholasLYang Nov 29, 2022
0ebdd45
Gotta build shim, not turbo in package.json
NicholasLYang Nov 29, 2022
8ae6056
Patched integration test
NicholasLYang Nov 29, 2022
6a39ab4
Reverted Cargo.toml
NicholasLYang Nov 29, 2022
c9f8cec
Cargo.lock too
NicholasLYang Nov 29, 2022
79365af
Fixing tests
NicholasLYang Nov 29, 2022
944c75b
Moving turbo binary to shim
NicholasLYang Nov 30, 2022
199f222
Fixed tests and CI
NicholasLYang Nov 30, 2022
d1bc327
Check for TURBO_BINARY_PATH
NicholasLYang Nov 30, 2022
384042d
Ported shell completions to Rust
NicholasLYang Nov 30, 2022
633ce85
Merge branch 'main' into nicholasyang/port-rest-of-cli
NicholasLYang Dec 1, 2022
4a46bf1
Fixed logic issue with TURBO_BINARY_PATH
NicholasLYang Dec 1, 2022
f56c487
Created .gitignore file with generated rust and header files
NicholasLYang Dec 1, 2022
5324492
Second try on cleaning up generated files
NicholasLYang Dec 1, 2022
6891488
More CI fixes
NicholasLYang Dec 1, 2022
588e958
Realized we still need some inference
NicholasLYang Dec 1, 2022
41cdbd8
Removed files that are edited by run-examples.sh
NicholasLYang Dec 1, 2022
e7dab55
Switched order of `cargo fmt --check` and `cargo clippy` because `car…
NicholasLYang Dec 1, 2022
c2d1dd4
Cleaning up changes.
NicholasLYang Dec 1, 2022
cd3ed1b
Fix one thing break another
NicholasLYang Dec 1, 2022
e991dce
Accept integration changes
NicholasLYang Dec 1, 2022
58081b6
Deleting more unused code
NicholasLYang Dec 1, 2022
6cf906d
Test fixes
NicholasLYang Dec 1, 2022
0ae6ff1
No longer need this line now that src/ffi.rs is gitignored
NicholasLYang Dec 1, 2022
55d53ac
PR feedback
NicholasLYang Dec 2, 2022
df4c32a
Update cli/internal/run/run.go
NicholasLYang Dec 2, 2022
7925937
Fixing typo
NicholasLYang Dec 5, 2022
f3de832
Merged main
NicholasLYang Dec 5, 2022
9ad9ec5
PR feedback
NicholasLYang Dec 5, 2022
2955728
PR feedback
NicholasLYang Dec 5, 2022
77d8890
Merge main
NicholasLYang Dec 5, 2022
4a1e3d0
Reverted workflow change
NicholasLYang Dec 5, 2022
e1c129e
Error on invalid unicode instead of lossy conversion
NicholasLYang Dec 5, 2022
0de7655
PR feedback
NicholasLYang Dec 5, 2022
c6c2741
Squashed commit of the following:
NicholasLYang Dec 5, 2022
508c3d2
Merged main
NicholasLYang Dec 5, 2022
c771bfa
Reverting changes that snuck in
NicholasLYang Dec 5, 2022
be01e3e
Merge branch 'main' into nicholasyang/port-rest-of-cli
NicholasLYang Dec 5, 2022
bd8fb9e
Fixing tests
NicholasLYang Dec 5, 2022
4eb22c3
Final fixes for tests
NicholasLYang Dec 5, 2022
b8a55fe
Moving shim to crates/turborepo
NicholasLYang Dec 6, 2022
c8a869e
More refactoring
NicholasLYang Dec 6, 2022
c1705c4
Merge branch 'main' into nicholasyang/barebones-shim-parser
NicholasLYang Dec 8, 2022
5f97cb2
First pass at splitting up shim and cli
NicholasLYang Dec 9, 2022
c2a2ca9
Fixing tests
NicholasLYang Dec 9, 2022
98e64df
Added detection for help flag so users can get help outside of a vali…
NicholasLYang Dec 9, 2022
6aa8d8a
Update crates/turborepo-lib/src/shim.rs
NicholasLYang Dec 9, 2022
3392ba0
Update crates/turborepo/Cargo.toml
NicholasLYang Dec 9, 2022
ed60b02
PR feedback and test fixing
NicholasLYang Dec 9, 2022
fa89bac
Set process's cwd
NicholasLYang Dec 9, 2022
aa0aeb6
Have to do inference even with TURBO_BINARY_PATH
NicholasLYang Dec 12, 2022
802c18f
Comments
NicholasLYang Dec 12, 2022
1d349e2
PR feedback
NicholasLYang Dec 12, 2022
e869784
Moved single_package back to RunArgs
NicholasLYang Dec 12, 2022
13e93b7
Merge branch 'main' into nicholasyang/barebones-shim-parser
NicholasLYang Dec 12, 2022
16cf12c
Removed print
NicholasLYang Dec 12, 2022
3f4e221
Fixing bugs
NicholasLYang Dec 12, 2022
d5f280d
Fixing rust tests
NicholasLYang Dec 12, 2022
618f7c3
Merge branch 'main' into nicholasyang/barebones-shim-parser
NicholasLYang Dec 12, 2022
6a36d15
fixing bug with verbosity and Go tests
NicholasLYang Dec 12, 2022
eb1dcf7
add local turbo version checking
chris-olszewski Dec 12, 2022
630c5ac
Merge remote-tracking branch 'nicholasyang/nicholasyang/barebones-shi…
chris-olszewski Dec 12, 2022
07d4c31
remove ffi inclusion
chris-olszewski Dec 12, 2022
075ab81
Fixed integration tests
NicholasLYang Dec 12, 2022
40e95db
Linting fixes
NicholasLYang Dec 12, 2022
7e7e070
Merge pull request #5 from chris-olszewski/nicholasyang/barebones-shi…
chris-olszewski Dec 12, 2022
f122f8e
Integrated local_turbo_supports_skip_shim
NicholasLYang Dec 12, 2022
5983c42
Fixed bug with cwd and restored yarn.lock
NicholasLYang Dec 12, 2022
543a8d2
Okay back to setting current_dir
NicholasLYang Dec 12, 2022
241477d
Fixed bug with canonicalization
NicholasLYang Dec 12, 2022
7157379
Removed print
NicholasLYang Dec 12, 2022
e1c1bfe
Merge branch 'main' into nicholasyang/barebones-shim-parser
NicholasLYang Dec 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Dependencies

On macOS: `brew install sponge jq`

1. Install [turbo/shim](https://github.com/vercel/turbo/blob/main/shim/README.md) build requirements
1. Install [turborepo crate](https://github.com/vercel/turbo/blob/main/crates/turborepo/README.md) build requirements

1. Run `pnpm install` at root

Expand Down
122 changes: 110 additions & 12 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ members = [
"crates/turbopack-swc-utils",
"crates/turbopack",
"crates/turbopack-tests",
"crates/turborepo",
"crates/turborepo-lib",
"crates/turbo-updater",
"shim",
"xtask",
]

Expand Down
2 changes: 1 addition & 1 deletion cli/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ SRC_FILES = $(shell find . -name "*.go" | grep -v "_test.go")
GENERATED_FILES = internal/turbodprotocol/turbod.pb.go internal/turbodprotocol/turbod_grpc.pb.go

turbo: libturbo.a
cargo build --manifest-path ../shim/Cargo.toml
cargo build --manifest-path ../crates/turborepo/Cargo.toml

libturbo.a: $(GENERATED_FILES) $(SRC_FILES) go.mod
go build -buildmode=c-archive -o libturbo.a ./cmd/turbo/...
Expand Down
12 changes: 6 additions & 6 deletions cli/cmd/turbo/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@ func main() {
os.Exit(1)
}

//export nativeRunWithTurboState
func nativeRunWithTurboState(turboStateString string) C.uint {
var turboState turbostate.CLIExecutionStateFromRust
err := json.Unmarshal([]byte(turboStateString), &turboState)
//export nativeRunWithArgs
func nativeRunWithArgs(argsString string) C.uint {
var args turbostate.ParsedArgsFromRust
err := json.Unmarshal([]byte(argsString), &args)
if err != nil {
fmt.Printf("Error unmarshalling turboState: %v\n Turbo state string: %v\n", err, turboStateString)
fmt.Printf("Error unmarshalling CLI args: %v\n Arg string: %v\n", err, argsString)
return 1
}
exitCode := cmd.RunWithTurboState(turboState, turboVersion)
exitCode := cmd.RunWithArgs(args, turboVersion)
return C.uint(exitCode)
}
4 changes: 3 additions & 1 deletion cli/integration_tests/link.t
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@ Setup

Link Test Run
$ ${TURBO} link --__test-run
Link test run successful
Repository inference failed: Unable to find `turbo.json` or `package.json` in current path
Running command as global turbo
Link test run successful
2 changes: 2 additions & 0 deletions cli/integration_tests/login.t
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@ Setup

Login Test Run
$ ${TURBO} login --__test-run
Repository inference failed: Unable to find `turbo.json` or `package.json` in current path
Running command as global turbo
Login test run successful
4 changes: 4 additions & 0 deletions cli/integration_tests/logout.t
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ Setup

Logout while logged in
$ ${TURBO} logout
Repository inference failed: Unable to find `turbo.json` or `package.json` in current path
Running command as global turbo
>>> Logged out

Logout while logged out
$ ${TURBO} logout
Repository inference failed: Unable to find `turbo.json` or `package.json` in current path
Running command as global turbo
>>> Logged out

6 changes: 4 additions & 2 deletions cli/integration_tests/no_args.t
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ Setup

Make sure exit code is 2 when no args are passed
$ ${TURBO}
Repository inference failed: Unable to find `turbo.json` or `package.json` in current path
Running command as global turbo
The build system that makes ship happen

Usage: turbo [OPTIONS] [COMMAND]
Expand Down Expand Up @@ -31,7 +33,7 @@ Make sure exit code is 2 when no args are passed
--team <TEAM> Set the team slug for API calls
--token <TOKEN> Set the auth token for API calls
--trace <TRACE> Specify a file to save a pprof trace
-v, --verbosity <VERBOSITY> verbosity
-v... verbosity
-h, --help Print help information

Run Arguments:
Expand All @@ -40,6 +42,7 @@ Make sure exit code is 2 when no args are passed
--concurrency <CONCURRENCY> Limit the concurrency of task execution. Use 1 for serial (i.e. one-at-a-time) execution
--continue Continue execution even if a task exits with an error or non-zero exit code. The default behavior is to bail
--dry-run [<DRY_RUN>] [possible values: text, json]
--single-package Run turbo in single-package mode
--filter <FILTER> Use the given selector to specify package(s) to act as entry points. The syntax mirrors pnpm's syntax, and additional documentation and examples can be found in turbo's documentation https://turbo.build/repo/docs/reference/command-line-reference#--filter
--force Ignore the existing cache (to force execution)
--global-deps <GLOBAL_DEPS> Specify glob of global filesystem dependencies to be hashed. Useful for .env and files
Expand All @@ -55,5 +58,4 @@ Make sure exit code is 2 when no args are passed
--remote-only Ignore the local filesystem cache for all tasks. Only allow reading and caching artifacts using the remote cache
--scope <SCOPE> Specify package(s) to act as entry points for task execution. Supports globs
--since <SINCE> Limit/Set scope to changed packages since a mergebase. This uses the git diff ${target_branch}... mechanism to identify which packages have changed
--single-package Run turbo in single-package mode
[2]