Improvement: add `--raw` flag to `run`, `deploy` command #1653

nopeNoshishi · 2024-03-02T06:07:15Z

Description of change

Implemented --raw flag for cargo-shuttle run, deploy command.

As a refactoring, I created a struct LogsArgs to organize the log() arguments.

How has this been tested? (if applicable)

deply

cargo run -p cargo-shuttle -- --wd examples/axum/hello-world/ deploy --raw

 INFO Starting deployment
 INFO shuttle_deployer::handlers: Deployer version: 0.39.0
 INFO shuttle_deployer::handlers: Deployment ID: e61942f0-ee66-4ed0-8161-352a789aa791
 INFO shuttle_deployer::handlers: Service ID: 01HQTD8NN8181CQ8RVF3VZYNY6
 INFO shuttle_deployer::handlers: Service name: hello-world-rocket-app
 INFO shuttle_deployer::handlers: Project ID: 01HQTD6YT05JQC532SJH7B2275
 INFO shuttle_deployer::handlers: Project name: hello-world-rocket-app
 INFO shuttle_deployer::handlers: Date: 2024-03-02T04:58:10Z
 INFO Entering Queued state
 INFO Waiting for queue slot
 INFO Entering Building state
 INFO Building project
 INFO shuttle_deployer::deployment::queue: Extracting files
 INFO shuttle_deployer::deployment::queue: Building deployment
 INFO shuttle_service::builder: {name="hello-world" deps="[\"shuttle-rocket '^0.39.0'\", \"shuttle-runtime '^0.39.0'\"]"} Compiling workspace member with shuttle dependencies
warning: Patch `shuttle-actix-web v0.39.0 (/usr/src/shuttle/services/shuttle-actix-web)` was not used in the crate graph.
Patch `shuttle-aws-rds v0.39.0 (/usr/src/shuttle/resources/aws-rds)` was not used in the crate graph.
Patch `shuttle-axum v0.39.0 (/usr/src/shuttle/services/shuttle-axum)` was not used in the crate graph.
Patch `shuttle-metadata v0.39.0 (/usr/src/shuttle/resources/metadata)` was not used in the crate graph.
Patch `shuttle-next v0.39.0 (/usr/src/shuttle/services/shuttle-next)` was not used in the crate graph.
Patch `shuttle-opendal v0.39.0 (/usr/src/shuttle/resources/opendal)` was not used in the crate graph.
Patch `shuttle-persist v0.39.0 (/usr/src/shuttle/resources/persist)` was not used in the crate graph.
Patch `shuttle-poem v0.39.0 (/usr/src/shuttle/services/shuttle-poem)` was not used in the crate graph.
Patch `shuttle-qdrant v0.39.0 (/usr/src/shuttle/resources/qdrant)` was not used in the crate graph.
Patch `shuttle-salvo v0.39.0 (/usr/src/shuttle/services/shuttle-salvo)` was not used in the crate graph.
Patch `shuttle-secrets v0.39.0 (/usr/src/shuttle/resources/secrets)` was not used in the crate graph.
Patch `shuttle-serenity v0.39.0 (/usr/src/shuttle/services/shuttle-serenity)` was not used in the crate graph.
Patch `shuttle-shared-db v0.39.0 (/usr/src/shuttle/resources/shared-db)` was not used in the crate graph.
Patch `shuttle-thruster v0.39.0 (/usr/src/shuttle/services/shuttle-thruster)` was not used in the crate graph.
Patch `shuttle-tide v0.39.0 (/usr/src/shuttle/services/shuttle-tide)` was not used in the crate graph.
Patch `shuttle-tower v0.39.0 (/usr/src/shuttle/services/shuttle-tower)` was not used in the crate graph.
Patch `shuttle-turso v0.39.0 (/usr/src/shuttle/resources/turso)` was not used in the crate graph.
Patch `shuttle-warp v0.39.0 (/usr/src/shuttle/services/shuttle-warp)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
    Finished release [optimized] target(s) in 0.12s
 INFO shuttle_deployer::deployment::queue: Running tests before starting up
warning: Patch `shuttle-actix-web v0.39.0 (/usr/src/shuttle/services/shuttle-actix-web)` was not used in the crate graph.
Patch `shuttle-aws-rds v0.39.0 (/usr/src/shuttle/resources/aws-rds)` was not used in the crate graph.
Patch `shuttle-axum v0.39.0 (/usr/src/shuttle/services/shuttle-axum)` was not used in the crate graph.
Patch `shuttle-metadata v0.39.0 (/usr/src/shuttle/resources/metadata)` was not used in the crate graph.
Patch `shuttle-next v0.39.0 (/usr/src/shuttle/services/shuttle-next)` was not used in the crate graph.
Patch `shuttle-opendal v0.39.0 (/usr/src/shuttle/resources/opendal)` was not used in the crate graph.
Patch `shuttle-persist v0.39.0 (/usr/src/shuttle/resources/persist)` was not used in the crate graph.
Patch `shuttle-poem v0.39.0 (/usr/src/shuttle/services/shuttle-poem)` was not used in the crate graph.
Patch `shuttle-qdrant v0.39.0 (/usr/src/shuttle/resources/qdrant)` was not used in the crate graph.
Patch `shuttle-salvo v0.39.0 (/usr/src/shuttle/services/shuttle-salvo)` was not used in the crate graph.
Patch `shuttle-secrets v0.39.0 (/usr/src/shuttle/resources/secrets)` was not used in the crate graph.
Patch `shuttle-serenity v0.39.0 (/usr/src/shuttle/services/shuttle-serenity)` was not used in the crate graph.
Patch `shuttle-shared-db v0.39.0 (/usr/src/shuttle/resources/shared-db)` was not used in the crate graph.
Patch `shuttle-thruster v0.39.0 (/usr/src/shuttle/services/shuttle-thruster)` was not used in the crate graph.
Patch `shuttle-tide v0.39.0 (/usr/src/shuttle/services/shuttle-tide)` was not used in the crate graph.
Patch `shuttle-tower v0.39.0 (/usr/src/shuttle/services/shuttle-tower)` was not used in the crate graph.
Patch `shuttle-turso v0.39.0 (/usr/src/shuttle/resources/turso)` was not used in the crate graph.
Patch `shuttle-warp v0.39.0 (/usr/src/shuttle/services/shuttle-warp)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
    Finished release [optimized] target(s) in 0.13s
     Running unittests src/main.rs (target/release/deps/hello_world-788f3aba457c3fce)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

 INFO shuttle_deployer::deployment::queue: Moving built executable
 INFO Releasing queue slot
 INFO Entering Built state
 INFO Starting deployment
 INFO Entering Loading state
 INFO Loading resources
 INFO shuttle_service::runner: {args="/opt/shuttle/shuttle-builds/hello-world-rocket-app/.shuttle-executables/e61942f0-ee66-4ed0-8161-352a789aa791 --port 24996"} Spawning runtime process
 INFO shuttle_proto::_runtime_client: connecting runtime client
 INFO shuttle_deployer::deployment::run: Loading resources
 INFO shuttle_deployer::persistence: {__self.project_id="01HQTD6YT05JQC532SJH7B2275"} Getting resources from resource-recorder
shuttle-runtime executable started (version 0.39.0)
===============================================================
Shuttle's default tracing subscriber is initialized!
To disable the subscriber and use your own,
turn off the default features for shuttle-runtime:

shuttle-runtime = { version = "...", default-features = false }
===============================================================
 INFO shuttle_deployer::persistence: Got no resources from resource-recorder
 INFO shuttle_deployer::persistence: {resources="Ok([])"} Local resources
 INFO shuttle_deployer::deployment::run: successfully loaded service
 INFO shuttle_deployer::persistence: Uploading resources to resource-recorder
 INFO shuttle_deployer::deployment::run: stopping old deployment (id 6705b576-c957-4e3c-af73-5c6197e40559)
 INFO Entering Running state
 INFO Starting service
Starting on 0.0.0.0:8000
 INFO shuttle_deployer::deployment::run: Runtime started successully
No resources are linked to this service

Service Name:  hello-world-rocket-app
Deployment ID: e61942f0-ee66-4ed0-8161-352a789aa791
Status:        running
Last Updated:  2024-03-02T04:58:10Z
URI:           https://hello-world-rocket-app.unstable.shuttleapp.rs

run

cargo run -p cargo-shuttle -- --wd shuttle-examples/rocket/hello-world run --raw

shuttle-runtime executable started (version 0.39.0)
===============================================================
Shuttle's default tracing subscriber is initialized!
To disable the subscriber and use your own,
turn off the default features for shuttle-runtime:
shuttle-runtime = { version = "...", default-features = false }
===============================================================
loading alpha service at /Users/****/examples/axum/hello-world

    Starting hello-world-axum-app on http://127.0.0.1:8080

Starting on 127.0.0.1:8080

modify: logs() argument modify: output log

nopeNoshishi · 2024-03-02T06:08:14Z

cargo-shuttle/src/args.rs

+    #[arg(short = 'F', long)]
+    pub format: Option<LogFormat>,
+}
+
+#[derive(ValueEnum, Clone, Debug, strum::Display, strum::EnumIter)]
+#[strum(serialize_all = "kebab-case")]
+pub enum LogFormat {
+    /// Don't display timestamps and log origin tags
+    Raw,
+}


I defined an enum LogFormat to allow logs commands to be output in various formats in the future. (This is why we call it --raw -> --format raw). If this change is not good, undo it.

Since raw is the only supported format, and the other commands take --raw, we could keep this as --raw for consistency.

I have standardized on --raw to be consistent with other commands. Thanks for your comments!

f24c072

nopeNoshishi · 2024-03-02T06:17:21Z

I tried to see what happens when I look at the logs in json format.

One problem: the important line of the LogItem was operating on a simple string, which was not easy to transfer to a JSON format. So, I gave up on this PR. For example, it may be possible to use the generic T: Serialize into line field. (However, I read the source code and found truncate to adjust the size, so there is much to consider.)

Trying json

// cargo-shuttle/src/args.rs
#[derive(ValueEnum, Clone, Debug, strum::Display, strum::EnumIter)]
#[strum(serialize_all = "kebab-case")]
pub enum LogFormat {
    /// Don't display timestamps and log origin tags
    Raw,
    Json
}

// cargo-shuttle/src/lib.rs
            for log in logs.into_iter() {
                match &args.format {
                    Some(LogFormat::Raw) => println!("{}", log.get_raw_line()),
                    Some(LogFormat::Json) => println!("{}", serde_json::to_string(&log)?),
                    None => println!("{log}"),
                }
            }

cargo run -p cargo-shuttle -- --wd shuttle-examples/rocket/hello-world logs -F json | jq

{
  "id": "d282df4b-a264-4839-9dca-dbfb42169867",
  "internal_origin": "Deployer",
  "timestamp": "2024-03-02T05:40:55.996918Z",
  "line": "\u001b[38;5;10m INFO\u001b[39m \u001b[38;5;12mget_deployment\u001b[39m"
}
{
  "id": "d282df4b-a264-4839-9dca-dbfb42169867",
  "internal_origin": "Deployer",
  "timestamp": "2024-03-02T05:48:35.786590Z",
  "line": "\u001b[38;5;10m INFO\u001b[39m \u001b[38;5;12mget_logs\u001b[39m"
}

…ormat

nopeNoshishi added 2 commits March 2, 2024 12:57

add: LogsArgs struct and Logformat enum

6dd6b80

modify: logs() argument modify: output log

add: --raw flag to deploy, run

7b74cd4

nopeNoshishi commented Mar 2, 2024

View reviewed changes

nopeNoshishi added 2 commits March 2, 2024 15:19

fix: common-tests

aa92521

Merge remote-tracking branch 'origin/main' into improvement/add/log-f…

d69f96f

…ormat

nopeNoshishi changed the title ~~Improvement/add/log format~~ Improvement: --raw flag for run, deploy command Mar 2, 2024

nopeNoshishi changed the title ~~Improvement: --raw flag for run, deploy command~~ Improvement: add --raw flag to run, deploy command Mar 2, 2024

fix: tiny

9bcdbd5

nopeNoshishi marked this pull request as ready for review March 2, 2024 06:45

nopeNoshishi mentioned this pull request Mar 4, 2024

[Improvement]: Add the --raw flag to the run and deploy commands #1632

Closed

nopeNoshishi added 2 commits March 9, 2024 22:56

fix: logs argument

f24c072

Merge remote-tracking branch 'origin/main' into improvement/add/log-f…

c5c944c

…ormat

jonaro00 approved these changes Mar 11, 2024

View reviewed changes

jonaro00 merged commit db4f2e6 into shuttle-hq:main Mar 11, 2024
30 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improvement: add `--raw` flag to `run`, `deploy` command #1653

Improvement: add `--raw` flag to `run`, `deploy` command #1653

nopeNoshishi commented Mar 2, 2024 •

edited

nopeNoshishi Mar 2, 2024

jonaro00 Mar 5, 2024

nopeNoshishi Mar 9, 2024

nopeNoshishi commented Mar 2, 2024 •

edited

Improvement: add --raw flag to run, deploy command #1653

Improvement: add --raw flag to run, deploy command #1653

Conversation

nopeNoshishi commented Mar 2, 2024 • edited

Description of change

How has this been tested? (if applicable)

nopeNoshishi Mar 2, 2024

Choose a reason for hiding this comment

jonaro00 Mar 5, 2024

Choose a reason for hiding this comment

nopeNoshishi Mar 9, 2024

Choose a reason for hiding this comment

nopeNoshishi commented Mar 2, 2024 • edited

Improvement: add `--raw` flag to `run`, `deploy` command #1653

Improvement: add `--raw` flag to `run`, `deploy` command #1653

nopeNoshishi commented Mar 2, 2024 •

edited

nopeNoshishi commented Mar 2, 2024 •

edited