feat: dedicated logger service #1225
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* [scaling] feat: capture deployment logs (#1015) * feat: init otlp-exporter * feat: DeploymentLayer to capture deployment logs * feat: send deployment logs over otlp * refactor: rename to logger * tests: for stored logs * feat: capture and return logs * feat: stream logs * refactor: finish main * tests: add claim * refactor: println to error * refactor: make field keys constants * chore: update lockfile * fix: revert persistence merge changes * ci: add logger to platform test * tests: try bumping test timeout to pass ci * [scaling] refactor: build files (#1018) * refactor: update source crates * refactor: remove test orchestrator * refactor: remove BACKEND_TAG * refactor: remove unused make targets * refactor: sort docker compose * feat: add builder * feat: add deployer * refactor: update gateway * feat: add logger * feat: add resource-recorder * refactor: make auth reachable * bug: db files * [scaling] feat: make a log exporter for runtime (#1041) * refactor: make LogRecorder to handle logs * feat: Otlp recorder * refactor: have a configurable log server for local runs * refactor: update codegen to take generic logger * feat: logger server in local runner * refactor: remove log streaming from runtime * refactor: make wasm compile * refactor: remove unused ParseError * refactor: use constants * refactor: severity number conversion * refactor: fix comments * refactor: clippy suggestions * fix: cherry-pick fixups * fix: more post-cherry-pick fixups * fix: proto generated formatting * fix: clippy * tests: remove old deployer log tests * fix: clippy * fix: from severity number impl bug * fix: tracing doc test, remove cfg * refactor: deployment id to UUID * fix: address docker-compose reviews * refactor: logger deployment id from ulid to string this way it will be easier to switch the deployment_id to ulid in the future * fix: print error for runtime invalid args * ci: enable deploying to unstable * ci: enable staging deployment * fix: debug logger implementation - send deployer logs to logger - instrument deployer run function - add service name to runtime logs - remove error traces from logger failing to send - skip connecting to logger for logger service in setup_tracing - skip sending empty logs inside logger - add todos * fix: don't connect logger to logger in setup_tracing * fix: logger default addr * fix: expose port for logger * fix: add logger to default network * WIP: fixing infinite loop in runtime logger * WIP: switch to trace service * WIP: fix first logger integration test * WIP: fix logger tests * tests: cleanup logger tests * feat: update codegen with new logger layer * refactor: remove unwraps in logger/dal.rs * tests: remove spawn_blocking * feat: update local logger, test nested span * refactor: remove logsservice in logger * WIP: add logger exporter layer to common * WIP: working config for logs from deployer/runtime * feat: add deployment_id attribute to runtime tracer * feat: pass deployment id to runtime loader * refactor: check inner spans for id, update tests * tests: update codegen tests * refactor: comments, lockfile, remove logger.rs * fix: codegen test, update registry type * docs: remove comments Co-authored-by: Pieter <pieter@chesedo.me> * fix: remove propagator from main codegen * tests: switch runtime integration test to trace svc * refactor: remove old code from common/tracing * refactor: remove severitynumber code * docs: clarify comment * tests: refactor logger integration test utils * tests: refactor runtime flag in logger, comments --------- Co-authored-by: Pieter <pieter@chesedo.me>
* deployer: routed get_logs call to shuttle-logger * deployer: routed get logs stream call to shuttle-logger * proto: added From impl for LogItem -> common LogItem * deployer: fixed proto LogItem conversion * fix: clippy * fix: unrelated openapi path bug * feat: connect and set logger client * feat: otlp_address env var for deployer local run * fix: correct expect message --------- Co-authored-by: oddgrd <29732646+oddgrd@users.noreply.github.com>
* logger: account for span name information * logger/tests: updated tests to account for spans * logger: get logs result should be ordered by timestamp * logger: fix tests * address O nit feedback
* codegen: removed the otel layer generation * proto: updated logger proto file * logger: add store_logs RPC impl * logger: fixed tests * logger: decreased the visibility for Log type * logger: remove the unwrap * proto/logger: differentiate between a fetched/stored log items * logger: simplify tests * runtime: removed tracing/otel dependencies * runtime: removed the tracing dependencies * codegen: cleanup tracing layers and tests * runtime/next: fix compilation * logger: remove claim verification on store_logs * logger: implemented from_stored for StoredLogItem convertion to Log * codegen: readd a missing import * codegen: fix tests * addressed P review * logger: removed integration test comment * logger/tests: remove the DeploymentPush claim scope * address orhun feedback * logger: fixed error message typo * address Johan review * logger/proto: remove dedup protobuf defs
…nto feat/shuttle-logger-service
* feat(deployer): send runtime logs to the logger service * refactor(deployer): replace outdated log types * fix(logger): use UTC timestamp and correct data for sending * feat(cargo-shuttle): handle the child process stdout * refactor(deployer): remove unnecessary Mutex from child process * feat(deployer): add nanos to log timestamp * refactor(cargo-shuttle): use println instead of handling stdout * test(deployer): update tests accordingly to runtime manager changes * test(runner): update tests accordingly to runtime manager changes * test(deployer): remove unnecessary layers * revert(deployer): add back layers This reverts commit e2edeac.
* feat(runtime): write next runtime logs to stdout * refactor(example): update the message in axum-wasm-expanded example * refactor(codegen): update accordingly to next runtime changes
* refactor(runtime): replace trace with println * feat(next): print out the wasm_path for next project
* feat: create batcher around logger client * feat: or send at interval * feat: hook up logger client * refactor: only match on the errors
…res (#1145) * deployer: routed get_logs call to shuttle-logger * proto: added From impl for LogItem -> common LogItem * logger-postgres: added a local postgres db for shuttle-logger * logger: starting it locally will start a local postgres container * docker-compose: remove logger volume * fix: update logger after rebase * tests: update logger integration tests * refactor: logger integration tests * docs: logger migration comments * refactor(logger): revert switch to native-tls * refactor: remove from impl in logger proto * fix: logger test names * refactor: remove unrelated changes * ci: shuttle-logger tests need docker * feat: address reviews, clean up docker/makefile --------- Co-authored-by: oddgrd <29732646+oddgrd@users.noreply.github.com>
…ructure (#1171) * refactor(deployer): new log structure * WIP revert state tracking * mostly finish DeploymentIdLayer and StateChangeLayer * fix: comments Co-authored-by: Pieter <pieter@chesedo.me> * format log lines. LogItem::new. * deployment_id in spans, LogRecorder for Batcher * Convertions, log retrieval. * derive default * add matching for deployment log end states * fmt errors correctly --------- Co-authored-by: Pieter <pieter@chesedo.me>
* feat(runtime): set up a tracing-subscriber as a default feature * style(codegen): update tracer subscriber warning message * refactor(codegen): simplify tracing setup code generation
…logger_uri arg (#1204) * chore(logger): remove the logger_uri dependency from runtime and codegen * misc(deployer): refactor persistence and fix tests * test(deployer): added connection to mocked logger required... ...by state change layer tests. * test(deployer): fixed the run queue tests and added... ...a common-tests method to instantiate mocked logger clients based on shuttle_proto::logger::Logger trait implementations. * chore(cargo-shuttle): remove the local logger server * chore(deployer): common-tests dep change from path to workspace
* refactor(codegen): create a fmt subscriber without time * refactor(runtime): emit resource tracker logs without timestmap
* feat(codegen): restore default log level, improve error messages * feedback
* fix(service): build failed because of missing common features * chore(logger): create a Default impl for Docker instance * misc(logger): add error information when log streaming fails * fix(proto): update logger generated proto
* fix: span names, log levels and messages * fix: codegen feature * improve log output * Update common/src/claims.rs Co-authored-by: Oddbjørn Grødem <29732646+oddgrd@users.noreply.github.com> --------- Co-authored-by: Oddbjørn Grødem <29732646+oddgrd@users.noreply.github.com>
* feat(codegen): enable default tracing for shuttle-next projects * misc(cargo-shuttle): fix lint issue * chore(codegen): remove env-filter leftovers * chore(runtime): remove unrelated change * misc(cargo-shuttle): fix cargo clippy error * misc(shuttle-next): remove colored dependency
* bug: only stream requested logs * refactor: stop spawns for closed streams
iulianbarbu
approved these changes
Sep 12, 2023
chesedo
commented
Sep 12, 2023
* feat: limit log line length to 2048 chars * log line touchups * feedback: reduce iterations and allocations when truncating * print at better time * cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
Moves logging to a dedicated service. This means deployers now only do the building and managing of deploys, thereby reducing their responsibilities.
This change comes with a whole lot of other affects though.
Features
Because of this change, these features are now possible:
stdoutso users are now in full control of theirtracinglayers. They may even choose to use another logging library, likelog, if they wish. Or even turntracingoff to reduce their dependencies and only useprintln!.Bugs
We believe these bugs are also fixed by this changes:
How has this been tested? (if applicable)
With new automated tests and staging