-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.service
subpkg
#483
.service
subpkg
#483
Commits on Mar 9, 2023
-
Adds a `piker storage` subcmd with a `-d` flag to wipe a particular fqsn's time series (both 1s and 60s). Obviously this needs to be extended much more but provides a start point.
Configuration menu - View commit details
-
Copy full SHA for fe0695f - Browse repository at this point
Copy the full SHA fe0695fView commit details -
Support startup-config overrides to
ahabd
superWith the addition of a new `elastixsearch` docker support in #464, adjustments were made to container startup sync logic (particularly the `trio` checkpoint sleep period - which itself is a hack around a sync client api) which caused a regression in upstream startup logic wherein container error logs were not being bubbled up correctly causing a silent failure mode: - `marketstore` container started with corrupt input config - `ahabd` super code timed out on startup phase due to a larger log polling period, skipped processing startup logs from the container, and continued on as though the container was started - history client fails on grpc connection with no clear error on why the connection failed. Here we revert to the old poll period (1ms) to avoid any more silent failures and further extend supervisor control through a configuration override mechanism. To address the underlying design issue, this patch adds support for container-endpoint-callbacks to override supervisor startup configuration parameters via the 2nd value in their returned tuple: the already delivered configuration `dict` value. The current exposed values include: { 'startup_timeout': 1.0, 'startup_query_period': 0.001, 'log_msg_key': 'msg', }, This allows for container specific control over the startup-sync query period (the hack mentioned above) as well as the expected log msg key and of course the startup timeout.
Configuration menu - View commit details
-
Copy full SHA for 7b196b1 - Browse repository at this point
Copy the full SHA 7b196b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 959e423 - Browse repository at this point
Copy the full SHA 959e423View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8c66f06 - Browse repository at this point
Copy the full SHA 8c66f06View commit details -
Configuration menu - View commit details
-
Copy full SHA for 05b67c2 - Browse repository at this point
Copy the full SHA 05b67c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for b078a06 - Browse repository at this point
Copy the full SHA b078a06View commit details -
Background docker-container logs processing
Previously we would make the `ahabd` supervisor-actor sync to docker container startup using pseudo-blocking log message processing. This has issues, - we're forced to do a hacky "yield back to `trio`" in order to be "fake async" when reading the log stream and further, - blocking on a message is fragile and often slow. Instead, run the log processor in a background task and in the parent task poll for the container to be in the client list using a similar pseudo-async poll pattern. This allows the super to `Context.started()` sooner (when the container is actually registered as "up") and thus unblock its (remote) caller faster whilst still doing full log msg proxying! Deatz: - adds `Container.cuid: str` a unique container id for logging. - correctly proxy through the `loglevel: str` from `pikerd` caller task. - shield around `Container.cancel()` in the teardown block and use cancel level logging in that method.
Configuration menu - View commit details
-
Copy full SHA for 7694419 - Browse repository at this point
Copy the full SHA 7694419View commit details -
Configuration menu - View commit details
-
Copy full SHA for bb723ab - Browse repository at this point
Copy the full SHA bb723abView commit details -
Configuration menu - View commit details
-
Copy full SHA for 56629b6 - Browse repository at this point
Copy the full SHA 56629b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for bfe3ea1 - Browse repository at this point
Copy the full SHA bfe3ea1View commit details -
Start
piker.service
sub-packageFor now just moves everything that was in `piker._daemon` to a subpkg module but a reorg is coming pronto!
Configuration menu - View commit details
-
Copy full SHA for 93c81fa - Browse repository at this point
Copy the full SHA 93c81faView commit details -
Configuration menu - View commit details
-
Copy full SHA for afac553 - Browse repository at this point
Copy the full SHA afac553View commit details -
Configuration menu - View commit details
-
Copy full SHA for dd87d11 - Browse repository at this point
Copy the full SHA dd87d11View commit details -
Configuration menu - View commit details
-
Copy full SHA for b226b67 - Browse repository at this point
Copy the full SHA b226b67View commit details -
Configuration menu - View commit details
-
Copy full SHA for 31f2b01 - Browse repository at this point
Copy the full SHA 31f2b01View commit details -
Configuration menu - View commit details
-
Copy full SHA for a2d4093 - Browse repository at this point
Copy the full SHA a2d4093View commit details -
Configuration menu - View commit details
-
Copy full SHA for eca048c - Browse repository at this point
Copy the full SHA eca048cView commit details -
Configuration menu - View commit details
-
Copy full SHA for f95ea19 - Browse repository at this point
Copy the full SHA f95ea19View commit details -
Configuration menu - View commit details
-
Copy full SHA for cec2967 - Browse repository at this point
Copy the full SHA cec2967View commit details -
Attempt to report
piker storage -d <fqsn>
errorsNot really sure there's much we can do besides dump Grpc stuff when we detect an "error" `str` for the moment.. Either way leave a buncha complaints (como siempre) and do linting fixups..
Configuration menu - View commit details
-
Copy full SHA for 441243f - Browse repository at this point
Copy the full SHA 441243fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6f92c6b - Browse repository at this point
Copy the full SHA 6f92c6bView commit details -
Fix final missed
marketstore
mod importThanks @esme! XD Also, do a linter pass and remove a buncha unused references.
Configuration menu - View commit details
-
Copy full SHA for cda7a54 - Browse repository at this point
Copy the full SHA cda7a54View commit details -
Configuration menu - View commit details
-
Copy full SHA for fbc12b1 - Browse repository at this point
Copy the full SHA fbc12b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6540c41 - Browse repository at this point
Copy the full SHA 6540c41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 31392af - Browse repository at this point
Copy the full SHA 31392afView commit details -
Configuration menu - View commit details
-
Copy full SHA for 75b7a8b - Browse repository at this point
Copy the full SHA 75b7a8bView commit details -
Add reconnect loop to
marketstore
startup testDue to making ahabd supervisor init more async we need to be more tolerant to mkts server startup: the grpc machinery needs to be up otherwise a client which connects to early may just hang on requests.. Add a reconnect loop (which might end up getting factored into client code too) so that we only block on requests once we know the client connection is actually responsive.
Configuration menu - View commit details
-
Copy full SHA for 2014019 - Browse repository at this point
Copy the full SHA 2014019View commit details -
Configuration menu - View commit details
-
Copy full SHA for aa36abf - Browse repository at this point
Copy the full SHA aa36abfView commit details -
Pull testing config dir from
tractor
runtime varsProvides a more correct solution (particularly for distributed testing) to override the `piker` configuration directory by reading the path from a specific `tractor._state._runtime_vars` entry that can be provided by the test harness. Also fix some typing and comments.
Configuration menu - View commit details
-
Copy full SHA for 5aaa7f4 - Browse repository at this point
Copy the full SHA 5aaa7f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79b0db4 - Browse repository at this point
Copy the full SHA 79b0db4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7cc9911 - Browse repository at this point
Copy the full SHA 7cc9911View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9a00c45 - Browse repository at this point
Copy the full SHA 9a00c45View commit details -
ahabd
: Harden cancellation teardown (again XD)Needed to move the startup sequence inside the `try:` block to guarantee we always do the (now shielded) `.cancel()` call if we get a cancel during startup. Also, support an optional `started_afunc` field in the config if backends want to just provide a one-off blocking async func to sync container startup. Add a `drop_root_perms: bool` to allow persisting sudo perms for testing or dyanmic container spawning purposes.
Configuration menu - View commit details
-
Copy full SHA for 15064d9 - Browse repository at this point
Copy the full SHA 15064d9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0772b4a - Browse repository at this point
Copy the full SHA 0772b4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 44a3115 - Browse repository at this point
Copy the full SHA 44a3115View commit details -
Configuration menu - View commit details
-
Copy full SHA for 97290fc - Browse repository at this point
Copy the full SHA 97290fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8ceaa27 - Browse repository at this point
Copy the full SHA 8ceaa27View commit details
Commits on Mar 10, 2023
-
Don't double send
enable_modules
anddebug_mode
in kwargs..This broke non-disti-mode actor tree spawn / runtime, seemingly because the cli entrypoint for a `piker chart` also sends these values down through the call stack independently? Pretty sure we don't need to send the `enable_modules` from the chart actor anyway.
Configuration menu - View commit details
-
Copy full SHA for 12883c3 - Browse repository at this point
Copy the full SHA 12883c3View commit details