feat: Spring Boot parity for observability, actuator & configuration#25
Merged
Merged
Conversation
added 2 commits
June 4, 2026 17:32
…actuator, and config
Brings pyfly's observability, actuator, and configuration to drop-in Spring Boot
parity (Micrometer metric names, full /actuator/* surface, Spring-style YAML
semantics), driven by a verified multi-agent audit.
Observability / metrics
- HTTP auto-instrumentation rewritten to Micrometer's `http_server_requests_seconds`
(Summary _count/_sum + _max gauge; optional histogram) tagged method/uri/status/
outcome/exception. `uri` is the TEMPLATED route (cardinality-safe), recovered from
the matched Starlette endpoint -> Route.path.
- MetricsFilter is now wired DIRECTLY in create_app (it was a bean built too late to
ever reach the chain -> metrics were silently never collected).
- Process/system meters with Micrometer names (process_uptime_seconds,
process_cpu_usage, system_cpu_count, process_start_time_seconds, process_files_*).
Actuator
- Actuator on by default (tri-state create_app(actuator_enabled=None) -> config),
Spring-exact web exposure (pyfly.management.endpoints.web.exposure.include default
"health,info"), base-path support, and PrometheusEndpoint now actually registered.
- /actuator/metrics: Micrometer JSON (dot meter names, COUNT/TOTAL_TIME/MAX,
availableTags, ?tag= drill-down); /actuator/prometheus pinned to version=0.0.4.
- New endpoints: configprops, mappings, scheduledtasks, threaddump, caches (+ /{name}),
conditions, httpexchanges (in-memory recorder); /actuator/beans contexts envelope.
- Generic route now supports selector sub-routes; loggers use Spring shape
(WARN/OFF names, GET/POST /loggers/{name} {configuredLevel}, groups).
- Shared pyfly.actuator.wiring used by BOTH Starlette and FastAPI adapters (the
FastAPI adapter was badly behind: no metrics filter, no exposure, no prometheus).
Health
- Severity-based status aggregation (UP/UNKNOWN/OUT_OF_SERVICE/DOWN), 503 for
DOWN/OUT_OF_SERVICE, show-details/show-components config, /health/{component|group}.
- Registered the DB and CQRS health indicators as beans (never were) and fixed
CqrsHealthIndicator (sync->async, HealthStatus) and OrchestrationHealthIndicator
(check()->health()) so they conform to the protocol and contribute to /actuator/health.
Configuration (Spring-style YAML, pyfly-namespaced)
- Relaxed binding (kebab-case YAML -> snake_case fields), env-var overrides now visible
to bind()/get_section() and type-coerced, effective_dict(), ordered property_sources(),
and secret masking (incl. URI userinfo passwords).
- Admin Configuration/Environment views: sorted, deep-flattened, source-attributed,
masked (Spring Boot Admin / /actuator/env shape). Admin Metrics view unified to the
Prometheus/Micrometer names (dropped the invented process.*/python.* scheme).
- Adopted Spring's `management.*` key structure under the pyfly namespace.
Fixes (test environment / installer)
- mongomock + beanie 2.x: compat shim so init_beanie's list_collection_names kwargs
don't blow up the in-memory Mongo tests (tests/data/conftest.py).
- install.sh find_python now probes versioned interpreters (python3.13/3.12) so it
doesn't wrongly abort on hosts whose bare `python3` is old (macOS/Homebrew/uv).
Full test suite green (3386 passed), ruff + ruff format + mypy clean.
…trics SSE, configprops kebab keys Completes the three deferred follow-ups from the Spring Boot parity work. @timed / @counted Micrometer parity - `timed()` now accepts a Micrometer dot.case meter name -> Prometheus `<name>_seconds` timer, tagged class/method/exception (+ extra_tags). - `counted()` -> Prometheus `<name>_total` counter, tagged class/method/result(success|failure)/exception (+ extra_tags). - Both default to Micrometer-style names (`method.timed`/`method.counted`). Admin metrics SSE is now push-based (was names-only) - MetricsProvider.get_metric_values() returns a {name: total-value} snapshot (summed across label sets); the /admin/api/sse/metrics stream now emits values. - The admin Metrics live trend is driven by the SSE stream (no per-tick REST polling); the measurements table remains a per-label snapshot. configprops - /actuator/configprops now emits kebab-case property keys (the YAML form users actually write, e.g. `event-loop`) instead of the bound dataclass field names. Tests updated/added (timed/counted, metrics SSE values, configprops kebab, integration). Full suite green (3388 passed); ruff + format + mypy clean.
Contributor
Author
|
Pushed the three remaining follow-ups (commit c7548a2):
Full suite: 3388 passed, ruff + format + mypy clean. This PR now covers the complete Spring Boot parity effort end-to-end. |
added 4 commits
June 4, 2026 18:01
…uilt-in ProcessCollector on Linux ProcessMetricsCollector emitted process_start_time_seconds, which prometheus_client's own ProcessCollector also exports on Linux (a no-op on macOS — hence green locally, red in CI). REGISTRY.register() raised 'Duplicated timeseries', the except swallowed it, and NO process meters registered. Fix: describe() returns empty (so registration never pre-collects/collides) and collect() skips any meter name another collector already provides. process_uptime_seconds, system_cpu_count, process_cpu_usage register cleanly; process_start_time_seconds is served by the built-in collector on Linux and by ours on macOS.
ancongui
pushed a commit
that referenced
this pull request
Jun 4, 2026
…lassName, guard prometheus endpoint (audit #153/#161/#162) #153 — TracingAutoConfiguration.tracer_provider now installs a BatchSpanProcessor with an exporter selected from config (pyfly.observability.tracing.exporter = otlp | console | none). When unset, OTLP is auto-selected iff an endpoint is configured (pyfly.observability.tracing.otlp.endpoint or the standard OTEL_EXPORTER_OTLP_ENDPOINT env var), with graceful fallback + a warning if the OTLP exporter package is missing; otherwise a single info line is logged so the drop is no longer silent. Previously the provider had no processor, so every @span trace was recorded and immediately discarded. #161 — threaddump now reports className as the frame's module (__name__) and methodName as the qualified function name (co_qualname), instead of setting both to the bare function name. Walks the frame chain so co_filename/co_name/module are read per frame. #162 — prometheus endpoint: enabled now reflects prometheus_client availability, and handle() returns a 503 text body instead of raising TypeError when the client is absent at request time. Tests: tests/observability/test_wave_observability.py (4), tests/actuator/test_wave_actuator.py (3). Deferred (observability wave-2): #154 trace/correlation IDs into log records, #155 W3C traceparent extraction into OTel context, #157 config-driven health groups, #158 firefly.health.status gauge, #159 common metric tags, #160 actuator beans unused by route builder. #156 (default web exposure incl. metrics/prometheus) intentionally NOT changed — pyfly deliberately mirrors Spring Boot's secure-by- default exposure (health,info), set in commit #25; auto-exposing metrics would be a security regression. Suite 3488 -> 3495 green; ruff + mypy clean.
ancongui
added a commit
that referenced
this pull request
Jun 5, 2026
* fix(di): DI-core keystones from full-parity audit (Wave 0) Fixes audit findings that affect every bean lifecycle: - #113 interface-typed @bean was registered under both its concrete and return type, so @post_construct / AOP weaving / every BeanPostProcessor ran TWICE on the same instance. Startup lifecycle, wiring, and pre_destroy passes now de-duplicate by instance identity and propagate the woven instance to all aliases. - #114 synchronous @around advice was silently dropped; _build_sync_wrapper now builds a sync around chain mirroring the async one. - #115 a synchronous @app_event_listener crashed startup (await on None); ApplicationEventBus.publish awaits only awaitable results. - #116 TRANSIENT @bean factory methods were ignored on resolution (rebuilt via __init__); Registration now carries a factory closure that the container prefers. - #117 conditional_on_property gains match_if_missing and Spring-correct empty-having_value semantics (present-and-not-"false"). - #119 app-event-type inference no longer mistakes the return annotation for the event type. Full suite 3397 passed; ruff + mypy clean. * fix(web): post-start dynamic wiring keystone (Wave 0, audit #40/#163) create_app() builds the WebFilter chain and collects routes BEFORE ApplicationContext.start(), so beans only instantiated during startup were silently dropped: security/session/CSRF WebFilters never ran, and @bean-produced controllers (orchestration, IDP, config-server) were unreachable. Both the Starlette and FastAPI adapters now re-run filter discovery and controller/ws/sse/OAuth2-login route collection inside the wrapped lifespan, after start(): - WebFilterChainMiddleware holds the live filter list so in-place additions are seen per request. - Filter/route discovery is factored into re-runnable, identity/path deduplicated closures; FastAPIControllerRegistrar.register_controllers is now idempotent. - The rescan is exposed via app.state.pyfly_install_dynamic_wiring for callers managing their own lifespan. This unblocks the security (#41/#42/#43/#44), admin (#66/#70), orchestration-web (#163), config-server (#83) and CORS (#204) fixes. Full suite 3399 passed; ruff + mypy clean. * fix(transactional): workflow + TCC + scheduling parity (audit #53-65) Orchestration was largely inert because nothing wired the declared beans: - #53 @saga/@workflow/@tcc beans are now auto-discovered into their registries via a new OrchestrationBeanPostProcessor (mirrors Java ensureScanned) — previously every declared orchestration was dead. - #54 @scheduled_saga/@scheduled_workflow/@scheduled_tcc are turned into live ScheduledTasks and the OrchestrationScheduler (already started by the context lifecycle) now fires them; register() spins up loops post-start. - #55 wait_for_any no longer swallows timeouts and captures the winning signal payload. - #56 TCC participant retry is N+1 attempts (was off-by-one) and falls back to class-level @tcc(retry_enabled, max_retries, backoff_ms). - #57 per-participant try/confirm/cancel errors + latency are recorded so TccResult.failed_participants() works. - #58 @on_workflow_error(suppress_error/error_types/step_ids) downgrades FAILED→COMPLETED. - #59 @workflow_step(condition=...) is evaluated via a new safe AST expression evaluator (no eval); false → step SKIPPED. - #60 async_ steps are fire-and-forget; #62 fire-and-forget run tasks hold a strong reference so they aren't GC-cancelled. - #61 workflow-level retry propagates to steps that declare none. - #63 @WaitForAll/@WaitForAny support timers; #64 named timer_id surfaced in the timer event; #65 warns on a compensatable step with no comp method. Full suite 3411 passed; ruff + mypy clean. * fix(security): wire JWT/OAuth2/CSRF/HttpSecurity filters + session + OIDC (audit #41-52) The keystone (#40/#163) re-scans WebFilter beans post-start, but several filters had no bean at all and were dead code: - #41 SecurityFilter (symmetric JWT, opt-in via pyfly.security.jwt.filter.enabled) and OAuth2ResourceServerFilter are now registered as WebFilter beans by the security auto-configs that own their JWTService/JWKSTokenValidator (avoids cross-config bean ordering); the post-start rescan adds them to the chain. - #42 CsrfFilter gains a CsrfFilterAutoConfiguration (pyfly.security.csrf.enabled) and its order is corrected to run before authentication (was -50, after). - #45 @pre_authorize/@post_authorize now enforce once the auth filter populates RequestContext.security_context (integration test added). - #46 RedisSessionStore round-trips dataclass attributes (SecurityContext) via a type-tagged JSON encoder/decoder instead of failing on json.dumps. - #47 HttpSecurityFilterAutoConfiguration builds the URL-rule filter from a user HttpSecurity bean. - #48 OAuth2 login validates the OIDC id_token against the provider JWKS + nonce before trusting claims; #49 surfaces token/userinfo failures as 401 (was a silently-stored anonymous session) and wraps httpx transport errors. - #50 security expression evaluator accepts double-quoted args; #51 fixes the session-filter ordering docstring; #52 session cookie gains a configurable Secure flag + sliding re-issue. Full suite 3415 passed; ruff + mypy clean. * fix(client): error mapping, retry filtering, lifecycle, timeout (audit #12-15,#18) - #12 (critical) the declarative client never checked HTTP status and returned 4xx/5xx error payloads as success. New src/pyfly/client/exceptions.py adds a ServiceClientException hierarchy + map_http_error (400→Validation, 401/403→ Authentication, 404→NotFound, 409→Conflict, 422→Unprocessable, 429→RateLimit, 5xx→Unavailable); the generated impl raises on status >= 400. - #13 retry now retries only transient failures (429/5xx-mapped, connection, timeout) — not 4xx or the circuit-open signal — and the wrapping order is fixed to circuit-breaker OUTSIDE / retry INSIDE so an OPEN circuit fails fast. - #14 HttpClientBeanPostProcessor gains start()/stop(); the context lifecycle now closes every per-bean httpx client (was a connection-pool leak). - #15 the post-processor builds adapters with the configured pyfly.client.timeout (declarative clients previously ignored it). - #18 a declarative method's param is sent as request headers (for per-call headers / auth injection) instead of as a query param. Remaining client parity features (#16 OAuth2 client-credentials, #17 service discovery, #19 request dedup, #20 client rate limiting) are net-new subsystems tracked for a follow-up pass. Full suite 3419 passed; ruff + mypy clean. * fix(idp): REST controller, provider selection, adapter bugs (audit #22,#23,#25,#26,#28,#29) - #22 (critical) the IdpAdapter bean was wired to nothing — no HTTP surface existed. New src/pyfly/idp/web.py adds an IdpController (/idp/login, /refresh, /logout, /introspect, /admin/users CRUD, /admin/roles) registered in the same auto-config as the adapter and mounted by the post-start route rescan. - #25 provider selection: pyfly.idp.provider chooses internal-db | keycloak | cognito | azure-ad and builds the adapter from config (was hardwired to internal-db); the bean is now exposed under the IdpAdapter port type. - #23 Cognito login computes SECRET_HASH = Base64(HMAC-SHA256(secret, user+id)) when a client secret is configured (auth previously failed for secret clients). - #26 Keycloak admin token is refreshed on expiry (tracked via expires_in) — it was cached forever and broke after the ~60s TTL. - #28 Cognito login/refresh guard a missing AuthenticationResult and surface a clean PermissionError; refresh keeps a rotated refresh token. - #29 internal-db assign_role populates the role catalogue + adds create_roles (list_roles was always empty). Remaining idp parity (deferred): #24 port capability expansion (MFA/sessions/ scopes/userinfo/register), #27 Azure AD local token parsing + MFA. Full suite 3424 passed; ruff + mypy clean. * fix(notifications): wire real providers + delivery correctness (audit #30,#31,#35,#36,#38) - #30 (critical) auto-config only ever registered Dummy providers and ignored any provider-selection config. It now reads pyfly.notifications.{email,sms,push}.provider and builds the real adapter (SendGrid/Resend/SMTP, Twilio, Firebase) from config, falling back to Dummy; providers are exposed under their port types so the services inject the port. - #31 the SMTP adapter now attaches EmailMessage.attachments (was silently dropped); #35 it also adds custom EmailMessage.headers. - #36 the Default{Email,Sms,Push}Service convert a provider exception into a structured FAILED NotificationResult instead of propagating. - #38 ResendEmailProvider honors a configured default_from sender. Deferred (net-new subsystems): #32 template-email engine, #33 channel preferences, #34 notification metrics, #37 FCM batch. Full suite 3428 passed; ruff + mypy clean. * fix(cli): add PyFlyApplication.run() so the CLI archetype boots (audit #1) The generated `cli` archetype's main() does asyncio.run(pyfly.run()), but PyFlyApplication had no run() — every generated CLI crashed with AttributeError. run() now starts the app, resolves the ShellRunnerPort (populated with @shell_command handlers during startup), dispatches the args (or sys.argv[1:], falling back to an interactive REPL), and shuts down. Full suite 3431 passed; ruff + mypy clean. * fix(cache): serializer, putIfAbsent, prefix-evict, stats, health, null-cache (audit #72-80) - #72 a new cache serializer tolerates datetime/Decimal/UUID/set/Pydantic so the Redis put path no longer throws TypeError on framework types. - #75 put_if_absent added to the protocol + both adapters (Redis SET NX, in-memory check-then-set). - #78 evict_by_prefix added (Redis SCAN+delete, in-memory prefix filter). - #76 both adapters track hit/miss/eviction counters and report hit_rate. - #79 RedisCacheAdapter.start() degrades gracefully (logs + marks unavailable) instead of aborting application startup when Redis is unreachable. - #74 new CacheHealthIndicator (put/get/evict probe + latency) registered by the cache auto-config so /actuator/health reports cache status. - #80 @cacheable/@cache distinguish a cached None from a miss via exists() (null caching / cache-penetration protection). Deferred: #73 SmartCacheAdapter (L1/L2 read-through), #77 cache metrics, #81/#82 CacheManager protocol completeness. Full suite 3437 passed; ruff + mypy clean. * fix(cqrs): metrics registry, deterministic cache key, correlation, validation metric (audit #94,#98,#99,#100) - #94 cqrs_metrics_service now receives the observability MetricsRegistry via optional injection, so CQRS metrics are actually recorded (was permanently no-op). - #100 Query.get_cache_key() uses a stable SHA-256 digest instead of the process-randomized builtin hash(), so the same query maps to the same key across processes/restarts. - #98 the command bus restores the prior correlation id in a finally block so it no longer leaks into the next command on the same task. - #99 a CqrsValidationException now records the validation-failure metric. Deferred (involved wiring): #93 EventDrivenCacheInvalidator, #95 @publish_domain_event, #96 CqrsMetricsEndpoint, #97 publisher destination/headers. Full suite 3439 passed; ruff + mypy clean. * fix(scheduling/resilience): sync time-limiter, Spring cron, scheduled-failure logging (audit #184,#185,#186) - #184 the synchronous time_limiter ran the callable in a worker thread and waits with a fractional timeout instead of SIGALRM, which truncated sub-second timeouts to whole seconds and crashed off the main thread. - #185 CronExpression accepts Spring-style 6-field (seconds-first) cron and the '?' day placeholder (via croniter second_at_beginning + normalization), not just standard 5-field cron. - #186 _invoke logs exceptions, so a failing cron / fixed-rate iteration (whose task the loop does not await) is reported instead of vanishing silently. Full suite 3443 passed; ruff + mypy clean. * fix(validation): ISO-4217 currency, IBAN length, card-scheme checks (audit #191,#192,#193) - #191 is_valid_currency_code checks membership of the active ISO 4217 set (was: any 3 uppercase letters). - #193 is_valid_iban enforces the country-specific total length per ISO 13616 (was: general format + mod-97 only). - #192 is_valid_credit_card requires a recognized scheme prefix (Visa, MC, Amex, Discover, Diners, JCB) before the Luhn check (was: Luhn only). Full suite 3446 passed; ruff + mypy clean. * fix(callbacks): SSRF domain allowlist + retry only transient failures (audit #190,#194) - #190 the dispatcher enforces config.authorized_domains before delivery: a target host not matching the allowlist is marked FAILED ("Domain not authorized") and no HTTP request is made (SSRF protection). - #194 retries are limited to transient failures (5xx/408/429 + transport errors) with exponential backoff capped at 5 min; a 4xx client error is now permanent (was: retry every non-2xx with fixed backoff). Full suite 3448 passed; ruff + mypy clean. * fix(data): soft-delete read paths + connection-pool config (audit #103,#107) - #103 SoftDeleteRepository overrides find_paginated, find_all_by_ids, find_all_by_spec and find_all_by_spec_paged to apply the not-deleted predicate (incl. the page-total count) — these inherited base readers previously leaked soft-deleted rows and miscounted pagination totals. - #107 the SQLAlchemy engine forwards configured connection-pool settings (pool.size/max-overflow/timeout/recycle/pre-ping) to create_async_engine; only url and echo were read before. Full suite 3450 passed; ruff + mypy clean. * fix(eventsourcing): fail on unhandled events, replay validation, snapshot interval (audit #146,#150,#151) - #146 AggregateRoot._dispatch raises EventHandlerException when an event has no registered handler (or on_<type> method) instead of silently swallowing it and corrupting reconstructed state. - #150 EventSourcedRepository.load validates each replayed envelope's aggregate_id / aggregate_type against the loaded aggregate. - #151 a snapshot is taken when the appended batch CROSSES a multiple of the snapshot interval (was: exact 'version % interval == 0', which is missed when a batch straddles the threshold). Full suite 3451 passed; ruff + mypy clean. * fix(rule-engine): fail on unknown actions, isolate action failures, validate 'not' (audit #215,#216,#222) - #215 the default evaluator raises NotImplementedError for unsupported action types ('call'/'calculate'/typos) instead of silently doing nothing. - #216 each action runs in isolation: a failing action records its error on the EvaluationResult and the remaining actions still execute. - #222 the 'not' operator requires exactly one child; empty/multiple children raise instead of silently using only the first. Full suite passed; ruff + mypy clean. * chore: gitignore local .audit/ remediation working files * fix(eda): @event_listener discovery, publish timing, serializer/consumer config (audit #134,#138,#140,#141) - #134 @event_listener now stamps discovery metadata and supports a bus-less form (@event_listener(["user.*"])); a new ApplicationContext._wire_event_listeners pass auto-subscribes decorated methods to the EventPublisher bean during startup (mirrors @message_listener) — previously it only worked with a hand-wired bus. - #140 @event_publisher AFTER/BOTH publishes a payload augmented with the method result instead of re-publishing the pre-call arguments. - #138 the EDA auto-config selects the serializer from pyfly.eda.serialization-format (json|avro|protobuf) and passes it to the Kafka/Redis adapters (Avro/Protobuf remain opt-in stubs). - #141 the Redis adapter receives the configured consumer-id. Deferred (cross-adapter dispatch refactor): #131 DLQ routing, #132 ErrorStrategy, #133 circuit-breaker wiring, #135 postgres outbox stall, #136 event filters, #137 metrics, #139 health probe. Full suite 3457 passed; ruff + mypy clean. * fix(config): placeholder env overrides + config-client merge order (audit #86,#87,#89) - #86 ConfigClient applies remote propertySources in reverse (lowest priority first) so the highest-priority source wins, matching Spring (was: forward order, so the lowest-priority source won). - #87/#89 inline ${...} placeholder resolution now consults the PYFLY_* relaxed env mapping (e.g. ${app.name} → PYFLY_APP_NAME) before the raw file data, so env overrides are visible to placeholders and win. Full suite passed; ruff + mypy clean. * fix(di): @config_properties classes are injectable beans bound from config (audit #118) The @config_properties decorator now marks the class injectable (__pyfly_injectable__), so the scanner registers it; ApplicationContext binds each registered __pyfly_config_prefix__ class from the active Config via a factory, making it constructor-injectable by type (Spring @EnableConfigurationProperties equivalent). Previously these classes were only usable when a @bean method manually called config.bind(). Full suite green; ruff + mypy clean. * fix(saga): honor configured compensation_policy (audit #170; #178 already done) - #170 SagaEngineProperties is now bound from pyfly.transactional.saga.* (was built with defaults), SagaEngine stores a default_compensation_policy, and execute() falls back to it when a caller does not override — the configured YAML compensation_policy was previously inert (engine always STRICT_SEQUENTIAL). - #178 (scheduled sagas) was already addressed by the OrchestrationBeanPostProcessor added in b2d0f4e, which schedules @scheduled_saga beans. Deferred (compensator/orchestrator internals): #171 compensation-error abort, #172 idempotency keys, #173 recovery scheduling, #174-177, #179. Full suite 3461 passed; ruff + mypy clean. * fix(server/websocket): serve_async honors full config + ws on_disconnect (audit #226,#232) - #226 a shared UvicornServerAdapter._build_kwargs feeds both serve() (uvicorn.run) and serve_async() (uvicorn.Config), so the async path honors SSL / keep-alive / backlog / graceful-timeout / concurrency limits that the blocking path already did (they were previously dropped from serve_async). - #232 the websocket endpoint logs unexpected handler exceptions (no longer swallowed) and invokes an on_disconnect lifecycle hook when the controller defines one. Full suite passed; ruff + mypy clean. * fix(web): RFC-compliant required params, CORS auto-config, exception-converter wiring (audit #202/#204/#206) #206 — Missing required QueryParam/Header/Cookie now raises a 400 (InvalidRequestException) instead of silently binding None. A param is optional only when it has a default or an Optional inner type (str | None). Surfaced two latent transactional filter params that were meant to be optional; their inner types are now honestly str | None. #204 — CORS is auto-configured from pyfly.web.cors.* via CORSConfig.from_config(), threaded into create_app for both the Starlette and FastAPI adapters when no explicit CORSConfig is passed. Secure-by-default (disabled unless pyfly.web.cors.enabled), mirroring Spring CorsAutoConfiguration. #202 — ExceptionConverterService is now wired into global_exception_handler. Non-PyFly exceptions are translated via the built-in chain (Pydantic, JSON, timeout) plus any user @bean ExceptionConverter, discovered at startup and stashed per-app on app.state. Falls back to 500 only when nothing matches. Tests: tests/web/test_wave_web.py (14). Suite 3462 -> 3476 green; ruff + mypy clean. * fix(ecm): provider-based storage/e-signature adapter selection + honor delete result (audit #120/#125) #120 — EcmAutoConfiguration now selects the storage adapter from pyfly.ecm.storage.provider (local | s3/aws | azure) and the e-signature adapter from pyfly.ecm.esignature.provider (noop | docusign | adobe | logalty), reading each provider's own config sub-keys. All producer beans (storage, metadata, folders, e-signature) now return their PORT types so DocumentService / ESignatureService inject abstractions; the container registers each bean under both the concrete type and the port, so port-typed injection resolves. Previously the adapter classes existed but were dead — only LocalFilesystem + NoOp were ever wired regardless of config. #125 — DocumentService.delete now returns storage_ok AND metadata_ok instead of discarding the storage delete result, so a failed blob delete is surfaced. Tests: tests/ecm/test_wave_ecm.py (12) incl. a real-ApplicationContext wiring test that round-trips an upload/download through the auto-wired ports. Deferred (per-module): #121 DocumentVersionPort, #122 signature lifecycle/proof ports (net-new subsystems), #126 version refetch, #127/#128/#129 provider get() polish, #130 folder hierarchy. #124 (enabled-by-default) intentionally NOT flipped: every sibling optional module (cache/notifications/eventsourcing/ rule-engine/callbacks/idp) is uniformly opt-in via enabled=true; flipping only ECM would make it the lone default-on module — a deliberate pyfly convention. Suite 3476 -> 3488 green; ruff + mypy clean. * fix(observability,actuator): wire span exporter, correct threaddump className, guard prometheus endpoint (audit #153/#161/#162) #153 — TracingAutoConfiguration.tracer_provider now installs a BatchSpanProcessor with an exporter selected from config (pyfly.observability.tracing.exporter = otlp | console | none). When unset, OTLP is auto-selected iff an endpoint is configured (pyfly.observability.tracing.otlp.endpoint or the standard OTEL_EXPORTER_OTLP_ENDPOINT env var), with graceful fallback + a warning if the OTLP exporter package is missing; otherwise a single info line is logged so the drop is no longer silent. Previously the provider had no processor, so every @span trace was recorded and immediately discarded. #161 — threaddump now reports className as the frame's module (__name__) and methodName as the qualified function name (co_qualname), instead of setting both to the bare function name. Walks the frame chain so co_filename/co_name/module are read per frame. #162 — prometheus endpoint: enabled now reflects prometheus_client availability, and handle() returns a 503 text body instead of raising TypeError when the client is absent at request time. Tests: tests/observability/test_wave_observability.py (4), tests/actuator/test_wave_actuator.py (3). Deferred (observability wave-2): #154 trace/correlation IDs into log records, #155 W3C traceparent extraction into OTel context, #157 config-driven health groups, #158 firefly.health.status gauge, #159 common metric tags, #160 actuator beans unused by route builder. #156 (default web exposure incl. metrics/prometheus) intentionally NOT changed — pyfly deliberately mirrors Spring Boot's secure-by- default exposure (health,info), set in commit #25; auto-exposing metrics would be a security regression. Suite 3488 -> 3495 green; ruff + mypy clean. * fix(i18n,resilience): MessageFormat quote-escaping + unified bulkhead permit accounting (audit #187/#189) #187 — ResourceBundleMessageSource._substitute now honors java.text.MessageFormat quote semantics: '' renders as a literal apostrophe, single-quoted text is copied literally (so '{0}' is not substituted), and {n}/{n,type,style} insert the positional arg (index parsed; format style is a documented non-locale-applied subset). Previously naive str.replace left ''/quoted placeholders wrong. #189 — Bulkhead now uses a single threading.Lock-guarded permit counter as the sole source of truth for BOTH sync and async decorated calls (the divergent asyncio.Semaphore is removed). The sync wrapper now goes through the same _acquire_slot/_release_slot primitive as the async path, so a Bulkhead shared across sync + async can no longer desynchronise its capacity accounting; the atomic check-and-increment also closes the async check/acquire TOCTOU. Tests: tests/i18n/test_wave_i18n.py (8), tests/resilience/test_wave_resilience.py (4). #188 (i18n MessageSource opt-in vs Spring default-on) intentionally NOT changed — the finding itself says opt-in is acceptable if intentional; every sibling optional module is uniformly opt-in via enabled=true (same convention as #124/#156). Suite 3495 -> 3507 green; ruff + mypy clean. * fix(plugins): extension-point registration/validation + unload unregisters extensions (audit #218/#219) #218 — ExtensionRegistry now records extension points: register_extension_point, has_extension_point, extension_point_ids. PluginManager.add() scans inner @extension_point classes (the decorated class is the point's interface type) and registers them BEFORE extensions, so register() validates that each contributed extension is an instance of its declared point type and raises ValueError otherwise — mirroring Java's DefaultExtensionRegistry. Previously the @extension_point decorator's metadata was pure dead code, never read by anything. Registration against an id with no declared point type stays lenient (backward-compatible) rather than raising on unknown point. #219 — PluginManager now tracks the (point_id, instance) pairs each plugin registers and adds remove(plugin_id) + unload_all(): unloading a plugin runs its unload hook, unregisters its extensions from the registry, and forgets it, so extensions no longer leak in the registry for the process lifetime after a plugin is unloaded (mirrors Java's unloadPlugin -> unregisterPluginExtensions). Tests: tests/plugins/test_wave_plugins.py (8). #220 (rule-engine audit trail) deferred — net-new subsystem (AuditTrailService/ entity/repository/controller + evaluate-with-audit); out of scope for this pass, documented like other net-new subsystem gaps. Suite 3507 -> 3515 green; ruff + mypy clean. * fix(orchestration-web): DLQ /count endpoint + in-flight default for GET /executions (audit #167/#169) #167 — DeadLetterController now exposes GET /api/orchestration/dlq/count, backed by a new DeadLetterService.count() and DeadLetterStore.count() (InMemory impl returns len). Java's DeadLetterController exposes /count for dashboards; pyfly previously 404'd on it and had no service-level count(). #169 — OrchestrationController.list_executions now defaults to in-flight (non-terminal) executions when no status filter is given, instead of returning the entire store including terminal history — matching Java's findInFlight() default. Explicit ?status=<S> still filters exactly. Uses the existing ExecutionStatus.is_terminal. Tests: tests/transactional/test_wave_orchestration_web.py (4). Deferred (orchestration-web): #163 already done (wave-0 route rescan). #164 WorkflowController 2->14 endpoints + WorkflowEngine cancel/suspend/resume/ find_by_correlation_id/find_by_status (large engine+REST expansion), #165 per-controller rest.enabled/health.enabled gates, #166 orchestration-metrics actuator endpoint, #168 DLQ retry real re-execution (needs engine re-enqueue). Suite 3515 -> 3519 green; ruff + mypy clean. * fix(starters): correct property keys so bundled adapters actually activate (audit #2/#3/#4/#6) Starter stacks set property keys their auto-configs never read, so the bundled adapters/services silently never activated (the old keys had ZERO consumers). Fixed to the keys the conditions actually gate on: #2 data/domain: pyfly.relational.enabled -> pyfly.data.relational.enabled pyfly.document.enabled -> pyfly.data.document.enabled #3 core: pyfly.eda.enabled -> pyfly.eda.provider=auto (EdaAutoConfiguration gates on pyfly.eda.provider; 'auto' resolves to the best installed broker, falling back to the in-memory bus) #4 application: pyfly.security-jwt.enabled / pyfly.security-password.enabled / pyfly.session-filter.enabled -> pyfly.security.enabled=true (JWT auth WebFilter stays opt-in via pyfly.security.jwt.filter.enabled) #6 core/web: pyfly.actuator.enabled / pyfly.actuator.metrics.enabled -> pyfly.web.actuator.enabled Scaffold (pyfly.yaml.j2): added pyfly.security.enabled to the security feature block so generated security apps wire JWTService + the password encoder. Left the 'eda' feature's messaging: block as-is — pyfly.messaging.provider drives the SEPARATE MessagingAutoConfiguration (MessageBrokerPort), a real distinct subsystem, and is covered by an existing test; it is not the EDA EventPublisher path. Updated the existing test_starters.py assertions (old keys) in lockstep. Tests: tests/starters/test_wave_starter_keys.py (8, functionally proving each auto-config's conditions now pass via ConditionEvaluator with deps installed) + 1 scaffold security test. Deferred: #5 test-slice decorators (net-new), #10 testcontainers (net-new), #11 inert pyfly.web/observability/metrics/tracing/resilience/aop.enabled keys (deliberate convention: those auto-configs activate on @conditional_on_class, not property flags). Suite 3519 -> 3528 green; ruff + mypy clean. * fix(testing,cli,shell): inject mock_bean, thread wizard package name, honor shell_option type/choices (audit #7/#8/#9) #7 — PyFlyTestCase.setup() now registers every mock_bean(...) descriptor's AsyncMock into the context container keyed on its bean type, so DI-resolved collaborators receive the mock. Previously the descriptors produced a mock on the test instance but it was never wired into the ApplicationContext, so context.get_bean(T) returned the real/absent bean. #8 — generate_project()/_build_context() accept an explicit package_name; the interactive 'pyfly new' wizard's collected package name is now threaded through instead of being discarded and recomputed from the project name. #9 — @shell_option(type=...) / @shell_argument(type=...) overrides are now honored by param inference (param_type = explicit type or inferred inner_type), and shell_option gained a choices= kwarg that flows onto the resolved ShellParam. Tests: tests/testing/test_wave_mock_inject.py (2), tests/cli/test_wave_new_package.py (4), tests/shell/test_wave_shell_option.py (4). Suite 3528 -> 3538 green; ruff + mypy clean. * fix(config-server): full overlay propertySources + configurable persistent backend root (audit #85/#88) #85 — ConfigServer.fetch now emits the standard Spring-Cloud-Config overlay set (highest priority first): {app}/{profile}, {app}/default, application/{profile}, application/default — deduped, skipping absent overlays, returning None only when all are absent. Previously it returned a single propertySource, so an app's default bundle and shared application config were never delivered. #88 — ConfigServerAutoConfiguration.config_backend now reads pyfly.config-server.backend.root (or .native.search-locations) and persists there, falling back to a throwaway tempdir only when nothing is configured. Previously every boot used a fresh mkdtemp() that served nothing and lost saves on restart. The bean now injects Config (same pattern as the actuator beans). Tests: tests/config_server/test_wave_config_server.py (7). Existing test_server_returns_property_sources stays green (single source -> [0] unchanged). Suite 3538 -> 3545 green; ruff + mypy clean. * fix(config): relaxed kebab/snake key + placeholder resolution, env-only key injection (audit #90/#92) #92 — _raw_get() and ${...} config-reference resolution now use relaxed (kebab/snake interchangeable) segment matching via a new _dict_get_relaxed helper, with an exact-match fast path first. So ${my-prop.sub-key} resolves a value stored under my_prop.sub_key (and vice versa), matching bind()'s relaxed semantics; previously get()/placeholders required literal key segments. #90 — effective_section()/bind() now inject env-only keys: a PYFLY_<PREFIX>_* environment variable with no corresponding file leaf is materialized into the section (scoped to the prefix, treating each '_' as a path separator, only adding absent leaves). So an env-only @config_properties field now binds, where before get() honored the env var but bind()/effective_section dropped it. The rootless ('') prefix is skipped (ambiguous without a section anchor), so the broad effective_dict() actuator/admin view is unchanged. Tests: tests/core/test_wave_config_relaxed.py (9). Full suite green across the broad _raw_get change. Suite 3545 -> 3554 green; ruff + mypy clean. * fix(config-server): mount ConfigServer HTTP routes (audit #83) The ConfigServer bean was built but never reachable over HTTP, so a config server served nothing. Added a Starlette adapter (config_server/adapters/starlette.py, the hexagonal starlette boundary) exposing Spring-Cloud-Config routes (GET/POST /{application}/{profile}[/{label}], GET /_list), and a config_server/wiring.py build_config_server_routes(context) that discovers the ConfigServer bean and mounts them when pyfly.config-server.enabled=true. Because the bean is only instantiated during ApplicationContext.start() (after create_app returns), the routes are added in BOTH the eager create_app path (for an already-started context) AND the post-start _install_dynamic_wiring rescan, deduped by route key — same keystone pattern as orchestration/admin controllers. Mirrored into the FastAPI adapter's _install_context_routes. base_path is read from pyfly.config-server.base-path (default root). Tests: tests/config_server/test_wave_config_server.py — adapter round-trip + routes-mounted-when-enabled (real ApplicationContext + TestClient) + absent-when- disabled. Hexagonal isolation test stays green (starlette only in the adapter). Suite 3554 -> 3557 green; ruff + mypy clean. * fix(config): import remote config from a config server at bootstrap (audit #84) PyFlyApplication.__init__ now invokes ConfigClient when pyfly.cloud.config.uri (or pyfly.config.import) is set, fetching remote config from a Spring-Cloud-Config server and deep-merging it as a HIGH-precedence source over local config. The flat dotted propertySource map is expanded to nested via _merge_dotted; the source is recorded in loaded_sources. Runs before the logging adapter/context exist (uses a stdlib logger). Fully optional and non-fatal by default — a missing httpx, unreachable server, or non-200 logs a warning and falls back to local config — unless pyfly.cloud.config.fail-fast=true. Guards against asyncio.run inside an already- running loop (skips with a warning). No-op (early return) when no uri is set, so every existing app is unaffected. Tests: tests/config_server/test_wave_remote_config.py (4, sync so asyncio.run works): merged-at-bootstrap, no-uri-skips, failure-non-fatal, fail-fast-propagates. config-server module now fully remediated (#83/#84/#85/#86/#87/#88/#89/#90/#92). Suite 3557 -> 3561 green; ruff + mypy clean. * fix(admin): enforce require_auth, TRACE/OFF log levels, mount beans SSE (audit #66/#69/#71) #66 — AdminRouteBuilder now wraps every {base}/api/* route (data, mutation, SSE, instance registry) with an auth guard: when pyfly.admin.require-auth is true, a request lacking an authenticated SecurityContext gets 401 and one missing every allowed role gets 403 (read from the request-scoped RequestContext). The SPA shell + static assets stay public so the dashboard can boot and then surface the 401s. No-op when require_auth is false (default), so existing behavior is unchanged. Routes are now built from a guarded spec list. #69 — LoggersProvider.set_level resolves TRACE (->5, below DEBUG) and OFF (->CRITICAL+1, disables) explicitly; both were advertised as valid levels but getattr(logging, 'TRACE'/'OFF') is None, so selecting them silently returned 400. #71 — the beans SSE generator (admin/api/sse.py beans_stream) is now reachable via GET {base}/api/sse/beans; it existed but was never routed. Tests: tests/admin/test_wave_admin_auth.py (10). Suite admin 115 -> 125 green; ruff + mypy clean. * fix(admin): server-mode instance registry, client self-registration, health rescan (audit #67/#68/#70) #67 — create_app now binds AdminServerProperties and, when pyfly.admin.server.enabled=true, builds an InstanceRegistry, seeds it from the configured instances via StaticDiscovery, and passes it to AdminRouteBuilder — so the /api/instances routes mount and settings.serverMode reports true. Previously server mode was permanently off (registry never constructed). #68 — AdminClientRegistration gained start()/stop() lifecycle methods that honor auto_register (now threaded from AdminClientProperties), so the ApplicationContext infrastructure lifecycle self-registers the app with the remote admin server at startup and deregisters at shutdown. register() swallows its own errors, so a down admin server never aborts startup. #70 — when the actuator is disabled but admin is enabled, the admin-owned HealthAggregator now gets a post-start indicator rescan (appended to _extra_post_start), so the admin health view reflects HealthIndicator beans instantiated during start() instead of a frozen empty pre-startup snapshot. Tests: tests/admin/test_wave_admin_wiring.py (6). Admin module fully remediated (#66/#67/#68/#69/#70/#71). Suite 3561 -> 3577 green; ruff + mypy clean. * docs(release): v26.06.01 — refresh docs to match remediation, bump version - Bump version 26.06.00 -> 26.06.01 (pyproject 26.6.1, __version__, README badge). - CHANGELOG: comprehensive v26.06.01 entry covering the ~130-finding parity/wiring remediation (web required-params + CORS + exception converters; config relaxed binding + env-only keys + remote import; config-server HTTP routes/overlays/root; admin auth + server-mode + client registration + log levels + beans SSE; tracing exporter; starter property keys; ECM provider selection; plugin extension-point validation/unload; orchestration DLQ count + in-flight default; bulkhead permit accounting; i18n MessageFormat; shell option type/choices; mock_bean injection; pyfly new package name). - Docs refreshed against current code across web, configuration, config-server, admin, observability, actuator, ecm, plugins, resilience, transactional, validation, shell, testing, cli, error-handling, web-filters; new module guides for i18n and websocket; indexes updated. - fix(config): remote-config import reads pyfly.app.name (was the inconsistent pyfly.application.name), matching the framework-wide key. Suite 3577 green; ruff + mypy clean. * style: fix import ordering in 3 wave test files (ruff I001, CI lint) --------- Co-authored-by: Andrés Contreras Guillén <ancongui@Andress-MacBook-Pro.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings pyfly's observability, actuator, and configuration to drop-in Spring Boot parity — identical Micrometer metric names, the full
/actuator/*endpoint surface, and Spring-style YAML config semantics. Driven by a verified multi-agent audit (79 findings). Spring's key structure is adopted under thepyfly.*namespace (e.g.pyfly.management.endpoints.web.exposure.include), keeping legacy keys working.Full suite green: 3386 passed; ruff + ruff format + mypy clean.
Observability / metrics
http_server_requests_seconds(Summary_count/_sum+_maxgauge; optional histogram viamanagement.metrics.distribution.percentiles-histogram), taggedmethod/uri/status/outcome/exception.uriis the templated route (cardinality-safe), recovered from the matched Starlette endpoint →Route.path.MetricsFilteris now wired directly increate_app— it was a bean built after the middleware chain was assembled, so HTTP metrics were silently never collected.process_uptime_seconds,process_cpu_usage,system_cpu_count,process_start_time_seconds,process_files_*.Actuator
create_app(actuator_enabled=None)→ configpyfly.management.enabled), Spring-exact exposure (default onlyhealth+info; opt in via exposure include), base-path support, andPrometheusEndpointnow actually registered (it never was)./actuator/metrics: Micrometer JSON — dot meter names,COUNT/TOTAL_TIME/MAXstatistics,availableTags,?tag=k:vdrill-down./actuator/prometheuspinned toversion=0.0.4.configprops,mappings,scheduledtasks,threaddump,caches(+/{name}),conditions,httpexchanges(in-memory recorder)./actuator/beansnow uses thecontextsenvelope.GET/POST /loggers/{name}{configuredLevel}, groups).pyfly.actuator.wiringis now used by both the Starlette and FastAPI adapters — which also fixed the FastAPI adapter (it had no metrics filter, no exposure model, no prometheus).Health
UP/UNKNOWN/OUT_OF_SERVICE/DOWN),503forDOWN/OUT_OF_SERVICE,show-details/show-componentsconfig, and/actuator/health/{component|group}.CqrsHealthIndicator(sync→async, returnsHealthStatus) andOrchestrationHealthIndicator(check()→health()) so they conform to the protocol and contribute to/actuator/health.Configuration (Spring-style YAML)
bind()/get_section()and type-coerced,effective_dict(), orderedproperty_sources(), and secret masking (including passwords embedded in URI userinfo)./actuator/envshape). Admin Metrics view unified to the real Prometheus/Micrometer names (dropped the inventedprocess.*/python.*scheme).Test-environment / installer fixes
beanie2.x callslist_collection_names(authorizedCollections=…, nameOnly=…), whichmongomock_motorproxies straight to mongomock and blows up. Added a test-only compat shim (tests/data/conftest.py) so the in-memory Mongo suite runs — 62 mongo tests now pass.find_pythonnow probes versioned interpreters (python3.14/3.13/3.12) so it no longer wrongly aborts on hosts whose barepython3is an old build (macOS/Homebrew/uv) — the end-to-end installer tests now pass.Verification
pytest: 3386 passed (was 3101 + new tests; previously 2 installer + 116 mongo failures, all now resolved).ruff check,ruff format --check,mypyall clean across 589 source files.uritags, process meters, Micrometer metrics JSON, ordered+masked/actuator/env, and the new endpoints all return correct Spring-shaped payloads.Decisions
management.*structure under thepyfly.*prefix (user decision), legacy keys retained.health+infoonly).Not included (follow-ups)
@Timed/@CountedMicrometer dot-naming; admin metrics SSE streams values (currently names-only, REST-polled);configpropsemits bound field names (snake) vs original kebab keys.